ANN system and method based on a hierarchically pipelined graph index of computed stores

Abstract

This invention provides a computation-storage-based hierarchical pipelined graph indexing (ANNS) system and method. The system includes: a heuristic shard selection module, implemented by modifying the FTL, which performs best-priority search on the centroid graph index, prefetches high-priority shards to the FPGA during the convergence phase, and uses adaptive threshold pruning to filter low-relevance shards; a pause-free intra-shard pipeline module, implemented on the FPGA, which generates a distance result per clock cycle through a two-stage fully parallel distance calculation engine, and uses a double-buffered streaming Top-k sorter to filter nearest neighbors within the shard in real time; and a global result aggregation module, implemented by modifying the FTL, which maintains the global Top-k using a max-heap and terminates collection early based on the order of the shard results. These three modules are organized into a three-stage pipeline, and different query stages can overlap. This invention reduces I / O bottlenecks, improves the efficiency of distance calculation and result filtering, and increases the overall system throughput.

Description

Technical Field

[0001] This invention relates to the field of computer system architecture technology, and more specifically, to a hierarchical pipeline graph indexing (ANNS) system and method based on computational storage. Background Technology

[0002] Approximate Nearest Neighbor Search (ANNS) refers to the efficient search for several data points in a high-dimensional vector space that are most similar to the query vector, thereby providing high-performance vector database queries.

[0003] Current mainstream ANNS solutions are primarily built on Dynamic Random Access Memory (DRAM) or Solid State Drives (SSDs). DRAM's advantage lies in its extremely low memory access latency, often down to the nanosecond level, while SSDs offer massive storage capacity and low cost. However, with the continuous expansion of data volumes, DRAM-based ANNS solutions struggle to handle the storage demands of billions of high-dimensional vectors. In SSD-based ANNS solutions, I / O latency and throughput quickly become the main bottlenecks for ANNS workloads. Current SSD-based ANNS engines often rely on fine-grained random I / O operations, leading to significant read / write amplification. Furthermore, the storage wall problem—the fact that current mainstream CPUs have processing speeds far exceeding SSD response times—causes DRAM-based ANNS architectures to perform poorly when directly migrated to SSDs, while SSD-based optimizations remain constrained by I / O bottlenecks.

[0004] The emergence of Computational Storage Devices (CSDs) has provided a revolutionary solution to I / O bottlenecks. By integrating processor cores or Field-Programmable Gate Arrays (FPGAs) into flash memory, CSDs enable applications to offload data processing tasks to the device, thereby minimizing data movement, alleviating performance limitations caused by storage barriers and I / O bottlenecks, and achieving lower latency and higher effective bandwidth. However, existing CSD-based ANNS solutions generally employ a coarse-grained producer-consumer model for scheduling, failing to effectively utilize the heterogeneous computing resources within the CSD.

[0005] In this field, the following terms are used.

[0006] ANNS: Approximate Nearest Neighbor Search, refers to the efficient search of data points in a high-dimensional vector space that are most similar to the query vector, thereby providing high-performance vector database queries.

[0007] SSD: Solid State Drive, refers to a storage device that uses flash memory chips to store data.

[0008] CSD: Computational Storage Device, refers to hardware that integrates computing resources in flash memory media. Its characteristic is that it enables applications to offload data processing tasks to the device.

[0009] FPGA: Field Programmable Gate Array, is a hardware reconfigurable architecture characterized by its reprogrammability and ability to perform high-performance, low-latency computing tasks.

[0010] FTL: Flash Translation Layer, is a key technology in SSDs used to map the host logical address space to the flash physical address space. Summary of the Invention

[0011] To address the shortcomings of existing technologies, the purpose of this invention is to provide a computationally stored hierarchical pipeline graph indexing (ANNS) system and method.

[0012] According to the present invention, a computation-storage-based hierarchical pipeline graph indexing (ANNS) system includes: A heuristic slice selection module is used to dynamically schedule slice search tasks and prefetch high-priority slices to the FPGA; The uninterrupted intra-slice pipeline module is used to calculate intra-slice vector distances in parallel and filter results in real time; The global results aggregation module is used to merge search results from various segments; The heuristic shard selection module is implemented by modifying the flash memory conversion layer, the uninterrupted shard pipeline module is implemented on the FPGA, and the global result aggregation module is implemented by modifying the flash memory conversion layer.

[0013] Preferably, the heuristic shard selection module includes: The center graph traversal unit is used to select candidate shards based on best-priority search on the shard center point graph index; The prefetch scheduling unit is used to trigger shard prefetching during the convergence phase of the best-priority search, prioritizing the scheduling of the shard closest to the query. Dynamic pruning unit is used to filter low-relevance fragments using an adaptive threshold.

[0014] Preferably, the prefetching scheduling unit performs the following operations: The candidate set is sorted by the distance from the centroid to the query point and divided into two halves. When at least one node in the first half of the sorted candidate set remains present in two consecutive centroid expansions, a heuristic prefetch operation is triggered, so that the FPGA prefetches the slice whose centroid is closest to the query point.

[0015] Preferably, the dynamic pruning unit performs the following operations: For each candidate partition center point, calculate its distance to the query vector. and with dynamic pruning threshold Comparison, among which The shortest distance between the current center point and the query vector. These are the pruning parameters; if If the candidate fragment is not selected, a pruning operation will be performed on it, excluding it from subsequent processing.

[0016] Preferably, the uninterrupted intra-segment pipeline module includes: A fully parallel distance calculation engine is used to traverse all vectors within a partition and calculate their Euclidean distance to the query vector in parallel. The result collection unit within a shard is used to collect the k vectors within the shard that are closest to the query vector.

[0017] Preferably, the fully parallel distance calculation engine employs a two-stage vector distance calculation engine: In the first stage, the dimensional loop is fully expanded on the FPGA, and the squared difference between the query vector and the candidate vector in each dimension is calculated. An array storing the squared differences of each dimension is generated in a single clock cycle. In the second stage, the squared difference array is filled with zeros to make its length a power of 2, and a balanced binary tree structure is constructed to achieve parallel addition operations. The squared distance is then merged layer by layer until the final squared distance is obtained.

[0018] Preferably, the result collection unit within a shard includes a streaming Top-k sorter that employs a double-buffered sorting mechanism. It maintains two sorted Top-k arrays that alternately receive new results. Each array retains no more than k recent results through insertion sort. After all shard data processing is completed, a merge operation is performed on the two arrays to generate an ordered single-shard Top-k result set.

[0019] Preferably, the global result aggregation module includes: The global Top-k collection unit is used to maintain a priority queue with a max-heap structure, storing the k vectors that are closest to the query vector in each partition result, with the top of the heap being the maximum distance in the global Top-k; The shard-level result collection early termination judgment unit is used to determine that if a result is greater than the current global Top-k maximum distance when collecting the results of a certain shard, the remaining results of that shard cannot enter the global Top-k, and the collection of the results of that shard is terminated.

[0020] The present invention provides a computation-storage-based hierarchical pipeline graph indexing (ANNS) method, comprising the following steps: Step S1: FTL selects the shards to be computed on the graph index based on best-first search and starts heuristic shard prefetching; Step S2: FPGA pipelined processing of slice-in-slice vectors to generate local Top-k results; Step S3: Aggregate the fragmentation results using FTL and determine if early termination is triggered; Preferably, steps S1, S2, and S3 are executed in a global pipeline manner: after step S1 starts shard prefetching, step S2 begins processing shard data; when shard results are generated, step S3 begins collecting results; after step S1 for the next query vector begins execution, step S2 continues to calculate the shard results for the current vector, step S3 continues to collect results, and step S1 begins graph index search for the next query vector. If the results for the current query vector have not yet been obtained when shard prefetching for the next query vector begins, step S1 is paused and execution waits for steps S2 and S3 to finish processing.

[0021] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention uses a heuristic shard selection module to pre-fetch high-priority shards during the optimal priority search convergence phase, and combines dynamic pruning thresholds to progressively filter low-relevance shards, thereby reducing invalid data reads caused by random I / O in traditional methods, and reducing flash memory access frequency and read / write amplification.

[0022] 2. This invention employs a two-stage fully pipelined distance calculation engine on an FPGA, outputting a distance result per clock cycle, in conjunction with a double-buffered streaming topology. The k-sorter performs distance calculation and result filtering in parallel, eliminating pipeline pauses and improving the efficiency of vector processing within slices.

[0023] 3. This invention organizes graph index search, intra-slice distance calculation and result aggregation into a three-stage pipeline. These three stages of different queries can be executed in an overlapping manner. While the FPGA is processing the current query, the FTL can start the graph index search for the next query at the same time, thereby improving the overall throughput of the system. Attached Figure Description

[0024] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the overall system in an embodiment of the present invention; Figure 2 This is a schematic diagram of data organization according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the FPGA slice-based vector search pipeline in an embodiment of the present invention; Figure 4 This is a schematic diagram of the global pipeline in an embodiment of the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0026] Before introducing the embodiments of the present invention, the terminology used in the present invention will be explained.

[0027] K-means is a clustering algorithm used to divide nodes in a vector database into K specified clusters based on the distance between vectors. Clusters can overlap, and the specific value of K is determined by the dataset size and storage sharding requirements. The K-means algorithm randomly samples K initial centroids from the dataset. Each centroid represents the initial center point of a cluster, and its dimension is consistent with the original vector data. It then performs iterative optimization, which can be divided into an allocation phase and an update phase. In the allocation phase, K-means calculates the distance (usually Euclidean distance) from each vector in the dataset to all centroids and assigns it to the shard containing the nearest centroid. In the update phase, the centroid positions are recalculated based on the mean of all vectors within the current cluster. The K-means algorithm repeats the allocation and update steps until the change in centroid position is less than a specified threshold. Finally, K-means uses the centroids as the center points of the shards, and each vector is assigned to a pre-specified number of shards based on the nearest proximity principle, ensuring redundant coverage to improve query recall.

[0028] Best-First Search (BFS) is a greedy search algorithm based on an ordered candidate set of specified length, typically no less than the desired number of results, k. Initially, the algorithm compares the query vector with the specified entry node, calculates the distance, and adds it to the candidate set (sorted in ascending order of distance). It then iteratively expands as follows: the node with the closest distance to the current node that has not been visited is retrieved from the queue, and its adjacency list is visited. Then, the distance between the query vector and each neighboring node is calculated, unvisited nodes are added to the queue, and if the candidate set becomes too long, the nodes with the largest distance to the query vector are removed. The search terminates when the adjacency lists of all nodes in the candidate set have been visited. Generally, BFS can be divided into two phases: the approach phase and the convergence phase. In the approach phase, the search region rapidly approaches the query vector, the candidate set is updated quickly, and very few become the final Top-k. The convergence phase exhibits drastically different behavior: the candidate set tends to stabilize, and subsequent modifications are minimal. At this stage, the partitions containing centroids that are close to the query vector have a significantly higher selection probability, which reflects that the algorithm is converging towards the optimal solution.

[0029] Vamana graph index is an ANNS (Approximate Nearest Neighbor Search) data structure based on a nearest neighbor graph. Each node's index consists of its own vector coordinates and an adjacency list. The adjacency list comprises the number of neighbors and their IDs. Each neighbor ID represents a unidirectional edge from that node to a neighboring node, and the number of neighbors a node has cannot exceed a specified degree. During a search, the Vamana graph index provides an initial node as the search entry point. Building the Vamana graph index first requires randomly creating unidirectional edges for each node and randomly selecting the initial node. Then, for each node, a best-first search is used to find the nearest neighbors, and a pruning strategy is applied to remove redundant edges, such as deleting edges that are far apart in locally dense connections and retaining the top-k shortest edges to control the out-degree (usually limited to 32-64 edges).

[0030] Before the search begins, this invention first divides the vector database into several pieces using the K-means method, and then builds a Vaman graph index based on the center points of each piece. For example... Figure 2 As shown, the vector data of each slice is stored contiguously in the flash memory of the CSD, while the graph index is stored in the on-chip memory of the CSD.

[0031] This embodiment provides a computationally stored, hierarchical pipelined graph index approximate nearest neighbor search system. Figure 1 This is a schematic diagram of the overall system of this embodiment. Figure 1As shown, the system comprises three main modules: a heuristic slice selection module, a pause-free intra-slice pipeline module, and a global result aggregation module. The heuristic slice selection module and the global result aggregation module are implemented by modifying the flash translation layer (FTL) in the solid-state drive (SSD), while the pause-free intra-slice pipeline module is implemented on an FPGA integrated into the compute storage device (CSD).

[0032] This invention performs vector database searches in the following modules: The heuristic slice selection module dynamically schedules slice search tasks and prefetches high-priority slices to the FPGA; A non-stop intra-slice pipeline module calculates intra-slice vector distances in parallel and filters results in real time. The global results aggregation module merges the search results from each segment.

[0033] The heuristic slice selection module is implemented by modifying the FTL, dynamically scheduling slice search tasks and prefetching high-priority slices to the FPGA; The uninterrupted intra-slice pipeline module is implemented on an FPGA, which calculates intra-slice vector distances in parallel and filters results in real time. The global result aggregation module merges the search results of each segment by modifying the FTL.

[0034] Specifically, the heuristic shard selection module includes: a central graph traversal unit, a prefetching and scheduling unit, and a dynamic pruning unit; Central graph traversal unit: Select candidate shards based on best-priority search on the shard center point graph index; Prefetch scheduling unit: Triggers shard prefetching during the convergence phase of best-priority search, prioritizing the scheduling of the shard closest to the query; The candidate set is sorted by distance from the centroid to the query point and then divided into two halves. When at least one node in the first half of the sorted candidate set remains present in two consecutive centroid expansions, it is considered that the explored centroid has covered the area near the query point, and the top-ranked slices are rarely evicted. At this point, a heuristic prefetch operation is performed, while the best-priority search continues until traversal is complete. This operation allows the FPGA to prefetch the slices whose centroids are closest to the query point.

[0035] Dynamic pruning unit: Filters low-relevance fragments using adaptive thresholds; For each candidate partition center point, calculate its distance to the query vector. ,Will The dynamic pruning threshold is compared with the shortest distance between the current centroid and the query vector among all current centroids. Proportional, calculated as follows ,in For pruning parameters, if If the candidate segment is not found to be a candidate segment, then pruning will be performed on it, excluding it from further processing. During the prefetching process, as the nearest candidate centroid is dynamically updated, The pruning threshold gradually decreases, thus enabling progressive filtering of distant segments.

[0036] Specifically, the uninterrupted intra-slice pipeline module includes: a fully parallel distance calculation engine and an intra-slice result collection unit; Fully parallel distance calculation engine: Traverses all vectors within a partition and calculates their Euclidean distance to the query vector in parallel, including a pipeline-optimized two-stage vector distance calculation engine; The two-stage vector distance calculation engine includes: For the vector being traversed within each slice, in the first stage, the pipeline fully expands the dimension loop on the FPGA while calculating the squared difference between the query vector and the candidate vector in each dimension, thereby generating an array storing the squared differences of each dimension in a single clock cycle. In the second stage, the squared difference array is first filled with zeros to make its length a power of 2, thus constructing a balanced binary tree structure to enable parallel addition operations. In each level of the tree, all elements in the upper half are added to the corresponding elements in the lower half, halving the effective array length. This merging process continues until a single value remains, which is the final squared distance. The above steps are highly pipelined within the FPGA, ensuring that a distance result is generated every clock cycle.

[0037] In-shard result collection unit: collects the k vectors closest to the query vector within the shard, including a streaming Top-k sorter and a threshold filtering unit; The streaming Top-k sorter comprises: two sequential arrays and a merge unit; The module employs a double-buffered sorting mechanism, maintaining two sorted Top-k arrays that alternately receive new results. Each array retains no more than k of its closest results through insertion sort. Initially, the elements in both ordered arrays are the maximum floating-point numbers. In each clock cycle, the fully parallel distance calculation engine generates a calculation result. If the receiving array for this result is full and its distance exceeds the maximum distance in that array, it is not inserted into the array. For each distance data point that needs to be inserted into the array, the selected array is scanned from back to front, and distance data points greater than the new result are shifted sequentially to the right until the correct insertion position is found, ensuring that the array elements are arranged in ascending order. After all the data fragments have been processed, the merge unit performs a merge operation on the two sorted arrays to generate a unified ordered Top-k result set of a single fragment arranged in ascending order of distance. like Figure 3 As shown, the read vector-distance calculation-insertion sort is highly pipelined in FPGA.

[0038] Specifically, the global result aggregation module includes: a global Top-k collection unit and a segment-level result collection early termination discrimination unit; Global Top-k collection unit: Maintains a priority queue with a max-heap structure, storing the k results from each partition that are closest to the query vector, i.e., the global Top-k, with the top of the heap being the maximum distance among the global Top-k; Early termination judgment unit for shard-level result collection: After the heuristic shard selection module, each shard result is arranged in ascending order of distance from the query vector. If, when collecting the results of a certain shard, there is a result that is greater than the maximum distance in the current global Top-k, it is determined that the remaining results of that shard cannot enter the global Top-k, and the collection of the results of that shard is terminated. This invention provides a hierarchical pipelined graph index approximate nearest neighbor search method based on computation and storage, characterized in that the framework implements query processing through the following steps: Step S1: The FTL search selects the shards to be computed based on the best-first search on the graph index and starts the heuristic shard prefetch; Step S2: FPGA pipelined processing of slice-in-slice vectors to generate local Top-k results; Step S3: Aggregate the fragmentation results using FTL and determine if early termination is triggered; like Figure 4 As shown, the above three steps can be pipelined into a global pipeline: During step S1, after the heuristic shard selection module starts shard prefetching, the uninterrupted intra-shard pipeline module will continuously process shard data, and step S2 will run. Once a shard result is generated, the global result aggregation module will collect the result, and step S3 will run. Furthermore, after step S1 of the next query vector begins, the uninterrupted intra-shard pipeline module can continue to calculate the shard result of the current vector, and the global result aggregation module can also continue to collect results. The heuristic shard selection module can start the graph index search for the next query vector until the shard prefetch of the next query vector starts. If the result of the current query vector has not yet been obtained, the heuristic shard selection module will pause and wait for the uninterrupted intra-shard pipeline module and the global result aggregation module to finish processing.

[0039] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A computation-storage-based hierarchical pipeline graph indexing (ANNS) system, characterized in that, include: A heuristic slice selection module is used to dynamically schedule slice search tasks and prefetch high-priority slices to the FPGA; The uninterrupted intra-slice pipeline module is used to calculate intra-slice vector distances in parallel and filter results in real time; The global results aggregation module is used to merge search results from various segments; The heuristic shard selection module is implemented by modifying the flash memory conversion layer, the uninterrupted shard pipeline module is implemented on the FPGA, and the global result aggregation module is implemented by modifying the flash memory conversion layer.

2. The ANNS system for hierarchical pipeline graph indexing based on computational storage according to claim 1, characterized in that, The heuristic slice selection module includes: The center graph traversal unit is used to select candidate shards based on best-priority search on the shard center point graph index; The prefetch scheduling unit is used to trigger shard prefetching during the convergence phase of the best-priority search, prioritizing the scheduling of the shard closest to the query. Dynamic pruning unit is used to filter low-relevance fragments using an adaptive threshold.

3. The ANNS system for hierarchical pipeline graph indexing based on computational storage according to claim 2, characterized in that, The prefetching scheduling unit performs the following operations: The candidate set is sorted by the distance from the centroid to the query point and divided into two halves. When at least one node in the first half of the sorted candidate set remains present in two consecutive centroid expansions, a heuristic prefetch operation is triggered, so that the FPGA prefetches the slice whose centroid is closest to the query point.

4. The ANNS system for hierarchical pipeline graph indexing based on computational storage according to claim 2, characterized in that, The dynamic pruning unit performs the following operations: For each candidate partition center point, calculate its distance to the query vector. and with dynamic pruning threshold Comparison, among which The shortest distance between the current center point and the query vector. These are the pruning parameters; if If the candidate fragment is not selected, a pruning operation will be performed on it, excluding it from subsequent processing.

5. The ANNS system for hierarchical pipeline graph indexing based on computational storage according to claim 1, characterized in that, The uninterrupted intra-segment pipeline module includes: A fully parallel distance calculation engine is used to traverse all vectors within a partition and calculate their Euclidean distance to the query vector in parallel. The result collection unit within a shard is used to collect the k vectors within the shard that are closest to the query vector.

6. The ANNS system for hierarchical pipeline graph indexing based on computational storage according to claim 5, characterized in that, The fully parallel distance calculation engine employs a two-stage vector distance calculation engine: In the first stage, the dimensional loop is fully expanded on the FPGA, and the squared difference between the query vector and the candidate vector in each dimension is calculated. An array storing the squared differences of each dimension is generated in a single clock cycle. In the second stage, the squared difference array is filled with zeros to make its length a power of 2, and a balanced binary tree structure is constructed to achieve parallel addition operations. The squared distance is then merged layer by layer until the final squared distance is obtained.

7. The ANNS system for hierarchical pipeline graph indexing based on computational storage according to claim 5, characterized in that, The result collection unit within a shard includes a streaming Top-k sorter that employs a double-buffered sorting mechanism. It maintains two sorted Top-k arrays that alternately receive new results. Each array retains no more than k of the most recent results through insertion sort. After all shard data has been processed, a merge operation is performed on the two arrays to generate an ordered single-shard Top-k result set.

8. The ANNS system for hierarchical pipeline graph indexing based on computational storage according to claim 1, characterized in that, The global result aggregation module includes: The global Top-k collection unit is used to maintain a priority queue with a max-heap structure, storing the k vectors that are closest to the query vector in each partition result, with the top of the heap being the maximum distance in the global Top-k; The shard-level result collection early termination judgment unit is used to determine that if a result is greater than the current global Top-k maximum distance when collecting the results of a certain shard, the remaining results of that shard cannot enter the global Top-k, and the collection of the results of that shard is terminated.

9. A computation-storage-based hierarchical pipeline graph indexing (ANNS) method, characterized in that, Includes the following steps: Step S1: FTL selects the shards to be computed on the graph index based on best-first search and starts heuristic shard prefetching; Step S2: FPGA pipelined processing of slice-in-slice vectors to generate local Top-k results; Step S3: FTL aggregates the fragmentation results and determines whether early termination is triggered.

10. The computation-storage-based hierarchical pipeline graph indexing (ANNS) method according to claim 1, characterized in that, Steps S1, S2, and S3 are executed in a global pipeline manner: after step S1 starts shard prefetching, step S2 begins processing sharded data; when sharding results are generated, step S3 begins collecting results; after step S1 for the next query vector begins execution, step S2 continues to calculate the sharding results for the current vector, step S3 continues to collect results, and step S1 begins graph index search for the next query vector. If the results for the current query vector have not yet been obtained when shard prefetching for the next query vector begins, step S1 is paused and execution waits for steps S2 and S3 to finish processing.

Images