A heterogeneous system for large-scale vector approximate nearest neighbor search based on GPU-NVMe architecture

Abstract

The application discloses a heterogeneous system for large-scale vector approximate nearest neighbor search based on a GPU-NVMe architecture, and has the characteristics that the system is based on a direct GPU-NVMe architecture, and a hybrid index combining clustering and graph algorithms is designed. The hybrid index utilizes the parallel processing capability of a GPU to efficiently process well-clustered vectors, and simultaneously uses a graph index as a compensation mechanism to improve the search accuracy for noise vectors. The application fully considers the collaborative optimization of hardware and software, deploys vector index data at the NVMe drive level, simultaneously cooperatively utilizes CPU and GPU hardware to perform a billion-level vector retrieval calculation process, and constructs a scalable large-scale retrieval system based on a direct GPU-NVMe architecture. Compared with the prior art, the application effectively solves the performance bottleneck in the billion-level vector data search, significantly improves the search throughput and accuracy, and reduces the search delay.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vector data management, and particularly to a large-scale vector approximate nearest neighbor search based on NVMe and GPU hardware, using NVMe devices to alleviate the memory space pressure in vector data management, using GPU hardware and CPU for parallel retrieval to accelerate the performance of super large-scale vector retrieval. BACKGROUND

[0002] Vector k-nearest neighbor search (k-NNS) as a basic technology has been widely used in various fields, including recommendation systems, search engines, image retrieval, and emerging artificial intelligence applications. The goal of the k-NNS problem is to find the k nearest vectors to a given query vector in a large-scale vector dataset. With the rapid development of the Internet and artificial intelligence, the amount of data in application scenarios such as recommendation systems, search engines, and multimedia processing is growing exponentially. Especially in recommendation systems, the features of users and goods are usually embedded in high-dimensional vector space for k-NNS to quickly match; while in information retrieval, the semantic similarity between documents and query words also relies on vectorization. However, the process of accurately searching for the top k results will be accompanied by high computational cost and unacceptable response delay. At this time, the computational cost and memory requirements of traditional methods in large-scale high-dimensional scenarios cannot meet the demand. In practical applications, as the dataset size grows further, completely accurate k-NNS search becomes infeasible. To overcome this challenge, existing technologies have proposed approximate nearest neighbor search (ANNS) solutions, which sacrifice part of the accuracy for significant performance improvement.

[0003] Based on the ANNS method, a large amount of information generated from various data types (such as images, text, video, etc.) is embedded in a multi-dimensional vector space, and through the use of context-aware features, efficient similarity search can be quickly performed. However, recent research and reports show that in many enterprise-level application scenarios, the size of the vector dataset has reached the level of billions. For example, a search engine needs to handle more than 100 billion vectors, an e-commerce service manages about 2 billion vectors, and a backend system also needs to support vector data up to 1 billion. After entering the era of deep learning and artificial intelligence, as the dataset size reaches billions or even hundreds of billions, ANNS systems face unprecedented challenges in terms of memory pressure, search performance, and search accuracy.

[0004] To address memory pressure, existing research has adopted two main approaches: quantization compression and / or the use of external storage (such as SSDs and PMEM). In quantization compression, high-dimensional vectors are decomposed into multiple low-dimensional subspaces, each of which is independently quantized. The vector is ultimately represented by the concatenation of these subspaces. This approach significantly reduces storage requirements, but the loss of information during the quantization process significantly impacts search result accuracy. Another approach addresses storage pressure through external storage, storing the majority of vector data on SSDs or PMEM and optimizing data access during the search process. Related research, such as SPANN and DiskANN, has demonstrated the potential of leveraging NVMe SSDs to alleviate memory pressure. However, the clustering quality of clustering-based SPANN degrades as data size grows, requiring the retrieval of more clusters, leading to I / O bandwidth and computational bottlenecks. Graph-based DiskANN, on the other hand, requires a large number of random I / O operations in external storage, significantly increasing search latency. Although much progress has been made in the above directions, the trade-off between performance and accuracy in large-scale ANNS systems remains a difficult problem that has not been completely solved.

[0005] At the same time, GPUs, as one of the most popular hardware platforms in high-performance parallel computing, offer new hope for improving the performance of ANNS. In recent years, several studies have attempted to leverage GPU acceleration in ANNS, including designing GPU-friendly graph structures and optimizing computations based on clustering methods. However, due to physical space and cost constraints on GPU on-chip memory, relying solely on GPUs to complete ANNS tasks with billions of vectors is unrealistic. Even so, it would be regrettable to forgo the enormous potential of GPUs for large-scale vector search. Furthermore, no existing technology can simultaneously combine GPUs and NVMe for large-scale vector search.

[0006] Existing research has significant limitations in areas such as memory pressure, I / O performance, balancing accuracy and performance, and hardware resource utilization. Especially when processing large-scale data, effectively combining hardware resources like GPUs and NVMe to simultaneously meet the demands of high performance and high accuracy remains a research area that urgently needs breakthroughs. Summary of the Invention

[0007] This paper addresses the shortcomings of existing large-scale vector search technologies, such as host memory space pressure and insufficient search performance, and provides a heterogeneous system for large-scale vector approximate nearest neighbor search based on a GPU-NVMe architecture. The system primarily comprises a hybrid index structure combining clustering and graph indexing, a hybrid index deployed on NVMe and host memory, a hybrid index search method based on CPU-GPU collaboration, and a GPU-NVMe heterogeneous architecture platform that supports large-scale approximate nearest neighbor vector search.

[0008] The specific technical solution for achieving the purpose of the present invention is:

[0009] A heterogeneous system for large-scale vector approximate nearest neighbor search based on a GPU-NVMe architecture, the system comprising:

[0010] A hybrid index that combines clustering and graph algorithms to build, deploy, and retrieve index data structures for organizing large-scale approximate nearest neighbor searches in this system;

[0011] A heterogeneous platform for large-scale approximate nearest neighbor search based on a direct GPU-NVMe architecture to support the deployment and retrieval of hybrid indexes;

[0012] The hybrid index combining clustering and graph algorithms includes: dividing a vector dataset into a clustered part and a noise part, and deploying it on a direct GPU-NVMe hardware architecture, performing vector approximate nearest neighbor retrieval, i.e., vector similarity retrieval, through a CPU-GPU collaborative computing mode, specifically including the following steps:

[0013] 1) The vector dataset is divided into a clustered portion and a noise portion. The clustered portion is deployed on an NVMe SSD array and retrieved using an inverted index, with GPU-accelerated vector similarity retrieval. The noise portion is retrieved using a graph index, and while the GPU processes the clustered data, the CPU performs similarity retrieval on the noise portion. The CPU and GPU retrieval are performed in parallel.

[0014] 2) During the deployment process, the clustered data is evenly divided into multiple clusters and deployed under the logical abstraction of the NVMe SSD driver layer, organizing the storage data structure in the form of logical blocks. At the same time, in the direct GPU-NVMe architecture, data transmission between the GPU and NVMe bypasses the host memory. That is, the clustered data deployed on the NVMe SSD can be directly transferred to the GPU device during the retrieval process, and the GPU device can be used to accelerate vector similarity retrieval.

[0015] The heterogeneous platform for large-scale approximate nearest neighbor search based on direct GPU-NVMe architecture includes:

[0016] The clustering part is stored on the NVMe device, the cluster centroid index and noise graph index reside in the host memory, and the graph vector index is retrieved using the CPU and the cluster vector index is retrieved using the GPU. During the search, data is directly transmitted between the NVMe device and the GPU device through P2P, bypassing the host memory and the traditional Linux system I / O software stack. The GPU device and the CPU device run in parallel during the search calculation process. The specific steps include:

[0017] 1) Receives a vector search request from the vector search application, searches for multiple nearest neighbor clusters in the cluster centroid index residing in the host memory, and then transfers the vector list of the corresponding clusters to the GPU device memory via PCIe P2P through the NVMe driver layer I / O;

[0018] 2) After the cluster vector list is transferred to the GPU device memory via PCIe P2P via NVMe I / O command, the CPU searches the noise graph index. Simultaneously, the GPU device performs similarity calculations between the vectors in the vector search request and the vectors in the cluster list.

[0019] 3) After the GPU calculation is completed, the similarity calculation results are transferred back to the host memory. The CPU device merges the noise graph index similarity search results and the similarity calculation results from the GPU to generate a vector approximate nearest neighbor search result and returns the result to the upper-level vector search application.

[0020] Furthermore, the vector data set is divided into a clustered part and a noise part, specifically comprising the following steps:

[0021] 1) Obtain the centroid of the vector dataset through a clustering algorithm, and initialize an empty member set and candidate set for the cluster represented by each centroid. Then, traverse all vector data in the vector dataset. During the traversal, obtain the top-K centroids closest to each vector data, and add the vector data to the member set of the cluster represented by the nearest top-1 centroid and the candidate set of the cluster represented by the nearest top-2 to top-K centroids.

[0022] 2) Through hierarchical balanced clustering, the member set is balanced and split into clusters. That is, the following operations are performed on each cluster to determine whether the number of elements in the cluster's member set exceeds a certain threshold. If so, the cluster is clustered again, and the member set is updated to the re-clustered member set. Each sub-cluster obtained after re-clustering shares the member set of the original parent cluster;

[0023] 3) For each cluster finally formed, that is, the member set cluster is split into subclusters in step 2), and a cluster list is constructed with a fixed size. Each cluster list is first filled with elements from the member set. If the number of elements in the member set is less than the fixed cluster list size, it is then filled with the candidate set. After the cluster list of all clusters is finally formed, the difference between all vector data sets and the cluster lists of all clusters is taken. The vectors in the difference are the noise part, and all clusters, that is, their cluster lists, are the cluster part.

[0024] The hybrid indexing in this invention helps leverage the advantages of both graph and cluster indexing to achieve higher performance and higher accuracy for large-scale vector approximate nearest neighbor searches. GPU devices accelerate the retrieval performance of the clustering portion of the hybrid indexing in this invention, while NVMe devices alleviate host memory pressure in traditional vector retrieval systems, reducing the deployment cost of the system in this invention. Overall, this invention achieves higher performance and lower cost for large-scale vector approximate nearest neighbor searches through hybrid indexing and a GPU-NVMe architecture platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A diagram of the hybrid index structure constructed for large-scale data by the present invention;

[0026] Figure 2 This is a schematic diagram of the GPU-NVMe platform architecture on which the present invention is based;

[0027] Figure 3 This is a diagram of the heterogeneous system architecture of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0029] The system of the present invention comprises:

[0030] 1. Hybrid Index Construction Method

[0031] The hybrid index construction method of the present invention first divides the vector data set into a cluster part and a noise part. The specific steps are as follows:

[0032] Distinguishing between clusters and noise: A clustering algorithm is used to cluster the vector dataset, obtaining the centroid of each cluster. Based on this centroid, vector data is assigned to clusters based on their similarity to the cluster centroid, forming the clusters. The remaining vector data is considered noise. The noise portion does not participate in clustering and is instead retrieved using a graph index.

[0033] Further processing of clustered data: During the clustering process, a hierarchical balanced clustering strategy is used to split clusters with excessively large member sets. Each resulting cluster consists of a member set and a candidate set. The cluster list is of fixed size and is first populated with the member set. If the member set is insufficient, vectors are selected from the candidate set to fill the list.

[0034] Noise processing: After clustering is complete, the vectors of all clusters and the noise vectors are subtracted to obtain the specific noise vectors. The clustered data is managed and stored using an inverted index, while the noise data is retrieved using a graph index.

[0035] 2. Hybrid index deployment method

[0036] During hybrid index deployment, the present invention deploys the clustering portion in an NVMe SSD storage array, while the noise portion is stored in host memory via a graph index. Clustering data and noise data are accelerated through collaborative computing using the GPU and CPU. The specific deployment steps are as follows:

[0037] Cluster data storage: Cluster data is evenly divided into multiple subclusters and stored in an NVMe SSD array, organized in logical blocks. In this storage architecture, data is transferred directly to the GPU device without passing through host memory, fully leveraging the GPU's computing power for acceleration.

[0038] Storage of noise data: The noise data is stored in the host memory through graph indexing, and the CPU is used to retrieve the noise vector.

[0039] 3. Hybrid index retrieval method under heterogeneous architecture

[0040] During the vector search process, this paper proposes a CPU-GPU collaborative computing search mode to improve search efficiency. The specific steps of the search process are as follows:

[0041] Accepting vector retrieval requests: After receiving a vector retrieval request from the vector search application layer, the system first searches the cluster centroid index residing in the host memory to find multiple nearest neighbor clusters.

[0042] Data transfer to the GPU: Through NVMe driver-layer I / O, the system transfers the corresponding clustered vector list directly to the GPU device memory via PCIe P2P (Peer-to-Peer). This process avoids the data transfer bottleneck through host memory and improves data processing efficiency.

[0043] CPU and GPU parallel computing: While the data is being transferred to the GPU device memory, the CPU starts to retrieve the graph index of the noise part, and the GPU is responsible for calculating the similarity between the cluster part vectors and the query vector. At this time, the retrieval work of CPU and GPU is carried out in parallel, which greatly improves the retrieval efficiency.

[0044] Merge the retrieval results: After the GPU completes the vector similarity calculation, the results are transmitted back to the host memory, and the CPU merges the similarity calculation results from the GPU and the retrieval results of the noise part graph index, and finally returns the approximate nearest neighbor search results of the vector.

[0045] 4. GPU-NVMe heterogeneous platform supporting large-scale approximate nearest neighbor vector retrieval

[0046] It includes a solid state disk storage array supporting NVMe protocol, a GPU device supporting P2P data transmission and a CPU device, the NVMe device supports SPDK user mode driver interface, the GPU device supports PCIe P2P DMA data transmission, and the CPU device supports in-memory vector graph index retrieval. Embodiments

[0047] Referring to Figure 1 , the noise vectors in the full data set X are distinguished from the cluster vectors. In this embodiment, the input vector data set X is first clustered. The vector data is clustered by using a clustering algorithm (such as K-means or DBSCAN, etc.), and the centroids C of multiple clusters are obtained. The clustering algorithm calculates the similarity (for example, using Euclidean distance or cosine similarity) between each vector and the centroid of the cluster, and assigns the vector data to the cluster that is most similar to it, forming a cluster list R. Those vectors that cannot be reasonably classified into any cluster, or have low similarity with all cluster centroids, are determined as noise part N. The vectors in the noise part will not participate in the subsequent clustering process, but will be retrieved by graph index. The specific steps are as follows:

[0048] Calculation of cluster centroid: The centroid of each cluster is calculated by the clustering algorithm. The centroid of the cluster is the average value of all vectors in the cluster, representing the center position of the cluster. Vector data assignment: according to the similarity of each vector and the centroid of each cluster, the most similar cluster is selected, and the vector data is assigned to the corresponding cluster. These vectors assigned to the cluster constitute the cluster part. Determination of noise part: those vectors that cannot be reasonably classified into any cluster, or have low similarity with all cluster centroids, are determined as noise part. The vectors in the noise part will not participate in the subsequent clustering process, but will be retrieved by graph index.

[0049] Referring to Figure 2, the system of the present invention includes a heterogeneous hardware architecture platform based on NVMe devices and GPU devices, and relies on it for implementation. Its NVMe device is an SSD storage array that supports the NVMe protocol, and the GPU device supports P2P data transmission of PCIe devices. By using the same mapping method as the DRAM host memory for the GPU device memory, exposing it to the PCIe bus through the Bar register, and then mapping the bus address to the virtual address space, direct I / O between NVMe and GPU devices is achieved. Furthermore, since commercial GPU hardware devices are usually PCIe 16x standard and NVMe devices are usually PCIe 4x standard, the hardware architecture of this embodiment uses multiple NVMe devices to connect to GPU devices to maximize PCIe bandwidth utilization.

[0050] See Figure 3 This embodiment uses a heterogeneous system for large-scale vector approximate nearest neighbor search based on the GPU-NVMe architecture to efficiently solve the problem of large-scale vector retrieval. The specific implementation is as follows:

[0051] When the system receives a search request from the vector search application layer (such as a recommendation system, a search engine, a large model application, etc.), it first reads and loads the cluster centroid index C_index from the host memory. The cluster centroid index is constructed by the clustering algorithm on the vector dataset, which represents the center point of each cluster. In this step, the system will search the query vector of the request through the cluster centroid index, and find multiple most similar clusters. Further, the NVMe side cluster list R data is transmitted to the GPU. After determining the closest multiple clusters, the cluster list L_map is obtained through the NVMe device address mapping table, and then the system transmits the I / O data to the GPU device memory through the NVMe drive layer. The data is directly transmitted through the PCIe P2P (Peer-to-Peer) technology, which can bypass the host memory, reduce the delay of data transmission, and avoid the bottleneck of traditional I / O software stack, thereby significantly improving the data processing efficiency. Further, CPU and GPU parallel computing, while data is transmitted to GPU device memory, CPU starts to search the noise part of the data. The noise part of the vector data is stored by graph index method and searched by CPU. GPU focuses on calculating the similarity between the cluster part of the vector and the query vector, and the GPU device performs cluster search calculation in parallel through multiple streams. At this time, the retrieval tasks of CPU and GPU are performed in parallel, thereby greatly improving the overall efficiency of vector retrieval. Further, merge the search results, after the GPU completes the similarity calculation, the GPU returns the calculation results to the host memory. At the same time, the CPU also completes the graph index search of the noise part. Then, the CPU merges the similarity calculation results from the GPU and the search results of the noise part, and finally forms the approximate nearest neighbor search results of the vector, and returns the merged results to the upper vector search application program.

[0052] The above specific implementation is only for further illustration of the present application, and is not used to limit the present application. Any equivalent implementation of the present application shall be included in the scope of the claims of the present application.

Claims

1. A heterogeneous system for large-scale vector approximate nearest neighbor search based on GPU-NVMe architecture, characterized by: The system comprises: A hybrid index that combines clustering and graph algorithms to build, deploy, and retrieve index data structures for organizing large-scale approximate nearest neighbor searches in this system; A heterogeneous platform for large-scale approximate nearest neighbor search based on a direct GPU-NVMe architecture to support the deployment and retrieval of hybrid indexes; The hybrid index combining clustering and graph algorithms includes: dividing a vector dataset into a clustered part and a noise part, and deploying it on a direct GPU-NVMe hardware architecture, performing vector approximate nearest neighbor retrieval, i.e., vector similarity retrieval, through a CPU-GPU collaborative computing mode, specifically including the following steps: 1) The vector dataset is divided into a clustered portion and a noise portion. The clustered portion is deployed on an NVMe SSD array and retrieved using an inverted index, with GPU-accelerated vector similarity retrieval. The noise portion is retrieved using a graph index, and while the GPU processes the clustered data, the CPU performs similarity retrieval on the noise portion. The CPU and GPU retrieval are performed in parallel. 2) During the deployment process, the clustered data is evenly divided into multiple clusters and deployed under the logical abstraction of the NVMe SSD driver layer, organizing the storage data structure in the form of logical blocks. At the same time, in the direct GPU-NVMe architecture, data transmission between the GPU and NVMe bypasses the host memory. That is, the clustered data deployed on the NVMe SSD can be directly transferred to the GPU device during the retrieval process, and the GPU device can be used to accelerate vector similarity retrieval. The heterogeneous platform for large-scale approximate nearest neighbor search based on direct GPU-NVMe architecture includes: The clustering part is stored on the NVMe device, the cluster centroid index and noise graph index reside in the host memory, and the graph vector index is retrieved using the CPU and the cluster vector index is retrieved using the GPU. During the search, data is directly transmitted between the NVMe device and the GPU device through P2P, bypassing the host memory and the traditional Linux system I / O software stack. The GPU device and the CPU device run in parallel during the search calculation process. The specific steps include: 1) Receives a vector search request from the vector search application, searches for multiple nearest neighbor clusters in the cluster centroid index residing in the host memory, and then transfers the vector list of the corresponding clusters to the GPU device memory via PCIe P2P through the NVMe driver layer I / O; 2) After the cluster vector list is transferred to the GPU device memory via PCIe P2P via NVMe I / O command, the CPU searches the noise graph index. Simultaneously, the GPU performs similarity calculations between the vectors in the vector search request and the vectors in the cluster list. 3) After the GPU calculation is completed, the similarity calculation results are transferred back to the host memory. The CPU device merges the noise graph index similarity search results and the similarity calculation results from the GPU to generate a vector approximate nearest neighbor search result and returns the result to the upper-level vector search application.

2. The heterogeneous system according to claim 1, characterized in that: The method of dividing the vector data set into a clustered part and a noise part specifically includes the following steps: 1) Obtain the centroid of the vector dataset through a clustering algorithm, and initialize an empty member set and candidate set for the cluster represented by each centroid. Then, traverse all vector data in the vector dataset. During the traversal, obtain the top-K centroids closest to each vector data, and add the vector data to the member set of the cluster represented by the nearest top-1 centroid and the candidate set of the cluster represented by the nearest top-2 to top-K centroids. 2) Through hierarchical balanced clustering, the member set is balanced and split into clusters. That is, the following operations are performed on each cluster to determine whether the number of elements in the cluster's member set exceeds a certain threshold. If so, the cluster is clustered again, and the member set is updated to the re-clustered member set. Each sub-cluster obtained after re-clustering shares the member set of the original parent cluster; 3) For each cluster finally formed, that is, the member set cluster is split into subclusters in step 2), and a cluster list is constructed with a fixed size. Each cluster list is first filled with elements from the member set. If the number of elements in the member set is less than the fixed cluster list size, it is then filled with the candidate set. After the cluster list of all clusters is finally formed, the difference between all vector data sets and the cluster lists of all clusters is taken. The vectors in the difference are the noise part, and all clusters, that is, their cluster lists, are the cluster part.

Images