Heterogeneous system for large-scale vector approximate nearest neighbor search based on GPU-NVMe architecture

Abstract

The invention discloses a large-scale vector approximate nearest neighbor search heterogeneous system based on a GPU-NVMe framework, which is characterized in that the system is based on a direct GPU-NVMe framework, and a hybrid index combining a clustering algorithm and a graph algorithm is designed. According to the hybrid index, vectors with good clustering are efficiently processed by utilizing the parallel processing capacity of the GPU, and meanwhile, the search accuracy of noise vectors is improved by taking a graph index as a compensation mechanism. According to the method, collaborative optimization of hardware and software is fully considered, vector index data deployment is implemented at an NVMe driving level, meanwhile, CPU and GPU hardware are cooperatively utilized to carry out a gigabit-level vector retrieval calculation process, and an extensible large-scale retrieval system on a direct GPU-NVMe architecture is constructed. Compared with the prior art, the method has the advantages that the performance bottleneck in the gigabit-level vector data search is effectively solved, the search throughput and accuracy are remarkably improved, and the search delay is reduced.

Description

Technical Field

[0001] The present invention relates to the field of vector data management, and in particular to a large-scale vector approximate nearest neighbor search based on NVMe and GPU hardware, which utilizes NVMe devices to alleviate the memory space pressure in vector data management, utilizes GPU hardware and CPU for parallel retrieval, and accelerates ultra-large-scale vector retrieval performance. Background Art

[0002] Vector k-nearest neighbor search (k-NNS) is a basic technology that has been widely used in many fields, including recommendation systems, search engines, image retrieval, and emerging artificial intelligence applications. The goal of the k-NNS problem is to find the nearest 𝑘 vectors to a given query vector in a large-scale vector dataset. With the rapid development of the Internet and artificial intelligence, the amount of data in application scenarios such as recommendation systems, search engines, and multimedia processing has grown exponentially. In particular, in recommendation systems, the features of users and products are usually embedded in high-dimensional vector spaces for k-NNS to use for fast matching; in information retrieval, the semantic similarity calculation between documents and query terms also relies on vectorized expressions. However, the process of accurately searching for the top 𝑘 results is accompanied by high computational costs and unacceptable response delays. At this point, the computational cost and memory requirements of traditional methods in large-scale high-dimensional scenarios can no longer meet the requirements. In practical applications, as the size of the dataset grows further, a completely accurate k-NNS search becomes no longer feasible. To overcome this challenge, existing technologies have proposed a solution called Approximate Nearest Neighbor Search (ANNS), which sacrifices some accuracy in exchange for significant performance improvements.

[0003] Based on the ANNS method, a large amount of information generated from multiple data types (such as images, text, videos, etc.) is embedded in a multidimensional vector space, and by utilizing context-aware features, efficient similarity searches can be performed quickly. However, recent studies and reports have shown that in many enterprise-level application scenarios, the scale of vector data sets has reached billions. For example, a search engine needs to process more than 100 billion vectors, an e-commerce service manages about 2 billion vectors, and a backend system also needs to support vector data of 1 billion. After entering the era of deep learning and artificial intelligence, as the scale of data sets reaches billions or even hundreds of billions, ANNS systems face unprecedented challenges in terms of memory pressure, search performance, and search accuracy.

[0004] To deal with the memory pressure problem, existing research has adopted two main methods: quantization compression and / or the use of external storage (such as SSD and PMEM). In the quantization compression technique, the high-dimensional vector is decomposed into multiple low-dimensional subspaces and quantized independently in each subspace, and the vector is finally represented by the cascaded results of these subspaces. This method significantly reduces the storage space requirements, but has a significant impact on the accuracy of search results due to the information loss in the quantization process. Another method is to solve the storage pressure through external storage, that is, storing most of the vector data on SSD or PMEM and optimizing the data access behavior during the search process. Related studies such as SPANN and DiskANN have demonstrated the potential of using NVMeSSD to alleviate memory pressure. However, the clustering quality of clustering-based SPANN will decrease as the data size grows, so more clusters need to be retrieved, resulting in I / O bandwidth and computing power becoming bottlenecks. And the graph-based DiskANN requires a large number of random I / O operations in external storage, which significantly increases the search latency. Although much progress has been made in the above directions, the trade-off between performance and accuracy in large-scale ANNS systems remains a difficult problem that has not been completely solved.

[0005] At the same time, GPU, as one of the most popular hardware in the field of high-performance parallel computing, has brought new hope for improving the performance of ANNS. In recent years, some studies have attempted to use GPU acceleration in ANNS, including designing GPU-friendly graph structures and optimizing calculations based on clustering methods. However, since the capacity of GPU on-chip memory is limited by physical space and cost, it is unrealistic to rely solely on GPU to complete ANNS tasks with billions of vectors. Even so, it is undoubtedly regrettable to give up the great potential of GPU in large-scale vector search. At the same time, there is no existing technology that can combine GPU and NVMe to implement large-scale vector retrieval.

[0006] Existing research has obvious limitations in terms of memory pressure, I / O performance, balance between accuracy and performance, and hardware resource utilization. Especially when processing large-scale data, how to effectively combine hardware resources such as GPU and NVMe to meet the needs of high performance and high accuracy at the same time is still a research direction that needs to be broken through. Summary of the invention

[0007] The present invention aims at the deficiencies in existing large-scale vector retrieval technologies, such as host memory space pressure, insufficient retrieval performance, etc., and provides a heterogeneous system for large-scale vector approximate nearest neighbor search based on GPU-NVMe architecture. The system mainly includes a hybrid index structure combining clustering and graph index, hybrid index deployment on NVMe, host memory deployment method and hybrid index retrieval method based on CPU-GPU collaboration, and a GPU-NVMe heterogeneous architecture platform supporting large-scale approximate nearest neighbor vector retrieval.

[0008] The specific technical solution for achieving the purpose of the present invention is: A heterogeneous system for large-scale vector approximate nearest neighbor search based on GPU-NVMe architecture, the system comprising: A hybrid index that combines clustering and graph algorithms to build, deploy, and retrieve index data structures for organizing large-scale approximate nearest neighbor searches in the system; A heterogeneous platform for large-scale approximate nearest neighbor search based on a direct GPU-NVMe architecture to support deployment and retrieval of hybrid indexes; The hybrid index combining clustering and graph algorithms includes: dividing a vector data set into a clustering part and a noise part, and deploying it on a direct GPU-NVMe hardware architecture, performing vector approximate nearest neighbor retrieval, i.e., vector similarity retrieval, through a CPU-GPU collaborative computing mode, specifically including the following steps: 1) The vector data set is divided into a clustering part and a noise part; the clustering data part is deployed on the NVMe SSD array, retrieved by inverted index, and the GPU device accelerates the vector similarity retrieval; the noise part is retrieved by graph index, and when the GPU processes the clustering data, the CPU is used to perform similarity retrieval of the noise vector part; the CPU and GPU retrieval are performed in parallel; 2) During the deployment process, the clustered data part is evenly divided into multiple groups of clusters and deployed under the logical abstraction of the NVMe SSD driver layer, and the storage data structure is organized in the form of logical blocks; at the same time, in the direct GPU-NVMe architecture, the data transmission between the GPU and NVMe bypasses the host memory, that is, the clustered data part deployed on the NVMe SSD can be directly transmitted to the GPU device during the retrieval process, and the GPU device is used to accelerate the vector similarity retrieval; The heterogeneous platform for large-scale approximate nearest neighbor search based on direct GPU-NVMe architecture includes: The clustering part is stored on the NVMe device, the cluster centroid index and the noise graph index reside in the host memory, and the CPU is used to retrieve the graph vector index and the GPU is used to retrieve the cluster vector index. During the search, the data between the NVMe device and the GPU device is directly transmitted in P2P, bypassing the host memory and the traditional Linux system I / O software stack. The GPU device and the CPU device run in parallel during the search calculation process, which specifically includes the following steps: 1) Receive vector search requests from the vector search application side, search for multiple nearest neighbor clusters in the cluster centroid index residing in the host memory, and then transfer the vector list of the corresponding cluster to the GPU device memory through PCIe P2P via NVMe driver layer I / O; 2) After the cluster vector list is transmitted to the GPU device memory via PCIe P2P via NVMe I / O command, the CPU searches in the noise graph index; at the same time, the GPU device performs similarity calculations on the vectors in the vector search request and the vectors in the cluster list; 3) After the GPU calculation is completed, the similarity calculation results are transferred back to the host memory, and the CPU device merges the noise graph index similarity search results and the similarity calculation results from the GPU to generate vector approximate nearest neighbor search results, and returns the results to the upper-level vector search application.

[0009] Further, the step of dividing the vector data set into a clustering part and a noise part specifically includes the following steps: 1) Obtain the centroid of the vector data set through a clustering algorithm, and initialize an empty member set and candidate set for each cluster represented by the centroid; then, traverse all the vector data in the vector data set; during the traversal, obtain the TOP-K nearest centroids for each vector data, and add the vector data to the member set of the cluster represented by the nearest TOP-1 centroid and the candidate set of the cluster represented by the nearest TOP-2 to TOP-K centroids; 2) Through hierarchical balanced clustering, the member set is split into clusters in a balanced manner, that is, the following operations are performed on each cluster to determine whether the number of elements in the member set of the cluster exceeds a certain threshold. If so, the cluster is clustered again, and the member set is updated to the member set after the re-clustering. Each sub-cluster obtained after the re-clustering shares the member set of the original parent cluster; 3) For each cluster finally formed, that is, in step 2), the member set cluster is split into sub-clusters, and a cluster list is constructed, and the cluster list size is fixed; each cluster list is first filled with elements in the member set. If the number of elements in the member set is less than the fixed cluster list size, it is then filled with the candidate set. After the cluster list of all clusters is finally formed, the difference between all vector data sets and the cluster list of all clusters is taken. The vector in the difference is the noise part, and all clusters, that is, their cluster lists, are the cluster parts.

[0010] The hybrid index in the present invention helps to take advantage of the advantages of both graph index and cluster index to achieve high performance and high accuracy large-scale vector approximate nearest neighbor search. GPU devices accelerate the retrieval performance of the clustering part of the hybrid index in the present invention, and NVMe devices relieve the host memory pressure in traditional vector retrieval systems, reducing the deployment cost of the system in the present invention. In general, the present invention achieves high performance and low cost large-scale vector approximate nearest neighbor search capabilities through hybrid index and GPU-NVMe architecture platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A hybrid index structure diagram constructed for large-scale data by the present invention; Figure 2 This is a schematic diagram of the GPU-NVMe platform architecture on which the present invention is based; Figure 3 This is a diagram of the heterogeneous system architecture of the present invention. DETAILED DESCRIPTION

[0012] The present invention is further described below through the accompanying drawings and specific embodiments.

[0013] The system of the present invention comprises: 1. Hybrid index construction method The hybrid index construction method of the present invention first divides the vector data set into a clustering part and a noise part. The specific steps are as follows: Distinguishing between clustering part and noise part: Clustering algorithm is used to cluster vector data sets and obtain the centroid of each cluster. On this basis, vector data is distributed to each cluster according to the similarity with the centroid of the cluster to form the cluster part; the remaining vector data is regarded as the noise part. The noise part does not participate in clustering, but is retrieved through graph indexing.

[0014] Further processing of clustered data: In the clustering process, a hierarchical balanced clustering strategy is used to split clusters with too large member sets. Each final cluster contains a member set and a candidate set. The size of the cluster list is fixed, and the cluster list is first filled with the member set. If the member set is insufficient, vectors are selected from the candidate set to fill it.

[0015] Processing of the noise part: After clustering is completed, the vector set of all clusters and the vector of the noise part are subtracted to obtain the specific vector of the noise part. The data of the cluster part is managed and stored by inverted index, and the noise part is retrieved by graph index.

[0016] 2. Hybrid index deployment method In the deployment process of hybrid index, the present invention deploys the clustering part in the NVMe SSD storage array, and the noise part is stored in the host memory through the graph index method. The clustering data and noise data are accelerated through the collaborative calculation of GPU and CPU. The specific deployment steps are as follows: Storage of cluster data: The cluster data is evenly divided into multiple sub-clusters and stored in the NVMe SSD array. The data is organized in logical blocks. In this storage architecture, data is directly transferred to the GPU device without passing through the host memory, making full use of the GPU's computing power for acceleration.

[0017] Storage of noise data: The data of the noise part is stored in the host memory through graph indexing, and the vector of the noise part is retrieved using the CPU.

[0018] 3. Hybrid index retrieval method under heterogeneous architecture In the vector retrieval process, the present invention proposes a CPU-GPU collaborative computing retrieval mode to improve the retrieval efficiency. The specific steps of the retrieval process are as follows: Accepting vector retrieval requests: After receiving a vector retrieval request from the vector search application layer, the system first searches the cluster centroid index residing in the host memory to find multiple nearest neighbor clusters.

[0019] Data transfer to GPU: Through NVMe driver layer I / O, the system directly transfers the vector list of the corresponding cluster to the GPU device memory through PCIe P2P (Peer-to-Peer). This process avoids the data transfer bottleneck through the host memory and improves data processing efficiency.

[0020] CPU and GPU parallel computing: While data is being transferred to the GPU device memory, the CPU begins to retrieve the graph index of the noise part, and the GPU is responsible for calculating the similarity between the clustering part vector and the query vector. At this time, the CPU and GPU retrieval work is carried out in parallel, greatly improving the retrieval efficiency.

[0021] Merge retrieval results: After the GPU completes the vector similarity calculation, it transfers the results back to the host memory. The CPU merges the similarity calculation results from the GPU and the retrieval results of the noise partial graph index, and finally returns the approximate nearest neighbor search results of the vector.

[0022] 4. GPU-NVMe heterogeneous platform supporting large-scale approximate nearest neighbor vector retrieval It includes solid-state drive storage arrays that support the NVMe protocol and GPU and CPU devices that support P2P data transmission. NVMe devices support the SPDK user-mode driver interface, GPU devices support PCIe P2P DMA data transmission, and CPU devices support in-memory vector graph index retrieval. Example

[0023] See also Figure 1 , distinguish the noise vectors from the clustering vectors in the full data set X. In this embodiment, the input vector data set X is first clustered. A clustering algorithm (such as K-means or DBSCAN, etc.) is used to cluster the vector data to obtain the centroids C of multiple clusters. The clustering algorithm calculates the similarity between each vector and the centroid of the cluster (for example, using Euclidean distance or cosine similarity), and assigns the vector data to the cluster that is most similar to it to form a cluster list R. Vectors that cannot be reasonably classified into any cluster, or vectors with low similarity to the centroids of all clusters, are judged to be the noise part N. The vectors of the noise part will not participate in the subsequent clustering processing, but will be retrieved through the graph index method. The specific steps are as follows: Calculation of cluster centroids: The centroid of each cluster is calculated through a clustering algorithm. The centroid of a cluster is the average value of all vectors in the cluster, representing the center position of the cluster. Allocation of vector data: According to the similarity between each vector and the centroid of each cluster, the most similar cluster is selected and the vector data is allocated to the corresponding cluster. These vectors allocated to the clusters constitute the cluster part. Determination of the noise part: Vectors that cannot be reasonably classified into any cluster, or vectors with low similarity to the centroids of all clusters, are judged to be noise parts. Vectors of the noise part will not participate in the subsequent clustering process, but will be retrieved through graph indexing.

[0024] See also Figure 2, the system of the present invention includes a heterogeneous hardware architecture platform based on NVMe devices and GPU devices, and relies on it for implementation. Its NVMe device is an SSD storage array that supports the NVMe protocol, and the GPU device supports PCIe device P2P data transmission. By using the same mapping method as the DRAM host memory, the GPU device memory is exposed to the PCIe bus through the Bar register, and then the bus address is mapped to the virtual address space, so as to realize the direct I / O between NVMe and GPU devices. Furthermore, since commercial GPU hardware devices are usually PCIe 16x standard and NVMe devices are usually PCIe 4x standard, the hardware architecture of this embodiment uses multiple NVMe devices to connect GPU devices to maximize the use of PCIe bandwidth.

[0025] See also Figure 3 This embodiment adopts a heterogeneous system for large-scale vector approximate nearest neighbor search based on GPU-NVMe architecture to efficiently solve the problem of large-scale vector retrieval. The specific implementation is as follows: When the system receives a retrieval request from the vector search application layer (such as a recommendation system, search engine, large model application, etc.), it first reads and loads the cluster centroid index C_index from the host memory. The cluster centroid index is constructed by the clustering algorithm on the vector data set and is used to represent the center point of each cluster. In this step, the system will search the requested query vector through the cluster centroid index to find multiple most similar clusters. Further, the cluster list R data on the NVMe side is transmitted to the GPU. After determining the closest multiple clusters, the logical block address of the cluster list on the NVMe device is obtained through the NVMe device address mapping table L_map of the cluster list, and then the system transmits the data to the GPU device memory through the NVMe driver layer I / O data. The data is directly transmitted through the PCIe P2P (Peer-to-Peer) technology, which can bypass the host memory, reduce the delay of data transmission, and avoid the bottleneck of the traditional I / O software stack, thereby significantly improving the data processing efficiency. Further, the CPU and GPU are calculated in parallel, and while the data is being transferred to the GPU device memory, the CPU begins to retrieve the data of the noise part. The vector data of the noise part is stored in the form of graph index and retrieved by the CPU. The GPU focuses on calculating the similarity between the vector of the clustering part and the query vector. The GPU device performs cluster search calculations in parallel through multiple streams. At this time, the retrieval tasks of the CPU and GPU are carried out in parallel, which greatly improves the overall efficiency of vector retrieval. Further, the retrieval results are merged, and after the GPU completes the similarity calculation, the GPU transfers the calculation results back to the host memory. At the same time, the CPU also completes the graph index retrieval of the noise part. Then, the CPU merges the similarity calculation results from the GPU and the retrieval results of the noise part, and finally forms an approximate nearest neighbor search result of the vector, and returns the merged results to the upper-level vector search application.

[0026] The above specific implementations are only for further explanation of the present invention, and are not intended to limit the present invention. Any equivalent implementations of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A heterogeneous system for large-scale vector approximate nearest neighbor search based on GPU-NVMe architecture, characterized in that: The system comprises: A hybrid index that combines clustering and graph algorithms to build, deploy, and retrieve index data structures for organizing large-scale approximate nearest neighbor searches in the system; A heterogeneous platform for large-scale approximate nearest neighbor search based on a direct GPU-NVMe architecture to support deployment and retrieval of hybrid indexes; The hybrid index combining clustering and graph algorithms includes: dividing a vector data set into a clustering part and a noise part, and deploying it on a direct GPU-NVMe hardware architecture, performing vector approximate nearest neighbor retrieval, i.e., vector similarity retrieval, through a CPU-GPU collaborative computing mode, specifically including the following steps: 1) The vector data set is divided into a clustering part and a noise part; the clustering data part is deployed on the NVMe SSD array, retrieved by inverted index, and the GPU device accelerates the vector similarity retrieval; the noise part is retrieved by graph index, and when the GPU processes the clustering data, the CPU is used to perform similarity retrieval of the noise vector part; the CPU and GPU retrieval are performed in parallel; 2) During the deployment process, the clustered data part is evenly divided into multiple groups of clusters and deployed under the logical abstraction of the NVMe SSD driver layer, and the storage data structure is organized in the form of logical blocks; at the same time, in the direct GPU-NVMe architecture, the data transmission between the GPU and NVMe bypasses the host memory, that is, the clustered data part deployed on the NVMe SSD can be directly transmitted to the GPU device during the retrieval process, and the GPU device is used to accelerate the vector similarity retrieval; The heterogeneous platform for large-scale approximate nearest neighbor search based on direct GPU-NVMe architecture includes: The clustering part is stored on the NVMe device, the cluster centroid index and the noise graph index reside in the host memory, and the CPU is used to retrieve the graph vector index and the GPU is used to retrieve the cluster vector index. During the search, the data between the NVMe device and the GPU device is directly transmitted in P2P, bypassing the host memory and the traditional Linux system I / O software stack. The GPU device and the CPU device run in parallel during the search calculation process, which specifically includes the following steps: 1) Receive vector search requests from the vector search application side, search for multiple nearest neighbor clusters in the cluster centroid index residing in the host memory, and then transfer the vector list of the corresponding cluster to the GPU device memory through PCIe P2P via NVMe driver layer I / O; 2) After the cluster vector list is transmitted to the GPU device memory via PCIe P2P via NVMe I / O command, the CPU searches in the noise graph index; at the same time, the GPU device performs similarity calculations on the vectors in the vector search request and the vectors in the cluster list; 3) After the GPU calculation is completed, the similarity calculation results are transferred back to the host memory, and the CPU device merges the noise graph index similarity search results and the similarity calculation results from the GPU to generate vector approximate nearest neighbor search results, and returns the results to the upper-level vector search application.

2. The heterogeneous system according to claim 1, characterized in that: The method of dividing the vector data set into a clustering part and a noise part specifically includes the following steps: 1) Obtain the centroid of the vector data set through a clustering algorithm, and initialize an empty member set and candidate set for each cluster represented by the centroid; then, traverse all the vector data in the vector data set; during the traversal, obtain the TOP-K nearest centroids for each vector data, and add the vector data to the member set of the cluster represented by the nearest TOP-1 centroid and the candidate set of the cluster represented by the nearest TOP-2 to TOP-K centroids; 2) Through hierarchical balanced clustering, the member set is split into clusters in a balanced manner, that is, the following operations are performed on each cluster to determine whether the number of elements in the member set of the cluster exceeds a certain threshold. If so, the cluster is clustered again, and the member set is updated to the member set after the re-clustering. Each sub-cluster obtained after the re-clustering shares the member set of the original parent cluster; 3) For each cluster finally formed, that is, in step 2), the member set cluster is split into sub-clusters, and a cluster list is constructed, and the cluster list size is fixed; each cluster list is first filled with elements in the member set. If the number of elements in the member set is less than the fixed cluster list size, it is then filled with the candidate set. After the cluster list of all clusters is finally formed, the difference between all vector data sets and the cluster list of all clusters is taken. The vector in the difference is the noise part, and all clusters, that is, their cluster lists, are the cluster parts.

Images