Vector retrieval method, apparatus, device, storage medium and program product

Abstract

This application discloses a vector retrieval method, apparatus, device, storage medium, and computer program, belonging to the field of data processing technology. The method is applied to a first processor included in a target node, which also includes a first storage medium storing a vector dataset. The method includes: obtaining the target graph node corresponding to the query vector from a graph index; based on the target graph node, reading data vectors corresponding to multiple candidate PQ vectors from the first storage medium to obtain multiple candidate data vectors, wherein these multiple candidate PQ vectors are selected from a PQ vector set based on their distance from the query vector, and the PQ vector set is determined based on the vector dataset; and determining the vector retrieval result of the query vector based on the multiple candidate data vectors. That is, this application can filter out relatively accurate candidate data vectors, thus only needing to read these candidate vector data from the first storage medium, reducing the number and size of I / O requests.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a vector retrieval method, apparatus, device, storage medium, and program product. Background Technology

[0002] Vector retrieval is the process of finding the top K (or top-K) data vectors in a given vector dataset that are closest to a query vector, according to a certain metric. Common vector retrieval algorithms primarily focus on the design of index structures, such as graph-based, hash-based, tree-based, and quantization-based algorithms. These algorithms largely rely on in-memory index structures to achieve fast and accurate searches. However, with the explosive growth of data vectors, the memory requirements of these algorithms are increasing, leading to higher costs for vector retrieval services.

[0003] In related technologies, the Scalable and Accurate Nearest Neighbor Search (SPANN) algorithm can be used to handle the nearest neighbor search problem in large-scale vector datasets, thereby reducing the high cost of large-scale vector retrieval services. The SPANN algorithm uses a clustering algorithm to partition the vector dataset into multiple posting lists, and then uses the centroids of these posting lists to construct a graph-based index (referred to as a graph index). Each graph node in the graph index corresponds to a posting list. After constructing the graph index, it is stored in memory, while the posting lists are stored on a solid-state disk (SSD). When processing query requests, the SPANN algorithm, based on the query vector, first traverses the graph index in memory to find graph nodes that are close to the query vector. Then, it loads the posting lists corresponding to these graph nodes from the SSD into memory and calculates the distance between each data vector in the posting list and the query vector. Finally, it finds the Top-K data vectors that are close to the query vector, thus obtaining the vector retrieval result for that query vector.

[0004] However, when the SPANN algorithm handles concurrent queries, each query request may involve loading multiple inverted lists from the SSD. These input / output (I / O) requests are coarse-grained and involve large amounts of data, increasing the overhead of I / O operations on the SSD and limiting its throughput. Furthermore, when the number of concurrent queries exceeds the processing capacity of the SSD, it will lead to increased I / O response latency, which in turn affects the timeliness of the vector retrieval service. Summary of the Invention

[0005] This application provides a vector retrieval method, apparatus, device, storage medium, and program product, which can reduce the number and size of I / O requests required during vector retrieval, improve the timeliness of vector retrieval services, and thus enhance the overall performance of vector retrieval services. The technical solution is as follows:

[0006] In a first aspect, a vector retrieval method is provided, applied to a first processor included in a target node, the target node further including a first storage medium storing a vector dataset, the vector dataset including multiple data vectors, the method comprising:

[0007] The target graph node corresponding to the query vector is obtained from the graph index, which includes multiple graph nodes, each of which is the centroid of a vector cluster in the vector dataset. The graph index is stored in the target node and located outside the first storage medium. Based on the target graph node, data vectors corresponding to multiple candidate product-quantized (PQ) vectors are read from the first storage medium to obtain multiple candidate data vectors. The multiple candidate PQ vectors are selected from the PQ vector set according to their distance from the query vector. The PQ vector set is determined based on the vector dataset. Based on the multiple candidate data vectors, the vector retrieval result of the query vector is determined.

[0008] Therefore, by traversing the graph index and analyzing the distance between the query vector and the PQ vector corresponding to the data vector in the vector cluster indicated by the graph node, multiple candidate PQ vectors and their corresponding data vectors can be identified. Thus, through these two layers of filtering operations—traversing the graph index and analyzing the distance between PQ vectors—the search range for vectors can be significantly narrowed. This allows only the data vectors corresponding to the candidate PQ vectors to be read from the first storage medium, thus determining the final vector retrieval result. Since this application only needs to read a small number of data vectors from the first storage medium, rather than reading data vectors from the first storage medium in units of inverted lists, the size and number of I / O requests initiated to the first storage medium are greatly reduced, alleviating system throughput bottlenecks and improving overall query performance.

[0009] In one possible implementation, the target node further includes a second storage medium storing the graph index, and the second storage medium has better input / output I / O performance than the first storage medium.

[0010] In one possible implementation, the second storage medium is the memory of the target node.

[0011] In one possible implementation, the target node further includes a third storage medium storing the PQ vector set.

[0012] In one possible implementation, the target node further includes a second processor, and the third storage medium is the memory of the second processor; the step of reading data vectors corresponding to multiple candidate product-quantized (PQ) vectors from the first storage medium based on the target graph node includes:

[0013] The system sends multiple vector identifiers to the second processor, wherein the multiple vector identifiers are vector identifiers of data vectors in the vector cluster to which the target graph node belongs; it receives vector identifiers of the multiple candidate PQ vectors sent by the second processor; wherein the vector identifiers of the multiple candidate PQ vectors are obtained by the second processor from the PQ vectors corresponding to the multiple vector identifiers based on the distance between the PQ vectors corresponding to the multiple vector identifiers and the query vector; and it reads the multiple candidate data vectors from the first storage medium based on the vector identifiers of the multiple candidate PQ vectors.

[0014] Therefore, this application uses a second processor to calculate the distance between the query vector and the PQ vector, which can quickly locate multiple candidate data vectors similar to the query vector, greatly reducing the number of data vectors that actually need to be read and calculated in the future.

[0015] In one possible implementation, the first storage medium is a solid-state drive (SSD).

[0016] In one possible implementation, the first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU), a neural network processor (NPU), a network processor (NP), or a field-programmable gate array (FPGA).

[0017] Therefore, when the second processor is a GPU, this application accelerates the calculation of the distance between the query vector and the PQ vector using the GPU. This fully utilizes the GPU's parallel computing capabilities and features such as high-speed caching and distributed memory to quickly locate multiple candidate PQ vectors most similar to the query vector, significantly accelerating the vector retrieval process and improving retrieval efficiency. This is of great significance for applications handling large-scale vector sets and those with high real-time requirements.

[0018] In one possible implementation, the target node further includes a second storage medium, which stores a vector list corresponding to each of the plurality of graph nodes, the vector list including vector identifiers of all data vectors in the vector cluster to which the corresponding graph node belongs;

[0019] Before sending the multiple vector identifiers to the second processor, the method further includes:

[0020] Read the list of target vectors corresponding to the target graph nodes from the second storage medium;

[0021] The plurality of vector identifiers are determined based on the vector identifiers in the target vector list.

[0022] It should be noted that the vector list corresponding to the graph node in this application only includes the vector identifiers of the data vectors in the vector cluster, and not the original data vectors. Therefore, compared with the inverted list in the SPANN scheme, the vector list in this application embodiment is a lightweight list that occupies very little storage space.

[0023] In one possible implementation, determining the plurality of vector identifiers based on the vector identifiers in the target vector list includes:

[0024] The vector identifiers in the target vector list are deduplicated to obtain the plurality of vector identifiers.

[0025] Considering that each graph node corresponds to a vector cluster, and when dividing a vector dataset into multiple vector clusters, different vector clusters may contain the same data vectors, meaning that the vector lists corresponding to different graph nodes may contain the same vector identifiers. Based on this, after determining the target vector list, the first processor can perform deduplication on the vector identifiers in the target vector list, obtaining multiple vector identifiers. At this point, the multiple vector identifiers are distinct, thus avoiding the second processor reading and processing duplicate data.

[0026] In one possible implementation, reading the plurality of candidate data vectors from the first storage medium based on the vector identifiers of the plurality of candidate PQ vectors includes:

[0027] Based on the vector identifiers of the plurality of candidate PQ vectors, at least one read request is generated, the read request being used to request the reading of at least one data vector located in the same storage area in the first storage medium; the at least one read request is sent to the first storage medium; and the plurality of candidate data vectors sent by the first storage medium are received.

[0028] In other words, when the data vector to be read is located in the same storage area in the first storage medium, multiple requests for the same storage area can be merged to eliminate redundant I / O, thereby reducing the number and size of I / O requests to be initiated to the first storage medium.

[0029] Secondly, a vector retrieval system is provided, the system including a first processor, a second processor and a first storage medium, the first storage medium storing a vector dataset, the vector dataset including multiple data vectors;

[0030] The first processor is configured to obtain the target graph node corresponding to the query vector from the graph index and send multiple vector identifiers to the second processor. The graph index includes multiple graph nodes, each graph node being the centroid of a vector cluster in the vector data set. The graph index is stored outside the first storage medium, and the multiple vector identifiers are vector identifiers of data vectors in the vector cluster to which the target graph node belongs.

[0031] The second processor is configured to filter out multiple candidate PQ vectors from the PQ vectors corresponding to the multiple vector identifiers based on the distance between the PQ vectors corresponding to the multiple vector identifiers and the query vector, and send the vector identifiers of the multiple candidate PQ vectors to the first processor.

[0032] The first processor is further configured to read data vectors corresponding to the plurality of candidate PQ vectors from the first storage medium based on the vector identifiers of the plurality of candidate PQ vectors, to obtain a plurality of candidate data vectors, and to determine the vector retrieval result of the query vector based on the plurality of candidate data vectors.

[0033] In one possible implementation, the system further includes a second storage medium that stores the graph index, and the second storage medium has better input / output I / O performance than the first storage medium.

[0034] In one possible implementation, the second storage medium is the memory of the host to which the first processor belongs.

[0035] In one possible implementation, the system further includes a third storage medium storing the PQ vector set.

[0036] In one possible implementation, the third storage medium is the memory of the host to which the second processor belongs.

[0037] In one possible implementation, the first storage medium is located in the host to which the first processor belongs, and the first storage medium is a solid-state drive (SSD).

[0038] In one possible implementation, the first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU), a neural network processor (NPU), a network processor (NP), or a field-programmable gate array (FPGA).

[0039] In one possible implementation, the system further includes a second storage medium storing vector lists corresponding to the plurality of graph nodes, the vector lists including vector identifiers of all data vectors in the vector cluster to which the corresponding graph node belongs; the first processor is further configured to:

[0040] Read the list of target vectors corresponding to the target graph nodes from the second storage medium;

[0041] The plurality of vector identifiers are determined based on the vector identifiers in the target vector list.

[0042] In one possible implementation, the first processor is further configured to:

[0043] The vector identifiers in the target vector list are deduplicated to obtain the plurality of vector identifiers.

[0044] In one possible implementation, the first processor is further configured to:

[0045] Based on the vector identifiers of the plurality of candidate PQ vectors, at least one read request is generated, the read request being used to request the reading of at least one data vector located in the same storage area in the first storage medium;

[0046] Send at least one read request to the first storage medium;

[0047] Receive the plurality of candidate data vectors sent by the first storage medium.

[0048] Thirdly, a vector retrieval device is provided, which has the function of implementing the vector retrieval method behavior described in the first aspect above. The vector retrieval device includes at least one module for implementing the vector retrieval method provided in the first aspect above.

[0049] Fourthly, a computer device is provided for implementing the vector retrieval method described in the first aspect.

[0050] Fifthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the vector retrieval method described in the first aspect.

[0051] Sixthly, a computer program product containing instructions is provided, which, when executed on a computer, causes the computer to perform the vector retrieval method described in the first aspect. Alternatively, a computer program is provided that, when executed on a computer, causes the computer to perform the vector retrieval method described in the first aspect.

[0052] The technical effects achieved by the second to sixth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here. Attached Figure Description

[0053] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application;

[0055] Figure 3 This is a flowchart illustrating a vector retrieval method provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of a multi-level index structure across devices provided in an embodiment of this application;

[0057] Figure 5 This is a schematic diagram of a GPU initialization process provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of a CPU and GPU collaboratively processing a query request, provided in an embodiment of this application.

[0059] Figure 7 This is a schematic diagram of a process for determining multiple candidate PQ vectors provided in an embodiment of this application;

[0060] Figure 8 This is a schematic diagram of a process for determining vector retrieval results provided in an embodiment of this application;

[0061] Figure 9 This is a schematic diagram illustrating the number of SSD pages read in a single query request, as provided in an embodiment of this application.

[0062] Figure 10 This is a schematic diagram illustrating the delay in processing a single query request, provided in an embodiment of this application.

[0063] Figure 11 This is a schematic diagram of the structure of a vector retrieval device provided in an embodiment of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0065] To facilitate understanding, before explaining the vector retrieval method provided in this application, the terminology, application background, and implementation environment involved in the embodiments of this application will be introduced first.

[0066] First, the terminology used in the embodiments of this application will be introduced.

[0067] 1. Product quantization (PQ)

[0068] PQ is an algorithm for vector compression and accelerated retrieval. Its core idea is to segment the original vector (divide it into subspaces) and cluster them, and then use the cluster center identifier (identity, ID, also known as codebook) to which each sub-vector belongs to represent the original vector.

[0069] The basic principle of the PQ algorithm can include the following steps (1)-(3).

[0070] (1) Segmentation and clustering: Divide the original vector into M sub-vectors (or subspaces) and cluster each sub-vector to obtain K cluster centers.

[0071] (2) Quantization representation: Each sub-vector can be represented by its cluster center ID. Therefore, a D-dimensional original vector can be compressed into an M-dimensional vector, where M is the number of sub-vectors and each dimension corresponds to a cluster center ID.

[0072] (3) Distance Calculation: During the retrieval process, it is necessary to calculate the distance between the query vector and the vectors in the data set. The PQ algorithm provides two distance calculation methods: symmetric distance computation (SDC) and asymmetric distance computation (ADC). SDC calculates the distance by quantizing the query vector and then calculating the distance between the quantized vectors. ADC, on the other hand, does not quantize the query vector; it directly calculates the distance between the query vector and each cluster center and uses these distances to estimate the distance between the query vector and the data set vectors.

[0073] In this embodiment of the application, the PQ vector is a vector processed by the PQ algorithm. Its dimension is lower than that of the original vector (i.e., the data vector before quantization), but it can retain the main information of the original vector and be used to accelerate the retrieval process.

[0074] The PQ vector consists of M cluster center IDs, with each ID corresponding to the cluster center to which a subvector belongs. These IDs are typically encoded as short vectors (e.g., when the number of cluster centers is 256, an ID can be represented by 8 bits), thereby further reducing storage costs.

[0075] As an example, suppose we have a 128-dimensional data vector x, which is divided into 8 sub-vectors (M=8), each with a dimension of 16 (D / M=16). Then, each sub-vector is clustered, resulting in 256 cluster centers (K=256). The process of quantizing this data vector to obtain the PQ vector is as follows: for each sub-vector of data vector x, find its corresponding cluster center and represent the sub-vector using the ID of that cluster center. For example, if the first sub-vector of data vector x belongs to the 3rd cluster center, then it is represented by ID=3. After quantization, x is represented as an 8-dimensional vector (M=8), where each dimension corresponds to a cluster center ID. For example, if all sub-vectors of data vector x belong to the 3rd, 5th, 7th, ..., 255th cluster centers respectively, then the PQ vector corresponding to data vector x is represented as: [3, 5, 7, ..., 255].

[0076] 2. PQ Distance Table

[0077] In the PQ algorithm, the PQ distance table is a key component, used to quickly calculate the distance between the query vector and the vectors in the vector dataset. The following will explain the construction and function of the PQ distance table.

[0078] (1) Construction of PQ distance table.

[0079] Assuming the vector dimension is D, the PQ algorithm first divides the original D-dimensional vector into M segments, each with a sub-dimension of D / M. Then, it performs K-means clustering on each segment of the training set, forming K clusters. Each segment will then have K cluster center vectors, each with a dimension of D / M. Each cluster center is represented by a cluster ID, for example, 0 to (K-1). After training, each sample in the training set can be quantized into a short M-dimensional vector, where each element is the cluster center number to which the sub-segment belongs. The Cartesian product of all cluster centers forms a dense partition of the entire space; this partition is the codebook for the PQ algorithm.

[0080] In practical applications, for a query vector, it is also divided into M segments, and then the distance is calculated between each segment and the K class center vectors of each segment. In this way, a total of M×K distances are obtained, which constitute the PQ distance table for the query vector. Each row in the PQ distance table represents M distances, corresponding to the distance between each segment of the query vector and the center vector of the corresponding class.

[0081] (2) The function of the PQ distance table.

[0082] During the query phase, for each data vector in the vector dataset, the distance between the query vector and the data vector can be quickly calculated using the PQ distance table based on its encoded PQ vector (i.e., the class center number to which each sub-vector belongs). Specifically, based on the PQ distance table of the query vector, the distance between each segment of the data vector and the corresponding segment of the query vector is found, and then these distances are added together to obtain the distance between the query vector and the corresponding PQ vector of the data vector.

[0083] As an example, suppose D = 128, M = 8, K = 256, meaning the sample vector has a dimension of 128. Divide it into 8 segments, each with 16 dimensions, and then perform 256-class K-means clustering on each segment. For a query vector q, divide it into 8 segments (q1, q2, ..., q8). Calculate the distances between q1, q2, ..., q8 and the 256 class center vectors of segments 1, 2, ..., 8 respectively, resulting in 8 × 256 distances. These distance values ​​constitute the PQ distance table for the query vector. For a data vector c in the vector dataset, suppose its corresponding PQ vector is [3, 2, 256, 3, 90, 45, 32, 1]. Based on the PQ distance table for the query vector, find the distance between each segment of the PQ vector corresponding to data vector c and the corresponding segment of query vector q. Then, sum these distances to obtain the distance between the PQ vectors corresponding to query vector q and data vector c.

[0084] 3. Inverted list (posting list)

[0085] Scalable and accurate nearest neighbor search (SPANN) uses clustering algorithms, such as K-means clustering, density-based spatial clustering of applications with noise (DBSCAN), and hierarchical balanced clustering (HBC), to divide the data vectors in the entire vector dataset into multiple clusters. Data vectors within each cluster are similar to each other, while data vectors between different clusters differ significantly.

[0086] Each cluster can be viewed as an inverted list, and each inverted list contains data vectors belonging to the same cluster.

[0087] Secondly, the application background of the embodiments of this application will be introduced.

[0088] Vector retrieval in high-dimensional space involves finding the most similar vector (or the top N most similar vectors) to a given query vector within a dataset composed of high-dimensional vectors. It has wide applications in computer science, data mining, information retrieval, machine learning, and recommender systems. Here, "high-dimensional" refers to the extremely high dimensionality of the vectors, often far exceeding the three or four dimensions that humans can intuitively comprehend. For example, in text processing or image recognition, a vector may contain thousands or even millions of dimensions.

[0089] In high-dimensional vector retrieval, similarity is typically used to measure the proximity between two vectors. Commonly used similarity metrics include Euclidean distance, cosine similarity, and Manhattan distance, with different metrics suitable for different application scenarios and data characteristics.

[0090] In recent years, with the rapid development of large language models (LLMs), the application of vector retrieval technology in retrieval augmented generation (RAG) has become an important research direction in the field of natural language processing (NLP). This technology aims to expand the knowledge base of LLMs, enhance contextual understanding capabilities, and improve the accuracy and richness of generated content by combining the advantages of vector retrieval and LLMs.

[0091] As an example, domain knowledge from various data formats (such as documents, images, videos, and languages) is embedded into a high-dimensional vector space using neural networks and stored as feature vectors in a vector database. When a user sends a query, the vector retrieval engine finds similar knowledge based on the semantics of the query and submits that knowledge along with the query to a large language model for more accurate reasoning.

[0092] Current vector retrieval algorithms primarily focus on index design, such as graph-based, hash-based, tree-based, and quantization-based methods. These methods largely rely on in-memory indexes to support fast and accurate searches. However, with the explosive growth of vector data, vector retrieval systems face the challenge of handling datasets containing millions or even billions of vectors. In such cases, in-memory vector retrieval algorithms require substantial memory space, leading to a significant increase in the total cost of ownership for data centers.

[0093] To reduce the cost of large-scale vector retrieval services, a straightforward approach is to deploy vector retrieval functionality on storage servers. This method stores the primary vector index on solid-state drives (SSDs) while using a small amount of dynamic random access memory (DRAM) as working memory.

[0094] See Figure 1 Data center storage servers are typically equipped with computing resources, including multi-core general-purpose central processing units (CPUs) and mid-to-high-end graphics processing units (GPUs), to accelerate storage-related services. When deploying applications such as RAG-based Large Oracle Models (LLMs), a distributed computing and storage strategy can be adopted. Compute nodes utilize high-end GPUs to perform LLM inference, while storage nodes utilize local computing resources (CPU / GPU) to provide vector retrieval services.

[0095] Upon receiving a user's inference request, the inference node first sends the request content to the storage node. Then, the storage node locally invokes the vector retrieval engine. Specifically, the local CPU and accelerator (GPU) work in concert to perform vector retrieval on an index structure stored on DRAM and SSD. After retrieving the most relevant knowledge for the query, the storage node returns it to the inference node. Finally, the inference node sends this knowledge as additional features to the query to the large language model, leveraging the massive GPUs for more accurate inference services.

[0096] In the above reasoning process, the retrieval process of the vector database directly affects the accuracy of the final reasoning result and the timeliness of the query. Regarding the retrieval process of the vector database, related technologies provide two solutions. To facilitate the introduction of the improvements and beneficial effects of the vector retrieval method provided in this application's embodiments in subsequent method examples, these two solutions will be briefly introduced below.

[0097] The first vector retrieval scheme: Suhas et al. proposed a disk-based approximate nearest neighbor (ANN) vector retrieval scheme (also known as the DiskANN scheme). This scheme explored the deployment of graph-based vector retrieval in storage servers for the first time and proposed a new graph indexing method to store both the index graph and the actual data vectors on SSDs, saving memory costs and providing a solution for large-scale vector retrieval at the billion-level.

[0098] However, the DiskANN solution needs to continuously read the graph index and data vector from the SSD when processing each query request, which will increase the query latency and thus limit its application in latency-sensitive scenarios.

[0099] The second vector retrieval scheme: Chen et al. proposed an efficient algorithm scheme for nearest neighbor search in high-dimensional data (i.e., SPANN). The SPANN scheme uses a clustering algorithm to partition the vector dataset into inverted lists, and then uses the centroids of the inverted lists (i.e., the centroids of the clusters) to construct a graph-based index (or graph index for short). In this graph index, nodes represent centroids, and edges between nodes are established based on the distance between centroids. The weight of the edge can be a distance value to represent the similarity between two centroids. After constructing the graph index, it is maintained in host memory, while a large number of inverted lists are stored on SSD. When processing query requests, the execution flow of the SPANN scheme is as follows: First, it traverses the graph index in memory to find the indices of some inverted lists that are close to the query vector (i.e., find the few centroids / most similar clusters closest to the query vector). Then, these inverted lists are loaded from SSD into memory, and the distances between the vectors contained within them and the query vector are traversed sequentially to obtain the final vector retrieval results.

[0100] However, the SPANN scheme requires sending a large number of concurrent coarse-grained input / output (I / O) requests when handling concurrent queries, which severely limits throughput and causes latency to increase significantly with the increase of threads.

[0101] As an example, to illustrate how the latency and throughput of the SPANN scheme vary with the number of threads, a detailed test and analysis of the SPANN scheme was conducted using a commercial high-end SSD and the SIFT1B dataset (a dataset containing 1 billion 128-dimensional vectors). The test and analysis results show that when using 1-3 threads, the latency of the SPANN scheme does not change significantly, and the throughput increases linearly; however, when using more than 3 threads, the throughput reaches saturation, and the latency increases significantly with the number of threads.

[0102] Furthermore, the query latency of the SPANN scheme is divided into two parts: index latency (denoted as HeadIndex-Latency) and read latency (denoted as PostingList-Latency). HeadIndex-Latency represents the time spent by SPANN traversing the graph index in memory when processing a query, while PostingList-Latency represents the time spent after traversing the graph index to obtain the IDs of the corresponding PostingLists, reading the corresponding PostingLists from SSD, calculating the distances between the query vector and the vectors contained in these PostingLists, and then sorting them to obtain the final nearest neighbors of the query vector. Testing and analysis revealed that as the number of threads increases, the graph traversal time corresponding to HeadIndex-Latency remains unchanged. This is because the graph index data is in memory, and the high bandwidth of memory can support multiple threads traversing the same graph index structure. However, the PostingList-Latency part grows exponentially. This is because when processing a query, after SPANN traverses the graph index, it needs to send a large number of concurrent coarse-grained I / O requests. As the number of threads increases, these concurrent I / O requests cause a system throughput bottleneck.

[0103] A detailed analysis of SPANN query latency reveals that HeadIndex-Latency is primarily limited by memory access speed, while PostingList-Latency is limited by I / O operation efficiency and storage system throughput.

[0104] Based on this, this application provides a vector retrieval method. By traversing the graph index and analyzing the distance between the query vector and the PQ vector corresponding to the data vector in the vector cluster indicated by the graph node, multiple candidate PQ vectors corresponding to data vectors are determined. Thus, through two layers of filtering operations—traversing the graph index and analyzing the distance of the PQ vectors—the retrieval range of vectors can be greatly narrowed. Therefore, only the data vectors corresponding to the candidate PQ vectors need to be read from the SSD to determine the final vector retrieval result. Since this application only needs to read a small number of data vectors from the SSD, instead of reading data vectors from the SSD in units of an inverted list as in related technologies, the size and number of I / O requests initiated to the SSD are greatly reduced, alleviating system throughput bottlenecks and thus improving overall query performance.

[0105] Finally, the implementation environment of the embodiments of this application will be described.

[0106] The vector retrieval method provided in this application involves an implementation environment including a first processor and a first storage medium. The first storage medium stores a vector dataset, which includes multiple data vectors. When executing the vector retrieval method provided in this application, the first storage medium retrieves the target graph node corresponding to the query vector from the graph index; based on the target graph node, it reads the data vectors corresponding to multiple candidate product-quantized (PQ) vectors from the first storage medium to obtain multiple candidate data vectors; and based on the multiple candidate data vectors, it determines the vector retrieval result of the query vector.

[0107] The graph index includes multiple graph nodes, each of which is the centroid of a vector cluster in the vector dataset. The graph index is stored in the target node and is located outside the first storage medium. Multiple candidate PQ vectors are selected from the PQ vector set according to their distance from the query vector. The PQ vector set is determined based on the vector dataset, and the PQ vectors included in the PQ vector set correspond one-to-one with the data vectors included in the vector dataset.

[0108] In some embodiments, the first processor and the first storage medium may be located in the target node, and the first storage medium is an SSD.

[0109] In some embodiments, the above-described implementation environment further includes a second storage medium storing a graph index, and the input / output I / O performance of the second storage medium is superior to that of the first storage medium.

[0110] Based on this, the first processor traverses the graph index in the second storage medium to determine the target graph node corresponding to the query vector, and then reads multiple candidate data vectors from the first storage medium based on the target graph node.

[0111] In some embodiments, the second storage medium is also located in the target node; in this case, the second storage medium can be the memory of the target node. Based on this, the first processor can quickly traverse the graph index from memory to determine the target graph node corresponding to the query vector.

[0112] In some embodiments, the second storage medium stores not only the graph index but also multiple vector lists corresponding to each graph node. Each vector list includes vector identifiers of all data vectors within the vector cluster to which the corresponding graph node belongs. Based on this, after determining the target graph node corresponding to the query vector, the first processor can also read the target vector list corresponding to the target graph node from the second storage medium; and determine multiple vector identifiers based on the vector identifiers in the target vector list.

[0113] In one possible implementation, consider that a vector list corresponds to a vector cluster. Since different vector clusters may contain the same data vectors, after determining the target vector list, the first processor can first deduplicate the vector identifiers in the target vector list, obtaining multiple vector identifiers. At this point, the multiple vector identifiers determined by the first processor are all different, thus avoiding repeatedly reading the same data during subsequent calculations and processing.

[0114] In some embodiments, the above-described implementation environment further includes a third storage medium having a PQ vector set. Based on this, after determining the target graph node, the first processor reads the PQ vectors corresponding to the data vectors in the vector cluster to which the target graph node belongs from the third storage medium, and determines multiple candidate PQ vectors by calculating the distances between these PQ vectors and the query. In this case, the first processor only needs to read the data vectors corresponding to the multiple candidate PQ vectors from the first storage medium.

[0115] In some embodiments, the above-described implementation environment further includes a second processor. In this case, the third storage medium is the memory of the host to which the second processor belongs.

[0116] In one possible implementation, after determining multiple vector identifiers based on the target graph nodes, the first memory can send these vector identifiers to a second processor. The second processor reads the PQ vectors corresponding to these vector identifiers from a third storage medium and calculates the distances between these PQ vectors and the query vectors to filter out multiple candidate PQ vectors. It then sends the vector identifiers of these candidate PQ vectors to the first processor. Based on the vector identifiers of the candidate PQ vectors, the first processor reads multiple candidate data vectors from the first storage medium.

[0117] In some embodiments, the first processor is a CPU, and the second processor is a GPU, a network processor (NP), or one or more integrated circuits capable of calculating PQ vector distance, such as a field-programmable gate array (FPGA).

[0118] Based on the implementation environment of the above example, the first processor and the second processor can be located on the same node or on different nodes, and this application embodiment does not impose any restrictions on this. That is, the target node mentioned above can include at least one computer device, which can be a server.

[0119] In some embodiments, the first processor, the second processor, and the first storage medium described above can form a vector retrieval system. The functions implemented by the first processor, the second processor, and the first storage medium in this vector retrieval system can be found in the description of the implementation environment above, and will not be repeated here.

[0120] Similarly, the vector retrieval system can be integrated into a single computer device or distributed across multiple computer devices; this application does not impose any restrictions on this.

[0121] Please refer to Figure 2 This application provides a schematic diagram of the structure of a general computer device, which can serve as a target node to execute the vector retrieval method provided in this application. Figure 2 As shown, the computer device includes at least one processor 201, a communication bus 202, a memory 203, and at least one communication interface 204.

[0122] Processor 201 can be a general-purpose central processing unit (CPU), network processor (NP), microprocessor, or one or more integrated circuits for implementing the solutions of this application, such as application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0123] The communication bus 202 is used to transmit information between the aforementioned components. The communication bus 202 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0124] The memory 203 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile optical disc, a Blu-ray disc, etc.), a magnetic disk storage medium, or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but not limited thereto. The memory 203 may exist independently and be connected to the processor 201 via the communication bus 202. The memory 203 may also be integrated with the processor 201.

[0125] In this embodiment, the memory 203 may further include an SSD. When executing the vector retrieval method provided in this embodiment, the vector dataset can be stored in the SSD.

[0126] Communication interface 204 uses any transceiver-like device for communicating with other devices or communication networks. Communication interface 204 includes a wired communication interface and may also include a wireless communication interface. The wired communication interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0127] As an example, processor 201 may include one or more CPUs, such as Figure 2 CPU0 and CPU1 are shown in the diagram.

[0128] As an example, a computer device may include multiple processors, such as Figure 2 The processors 201 and 205 are shown. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" may refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0129] In this embodiment of the application, the first processor can be processor 201 and the second processor can be processor 205. Therefore, the vector retrieval method provided in this embodiment of the application can be completed collaboratively by processor 201 and processor 205.

[0130] As an example, the first processor can be a CPU, and the second processor can be a GPU, NP, or FPGA, etc. With the development of processor technology and hardware devices, the first and second processors can also be other newer and better processors, and the specific types of these processors are not limited in the embodiments of this application.

[0131] In some embodiments, the computer device may further include output devices and input devices. The output device communicates with the processor 201 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with the processor 201 and can receive user input in various ways. For example, the input device may be a mouse, keyboard, touchscreen device, or sensing device, etc.

[0132] In some embodiments, memory 203 stores program code 210 for executing the scheme of this application, and processor 201, as a first processor, can execute the program code 210 stored in memory 203. The program code 210 may include one or more software modules, and the computer device can implement the following by using processor 201 and the program code 210 in memory 203. Figure 3 The vector retrieval method provided in the embodiment.

[0133] It should be understood that the application scenarios and hardware architectures described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the emergence of new application scenarios or the evolution of implementation architecture, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0134] Next, the vector retrieval method provided in the embodiments of this application will be explained in detail.

[0135] Figure 3 This is a flowchart illustrating a vector retrieval method provided in an embodiment of this application. The method is applied to a first processor included in a target node, which also includes a first storage medium. The first storage medium stores a vector dataset, which includes multiple data vectors. Please refer to... Figure 3 The method includes the following steps.

[0136] Step 301: Obtain the target graph node corresponding to the query vector from the graph index. The graph index includes multiple graph nodes, each of which is the centroid of a vector cluster in the vector dataset. The graph index is stored in the target node and is located outside the first storage medium.

[0137] In one possible implementation, the target node further includes a second storage medium that stores a graph index, and the input / output I / O performance of the second storage medium is superior to that of the first storage medium.

[0138] As an example, the second storage medium is the target node's memory. This memory can be RAM, and the first processor and the memory are connected and communicate via a bus.

[0139] It should be noted that the memory of the target node can be remote memory or near memory, or it can be the memory of the first processor. This application embodiment does not limit this, and aims to illustrate that the graph index is stored on a medium outside the first storage medium and can be quickly read by the first processor, but does not limit the specific type and location of the storage medium.

[0140] In one possible implementation, the first processor can be a CPU, and the first storage medium can be an SSD.

[0141] For step 301 above, this application embodiment does not limit the method of constructing a graph index based on the vector dataset; its implementation process can refer to other vector retrieval algorithms based on graph indexes. Each graph node in the graph index is the centroid of a vector cluster, which is the center vector of the vector cluster.

[0142] In one possible implementation, step 301 can be implemented as follows: determining the distance between the query vector and multiple graph nodes; and determining the target graph node from the multiple graph nodes based on the distance between the query vector and the multiple graph nodes.

[0143] As an example, the target graph node can be any graph node whose distance to the query vector is greater than a distance threshold. This distance threshold can be determined using a dynamic pruning strategy.

[0144] The dynamic pruning strategy sets a distance threshold based on a multiple of the nearest centroid distance. This threshold is dynamically adjusted and is typically determined based on the distance to the nearest centroid found during the query process. Based on this, after calculating the distances between the query vector and multiple graph nodes (i.e., multiple centroids), graph nodes with distances less than or equal to the threshold are identified as target graph nodes, while other centroids that are farther away are considered unlikely to be nearest neighbors and are filtered out.

[0145] Therefore, during the traversal of the graph index, based on dynamic pruning techniques, only graph nodes with a distance less than or equal to this threshold are retained as target graph nodes. This reduces the amount of data requiring further processing, thereby improving search efficiency.

[0146] Step 302: Based on the target graph node, read the data vectors corresponding to multiple candidate product quantized PQ vectors from the first storage medium to obtain multiple candidate data vectors. These multiple candidate PQ vectors are selected from the PQ vector set according to their distance from the query vector. The PQ vector set is determined based on the vector dataset.

[0147] In some embodiments, the target node further includes a third storage medium storing a PQ vector set.

[0148] It should be noted that the PQ vector set is determined based on the vector dataset. The number of PQ vectors included in the PQ vector set is the same as the number of data vectors included in the vector dataset, and each data vector corresponds to one PQ vector.

[0149] The correspondence between the quantity vector and the PQ vector can be linked through a single vector identifier, where one vector identifier corresponds to one data vector and one PQ vector, and the PQ vector is obtained by quantizing the product of the data vector. Alternatively, the correspondence between the data vector and the PQ vector can be linked through two vector identifiers, where a mapping exists between the identifier of the data vector and the identifier of its corresponding PQ vector. Based on the identifier of the data vector and this mapping relationship, the corresponding PQ vector can be uniquely identified. This application does not limit the method of associating the data vector with its corresponding PQ vector.

[0150] In one possible implementation, step 302 is performed as follows: The first processor reads the PQ vector corresponding to the vector identifier of the data vector in the vector cluster to which the graph node belongs from the third storage medium, and calculates the distance between these PQ vectors and the query vector, so as to filter out multiple candidate PQ vectors from the PQ vectors corresponding to these vector identifiers. Then, the data vectors corresponding to the multiple candidate PQ vectors are read from the first storage medium to obtain multiple candidate data vectors.

[0151] In some embodiments, the target node further includes a second processor, in which case the aforementioned third storage medium may be the memory of the second processor. That is, the first processor may assign the task of filtering multiple PQ vectors from the PQ vector set to the second processor for execution.

[0152] As an example, the second processor can be a GPU, NPU, NP, or FPGA, etc., and this application embodiment does not limit this.

[0153] Taking the GPU as the second processor as an example, the third storage medium can be the GPU's video memory, i.e., high bandwidth memory (HBM). HBM achieves a high-capacity, high-bandwidth, low-latency, and low-power memory solution by stacking multiple dynamic random access memory (DRAM) chips together and packaging them with the GPU. The role of HBM is similar to a data "transfer station," storing data in the frame buffer and waiting for the GPU to access it.

[0154] When calculating the distance between the query vector and the PQ vector, common methods include Euclidean distance and cosine similarity. These methods involve numerous vector multiplications and additions, resulting in significant computational overhead. GPUs, with their powerful parallel computing capabilities, can significantly accelerate the distance calculation process between the query vector and the PQ vector. Furthermore, GPU-based vector retrieval can drastically improve retrieval efficiency and reduce response time. This is particularly important for applications with high real-time requirements (such as search engines and recommendation systems).

[0155] Therefore, to accelerate the vector retrieval process, the first processor can send the vector identifiers of the data vectors in the vector cluster to which the target graph node belongs to the second processor, which then determines multiple candidate PQ vectors. When the second processor is a GPU, as explained above, using the GPU to calculate the distance between the query vector and the PQ vector can significantly improve the retrieval efficiency of multiple candidate PQ vectors.

[0156] In one possible implementation, when the first processor performs step 302 above, its implementation process may include the following steps (1)-(3).

[0157] (1) Send multiple vector identifiers to the second processor. The multiple vector identifiers are the vector identifiers of the data vectors in the vector cluster to which the target graph node belongs.

[0158] As explained earlier, data vectors and query vectors are high-dimensional vectors with a large data volume. Therefore, the first processor can determine the identifier of the data vector that the second processor needs to calculate based on the target graph node, instead of directly sending the data vector corresponding to the target graph node to the second processor for calculation. In this way, the first and second processors determine multiple candidate data vectors by exchanging vector identifiers, rather than vector data, reducing the amount of data exchanged and improving the data transmission rate.

[0159] In some embodiments, the second storage medium for storing the graph index may also store vector lists corresponding to multiple graph nodes in the graph index, each vector list including vector identifiers of all data vectors in the vector cluster to which the corresponding graph node belongs.

[0160] It should be noted that the vector list corresponding to the graph node in this embodiment only includes the vector identifier of the data vector in the vector cluster, and not the original data vector. Therefore, compared with the inverted list in the SPANN scheme, the vector list in this embodiment is a lightweight list that occupies very little storage space.

[0161] In one possible implementation, the process by which the first processor determines multiple vector identifiers can be as follows: the first processor reads a list of target vectors corresponding to the target graph nodes from the second storage medium based on the target graph nodes corresponding to the query vectors; and determines multiple vector identifiers based on the vector identifiers in the list of target vectors.

[0162] It should be noted that the first processor can directly determine the vector identifiers in the target vector list as multiple vector identifiers, meaning the number of multiple vector identifiers is equal to the total number of vector identifiers included in the target vector list. Alternatively, the first processor can filter the vector identifiers in the target vector list to determine only some vector identifiers as multiple vector identifiers.

[0163] In one possible implementation, each graph node corresponds to a vector cluster. When dividing the vector dataset into multiple vector clusters, different vector clusters may contain the same data vectors; that is, the vector lists corresponding to different graph nodes may contain the same vector identifiers. Based on this, after determining the target vector list, the first processor can perform deduplication on the vector identifiers in the target vector list to obtain multiple vector identifiers. At this time, the multiple vector identifiers are different from each other, thereby avoiding the second processor reading and processing duplicate data.

[0164] (2) Receive vector identifiers of multiple candidate PQ vectors sent by the second processor.

[0165] Among them, the vector identifiers of multiple candidate PQ vectors are obtained by the second processor from the PQ vectors corresponding to multiple vector identifiers based on the distance between the PQ vectors corresponding to the multiple vector identifiers and the query vector.

[0166] In one possible implementation, after receiving multiple vector identifiers sent by the first processor, the second processor, based on the association between the data vector and the PQ vector, first determines multiple PQ vectors corresponding to the multiple vector identifiers from the PQ vector set (to distinguish them from the finally determined candidate PQ vectors, these multiple PQ vectors can be referred to as first PQ vectors). Then, it calculates the distance between the multiple first PQ vectors and the query vector, sorts the calculated distance values, and determines the top-K first PQ vectors in the sorting results as candidate PQ vectors, thus obtaining multiple candidate PQ vectors.

[0167] It should be understood that the multiple candidate PQ vectors are obtained by filtering based on the multiple vector identifiers sent by the first processor. Therefore, each candidate PQ vector is associated with one of the multiple vector identifiers. Based on this, the second processor's determination of the multiple candidate PQ vectors can also be seen as filtering the multiple vector identifiers sent by the first processor. The vector identifiers of the multiple PQ vectors fed back by the second processor to the first processor can be a subset of the multiple vector identifiers sent by the first processor.

[0168] Therefore, this embodiment of the application uses a second processor to calculate the distance between the query vector and the PQ vector, which can quickly locate multiple candidate data vectors similar to the query vector, greatly reducing the number of data vectors that actually need to be read and calculated in the future.

[0169] Furthermore, when the second processor is a GPU, this embodiment of the application accelerates the calculation of the distance between the query vector and the PQ vector using the GPU. This fully utilizes the parallel computing capabilities, high-speed cache, and distributed memory features of the GPU to quickly locate multiple candidate PQ vectors most similar to the query vector, significantly accelerating the vector retrieval process and improving retrieval efficiency. This is of great significance for processing large-scale vector sets and applications with high real-time requirements.

[0170] (3) Based on the vector identifiers of multiple candidate PQ vectors, read multiple candidate data vectors from the first storage medium.

[0171] In one possible implementation, the process of reading multiple candidate data vectors from the first storage medium can be as follows: generating at least one read request based on the vector identifiers of the multiple candidate PQ vectors; sending the at least one read request to the first storage medium; and receiving the multiple candidate data vectors sent by the first storage medium.

[0172] The read request is used to request the reading of at least one data vector located in the same storage area of ​​the first storage medium.

[0173] In other words, when the data vector to be read is located in the same storage area in the first storage medium, multiple requests for the same storage area can be merged to eliminate redundant I / O, thereby reducing the number and size of I / O requests to be initiated to the first storage medium.

[0174] Step 303: Based on multiple candidate data vectors, determine the vector retrieval result of the query vector.

[0175] The vector retrieval results may include all candidate data vectors or a subset of multiple candidate data vectors; this application embodiment does not impose any restrictions on this.

[0176] In one possible implementation, step 303 can be implemented by: calculating the distances between the query vector and multiple candidate data vectors, sorting the calculated distance values ​​in ascending order, and then determining the candidate data vectors corresponding to the Top-K distance values ​​in the sorting results as the vector retrieval results of the query vector.

[0177] In summary, this embodiment of the application determines the data vectors corresponding to multiple candidate PQ vectors by traversing the graph index and analyzing the distance between the query vector and the PQ vector corresponding to the data vector in the vector cluster indicated by the graph node. Thus, through two layers of filtering operations—traversing the graph index and analyzing the distance of the PQ vectors—the search range for vectors can be significantly narrowed. Therefore, only the data vectors corresponding to the candidate PQ vectors need to be read from the first storage medium to determine the final vector retrieval result. Since this application only needs to read a small number of data vectors from the first storage medium, rather than reading data vectors from the first storage medium in units of inverted lists, the size and number of I / O requests initiated to the first storage medium are greatly reduced, alleviating system throughput bottlenecks and thus improving overall query performance.

[0178] To facilitate understanding, the following will take a vector retrieval system consisting of a CPU as the first processor, a GPU as the second processor, and an SSD as the first storage medium as an example to provide supplementary explanations of the index structure in the vector retrieval scheme of this application embodiment, as well as the implementation process of CPU and GPU collaboratively processing query requests.

[0179] It should be noted that the following examples and supplementary explanations belong to the same inventive concept as the above method embodiments. The related steps can be substituted and combined with each other, and the specific meanings of some parameters can also be referred to the above method embodiments, which will not be repeated in the following examples.

[0180] Before executing the vector retrieval process, this application embodiment needs to construct a multi-level index structure, which can greatly reduce data transmission between devices and improve the utilization efficiency of computing units.

[0181] See Figure 4 Similar to the SPANN indexing method, Hierarchical Balanced Clustering (HBC) is used to cluster the original vector dataset into multiple vector clusters, ensuring that the vectors within each cluster are similar. Each vector cluster contains a centroid vector (i.e., the centroid information) calculated during clustering and multiple pairs consisting of <vector ID, high-dimensional vector>. Then, a graph-based index structure is constructed using the centroid vectors of each cluster, resulting in a graph index. Because each data vector may be placed into multiple vector clusters during HBC clustering, the final inverted list size is approximately 8 times larger than the original vector dataset on average. This means that even with PQ compression, the compressed inverted list is still significantly larger than the original vector dataset.

[0182] However, unlike SPANN, this embodiment does not retain the high-dimensional vectors in the original inverted list when determining the vector list corresponding to each graph node. Instead, it only saves the lightweight vector IDs. In this way, each vector list consists of a series of vector IDs, occupying very little storage space. Furthermore, the original data vectors are PQ quantized to obtain the PQ vector set corresponding to the vector dataset.

[0183] After completing the above construction operations, these data need to be distributed across different storage media according to their characteristics. Since the number of vector lists constructed in this embodiment is much smaller than the number of original data vectors, the storage space of the graph index structure built from the central vector is relatively small, and the vector list composed of vector IDs also occupies very little storage space. Therefore, the graph index and the vector list corresponding to each graph node can be maintained in the CPU's memory. Simultaneously, to eliminate the data transfer overhead when the GPU calculates the PQ distance, the lightweight PQ vectors are maintained in the GPU's video memory (i.e., HBM). Finally, the original vector dataset is stored in an SSD.

[0184] In this way, the CPU only needs to transmit a lightweight vector ID to the GPU. After receiving the vector ID to be calculated, the GPU can directly read the corresponding PQ vector locally. After the GPU calculates and sorts, the CPU only needs to read a small amount of data vectors from the SSD and reorder them to output the final vector retrieval result.

[0185] In some embodiments, after completing the above construction operations, the CPU needs to copy the PQ vector set to the GPU. The GPU initialization process will then be illustrated with an example. See [link to documentation]. Figure 5 The GPU initialization process may include the following steps (1)-(4).

[0186] (1) Initialize the asynchronous read mechanism.

[0187] By configuring std::vector<aio_context_t> m_iocp, the vector retrieval system sets up an asynchronous I / O context structure for each worker thread (used to manage and track asynchronous I / O requests), supporting non-blocking data loading. This setup allows the system to perform SSD I / O operations without blocking the main computation process, thereby optimizing overall read speed and program responsiveness.

[0188] (2) Load the PQ vector set and transfer it to the GPU.

[0189] The vector retrieval system first loads the PQ vector set (denoted as PQVectorDataSet) from the storage medium into the CPU memory. The PQVectorDataSet structure contains a std::shared_ptr <vectorset>The PQVectorSet is used to manage vector data on the CPU, and a void* d_PQVectorSet points to the vector data in GPU memory. Then, using the cudaMalloc and cudaMemcpy functions of the compute unified device architecture (CUDA), the vector retrieval system copies the PQ vector set from CPU memory to GPU memory to ensure that the GPU can efficiently access and process this data during the search process.

[0190] (3) Load the PostingList index (i.e., the graph index and the vector list corresponding to multiple graph nodes) and the PQ quantizer.

[0191] The vector list structure provided in this application includes several key components, such as std::vector. <int>MetaPostingList and std::vector <std::unique_ptr<int[]> `FullPostingList` stores the metadata and a list of vector IDs for each vector list. The vector retrieval system loads these indexes by parsing the storage format and constructs internal data structures to efficiently manage them, thus supporting fast vector retrieval. `std::shared_ptr` <sptag::common::iquantizer>The quantizer is used to store and manage quantizers, which consist of a series of trained codewords used to calculate the approximate distance between the query vector and the database vector. After loading the quantizer, the system enables the asymmetric distance (ADC) calculation process, which increases the accuracy of the calculation during queries.

[0192] (4) Allocate data structures stored in GPU memory.

[0193] Several key GPU memory pointers are defined in the GPU data structures, such as uint8_t*d_table, int*d_vectorIDs, float*d_dist, and float*h_dist. These structures are used to store the PQ distance table, vector list, distance values ​​calculated on the GPU, and their mirror images on the host side.

[0194] In this way, by pre-allocating this memory during the initialization phase, the vector retrieval system avoids dynamic memory allocation under high pressure, reduces the risk of memory fragmentation, and optimizes GPU performance.

[0195] After completing the offline construction of the above index structure, this application embodiment also provides a CPU-GPU collaborative query processing flow for query vectors.

[0196] See Figure 6 Based on the query vector carried in the query request, the CPU first traverses the graph index and determines the target graph nodes corresponding to the query vector, such as graph node 1, graph node 2, and graph node 3, according to the distance between the query vector and multiple graph nodes. Then, the CPU filters the vector list corresponding to the target graph node to determine multiple vector IDs. Since the same vector IDs may exist in different vector lists, the CPU needs to merge the vector IDs in the vector list corresponding to the target graph node and remove duplicate vector IDs to obtain multiple vector IDs, such as V0, V1, V2, V3, V4, and V5, and asynchronously transmit these multiple vector IDs to the GPU.

[0197] After receiving multiple vector IDs, the GPU first reads the PQ vector from the HBM based on the multiple vector IDs, and then calculates the distance between the PQ vector and the query vector. After the calculation is completed, the results are sorted, and the vector IDs that are closest to the query vector are returned to the CPU, for example, V0, V2, and V5 are returned to the CPU.

[0198] Because PQ technology sacrifices some precision while compressing vectors, to ensure high recall, the actual vector data needs to be read and sorted. After receiving multiple vector IDs from the GPU, the CPU uses a redundancy-aware I / O engine to check if any CPU requests to the SSD are for the same page. If so, the requests are merged to eliminate redundant I / O. Finally, distance calculations are performed on the read data vectors 2, 0, and 5, and the data vectors are reordered according to the results to obtain the final result. After the above process, the final output vector retrieval results for the query vectors are: data vector V0 and data vector V2.

[0199] In one possible implementation, see Figure 7 The process of CPU-GPU determining candidate data vectors may include the following steps (1)-(7).

[0200] (1) Traverse the graph index to obtain the vector list of the target graph nodes.

[0201] The vector retrieval system first traverses the graph index in CPU memory, searches for and queries the cluster centers with the closest vector distance, and then obtains the list of vectors corresponding to these graph nodes.

[0202] (2) Construct the ADC distance table.

[0203] Before the next search step begins, the vector retrieval system constructs a PQ distance table, such as an ADC distance table. The ADC distance table contains the distances between the query vector and all possible centroid vectors; this data is pre-calculated and stored to accelerate subsequent distance lookup operations.

[0204] (3) Use dynamic pruning strategy to filter graph nodes.

[0205] The dynamic pruning strategy is used to filter out centroids that cannot be the nearest neighbors during the query process. By setting a distance threshold based on a multiple of the nearest centroid distance, the vector retrieval system only retains centroids whose distance is less than or equal to this threshold as the final target graph nodes.

[0206] This part is typically executed on the CPU to quickly filter and reduce the computational burden on the GPU.

[0207] (4) Determine the vector list.

[0208] For each centroid selected through the pruning strategy, the vector retrieval system collects all its associated vector IDs. This step involves efficient access and management of memory, using instructions such as _mm_prefetch (used to load data from memory into the cache before the CPU actually needs it) to optimize access speed.

[0209] In addition, a deduplication mechanism can be added to ensure that the same vectors are not processed repeatedly, which can be achieved through hash tables or Bloom filters.

[0210] (5) Transfer the ADC distance table and vector list to the GPU.

[0211] To perform distance calculations on the GPU, both the ADC distance table and the vector list need to be transferred to the GPU. This is achieved using the cudaMemcpy function, which involves data transfer from main memory to GPU memory.

[0212] (6) GPU calculates vector distance.

[0213] On the GPU, kernel functions written in CUDA are used to calculate the distance between the query vector and the target vector in parallel. Each GPU thread processes one dimension of the vector, looks up the distance in the ADC distance table for each dimension, adds them together, and stores the result in GPU memory. This step fully utilizes the high parallel processing power of the GPU, significantly accelerating the distance calculation process.

[0214] (7) Transmit the distance value to the CPU to determine the candidate data vector.

[0215] The calculated distance values ​​need to be transferred from the GPU back to the CPU, which is also done using the cudaMemcpy function. The returned distance values ​​will be used for the next step of sorting the results. All calculated distance values ​​and their corresponding vector IDs are added to the search results set and sorted by distance value to determine candidate data vectors.

[0216] In one possible implementation, see Figure 8 The process of the CPU reading multiple candidate data vectors from the SSD and determining the vector retrieval results may include the following steps (1)-(4).

[0217] (1) Storage I / O request.

[0218] Based on the filtered results of the vector list, asynchronous read requests are generated. These requests target data vectors stored on disk (i.e., SSD) and need to be loaded precisely for subsequent accurate distance calculations. The vector retrieval system creates a list of "AsyncReadRequest" objects, each containing the specific offset and size of the data to be loaded (i.e., the data vector to be read). To ensure data alignment, the request offset is typically adjusted to a multiple of the system page, which helps optimize disk read performance.

[0219] (2) Initiate an I / O request.

[0220] This step utilizes Linux system calls such as "io_setup" and "io_submit" to manage asynchronous I / O operations. These calls are based on Linux's AIO (asynchronous I / O) functionality, allowing applications to perform disk I / O operations without blocking the main thread. Through "io_submit," the vector retrieval system can submit previously prepared "AsyncReadRequests" in batches to the I / O subsystem, where these requests will be queued and processed by the operating system.

[0221] (3) Callback processing after completion.

[0222] Each "AsyncReadRequest" is associated with a callback function that is triggered upon completion of the corresponding I / O request. The callback function's task is to perform an accurate distance calculation using the read data. This typically involves extracting vector data from a buffer and then calculating the distance between the target vector and the query vector using Euclidean distance (or another distance method). Once the calculation is complete, the result is updated in the query result set.

[0223] (4) Process and sort I / O results.

[0224] Once all asynchronous I / O requests have been processed, the system will sort the results according to the newly calculated distance values. Sorting typically occurs after all callback functions have finished executing, ensuring all results are updated with the latest calculations. Finally, the sorted result set will be returned to the user or application, completing the entire query process.

[0225] Furthermore, the embodiments of this application also compare the SPANN scheme and perform performance analysis and verification on the experimental dataset SIFT1B.

[0226] in, Figure 9 This paper illustrates a comparison of the number of pages that need to be read from SSD when processing a single query using the SIFT1B dataset. Compared to the SPANN scheme, the technical solution provided in this application embodiment can reduce the number of pages that need to be read from SSD by 90%-92%.

[0227] in addition, Figure 10 The comparison of queries per second (QPS) and latency when processing a single query using the SIFT1B dataset is shown. Compared to the SPANN scheme, the technical solution provided in this application improves throughput by 9-12X while keeping the query latency relatively unchanged.

[0228] It should be noted that the above Figures 4-8 This is merely an illustrative example of the technical solution of this application and does not constitute a limitation on the implementation process of the technical solution of this application.

[0229] Figure 11 This is a schematic diagram of a vector retrieval device provided in an embodiment of this application. This vector retrieval device can be implemented by software, hardware, or a combination of both, and can be part or all of the first processor included in the target node mentioned above. See also... Figure 11 The vector retrieval device includes: a graph node determination module 1101, a candidate vector determination module 1102, and a retrieval result output module 1103.

[0230] The graph node determination module 1101 is used to obtain the target graph node corresponding to the query vector from the graph index. The graph index includes multiple graph nodes, each of which is the centroid of a vector cluster in the vector dataset. The graph index is stored in the target node and is located outside the first storage medium. For detailed implementation process, please refer to the above. Figure 3 The content of step 301 in the illustrated embodiment will not be repeated here.

[0231] The candidate vector determination module 1102 is used to read the data vectors corresponding to multiple candidate product-quantized PQ vectors from the first storage medium based on the target graph nodes, thereby obtaining multiple candidate data vectors. These candidate PQ vectors are selected from a PQ vector set based on their distance from the query vector. The PQ vector set is determined based on the vector dataset. For detailed implementation details, please refer to the above. Figure 3 The content of step 302 in the illustrated embodiment will not be repeated here.

[0232] The retrieval result output module 1103 is used to determine the vector retrieval result of the query vector based on multiple candidate data vectors. For detailed implementation details, please refer to the above. Figure 3 The content of step 303 in the illustrated embodiment will not be repeated here.

[0233] In one possible implementation, the target node further includes a second storage medium that stores a graph index, and the input / output I / O performance of the second storage medium is superior to that of the first storage medium.

[0234] In one possible implementation, the second storage medium is the memory of the target node.

[0235] In one possible implementation, the target node also includes a third storage medium that stores the PQ vector set.

[0236] In one possible implementation, the target node further includes a second processor, and the third storage medium is the memory of the second processor; the candidate vector determination module 1102 includes:

[0237] The identifier sending unit is used to send multiple vector identifiers to the second processor. The multiple vector identifiers are the vector identifiers of the data vectors in the vector cluster to which the target graph node belongs.

[0238] The identifier receiving unit is used to receive the vector identifiers of multiple candidate PQ vectors sent by the second processor; wherein, the vector identifiers of the multiple candidate PQ vectors are obtained by the second processor from the PQ vectors corresponding to the multiple vector identifiers based on the distance between the PQ vectors corresponding to the multiple vector identifiers and the query vector.

[0239] The vector reading unit is used to read multiple candidate data vectors from the first storage medium based on the vector identifiers of multiple candidate PQ vectors.

[0240] In one possible implementation, the first storage medium is a solid-state drive (SSD).

[0241] In one possible implementation, the first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU), a neural network processor (NPU), a network processor (NP), or a field-programmable gate array (FPGA).

[0242] In one possible implementation, the target node further includes a second storage medium storing multiple vector lists corresponding to graph nodes, each vector list including vector identifiers of all data vectors in the vector cluster to which the corresponding graph node belongs; the vector retrieval device further includes:

[0243] The identifier reading module is used to read the list of target vectors corresponding to the target graph nodes from the second storage medium;

[0244] The identifier determination module is used to determine multiple vector identifiers based on the vector identifiers in the target vector list.

[0245] In one possible implementation, the identifier determination module is specifically used for:

[0246] The vector identifiers in the target vector list are deduplicated to obtain multiple vector identifiers.

[0247] In one possible implementation, the vector reading unit is specifically used for:

[0248] Based on the vector identifiers of multiple candidate PQ vectors, at least one read request is generated. The read request is used to request the reading of at least one data vector located in the same storage area in the first storage medium.

[0249] Send at least one read request to the first storage medium;

[0250] Receive multiple candidate data vectors sent by the first storage medium.

[0251] In summary, this embodiment of the application determines the data vectors corresponding to multiple candidate PQ vectors by traversing the graph index and analyzing the distance between the query vector and the PQ vector corresponding to the data vector in the vector cluster indicated by the graph node. Thus, through two layers of filtering operations—traversing the graph index and analyzing the distance of the PQ vectors—the search range for vectors can be significantly narrowed. Therefore, only the data vectors corresponding to the candidate PQ vectors need to be read from the first storage medium to determine the final vector retrieval result. Since this application only needs to read a small number of data vectors from the first storage medium, rather than reading data vectors from the first storage medium in units of inverted lists, the size and number of I / O requests initiated to the first storage medium are greatly reduced, alleviating system throughput bottlenecks and thus improving overall query performance.

[0252] It should be noted that the vector retrieval device provided in the above embodiments, when determining the vector retrieval result of the query vector from the vector dataset, is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the vector retrieval device and the vector retrieval method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0253] This application also provides a computer-readable storage medium storing instructions that, when executed on a processor, cause the processor to perform the steps of the vector retrieval method shown in the above embodiments.

[0254] This application also provides a computer program product containing instructions that, when executed on a processor, cause the processor to perform the method for determining the model output result as shown in the above embodiments. Alternatively, a computer program is provided that, when executed on a processor, causes the processor to perform the steps of the vector retrieval method shown in the above embodiments.

[0255] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0256] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0257] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0258] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.< / sptag::common::iquantizer> < / int> < / vectorset>

Claims

1. A vector retrieval method, characterized in that, The method applies to a first processor included in a target node, the target node further including a first storage medium storing a vector dataset, the vector dataset including multiple data vectors, the method comprising: The target graph node corresponding to the query vector is obtained from the graph index, which includes multiple graph nodes. Each graph node is the centroid of a vector cluster in the vector dataset. The graph index is stored in the target node and is located outside the first storage medium. Based on the target graph node, multiple candidate product quantization (PQ) vectors are read from the first storage medium to obtain multiple candidate data vectors. The multiple candidate PQ vectors are selected from the PQ vector set according to their distance from the query vector. The PQ vector set is determined based on the vector dataset. Based on the multiple candidate data vectors, the vector retrieval result of the query vector is determined.

2. The method as described in claim 1, characterized in that, The target node further includes a second storage medium that stores the graph index, and the input / output I / O performance of the second storage medium is better than that of the first storage medium.

3. The method as described in claim 2, characterized in that, The second storage medium is the memory of the target node.

4. The method according to any one of claims 1-3, characterized in that, The target node also includes a third storage medium, which stores the PQ vector set.

5. The method as described in claim 4, characterized in that, The target node further includes a second processor, and the third storage medium is the memory of the second processor; The step of reading data vectors corresponding to multiple candidate product-quantized (PQ) vectors from the first storage medium based on the target graph node includes: Send multiple vector identifiers to the second processor, wherein the multiple vector identifiers are vector identifiers of data vectors in the vector cluster to which the target graph node belongs; The second processor receives vector identifiers of the plurality of candidate PQ vectors; wherein the vector identifiers of the plurality of candidate PQ vectors are obtained by the second processor from the PQ vectors corresponding to the plurality of vector identifiers based on the distance between the PQ vectors corresponding to the plurality of vector identifiers and the query vector; Based on the vector identifiers of the plurality of candidate PQ vectors, the plurality of candidate data vectors are read from the first storage medium.

6. The method according to any one of claims 1-5, characterized in that, The first storage medium is a solid-state drive (SSD).

7. The method as described in claim 5, characterized in that, The first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU), a neural network processor (NPU), a network processor (NP), or a field-programmable gate array (FPGA).

8. The method as described in claim 5, characterized in that, The target node further includes a second storage medium, which stores a vector list corresponding to each of the plurality of graph nodes. The vector list includes vector identifiers of all data vectors in the vector cluster to which the corresponding graph node belongs. Before sending the multiple vector identifiers to the second processor, the method further includes: Read the list of target vectors corresponding to the target graph nodes from the second storage medium; The plurality of vector identifiers are determined based on the vector identifiers in the target vector list.

9. The method as described in claim 8, characterized in that, The step of determining the plurality of vector identifiers based on the vector identifiers in the target vector list includes: The vector identifiers in the target vector list are deduplicated to obtain the plurality of vector identifiers.

10. The method as described in claim 5, characterized in that, The step of reading the multiple candidate data vectors from the first storage medium based on the vector identifiers of the multiple candidate PQ vectors includes: Based on the vector identifiers of the plurality of candidate PQ vectors, at least one read request is generated, the read request being used to request the reading of at least one data vector located in the same storage area in the first storage medium; Send at least one read request to the first storage medium; Receive the plurality of candidate data vectors sent by the first storage medium.

11. A vector retrieval system, characterized in that, The system includes a first processor, a second processor, and a first storage medium, wherein the first storage medium stores a vector dataset, and the vector dataset includes multiple data vectors; The first processor is configured to obtain the target graph node corresponding to the query vector from the graph index and send multiple vector identifiers to the second processor. The graph index includes multiple graph nodes, each graph node being the centroid of a vector cluster in the vector data set. The graph index is stored outside the first storage medium, and the multiple vector identifiers are vector identifiers of data vectors in the vector cluster to which the target graph node belongs. The second processor is configured to filter out multiple candidate PQ vectors from the PQ vectors corresponding to the multiple vector identifiers based on the distance between the PQ vectors corresponding to the multiple vector identifiers and the query vector, and send the vector identifiers of the multiple candidate PQ vectors to the first processor. The first processor is further configured to read data vectors corresponding to the plurality of candidate PQ vectors from the first storage medium based on the vector identifiers of the plurality of candidate PQ vectors, to obtain a plurality of candidate data vectors, and to determine the vector retrieval result of the query vector based on the plurality of candidate data vectors.

12. The system as claimed in claim 11, characterized in that, The system further includes a second storage medium that stores the graph index, and the input / output I / O performance of the second storage medium is superior to that of the first storage medium.

13. The system as described in claim 12, characterized in that, The second storage medium is the memory of the host to which the first processor belongs.

14. The system according to any one of claims 11-13, characterized in that, The system also includes a third storage medium that stores the PQ vector set.

15. The system as described in claim 14, characterized in that, The third storage medium is the memory of the host to which the second processor belongs.

16. The system according to any one of claims 11-15, characterized in that, The first storage medium is located in the host to which the first processor belongs, and the first storage medium is a solid-state drive (SSD).

17. The system as claimed in claim 16, characterized in that, The first processor is a central processing unit (CPU), and the second processor is a graphics processing unit (GPU), a neural network processor (NPU), a network processor (NP), or a field-programmable gate array (FPGA).

18. The system according to any one of claims 11-17, characterized in that, The system further includes a second storage medium, which stores vector lists corresponding to the plurality of graph nodes respectively. The vector lists include vector identifiers of all data vectors in the vector cluster to which the corresponding graph node belongs. The first processor is further configured to: Read the list of target vectors corresponding to the target graph nodes from the second storage medium; The plurality of vector identifiers are determined based on the vector identifiers in the target vector list.

19. The system as claimed in claim 18, characterized in that, The first processor is further configured to: The vector identifiers in the target vector list are deduplicated to obtain the plurality of vector identifiers.

20. The system according to any one of claims 11-19, characterized in that, The first processor is further configured to: Based on the vector identifiers of the plurality of candidate PQ vectors, at least one read request is generated, the read request being used to request the reading of at least one data vector located in the same storage area in the first storage medium; Send at least one read request to the first storage medium; Receive the plurality of candidate data vectors sent by the first storage medium.

21. A computer device, characterized in that, The computer device is used to perform the method according to any one of claims 1-10.

22. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-10.

23. A computer program product, characterized in that, The computer program product stores computer instructions, which, when executed by a processor, implement the method described in any one of claims 1-10.

Images