A vector retrieval apparatus based on approximate nearest neighbor search

Abstract

The application provides a vector retrieval device based on approximate nearest neighbor search, comprising a first storage, a second storage, a third storage, a fourth storage, a large cluster calculation LCM module, a quantization lookup table calculation QPM module, and a final quantization similarity calculation and result calculation DCM module; the device can be connected with a memory such as HBM and DDR, instead of only being able to use BRAM on-chip storage of an FPGA to store all data, and can be applied to more complex scenes; the device adopts a parallel strategy of more efficient large cluster center calculation and quantization lookup table calculation, and a faster Top-k calculation module, and is faster and lower in power consumption.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a vector retrieval device based on approximate nearest neighbor search. BACKGROUND

[0002] Vector retrieval is the basis of many intelligent systems in today's Internet era, including recommendation systems, search engines, etc. At the same time, with the rapid development of large language models (LLM), retrieval augmented generation (RAG) is widely considered as one of the most effective technical routes to solve the problem of large model illusion, data obsolescence, etc., and can solve the problem of data acquisition and privacy protection when large models are applied to private knowledge bases of individuals, enterprises, etc. According to the query vector provided by the user, the retrieval system needs to respond quickly based on a large number of vectors in the database and provide real-time user feedback. Therefore, this puts forward very high requirements on the speed and accuracy of retrieval.

[0003] Traditional vector nearest neighbor retrieval needs to calculate the similarity between the query vector and all vectors in the database, and perform sorting to finally obtain a number of vectors most similar to the query vector, i.e. the retrieval result. Due to the need for massive similarity calculation and sorting calculation, the defects of retrieval speed, resource consumption, etc. in the actual application scene are often unacceptable. Therefore, approximate nearest neighbor search (ANNS) is widely used at present to improve retrieval efficiency. This kind of method pre-arranges all vectors in the database according to the coordinate rule by constructing a specific database index structure, thereby reducing the number or complexity of similarity calculation, sacrificing a small part of accuracy to obtain higher retrieval speed.

[0004] In terms of algorithms, approximate nearest neighbor search mainly includes methods based on quantization, graph, tree, etc. Among them, a class of algorithms based on inverted index and quantization, including IVF-PQ (Inverted File Product Quantification), OPQ (Optimized Product Quantification), etc., has the advantages of small calculation amount, fast speed, small memory occupation, etc., and has broad application prospects. The main process of this kind of method in the retrieval process is:

[0005] (1) Select k1 centers closest to the query vector from the nlist large clustering centers of the vector dataset;

[0006] (2) Divide the query vector into m blocks and calculate the distance of each small block to k *a similarity of the quantized clustering centers, to obtain a quantized lookup table;

[0007] (3) According to the clustering selected in (1), the vectors in the corresponding part of the data set are read, the corresponding values are selected from the lookup table constructed in (2) according to the ID values of each small block of the vector, and the sum is obtained, to obtain the calculation result, and the most similar k vectors are selected, which are the final search results.

[0008] However, general computing devices, such as CPUs and GPUs, often have the disadvantages of high power consumption, waste of computing resources, and slow speed when directly calculating for such vector retrieval problems. For end-side deployment of large model external vector databases and other application scenarios, there is an urgent need for efficient hardware solutions with lower power consumption and lower latency, such as more optimized vector retrieval through FPGA, ASIC and other devices. Among them, the existing FPGA or ASIC-based related work mainly includes two categories: quantization-based schemes and graph retrieval-based schemes. Representative works mainly include:

[0009] (1) In 2018, Jialiang Zhang et al. (Reference: J. Zhang, S. Khoram, and J. Li, “Efficient large-scale approximate nearest neighbor search on opencl fpga,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018) proposed an OpenCL-based FPGA implementation scheme, but the optimization in the fine hardware structure was insufficient and the power consumption problem during retrieval was not solved;

[0010] (2) In 2019, Ameer MS Abdelhadi et al. (Reference: A.M. Abdelhadi, C.-S. Bouganis, and G.A. Constantinides, “Accelerated approximate nearest neighbors search through hierarchical product quantization,” in 2019 International Conference on Field-Programmable Technology (ICFPT). IEEE, 2019, pp. 90–98) proposed an FPGA implementation scheme based on hierarchical quantization, which uses on-chip BRAM to store lookup table data. Although it achieves speed improvement, it can only handle very small index structures and cannot meet the needs of most application scenarios.

[0011] (3) In 2022, Yejin Lee et al. (Reference: Y.Lee, H.Choi, S.Min, H.Lee, S.Beak, D.Jeong, JWLee, and TJHam, “Anna: Specialized architecture for approximate nearest neighbor search,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2022, pp.169–183) proposed a hardware solution for inverted index quantization search based on ASIC called ANNA. Its main drawback is still insufficient hardware detail optimization, and its advantages in power consumption and speed are not obvious.

[0012] (4) Graph index-based schemes. For example, in 2023, Shulin Zeng et al. (Reference: S. Zeng, Z. Zhu, J. Liu, H. Zhang, G. Dai, Z. Zhou, S. Li, X. Ning, Y. Xie, H. Yang, and Y. Wang, “Df-gas: a distributed fpga-as-a-service architecture towards billion-scale graph-based approximate nearest neighbor search.” Association for Computing Machinery, 2023, pp. 283–296) proposed a graph retrieval-based FPGA implementation scheme called DF-GAS. However, this type of method has significant shortcomings in terms of hardware resource overhead and power consumption. For example, when deploying large databases, multiple FPGA devices need to process them simultaneously.

[0013] In summary, existing FPGA / ASIC-based hardware solutions generally suffer from the following problems: (1) high retrieval latency; (2) high hardware resource overhead and high power consumption; (3) limited application scenarios and inability to achieve effective edge vector retrieval; (4) often adopting design methods and platforms such as OpenCL, lacking fine hardware structure optimization. Summary of the Invention

[0014] Purpose of the Invention: Vector retrieval schemes based on inverted indexes and quantization offer significant advantages over traditional direct retrieval in terms of computational complexity and memory usage, while maintaining sufficient accuracy for most application scenarios, making them one of the mainstream vector retrieval schemes currently available. However, this method suffers from drawbacks such as low speed and high power consumption on general-purpose computing devices like CPUs and GPUs, and research on hardware acceleration for this scheme using FPGAs and ASICs remains limited. This invention addresses these issues by proposing a vector retrieval device based on approximate nearest neighbor search, which can reduce the power consumption of a single vector retrieval by more than 80% while maintaining accuracy and significantly reducing retrieval latency. The hardware device proposed in this invention represents the lowest power consumption solution for a single retrieval to date.

[0015] Specifically, this invention provides a vector retrieval device based on approximate nearest neighbor search, comprising a first storage, a second storage, a third storage, a fourth storage, a large clustering calculation LCM module, a quantized lookup table calculation QPM module, and a final quantized similarity calculation and result calculation DCM module;

[0016] The first, second, third, and fourth storage units are hardware memories used to store pre-prepared input data or intermediate results calculated by other modules.

[0017] The Large Clustering Calculation (LCM) module takes the query vector and the coordinates of nlist large cluster centers in the first storage (the coordinates of the large cluster centers are obtained in the pre-construction stage of the dataset) as input, outputs k1 selected large clusters, finds the k1 large cluster centers that are most similar to the query vector and saves them to the second storage, thereby initially defining the distribution range of the candidate target vector;

[0018] The Quantization Lookup Table (QPM) module inputs the query vector and the coordinates of the sub-vector cluster centers in the first storage (the coordinates of the sub-vector cluster centers are obtained during the pre-construction phase of the dataset), calculates the similarity between the sub-vectors to obtain a complete Quantization Lookup Table, and saves it to the third storage.

[0019] The final quantization similarity calculation and result calculation DCM module reads the corresponding part of the quantized vector dataset in the fourth storage (the part corresponding to the quantized vector dataset is obtained in the dataset pre-construction stage) based on the k1 largest clusters most similar to the query vector in the second storage, and calculates the final result based on the quantization lookup table in the third storage to obtain the k2 vectors in the dataset most similar to the query vector.

[0020] The large clustering calculation LCM module includes a D-dimensional vector similarity calculation module and a Top-k calculation module A;

[0021] The D-dimensional vector similarity calculation module is used to calculate vector similarity;

[0022] The Top-k calculation module A is used to find the k1 largest cluster centers that are most similar to the query vector.

[0023] The final quantized similarity calculation and result calculation DCM module includes a quantized vector calculation module and a Top-k calculation module B;

[0024] The quantized vector calculation module is used to calculate the final quantized similarity between the vectors in the quantized vector dataset and the query vector based on the k1 largest clusters most similar to the query vector in the second storage, the quantized lookup table in the third storage, and the quantized vector dataset in the fourth storage.

[0025] The Top-k calculation module B is used to obtain the k2 vectors in the dataset that are most similar to the query vector, based on the output of the quantized vector calculation module.

[0026] The Quantization Lookup Table (QPM) calculation module includes a D / m-dimensional vector similarity calculation module and a register.

[0027] The D / m-dimensional vector similarity calculation module is used to calculate the similarity between D / m-dimensional sub-vectors;

[0028] The register is used to temporarily store the output results of the D / m-dimensional vector similarity calculation module.

[0029] The device of the present invention also includes a storage control module;

[0030] The storage control module is used to read quantized vector data from the fourth storage based on the data in the second storage, thereby controlling the reading of data from the third storage.

[0031] In the Large Clustering Calculation (LCM) module, the input query vector and the coordinates of the large cluster center are first processed by the D-dimensional vector similarity calculation module, and then by the Top-k calculation module A to obtain k1 results.

[0032] In the large clustering calculation LCM module, the number of clock cycles required to calculate similarity and Top-k is N1 and N2, respectively.

[0033] The Top-k calculation module A and Top-k calculation module B adopt either a Top-k I structure or a Top-k II structure;

[0034] Both the Top-k I and Top-k II structures contain real-time updated lists, updated once per calculation cycle, used to store the k1 or k results with the highest similarity at the moment; the initial value of the list before the calculation begins is the original list, which is set according to the actual calculation requirements.

[0035] The Top-k I structure processes one input value per clock cycle. The input value is compared with the original k1 values ​​in the Top-k I structure. Each value in the list corresponds to a comparator, for a total of k1 comparators.

[0036] The comparator is used to compare the input value with the value at the corresponding position in the list. If the input value is greater than the value at the corresponding position in the list, the comparator outputs 1; otherwise, it outputs 0.

[0037] After the comparator is an adder that sums the outputs of k1 comparators to get the score of the input value, which is the rank of the new input value in the new list. Then, based on the list before the update, the new input, and the score, it goes through a set of selection modules, with each position in the list corresponding to a selection module, for a total of k1 selection modules.

[0038] The selection module is used to select the correct new value from the list before the update and the new input, and replace the original value at the corresponding position in the list to obtain the updated list. For the first position, i.e. the minimum value Top-0, the possible new value is the original Top-0 or the input value. For the i-th position, i>0, the possible new values ​​are the original Top-i, Top-(i-1), or the input value. The multiplexer in the selection module determines the output value based on the score of the input value. The input with a score of 'score' is inserted at the new position Top-score, and the remaining positions are arranged sequentially with the values ​​in the list before the update.

[0039] The Top-k II structure adopts a method of simultaneously processing multiple inputs. It uses a pipelined structure to process p3 inputs per cycle, specifically employing a two-stage design:

[0040] The first stage processes the p3 input values, obtains the pairwise relationship between the p3 values, compares them, and sums them to get the score of each input. Then, the output of the first stage is obtained through a set of multiplexers, which is the p3 inputs arranged in order.

[0041] The second stage updates the Top-k list based on the input and the original Top-k list. The output of the first stage is used as the input for the second stage. The two sets of elements to be compared are: p4 input values ​​and k values ​​from the original Top-k list, where p4 = min{p3,k}. The p4 input values ​​and the k values ​​from the original Top-k list are already sorted sequences. The scores of the p4 input values ​​and the k values ​​from the original Top-k list within their respective lists are obtained by directly reading their positions. The comparison results are then added to the scores of the p4 input values ​​and the k values ​​from the original Top-k list. The positions of k values ​​in the original list are used to obtain the final scores of the p4 input values ​​and the final scores of the k values ​​in the original Top-k list. In the second stage selection module, a total of 2p4+1 elements are input and classified according to the size relationship between i and p3. When i≤p4, the p4 inputs and the original Top-0 to Top-i are used as the inputs of the selector module. In order to maintain the list size, a total of (p4-i+1) Top-i elements are added. When i>p4, the Top list before the update only retains Top-(i-p4) to Top-i, and the total number of input elements is 2p4+1.

[0042] The first storage 1, the second storage 2, the third storage 3, and the fourth storage 4 may be combined or stored separately to store different data.

[0043] In the QPM (Quantized Lookup Table) module, the similarity between D / m-dimensional sub-vectors is calculated. The input query vector is divided into m sub-vectors, each corresponding to k in the first storage. *For each quantized sub-cluster center, calculate the sub-vector and its corresponding k. * The similarity of the sub-cluster centers is quantized; the number of clock cycles required for the QPM module to calculate the quantized lookup table is N3; in order to maximize the parallel computing efficiency of the LCM module and the QPM module, their parallelism ratio should be close to (N1+N2) / N3.

[0044] This device includes a highly parallel Top-k computation module, employs parallelized clustering and quantized lookup table computation modules, and is suitable for different data precision types. It is applicable to high-dimensional vector datasets of various sizes.

[0045] Compared with existing FPGA and ASIC-based solutions, it offers the following advantages:

[0046] (1) This device does not require the use of CPU or other devices for cross-platform computing and can fully realize vector retrieval;

[0047] (2) This device can be connected to memory such as HBM and DDR, instead of only using the on-chip BRAM of the FPGA to store all data, and can be applied to more complex scenarios;

[0048] (3) This device adopts a more efficient parallel strategy for large cluster center calculation and quantized lookup table calculation, as well as a faster Top-k calculation module, which is faster and consumes less power.

[0049] (4) This device is designed for quantization-based solutions rather than graph retrieval solutions. Its advantages are that it is more lightweight and has higher efficiency in utilizing computing resources. Attached Figure Description

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0051] Figure 1 This is an overall architectural diagram of the device of the present invention.

[0052] Figure 2 This is a schematic diagram of the Top-k I module architecture.

[0053] Figure 3 This is a schematic diagram of the Top-k II module architecture. Detailed Implementation

[0054] In a specific embodiment of the present invention, a vector retrieval device based on approximate nearest neighbor search is provided, such as... Figure 1 As shown, the structure of this device includes a large clustering calculation LCM module, a quantization lookup table calculation QPM module, and a final quantization similarity calculation and result calculation DCM module.

[0055] The main functions of the three modules include: (1) The LCM module takes the query vector and the coordinates of nlist large cluster centers in the first storage 1 as input, and outputs k1 selected large clusters. Its function is to find the k1 large cluster centers that are most similar to the query vector and save them to the second storage 2, thereby initially defining the distribution range of the candidate target vector; (2) The QPM module takes the query vector and the coordinates of the sub-vector cluster centers in the first storage 1 as input, and calculates the similarity between the sub-vectors to obtain a complete quantized lookup table and saves it to the third storage 3; (3) The DCM module reads the corresponding part of the quantized vector dataset in the fourth storage 4 according to the large clusters in the second storage 2, and calculates the final result according to the lookup table in the third storage 3, that is, the k2 vectors in the dataset that are most similar to the query vector. Among them, LCM and DCM each contain a Top-k calculation module, which selects the maximum value according to the results calculated in the LCM module and DCM module respectively.

[0056] (I) Efficient Parallel Design of LCM and QPM Modules

[0057] To improve the parallel efficiency of the LCM and QPM modules, this invention adopts a fine-grained resource allocation, enabling the LCM and QPM modules to complete calculations in similar timeframes, thus allowing them to enter the DCM module's calculations almost simultaneously and reducing waste caused by the time difference between the completion of calculations by the LCM and QPM modules.

[0058] In the LCM module, the input query vector and the coordinates of the large cluster center (located in the first storage 1) first pass through a D-dimensional vector similarity calculation module. This part can be a structure that implements one or more arbitrary types of vector similarity calculations, such as Euclidean distance, cosine distance, or inner product distance. Then, it passes through a Top-k calculation structure to obtain k1 results. The number of clock cycles required for similarity calculation and Top-k calculation in the LCM module are N1 and N2, respectively. In the QPM module, the similarity between D / m-dimensional sub-vectors is calculated. The input query vector is divided into m sub-vectors, each corresponding to the k values ​​of its respective block. * Each sub-cluster center is quantized, and its similarity is calculated separately. This part functions similarly to the D-dimensional distance calculation module in the LCM module. The QPM module consumes N³ clock cycles. To maximize the parallel computation efficiency of the LCM and QPM modules, after appropriate parallelism design and resource allocation, their parallelism ratio should be close to (N1+N2) / N³.

[0059] The following example illustrates the use of Euclidean distance and Top-k I:

[0060] The LCM module first calculates the distance in each dimension, then uses a multi-level tree-structured accumulation module to obtain the final distance between a pair of D-dimensional vectors. It contains a total of D-1 adders and D multipliers. To optimize the clock frequency of the distance calculation section, a pipeline is inserted between each adder in the accumulation module. Therefore, the distance calculation in this module requires a total of N1 = log2D clock cycles. Since the QPM module needs to execute m·k... * For vector distance calculation, to save resources, the distance calculation and accumulation modules are reused, therefore k is required. * The calculation of the quantization lookup table is completed in / p2 cycles. Therefore, the calculation in the corresponding Top-k module adopts a serial list update of Top-k I: that is, each cycle updates the new list after obtaining an input value, and after N2 = nlist cycles, the final k1 results are obtained. Therefore, the calculation of the LCM module consumes a total of log2D + nlist clock cycles.

[0061] In the QPM module, each pair of vectors requires D multipliers, and each accumulation module requires D / m-1 adders, totaling Dm adders for m computation units. To conserve computational resources, a pipelined structure is adopted for module reuse. Each cycle sequentially inputs a set of quantized cluster center coordinates for calculation, totaling k... * The total time required to complete the QPM module for the group is N³ = k * +log2D / m clock cycles.

[0062] Since log2D and log2D / m are relative to k * Since nlist and nlist are often small values, it can be approximated that the ratio of the time required for the LCM module and the QPM module to complete all calculations is nlist / k. * The parallelism ratio of the corresponding LCM module and QPM module should be nlist / k*. Let k * For example, with 512, nlist=256, k1=10, and p1=1, the parallelism p2 of the QPM module should be 2. In this case, the LCM module requires a total of 2D+15 adders and D multipliers, while the QPM module requires a total of 4D-2m adders and 2D multipliers.

[0063] (II) Top-k Module Design

[0064] For the two Top-k modules in LCM and DCM, two different design architectures can be selectively adopted based on a comprehensive consideration of hardware resources and computing speed in practical applications.

[0065] 1. Top-k I module

[0066] The Top-k I module processes one input value per clock cycle, such as Figure 2 As shown. Top represents the list before the update, and New Top represents the list after the update. The input value is compared with the original k1 values ​​in Top-k I, corresponding to k1 comparators (...). Figure 2 In the first position (C), if the input value is greater than the value at that position in the list, the comparator outputs 1; otherwise, it outputs 0. Following the comparator is an adder that sums the outputs of all comparators to obtain the score for the input value, which is the rank of the new input value in the new list. Then, based on the original list, the new input, and the score, a set of selection modules is used, with one module corresponding to each position in the list. For the first position (i.e., the minimum value Top-0), the score is calculated as follows: Figure 2 In the case of Case i = 0, the possible new values ​​are the original Top-0 or the input value. For the i-th position, i > 0 (corresponding to...). Figure 2 In the case of Case i>0), the possible new values ​​are the original Top-i, Top-(i-1), or the input value. The multiplexer in the selection module determines the output value based on the score of the input value. The input with a score of 'score' is inserted into the new Top-score position, and the remaining positions are filled by the values ​​in the original Top list.

[0067] 2. Top-k II module

[0068] In practical applications, ANNS computations may require processing a large number of vectors (often exceeding millions). Therefore, processing only one input per cycle would result in significant latency for this module. To address this issue, this device proposes a Top-k II module that processes multiple inputs simultaneously. Through a pipelined architecture, it can process p3 inputs per cycle. Compared to the scheme that processes one input per cycle, this approach can increase the speed by up to p3 times.

[0069] The Top-k II module is capable of handling high-parallel inputs on p3, so a two-stage design is adopted to optimize the clock frequency.

[0070] Stage 1 processes the p3 input values, requiring the determination of pairwise relationships between these values. After comparison, the scores for each input are summed, and then the output of Stage 1, i.e., the p3 arranged inputs, is obtained through a set of multiplexers. This part should be designed to determine whether to add one or more pipeline stages based on the required clock frequency and the size of the input list.

[0071] Phase 2 is used to update the new Top-k list based on the input and the original Top-k list. In this phase, the output of Phase 1 is used as the input. Since the two sets of elements to be compared are: (1) p4 input values ​​(p4 = min{p3,k}); (2) k values ​​in the original Top-k list, both sets of data are sorted sequences. Therefore, the score within the list can be obtained by directly reading the position. The result of the comparison plus its position in the original list is the final score. In the selector module of this phase, the total number of input elements is 2p4+1, which are classified according to the size relationship between i and p3. When i≤p4, the p4 input elements and the original Top-0 to Top-i are used as the input of the selector module. In order to maintain the list size, the Top-i is filled with a total of (p4-i+1) elements. When i > p4, since the elements Top-0 to Top-(i-p4-1) cannot appear in the i-th position of the new list, the original Top list in the input only retains Top-(i-p4) to Top-i, and the total number of input elements is 2p4+1.

[0072] like Figure 3 The diagram shows an embodiment of the Top-k II structure: in stage 1, a comparator is retained on one side of the diagonal. Figure 3 The value of comparison is obtained by logically inverting the value at the symmetrical position on the other side (C). Figure 3 In stage 2, the input list and the original list are compared one by one, requiring a total of p3(p3-1) / 2 comparators. Figure 3 In Phase 2, Top represents the list before the update, and New Top represents the list after the update.

[0073] Figure 1 In practice, the first storage (1), second storage (2), third storage (3), and fourth storage (4) can be combined or separated to store different data, and can replace [other storage methods]. Figure 1 The location and arrangement of memory in the architecture diagram.

[0074] The Top-k computation in the LCM module can adopt either the Top-k I architecture or the Top-k II architecture, depending on the parallelism of the module, and they are interchangeable.

[0075] In one embodiment of this invention, the hardware architecture of the device is written in Verilog, using Xilinx Alveo U280, and simulated and implemented using Vivado 2022.2. SIFT 1M, the most commonly used vector approximate nearest neighbor search algorithm, is used for testing, containing one million 128-dimensional vectors. The IVF-PQ algorithm is used to construct the index. Euclidean distance is used to measure vector similarity. The LCM module uses a Top-k I module, and the DCM module uses a Top-k II module. In this embodiment, the parallelism of the DCM module is p3=44, and the data precision is Int16 for comparison and verification. Under the condition of Recall@10=94%, k1 is set to 20 in the LCM module, that is, 20 clusters need to be output, and a total of about 19,000 vectors need to be processed in the DCM module. Setting k... * =1024, m=32.

[0076] As a control, the results of CPU (Intel Core i9-13900K) and GPU (NVIDIA RTX 4090) were tested under the same algorithm, parameters, recall, and dataset conditions. The results of ANNA (Reference: Y. Lee, H. Choi, S. Min, H. Lee, S. Beak, D. Jeong, JW Lee, and TJ Ham, “Anna: Specialized architecture for approximate nearest neighbor search,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2022, pp. 169–183) under the same conditions were compared with the effects of the present invention.

[0077] Table 1

[0078]

[0079] As can be seen from the table above, the device of the present invention can adjust the parallelism and data precision according to the needs, and optimize resource allocation and retrieval speed. Compared with other implementation schemes, the device can achieve the same retrieval recall rate (94%): (1) Compared with the current advanced CPU (Intel Core i9-13900K), the latency is reduced by more than 99.2% and the power consumption of a single retrieval is reduced by more than 99.9%; (2) Compared with the current advanced GPU (NVIDIA RTX 4090), the latency is reduced by more than 95.8% and the power consumption of a single retrieval is reduced by more than 99.2%; (3) Compared with the latest hardware design schemes based on quantization vector retrieval (FPGA, ASIC), the latency is reduced by more than 80% and the power consumption of a single retrieval is reduced by more than 87%.

[0080] This invention provides a vector retrieval device based on approximate nearest neighbor search. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A vector retrieval device based on approximate nearest neighbor search, characterized in that, It includes a first storage, a second storage, a third storage, a fourth storage, a large clustering calculation LCM module, a quantized lookup table calculation QPM module, and a final quantized similarity calculation and result calculation DCM module; The first, second, third, and fourth storage are hardware memories used to store pre-prepared input data or intermediate calculation results; The Large Clustering Calculation (LCM) module takes the query vector and the coordinates of nlist large cluster centers in the first storage as input, outputs k1 selected large clusters, finds the k1 large cluster centers most similar to the query vector and saves them to the second storage, thereby initially defining the distribution range of the candidate target vector; The quantization lookup table calculation module inputs the query vector and the coordinates of the sub-vector cluster centers in the first storage, calculates the similarity between the sub-vectors to obtain the complete quantization lookup table, and saves it to the third storage; The final quantization similarity calculation and result calculation DCM module reads the part corresponding to the quantized vector dataset in the fourth storage based on the k1 largest clusters most similar to the query vector in the second storage, and calculates the final result based on the quantization lookup table in the third storage to obtain the k2 vectors in the dataset most similar to the query vector. The large clustering calculation LCM module includes a D-dimensional vector similarity calculation module and a Top-k calculation module A; The D-dimensional vector similarity calculation module is used to calculate vector similarity; The Top-k calculation module A is used to find the k1 largest cluster centers that are most similar to the query vector; The Quantization Lookup Table (QPM) calculation module includes a D / m-dimensional vector similarity calculation module and a register. The D / m-dimensional vector similarity calculation module is used to calculate the similarity between D / m-dimensional sub-vectors; The register is used to temporarily store the output results of the D / m-dimensional vector similarity calculation module.

2. The apparatus according to claim 1, characterized in that, The final quantized similarity calculation and result calculation DCM module includes a quantized vector calculation module and a Top-k calculation module B; The quantized vector calculation module is used to calculate the final quantized similarity between the vectors in the quantized vector dataset and the query vector based on the k1 largest clusters most similar to the query vector in the second storage, the quantized lookup table in the third storage, and the quantized vector dataset in the fourth storage. The Top-k calculation module B is used to obtain the k2 vectors in the dataset that are most similar to the query vector, based on the output of the quantized vector calculation module.

3. The apparatus according to claim 2, characterized in that, It also includes a storage control module; The storage control module is used to read quantized vector data from the fourth storage based on the data in the second storage, thereby controlling the reading of data from the third storage.

4. The apparatus according to claim 3, characterized in that, In the Large Clustering Calculation (LCM) module, the input query vector and the coordinates of the large cluster center are first processed by the D-dimensional vector similarity calculation module, and then by the Top-k calculation module A to obtain k1 results. In the large clustering calculation LCM module, the number of clock cycles required to calculate similarity and Top-k is N1 and N2, respectively.

5. The apparatus according to claim 4, characterized in that, The Top-k calculation module A and Top-k calculation module B adopt either a Top-k I structure or a Top-k II structure; Both the Top-k I and Top-k II structures contain real-time updated lists, updated once per calculation cycle, used to store the k1 or k results with the highest similarity at the moment; the initial value of the list before the calculation begins is the original list, which is set according to the actual calculation requirements.

6. The apparatus according to claim 5, characterized in that, The Top-k I structure processes one input value per clock cycle. The input value is compared with the original k1 values ​​in the Top-k I structure. Each value in the list corresponds to a comparator, for a total of k1 comparators. The comparator is used to compare the input value with the value at the corresponding position in the list. If the input value is greater than the value at the corresponding position in the list, the comparator outputs 1; otherwise, it outputs 0. After the comparator is an adder that sums the outputs of k1 comparators to get the score of the input value, which is the rank of the new input value in the new list. Then, based on the list before the update, the new input, and the score, it goes through a set of selection modules, with each position in the list corresponding to a selection module, for a total of k1 selection modules. The selection module is used to select the correct new value from the list before the update and the new input, and replace the original value at the corresponding position in the list to obtain the updated list. For the first position, i.e. the minimum value Top-0, the possible new value is the original Top-0 or the input value. For the i-th position, i>0, the possible new values ​​are the original Top-i, Top-(i-1), or the input value. The multiplexer in the selection module determines the output value based on the score of the input value. The input with a score of 'score' is inserted at the new position Top-score, and the remaining positions are arranged sequentially with the values ​​in the list before the update.

7. The apparatus according to claim 6, characterized in that, The Top-k II structure adopts a method of simultaneously processing multiple inputs. It uses a pipelined structure to process p3 inputs per cycle, specifically employing a two-stage design: The first stage processes the p3 input values, obtains the pairwise relationship between the p3 values, compares them, and sums them to get the score of each input. Then, the output of the first stage is obtained through a set of multiplexers, which is the p3 inputs arranged in order. The second stage is used to update the new Top-k list based on the input and the original Top-k list. The output of the first stage is used as the input for the second stage. The two sets of elements to be compared are: p4 input values ​​and k values ​​in the original Top-k list, where p4 = min{p3,k}. The p4 input values ​​and the k values ​​in the original Top-k list are sorted sequences. The scores of the p4 input values ​​and the k values ​​in the original Top-k list within their respective lists are obtained by directly reading their positions. The results of the comparison are added to the positions of the p4 input values ​​and the k values ​​in the original Top-k list in the original list to obtain the final scores of the p4 input values ​​and the k values ​​in the original Top-k list. In the second stage selection module, a total of 2p4+1 elements are input, and they are classified according to the size relationship between i and p3. When i≤p4, p4 inputs and the original Top-0 to Top-i are used as inputs to the selector module. In order to maintain the size of the list, a total of (p4-i+1) Top-i elements are filled. When i>p4, the Top list before the update only retains Top-(i-p4) to Top-i, and the total number of input elements is 2p4+1.

8. The apparatus according to claim 7, characterized in that, The first storage, second storage, third storage, and fourth storage may be combined or stored separately to store different data.

Images