Vector retrieval device based on approximate nearest neighbor search

Abstract

The invention provides a vector retrieval device based on approximate nearest neighbor search. The vector retrieval device comprises a first storage, a second storage, a third storage, a fourth storage, a large clustering calculation LCM module, a quantitative lookup table calculation QPM module and a final quantitative similarity calculation and result calculation DCM module. The device can be connected with memories such as an HBM and a DDR instead of only adopting a BRAM on-chip memory of an FPGA to store all data, and can be applied to more and more complex scenes; according to the device, a more efficient parallel strategy of large clustering center calculation and quantitative lookup table calculation and a higher-speed Top-k calculation module are adopted, so that the speed is higher, and the power consumption is lower.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a vector retrieval device based on approximate nearest neighbor search. Background Art

[0002] Vector retrieval is the basis of many intelligent systems in today's Internet era, including recommendation systems, search engines, etc. At the same time, with the rapid development of large language models (LLM), retrieval augmented generation (RAG) is widely regarded as one of the most effective technical routes to solve problems such as large model hallucination and data obsolescence, and can solve the data acquisition and privacy protection problems when large models are applied to private knowledge bases such as individuals and enterprises. According to the query vector provided by the user, the retrieval system needs to respond quickly based on a large number of vectors in the database and provide real-time user feedback. Therefore, this places high demands on the speed and accuracy of retrieval.

[0003] Traditional vector nearest neighbor search requires calculating the similarity between the query vector and all vectors in the database, and sorting them, and finally obtaining several vectors that are most similar to the query vector, i.e., the search results. Since this method requires massive similarity calculations and sorting calculations, its defects in search speed and resource consumption are often unacceptable in actual application scenarios. Therefore, Approximate Nearest Neighbor Search (ANNS) is widely used to improve search efficiency. This type of method constructs a specific database index structure and arranges all vectors in the database according to the coordinate law in advance, thereby reducing the number or complexity of similarity calculations and sacrificing a small amount of accuracy in exchange for higher search speed.

[0004] In terms of algorithms, approximate nearest neighbor search mainly includes methods based on quantization, graph, and tree. Among them, algorithms based on inverted index and quantization, including IVF-PQ (Inverted File Product Quantification) and OPQ (Optimized Product Quantification), have the advantages of small computational complexity, fast speed, and small memory usage, and have broad application prospects. The main process of this type of method in the retrieval process is:

[0005] (1) From the nlist large cluster centers of the vector data set, select k1 centers that are closest to the query vector;

[0006] (2) Divide the query vector into m blocks and calculate the k *The similarity of the quantitative cluster centers is obtained to obtain a quantitative lookup table;

[0007] (3) Based on the cluster selected in (1), read the vectors in the corresponding part of the data set, select the corresponding values ​​from the lookup table constructed in (2) according to the ID values ​​of each small block of the vector and sum them up to obtain the calculation result, and select the most similar k vectors among them, which are the final retrieval results.

[0008] However, general computing devices, such as CPUs and GPUs, often have disadvantages such as high power consumption, waste of computing resources, and slow speed when performing direct calculations for such vector retrieval problems. For application scenarios such as the end-side deployment of large model external vector databases, there is an urgent need for efficient hardware solutions with lower power consumption and lower latency, such as more optimized vector retrieval through devices such as FPGAs and ASICs. Among them, the existing FPGA- or ASIC-based related work mainly includes two categories: quantization-based solutions and graph retrieval-based solutions. Representative works mainly include:

[0009] (1) In 2018, Jialiang Zhang et al. (reference: J. Zhang, S. Khoram, and J. Li, “Efficient large-scale approximate nearest neighbor search on opencl fpga,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018) proposed an FPGA implementation based on OpenCL, but the fine hardware structure was not optimized enough and the power consumption problem during retrieval was not solved;

[0010] (2) In 2019, Ameer MS Abdelhadi et al. (reference: AM Abdelhadi, C.-S. Bouganis, and GA Constantinides, “Accelerated approximate nearest neighbors search through hierarchical product quantization,” in 2019 International Conference on Field-Programmable Technology (ICFPT). IEEE, 2019, pp. 90–98) proposed an FPGA implementation based on hierarchical structure quantization, which completely uses on-chip BRAM to store lookup table data. Although the speed is improved, it can only process very small index structures and cannot meet most application scenarios;

[0011] (3) In 2022, Yejin Lee et al. (reference: Y. Lee, H. Choi, S. Min, H. Lee, S. Beak, D. Jeong, J. W. Lee, and T. J. Ham, “Anna: Specialized architecture for approximate nearest neighbor search,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2022, pp. 169–183) proposed a hardware solution for inverted quantization retrieval based on ASIC implementation called ANNA. Its main disadvantage is still the insufficient optimization of hardware details, and the advantages of power consumption and speed are not obvious;

[0012] (4) Graph index-based solutions. For example, in 2023, Shulin Zeng et al. (reference: S. Zeng, Z. Zhu, J. Liu, H. Zhang, G. Dai, Z. Zhou, S. Li, X. Ning, Y. Xie, H. Yang, and Y. Wang, “Df-gas: a distributed FPGA-as-a-service architecture towards billion-scale graph-based approximate nearest neighbor search.” Association for Computing Machinery, 2023, pp. 283–296) proposed a graph retrieval-based FPGA implementation solution called DF-GAS. However, this type of method has significant shortcomings in terms of hardware resource overhead and power consumption. For example, when deploying a large database, multiple FPGA devices are required to process simultaneously.

[0013] In summary, existing FPGA / ASIC-based hardware solutions generally have the following problems: (1) high retrieval latency; (2) large hardware resource overhead and high power consumption; (3) limited application scenarios and inability to achieve effective end-side vector retrieval; (4) often using design methods and platforms such as OpenCL, but lacking sophisticated hardware structure optimization. Summary of the invention

[0014] Purpose of the invention: The vector retrieval scheme based on inverted index and quantization has significant advantages over traditional direct retrieval in terms of computational complexity and memory usage, and its accuracy can meet the needs of most application scenarios. It is one of the current mainstream vector retrieval schemes. However, this method has disadvantages such as low speed and high power consumption on general computing devices such as CPU and GPU, and research on hardware acceleration such as FPGA and ASIC for this scheme is still limited. In response to the above problems, the present invention proposes a vector retrieval device based on approximate nearest neighbor search, which can reduce the power consumption required for a single vector retrieval by more than 80% while ensuring accuracy, and significantly reduce the delay required for retrieval. The hardware device proposed by this invention is the solution with the lowest power consumption for a single retrieval at present.

[0015] The present invention specifically provides a vector retrieval device based on approximate nearest neighbor search, comprising a first storage, a second storage, a third storage, a fourth storage, a large cluster calculation LCM module, a quantitative lookup table calculation QPM module, and a final quantitative similarity calculation and result calculation DCM module;

[0016] The first storage, the second storage, the third storage and the fourth storage are hardware storages, which are used to store pre-prepared input data or intermediate results calculated by other modules;

[0017] The large cluster calculation LCM module inputs the query vector and the coordinates of nlist large cluster centers in the first storage (the coordinates of the large cluster centers are obtained in the pre-construction stage of the data set), outputs k1 selected large clusters, finds the k1 large cluster centers that are most similar to the query vector and saves them to the second storage, thereby preliminarily defining the distribution range of the candidate target vector;

[0018] The quantization lookup table calculation QPM module inputs the query vector and the coordinates of the subvector cluster centers in the first storage (the coordinates of the subvector cluster centers are obtained in the pre-construction stage of the data set), calculates the similarity between the subvectors to obtain a complete quantization lookup table, and saves it to the third storage;

[0019] The final quantization similarity calculation and result calculation DCM module reads the corresponding part of the quantized vector data set in the fourth storage (the corresponding part of the quantized vector data set is obtained in the pre-construction stage of the data set) according to the k1 large clusters most similar to the query vector in the second storage, and calculates the final result according to the quantization lookup table in the third storage to obtain the k2 vectors in the data set that are most similar to the query vector.

[0020] The large cluster calculation LCM module includes a D-dimensional vector similarity calculation module and a Top-k calculation module A;

[0021] The D-dimensional vector similarity calculation module is used to calculate vector similarity;

[0022] The Top-k calculation module A is used to find k1 large cluster centers that are most similar to the query vector.

[0023] The final quantization similarity calculation and result calculation DCM module includes a quantization vector calculation module and a Top-k calculation module B;

[0024] The quantized vector calculation module is used to calculate the final quantized similarity between the vector in the quantized vector data set and the query vector according to the k1 large clusters in the second storage that are most similar to the query vector, the quantized lookup table in the third storage, and the quantized vector data set in the fourth storage;

[0025] The Top-k calculation module B is used to obtain k2 vectors in the data set that are most similar to the query vector according to the results output by the quantization vector calculation module.

[0026] The quantization lookup table calculation QPM module includes a D / m-dimensional vector similarity calculation module and a register;

[0027] The D / m-dimensional vector similarity calculation module is used to calculate the similarity between D / m-dimensional sub-vectors;

[0028] The register is used to temporarily store the output result of the D / m-dimensional vector similarity calculation module.

[0029] The device of the present invention also includes a storage control module;

[0030] The storage control module is used to read the quantized vector data in the fourth storage according to the data in the second storage, thereby controlling the reading of the data in the third storage.

[0031] In the large cluster calculation LCM module, the input query vector and the large cluster center coordinates first pass through the D-dimensional vector similarity calculation module, and then pass through the Top-k calculation module A to obtain k1 results;

[0032] The number of clock cycles spent on calculating similarity and Top-k in the large cluster calculation LCM module are N1 and N2 respectively.

[0033] The Top-k computing module A and the Top-k computing module B adopt a Top-k I structure or a Top-k II structure;

[0034] The Top-k I structure and the Top-k II structure both contain lists that are updated in real time, updated once in each calculation cycle, and are used to store the k1 or k results with the highest current similarity; the initial value of the list before the calculation starts is the original list, which is set according to the actual calculation requirements.

[0035] The Top-k I structure processes an input value in each clock cycle, and compares the input value with the original k1 values ​​in the Top-k I structure. Each value in the list corresponds to a comparator, for a total of k1 comparators.

[0036] The comparator is used to compare the input value with the value at the corresponding position in the list. If the input value is greater than the value at the corresponding position in the list, the comparator outputs 1, otherwise it outputs 0;

[0037] After the comparator is an adder, which sums the outputs of k1 comparators to get the score of the input value, which is the position of the new input value in the new list. Then, according to the list before the update, the new input, and the score, it passes through a set of selection modules. Each position in the list corresponds to a selection module, totaling k1 selection modules.

[0038] The selection module is used to select the correct new value from the list before update and the new input, and replace the original value of the corresponding position in the list to obtain the updated list; for the first position, that is, the minimum value Top-0, the possible new value is the original Top-0 or the input value; for the i-th position, i>0, the possible new value is the original Top-i, Top-(i-1) or the input value; the multiplexer in the selection module determines the output value according to the score of the input value, and the input with the score score is inserted in the new position Top-score, and the remaining positions are arranged in sequence according to the values ​​in the list before update.

[0039] The Top-k II architecture adopts a method of processing multiple inputs simultaneously, and uses a pipelined structure to process p3 inputs in each cycle. Specifically, a two-stage design is adopted:

[0040] The first stage is used to process the p3 input values, obtain the size relationship between the p3 values, compare them with each other and sum them to get the score of each input, and then obtain the output result of the first stage, that is, the arranged p3 inputs, through a set of multiplexers;

[0041] The second stage is used to update the new Top-k list according to the input and the Top-k list before the update. The output of the first stage is used as the input of the second stage. The two groups of elements to be compared are: the input p4 values ​​and the k values ​​in the original Top-k list, p4 = min{p3,k}, the input p4 values ​​and the k values ​​in the original Top-k list are sorted ordered series, and the positions are directly read to obtain the scores of the input p4 values ​​and the k values ​​in the original Top-k list in the corresponding lists. The comparison results are added to the input p4 values ​​and the scores of the k values ​​in the original Top-k list. The positions of the k values ​​in the original list are used to obtain the final scores of the p4 input values ​​and the final scores of the k values ​​in the original Top-k list; in the selection module of the second stage, a total of 2p4+1 elements are input and classified according to the size relationship between i and p3; when i≤p4, the p4 inputs and the original Top-0~Top-i are used as the input of the selector module. In order to maintain the list size, Top-i is filled with a total of (p4-i+1); when i>p4, the Top list before update only retains Top-(i-p4)~Top-i, and the total input elements are 2p4+1.

[0042] The first storage 1, the second storage 2, the third storage 3, and the fourth storage 4 store different data in a combined or separate manner.

[0043] In the quantized lookup table calculation QPM module, the similarities between D / m-dimensional sub-vectors are calculated respectively; the input query vector is divided into m sub-vectors, each of which corresponds to k in the first storage. *quantized sub-cluster centers, and calculate the sub-vectors and the corresponding k * The similarity of the quantized sub-cluster centers is calculated; the number of clock cycles spent by the QPM module for quantized lookup table calculation is N3; in order to maximize the parallel computing efficiency of the LCM module and the QPM module, the ratio of their parallelism should be close to (N1+N2) / N3.

[0044] The device includes a highly parallel Top-k computing module, uses parallel clustering computing and quantized lookup table computing modules, and is suitable for different data precision types. The device is suitable for high-dimensional vector data sets of various sizes.

[0045] Compared with existing FPGA and ASIC-based solutions, the benefits include:

[0046] (1) This device does not require the use of CPU and other devices for cross-platform calculations and can fully realize vector retrieval;

[0047] (2) This device can be connected to HBM, DDR and other memories, rather than just using FPGA's BRAM on-chip storage to store all data, and can be applied to more complex scenarios;

[0048] (3) This device adopts a more efficient parallel strategy for large cluster center calculation and quantitative lookup table calculation, as well as a higher-speed Top-k calculation module, which is faster and consumes less power.

[0049] (4) This device is aimed at quantization-based solutions rather than graph retrieval solutions. Its advantage is that it is more lightweight and has higher efficiency in computing resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0051] Figure 1 It is the overall architecture diagram of the device of the present invention.

[0052] Figure 2 This is a schematic diagram of the Top-k I module architecture.

[0053] Figure 3 This is a schematic diagram of the Top-k II module architecture. DETAILED DESCRIPTION

[0054] In a specific embodiment of the present invention, a vector retrieval device based on approximate nearest neighbor search is provided, such as Figure 1 As shown, the structure of the device includes a large cluster calculation LCM module, a quantitative lookup table calculation QPM module, and a final quantitative similarity calculation and result calculation DCM module.

[0055] The main functions of the three modules include: (1) The LCM module inputs the query vector and the coordinates of the nlist large cluster centers in the first storage 1, and outputs k1 selected large clusters. Its function is to find the k1 large cluster centers that are most similar to the query vector and save them in the second storage 2, so as to preliminarily define the distribution range of the candidate target vector; (2) The QPM module inputs the query vector and the coordinates of the sub-vector cluster centers in the first storage 1, and its function is to calculate the similarity between the sub-vectors to obtain a complete quantization lookup table, and save it to the third storage 3; (3) The DCM module reads the corresponding part of the quantized vector data set in the fourth storage 4 according to the large clusters in the second storage 2, and calculates the final result according to the lookup table in the third storage 3, that is, the k2 vectors in the data set that are most similar to the query vector. Among them, LCM and DCM each contain a Top-k calculation module, which selects the maximum value according to the results calculated in the LCM module and the DCM module respectively.

[0056] (I) Efficient parallel design of LCM and QPM modules

[0057] In order to improve the parallel efficiency of the LCM module and the QPM module, the present invention adopts a fine resource allocation, so that the LCM module and the QPM module complete the calculation in a similar time, and enter the calculation of the DCM module at almost the same time, thereby reducing the waste caused by the time difference between the end of the calculation of the LCM module and the QPM module.

[0058] In the LCM module, the input query vector and the coordinates of the large cluster center (located on the first storage 1) first pass through the D-dimensional vector similarity calculation module, which can be a structure that implements one or more arbitrary types of vector similarity calculations such as Euclidean distance, cosine distance, inner product distance, etc. Then, after passing through a Top-k calculation structure, k1 results are obtained. The number of clock cycles spent on calculating similarity and Top-k in the LCM module are N1 and N2 respectively. In the QPM module, the similarities between the D / m-dimensional sub-vectors are calculated separately. The input query vector is divided into m sub-vectors, each corresponding to the k sub-vectors of the block part. * quantized sub-cluster centers and calculate the similarity respectively. The function of this part is similar to the D-dimensional distance calculation module in the LCM module. The number of clock cycles spent by the QPM module is N3. In order to maximize the parallel computing efficiency of the LCM module and the QPM module, after appropriate parallelism design and resource allocation, the parallelism ratio should be close to (N1+N2) / N3.

[0059] The following is an example using the Euclidean distance and Top-k I solution:

[0060] The LCM module first calculates the distance in each dimension, and then obtains the final distance between a pair of D-dimensional vectors through a multi-level tree-structured accumulation module. It contains a total of D-1 adders and D multipliers. In order to optimize the clock frequency of the distance calculation part, a pipeline is inserted between each level of adders in the accumulation module. Therefore, the distance calculation in this module requires a total of N1=log2D clock cycles. Since the QPM module needs to execute m·k * For vector distance calculation, the distance calculation and accumulation modules are reused to save resources, so k * / p2 cycles to complete the calculation of the quantization lookup table. Therefore, the calculation in the corresponding Top-k module adopts the serial list update of Top-k I: that is, each cycle updates a new list after obtaining an input value, and after N2=nlist cycles, the final k1 results are obtained. Therefore, the calculation of the LCM module consumes a total of log2D+nlist clock cycles.

[0061] In the QPM module, each pair of vectors also requires D multipliers, and each subsequent accumulation module requires D / m-1 adders, for m computing units, a total of Dm adders. In order to save computing resources, a pipelined structure is adopted here for module reuse. Each cycle inputs a set of quantized cluster center coordinates for calculation, a total of k * The total cost of completing the QPM module is N3 = k * +log2D / m clock cycles.

[0062] Since log2D and log2D / m are relative to k * and nlist are often small values, so it is roughly assumed that the time required for the LCM module and the QPM module to complete all calculations is nlist / k * , the corresponding parallelism ratio of the LCM module and the QPM module should be nlist / k*. * =512, nlist=256, k1=10, p1=1, for example, the parallelism p2 of the QPM module should be 2. In this case, the LCM module requires a total of 2D+15 adders and D multipliers, and the QPM requires a total of 4D-2m adders and 2D multipliers.

[0063] (II) Top-k module design

[0064] For the two Top-k modules in LCM and DCM, two different design architectures can be selectively adopted based on the comprehensive consideration of hardware resources and computing speed in practical applications.

[0065] 1. Top-k I module

[0066] The Top-k I module processes one input value per clock cycle, such as Figure 2 As shown in the figure, Top is the list before the update, and New Top is the list after the update. The input value is compared with the original k1 values ​​in Top-k I, corresponding to k1 comparators ( Figure 2 C in the list), if the input value is greater than the value at that position in the list, the comparator outputs 1, otherwise it outputs 0. After the comparator is an adder, the outputs of all comparators are summed to get the score of the input value, which is the position of the new input value in the new list. Then, according to the original list, the new input, and the score, a set of selection modules are passed, and each position in the list corresponds to a selection module. For the first position (i.e. the minimum value Top-0, corresponding Figure 2 For Case i=0), the possible new value is the original Top-0 or the input value. For the i-th position, i>0 (corresponding to Figure 2 In Case i>0), the possible new values ​​are the original Top-i, Top-(i-1) or the input value. The multiplexer in the selection module determines the output value based on the score of the input value. The input with the score is inserted in the new Top-score position, and the remaining positions are arranged in sequence with the values ​​in the original Top list.

[0067] 2. Top-k II module

[0068] Since ANNS calculations may need to process a large number of vectors (often more than one million) in actual applications, if only one input is processed per cycle, the module will cause a high delay. To solve this problem, the device proposes a Top-k II module, which takes the approach of processing multiple inputs simultaneously. Through a pipelined structure, it can process p3 inputs per cycle. Compared with the solution of processing one input per cycle, this solution can increase the speed to p3 times the original.

[0069] The Top-k II module is capable of handling the higher parallel input of p3, so a dual-stage design is adopted to optimize the clock frequency.

[0070] Stage 1 is used to process the p3 input values, and the size relationship between these values ​​needs to be obtained. After mutual comparison, the sum is calculated to obtain the score of each input, and then the output result of stage 1, that is, the arranged p3 inputs, is obtained through a set of multiplexers. This part should choose whether to add one or more pipelines in the structure according to the clock frequency required by the design and the size of the input list.

[0071] Phase 2 is used to update the new Top-k list based on the input and the original Top-k list. In this phase, the output of Phase 1 is used as input. Since the two sets of elements to be compared are: (1) the p4 input values ​​(p4 = min{p3, k}); (2) the k values ​​in the original Top-k list, both sets of data are already sorted ordered series, so the position can be directly read to get the score in the list. The result of the comparison plus the position in the original list is the final score. In the selector module of this phase, the total input is 2p4+1 elements, which are classified according to the size relationship between i and p3. When i≤p4, the p4 inputs and the original Top-0 to Top-i are used as the input of the selector module. In order to maintain the list size, Top-i is filled with a total of (p4-i+1). When i>p4, since the elements Top-0 to Top-(i-p4-1) cannot appear in the i-th position of the new list, the original Top list in the input only retains Top-(i-p4) to Top-i, and the total input elements are 2p4+1.

[0072] like Figure 3 As shown, it is an embodiment of the Top-k II structure: in stage 1, the comparator on one side of the diagonal is retained ( Figure 3 C in the figure), the other side performs logical inversion according to the value of the symmetrical position to obtain the comparison value ( Figure 3 In the stage 2, the input list and the original list are compared one by one, which requires a total of p4·k comparators. Figure 3 The Top in stage 2 is the list before the update, and the New Top is the list after the update.

[0073] Figure 1 The first storage 1, the second storage 2, the third storage 3, and the fourth storage 4 in the actual use can be combined or separated to store different data, which can replace Figure 1 The location and arrangement of memory in the architecture diagram.

[0074] For Top-k calculation in LCM module, the Top-k I architecture or the Top-k II architecture can be used, depending on the degree of parallelism of the module. They are alternative solutions.

[0075] In one embodiment of the present invention, the hardware architecture of the present device is written based on Verilog, Xilinx AlveoU280 is used, and simulation and implementation are performed based on Vivado 2022.2. SIFT 1M, the most commonly used vector approximate nearest neighbor search, is used for testing, which contains one million 128-dimensional vectors. The IVF-PQ algorithm is used to build an index. The Euclidean distance is used to measure the similarity of vectors. The Top-k I module is used in the LCM module, and the Top-k II module is used in the DCM module. When testing this embodiment, the DCM module parallelism of p3=44 and the data accuracy of Int16 are used for comparison and verification. Under the condition of Recall@10=94%, k1=20 is set in the LCM module, that is, 20 clusters need to be output, and a total of about 19,000 vectors need to be processed in the DCM module. Set k * =1024, m=32.

[0076] As a control, the results of CPU (Intel Core i9-13900K) and GPU (NVIDIARTX 4090) under the same algorithm, parameters, recall rate, and data set conditions were tested, and the results of ANNA (reference: Y.Lee, H.Choi, S.Min, H.Lee, S.Beak, D.Jeong, JWLee, and TJHam, “Anna: Specialized architecture forapproximate nearest neighbor search,” in 2022IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2022, pp.169–183) under the same conditions were cited and compared with the effects of the present invention.

[0077] Table 1

[0078]

[0079] It can be concluded from the above table that the device of the present invention can adjust the parallelism and data accuracy according to the needs, optimize the resource allocation and retrieval speed. Compared with other implementation schemes, under the premise of the same retrieval recall rate (94%), the present device can: (1) reduce the delay by more than 99.2% and the power consumption of a single retrieval by more than 99.9% compared with the current advanced CPU (Intel Core i9-13900K); (2) reduce the delay by more than 95.8% and the power consumption of a single retrieval by more than 99.2% compared with the current advanced GPU (NVIDIARTX 4090); (3) reduce the delay by more than 80% and the power consumption of a single retrieval by more than 87% compared with the latest hardware design scheme of quantization-based vector retrieval (FPGA, ASIC).

[0080] The present invention provides a vector retrieval device based on approximate nearest neighbor search. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A vector retrieval device based on approximate nearest neighbor search, characterized in that: It includes a first storage, a second storage, a third storage, a fourth storage, a large cluster calculation LCM module, a quantitative lookup table calculation QPM module, and a final quantitative similarity calculation and result calculation DCM module; The first storage, the second storage, the third storage and the fourth storage are hardware storages, which are used to store pre-prepared input data or intermediate results of calculations; The large cluster calculation LCM module inputs the query vector and the coordinates of nlist large cluster centers in the first storage, outputs k1 selected large clusters, finds the k1 large cluster centers that are most similar to the query vector and saves them to the second storage, thereby preliminarily defining the distribution range of the candidate target vector; The quantization lookup table calculation QPM module inputs the query vector and the coordinates of the subvector cluster center in the first storage, calculates the similarity between the subvectors to obtain a complete quantization lookup table, and saves it to the third storage; The final quantization similarity calculation and result calculation DCM module reads the corresponding part of the quantized vector data set in the fourth storage according to the k1 large clusters most similar to the query vector in the second storage, and calculates the final result according to the quantization lookup table in the third storage to obtain the k2 vectors in the data set that are most similar to the query vector.

2. The device according to claim 1, characterized in that The large cluster calculation LCM module includes a D-dimensional vector similarity calculation module and a Top-k calculation module A; The D-dimensional vector similarity calculation module is used to calculate vector similarity; The Top-k calculation module A is used to find k1 large cluster centers that are most similar to the query vector.

3. The device according to claim 2, characterized in that The final quantization similarity calculation and result calculation DCM module includes a quantization vector calculation module and a Top-k calculation module B; The quantized vector calculation module is used to calculate the final quantized similarity between the vector in the quantized vector data set and the query vector according to the k1 large clusters in the second storage that are most similar to the query vector, the quantized lookup table in the third storage, and the quantized vector data set in the fourth storage; The Top-k calculation module B is used to obtain k2 vectors in the data set that are most similar to the query vector according to the results output by the quantization vector calculation module.

4. The device according to claim 3, characterized in that The quantization lookup table calculation QPM module includes a D / m-dimensional vector similarity calculation module and a register; The D / m-dimensional vector similarity calculation module is used to calculate the similarity between D / m-dimensional sub-vectors; The register is used to temporarily store the output result of the D / m-dimensional vector similarity calculation module.

5. The device according to claim 4, characterized in that Also included is a storage control module; The storage control module is used to read the quantized vector data in the fourth storage according to the data in the second storage, thereby controlling the reading of the data in the third storage.

6. The device according to claim 5, characterized in that In the large cluster calculation LCM module, the input query vector and the large cluster center coordinates first pass through the D-dimensional vector similarity calculation module, and then pass through the Top-k calculation module A to obtain k1 results; The number of clock cycles spent on calculating similarity and Top-k in the large cluster calculation LCM module are N1 and N2 respectively.

7. The device according to claim 6, characterized in that The Top-k computing module A and the Top-k computing module B adopt a Top-k I structure or a Top-k II structure; The Top-k I structure and the Top-k II structure both contain lists that are updated in real time, updated once in each calculation cycle, and are used to store the k1 or k results with the highest current similarity; the initial value of the list before the calculation starts is the original list, which is set according to the actual calculation requirements.

8. The device according to claim 7, characterized in that The Top-k I structure processes an input value in each clock cycle, and compares the input value with the original k1 values ​​in the Top-k I structure. Each value in the list corresponds to a comparator, for a total of k1 comparators. The comparator is used to compare the input value with the value at the corresponding position in the list. If the input value is greater than the value at the corresponding position in the list, the comparator outputs 1, otherwise it outputs 0; After the comparator is an adder, which sums the outputs of k1 comparators to get the score of the input value, which is the position of the new input value in the new list. Then, according to the list before the update, the new input, and the score, it passes through a set of selection modules. Each position in the list corresponds to a selection module, totaling k1 selection modules. The selection module is used to select the correct new value from the list before update and the new input, and replace the original value of the corresponding position in the list to obtain the updated list; for the first position, that is, the minimum value Top-0, the possible new value is the original Top-0 or the input value; for the i-th position, i>0, the possible new value is the original Top-i, Top-(i-1) or the input value; the multiplexer in the selection module determines the output value according to the score of the input value, and the input with the score score is inserted in the new position Top-score, and the remaining positions are arranged in sequence according to the values ​​in the list before update.

9. The device according to claim 8, characterized in that The Top-k II architecture adopts a method of processing multiple inputs simultaneously, and uses a pipelined structure to process p3 inputs in each cycle. Specifically, a two-stage design is adopted: The first stage is used to process the p3 input values, obtain the size relationship between the p3 values, compare them with each other and sum them to get the score of each input, and then obtain the output result of the first stage, that is, the arranged p3 inputs, through a set of multiplexers; The second stage is used to update the new Top-k list according to the input and the Top-k list before the update. The output of the first stage is used as the input of the second stage. The two groups of elements to be compared are: the input p4 values ​​and the k values ​​in the original Top-k list, p4 = min{p3, k}, the input p4 values ​​and the k values ​​in the original Top-k list are sorted ordered series, and the positions of the input p4 values ​​and the k values ​​in the original Top-k list are directly read to obtain the scores of the input p4 values ​​and the k values ​​in the original Top-k list in the same list. The comparison results are added to the positions of the input p4 values ​​and the k values ​​in the original Top-k list in the original list to obtain the final scores of the input p4 values ​​and the final scores of the k values ​​in the original Top-k list. In the selection module of the second stage, a total of 2p4+1 elements are input and classified according to the size relationship between i and p3; when i≤p4, p4 inputs and the original Top-0~Top-i are used as the input of the selector module. In order to maintain the list size, Top-i is filled with a total of (p4-i+1); when i>p4, the Top list before update only retains Top-(i-p4)~Top-i, and the total input elements are 2p4+1.

10. The device according to claim 9, characterized in that The first storage 1, the second storage 2, the third storage 3, and the fourth storage 4 store different data in a combined or separate manner.

Images