Hardware accelerator for efficient vector retrieval and efficient vector retrieval method and device

Abstract

The invention discloses a hardware accelerator used for efficient vector retrieval and an efficient vector retrieval method and device, and belongs to the field of data retrieval, a vector calculation unit arranged in the hardware accelerator can realize parallel multi-vector operation, and a matrix calculation unit improves the efficiency of vector calculation and improves the efficiency of vector retrieval. And the graph data cache unit effectively solves the problem that the calculation unit is idle in the execution process of the ANNS algorithm, and the resource utilization rate is remarkably improved. Furthermore, in an efficient vector retrieval method proposed on the basis of the hardware accelerator, through a high-reachability neighbor graph construction technology, including neighbor graph generation, reverse graph edge fusion and edge rearrangement and screening based on the number of unfavorable paths, a graph structure with strong connectivity and controllable out-degree can be formed, dynamic memory allocation is reduced, and the probability of data transmission is lowered. The memory access mode is more regular and predictable, and the intelligent switching and load balancing of the computing units in the hardware accelerator are combined, so that the resource utilization is optimized, and the more uniform computing load is realized.

Description

Technical Field

[0001] This invention belongs to the field of data retrieval, and more specifically, relates to a hardware accelerator, a high-efficiency vector retrieval method, and an apparatus for efficient vector retrieval. Background Technology

[0002] With the advent of the AI ​​era, Approximate Nearest Neighbor Search (ANNS) plays a crucial role in various disciplines of data mining and artificial intelligence. In recent years, the amount of data requiring processing in related fields has skyrocketed, and the computational cost of exact nearest neighbor search is often too high, necessitating the use of approximation techniques. Due to its excellent balance of performance and recall, graph-based ANNS methods have received considerable attention in this field. Many existing vector-related computations utilize graph-based ANNS, such as vector retrieval, vector comparison in large-scale model inference, and recommendation algorithms.

[0003] In graph-based ANNS, the execution of the algorithm is mainly divided into two parts: the first is the construction of the nearest neighbor graph, and the second is the execution of the ANNS search algorithm.

[0004] In the first part, the main approach is to filter nodes and edges from the dataset and assemble suitable nodes and edges into a nearest neighbor graph. Traditional nearest neighbor graphs include: Delaunay graphs (DG), relative adjacency graphs (RNG), k-nearest neighbor graphs (KNNG), and minimum spanning trees (MST). However, it has been concluded that the quality of the nearest neighbor graph is not necessarily positively correlated with search efficiency. Therefore, current cutting-edge approaches use these traditional nearest neighbor graphs as base graphs based on past performance, and empirically and heuristically optimize these base graphs. This approach also has certain drawbacks. Currently, most base graphs do not control the in-degree and out-degree of nodes, leading to excessive dynamic memory scheduling when the nearest neighbor graph is accessed during actual search operations.

[0005] In the second part, most current ANNS search algorithms adapt to the characteristics of the nearest neighbor graph used in the current system. For example, the HNSW algorithm allows the search algorithm to perform a top-down search based on the structure of the HNSW nearest neighbor graph. To improve the efficiency of ANNS, existing research has proposed many nearest neighbor graphs and search algorithms to accelerate ANNS. However, due to neglecting the parallelism available in ANNS, existing solutions suffer from insufficient utilization of multi-core processors, resulting in slow ANNS search speeds. Summary of the Invention

[0006] In view of the above-mentioned defects or improvement needs of the prior art, the present invention provides a hardware accelerator, a high-efficiency vector retrieval method and apparatus for efficient vector retrieval, thereby solving the technical problems of low effective parallelism, excessive dynamic memory scheduling and low search efficiency in vector retrieval on multi-core processors.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a hardware accelerator for efficient vector retrieval is provided, comprising: Vector computation unit, matrix computation unit, and graph data caching unit; The vector calculation unit is used to calculate the similarity between vectors, and the vector calculation unit has a built-in configurable vector register, which is used to dynamically adjust the vector processing width according to the sparsity of the vector to be calculated. The matrix calculation unit is used to organize the vectors to be calculated into a matrix form and then perform batch similarity calculation. The matrix calculation unit has a built-in hardware compressor. When the matrix obtained by organizing the vectors to be calculated is detected to be a sparse matrix, the hardware compressor is used to convert the matrix into a block compression format before calculation. The graph data caching unit includes a cache for storing the nearest neighbor graph formed by each data point in the vector database, and has a built-in access mode model. The access mode model is used to schedule the current vector computing task to the vector computing unit or the matrix computing unit based on the access frequency and edge connection density of each node in the nearest neighbor graph formed by each data point in the vector database, as well as the computing load data of the matrix computing unit and the vector computing unit.

[0008] According to the aforementioned hardware accelerator, when the sparsity of the vector to be computed is higher than a preset threshold, the configurable vector register sets the vector processing width to a first vector width; otherwise, the configurable vector register sets the vector processing width to a second vector width; wherein, the first vector width is greater than the second vector width.

[0009] According to a second aspect of the present invention, an efficient vector retrieval method based on the above-described hardware accelerator is provided, comprising: Read each data point from the vector database, organize each data point into a vector data matrix, and perform matrix multiplication on the vector data matrix and the transpose of the vector data matrix based on the matrix calculation unit to obtain the vector distance between each pair of data points. For each data point, the vector calculation unit sorts other data points in ascending order of vector distance, selects the K points with the smallest vector distance as the neighbors of the corresponding data point, and constructs a nearest neighbor graph based on each data point and its neighbors. The nearest neighbor graph is then stored in the cache of the graph data cache unit. Specifically, for each node in the nearest neighbor graph, the node ID, vector, cache offset, and neighbor ID of the corresponding node are stored together in the cache. Receive a query vector from the user, and perform an approximate nearest neighbor search on the nearest neighbor graph based on the query vector to obtain multiple nodes that are most similar to the query vector.

[0010] Based on the above-described efficient vector retrieval method, the nearest neighbor graph is stored in the cache of the graph data caching unit, and then the method further includes: Construct a reverse graph of the nearest neighbor graph and merge the reverse graph into the nearest neighbor graph. Based on the number of unfavorable paths from each node to its neighbors in the merged nearest neighbor graph, reorder and filter the neighbors of each node to obtain a nearest neighbor graph that satisfies the connectivity objective and update the cache. Specifically, for any node A and its neighbor node B, if there exists another path from node A to node B that includes an intermediate node C, and the sum of the distance from node A to node C and the distance from node C to node B is less than the distance from node A to node B, then the path including the intermediate node C is an unfavorable path from node A to node B.

[0011] According to the above-mentioned efficient vector retrieval method, the reordering and filtering of neighbors of each node based on the number of unfavorable paths of each node in the merged nearest neighbor graph specifically includes: For each node in the merged nearest neighbor graph, the neighbors of the corresponding node are reordered in descending order of the number of unfavorable paths from the corresponding node to each of its neighbors, and the neighbors at the top are retained based on the preset out-degree number.

[0012] Based on the above efficient vector retrieval method, the approximate nearest neighbor search includes a multi-round graph query process, wherein any round of graph query process includes: Step S1: Randomly select multiple starting nodes from the nearest neighbor graph, and calculate the distance between the query vector and the starting nodes based on the vector calculation unit; Step S2: Add the starting node to the candidate list. After initializing the top-M list, select the M nodes with the smallest distance to the query vector from the candidate list and place them in the top-M list. Step S3: Obtain the node that is added to the top-M list for the first time, and add all the neighbors of the node to the candidate list; Step S4: Determine the node that is added to the candidate list for the first time and mark it as a node to be calculated; construct a vector calculation task for all nodes to be calculated, and call the vector calculation unit or the matrix calculation unit to execute the vector calculation task to obtain the distance between all nodes to be calculated and the query vector; Step S5: Sort all nodes in the candidate list in ascending order of distance from the query vector, and compare the sorting result with the top-M list; if the top M nodes in the sorting result are inconsistent with the top-M list, update the top-M list and jump to step S3; otherwise, output the top-M list.

[0013] Based on the above efficient vector retrieval method, the vector calculation unit or the matrix calculation unit is invoked to execute the vector calculation task, obtaining the distances between all nodes to be calculated and the query vector, specifically including: Obtain the current access frequency and edge connection density of each node in the nearest neighbor graph, as well as the current computational load data of the matrix calculation unit and the vector calculation unit; Using the access pattern model of the graph data caching unit, based on the current access frequency and edge connection density of each node in the nearest neighbor graph, as well as the current computational load data of the matrix computation unit and the vector computation unit, the vector computation task is scheduled to the vector computation unit or the matrix computation unit, so that the vector computation task is executed based on the scheduled computation unit, and the distance between all nodes to be computed and the query vector is obtained.

[0014] According to a third aspect of the present invention, a high-efficiency vector retrieval apparatus based on the above-described hardware accelerator is provided for performing the method described in the second aspect.

[0015] According to a fourth aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the second aspect.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: The hardware accelerator provided in this embodiment of the invention includes a vector computation unit, a matrix computation unit, and a graph data caching unit. The vector computation unit calculates the similarity between vectors and includes a built-in configurable vector register. This register dynamically adjusts the vector processing width based on the sparsity of the vectors to be computed, enabling parallel multi-vector operations. The matrix computation unit organizes the vectors into a matrix and performs batch similarity calculations. It also includes a built-in hardware compressor that converts the matrix into a block-compressed format before computation when the matrix is ​​detected to be sparse, improving the efficiency of vector computation. The graph data caching unit stores the nearest neighbor graph formed by each data point in the vector database and includes an access mode model. This model schedules the current vector computation task to either the vector computation unit or the matrix computation unit based on the access frequency of each node in the nearest neighbor graph, the edge connection density, and the computational load data of the matrix and vector computation units. This effectively solves the problem of idle computation units during the execution of the ANNS algorithm and significantly improves resource utilization.

[0017] Furthermore, the efficient vector retrieval method proposed based on the aforementioned hardware accelerator utilizes highly accessible nearest neighbor graph construction techniques, including nearest neighbor graph generation, reverse graph edge fusion, and edge rearrangement and filtering based on the number of unfavorable paths. This results in a strongly connected graph structure with controllable out-degree, reducing dynamic memory allocation and making memory access patterns more regular and predictable. Combined with the intelligent switching and load balancing of computing units (i.e., vector computing units and matrix computing units) in the hardware accelerator, resource utilization is optimized, achieving a more uniform computing load and thus improving the memory utilization of multi-core processors. Attached Figure Description

[0018] Figure 1 A schematic diagram of the structure of a hardware accelerator for efficient vector retrieval provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the nearest neighbor graph construction method provided in an embodiment of the present invention. Figure 3 This is a flowchart illustrating the approximate nearest neighbor search method provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0020] This invention provides a hardware accelerator for efficient vector retrieval, such as... Figure 1 As shown, the hardware accelerator includes: Vector computation unit, matrix computation unit, and graph data caching unit.

[0021] The vector computation unit is used to calculate the similarity between vectors. A key feature is its built-in configurable vector register, which dynamically adjusts the vector processing width based on the sparsity of the vectors to be computed. Here, the sparsity of the vectors to be computed can be determined based on the number of zeros in the vector; more zeros indicate greater sparsity. In some embodiments, when the sparsity of the vectors to be computed exceeds a preset threshold, the configurable vector register can set the vector processing width to a first vector width (e.g., 128 bits or more) for batch processing; otherwise, it can set the vector processing width to a second vector width (e.g., 64 bits) to optimize energy efficiency. This dynamic adjustment mechanism is implemented through a hardware state machine, enabling adaptive adjustment of the vector processing width and vector computation within a single clock cycle.

[0022] The matrix computation unit is used to organize the vectors to be calculated into a matrix form before performing batch similarity calculations. Specifically, after organizing the vectors into a matrix form, matrix multiplication is performed on this matrix and its transpose, enabling batch vector similarity calculations. Furthermore, the matrix computation unit incorporates a hardware compressor. When the matrix formed by organizing the vectors to be calculated is detected as a sparse matrix, this compressor converts the matrix into a block-compressed format (such as CSR, compressed sparse row storage) before computation. The hardware compressor can achieve real-time compression and decompression of matrix data through a hardware pipeline. During matrix multiplication, only non-zero data blocks need to be computed in parallel, effectively avoiding the resource waste caused by zero-value computation.

[0023] The graph data caching unit contains a cache for storing the nearest neighbor graph formed by each data point (one data point corresponds to one vector) in the vector database. It handles cache management and data prefetching of node data in the nearest neighbor graph. Multiple levels of caching can be configured to store the graph data. For example... Figure 1As shown, the cache can store the node ID, vector (i.e., the data point corresponding to that node in the vector database), cache offset, and neighbor ID for each node in the nearest neighbor graph. The cache can also store the TopM list of nodes most similar to the query vector obtained from subsequent approximate nearest neighbor searches. Furthermore, the graph data cache unit has a built-in access pattern model. This model can schedule the current vector computation task to either the vector computation unit or the matrix computation unit based on the access frequency, edge connection density, and computational load data of each node in the nearest neighbor graph formed by each data point in the vector database. This effectively solves the problem of idle computing units in multi-core processors and significantly improves resource utilization. Specifically, the access pattern model can make scheduling decisions based on preset decision rules, combining the feature combinations of the access frequency, edge connection density, and computational load data of each node in the nearest neighbor graph. Here, the decision rules define the scheduling object corresponding to each feature combination. For example, when the matrix formed by the vectors of the node to be computed is a sparse matrix and the load on the matrix computation unit is moderate, scheduling is performed to the matrix computation unit; when the node to be computed has a high access frequency and a high edge connection density, and the load on the matrix computation unit is low, scheduling is performed to the matrix computation unit; when the node to be computed has a low access frequency and a low edge connection density, and the load on the vector computation unit is low, scheduling is performed to the vector computation unit. Alternatively, when the node to be computed has a high edge connection density, it is preferentially scheduled to the matrix computation unit regardless of the access frequency; when the node to be computed has a low edge connection density, it is preferentially scheduled to the vector computation unit regardless of the access frequency.

[0024] Based on any of the above embodiments, this invention also provides an efficient vector retrieval method. This method mainly includes a nearest neighbor graph construction stage and an approximate nearest neighbor search stage. Specifically, as... Figure 2As shown, in the nearest neighbor graph construction stage, each data point in the vector database (a single data point corresponds to a vector in the vector database, such as a text vector, image vector, etc.) can be read, organized into a vector data matrix, and matrix multiplication is performed on the vector data matrix and its transpose using the matrix computing unit in the hardware accelerator to obtain the vector distance between each pair of data points. During the construction of the nearest neighbor graph and subsequent similar nearest neighbor search, the distance calculation method can be selected according to actual needs, such as Euclidean distance, Manhattan distance, cosine similarity, etc., and this embodiment of the invention does not specifically limit this. To fully utilize the batch computing capability and compression computing capability for sparse matrices of the matrix computing unit when calculating the pairwise distance between data points in the vector database, the computational efficiency of this stage can be improved. Specifically, the data points can be organized into a matrix and then matrix multiplication can be performed. When dealing with a matrix whose sparsity exceeds a threshold, the matrix can be compressed using a hardware compressor to reduce redundant operations. Finally, a symmetric matrix D is formed, where D[i,j] represents the distance between data point i and data point j.

[0025] Subsequently, for each data point, the vector computation unit sorts the other data points in ascending order of vector distance and selects the K points with the smallest vector distance as the neighbors of the corresponding data point, thereby constructing a nearest neighbor graph based on each data point and its neighbors. The constructed nearest neighbor graph can be stored in the cache of the graph data caching unit.

[0026] To improve the performance of subsequent similar nearest neighbor searches, the nearest neighbor graph needs to have high reachability. One metric for improving reachability is how many nodes can be reached from a given node within a specified number of path traversals. Specifically, the average number of 2-hop nodes (N2hop) across all nodes in the graph can be used as a metric. Here, the number of 2-hop nodes for a given node is the number of nodes reachable in two traversals from that node. Therefore, to improve the reachability of the nearest neighbor graph, in some embodiments, the edges of the constructed nearest neighbor graph can be reordered and filtered. Specifically, a reverse graph of the current nearest neighbor graph (i.e., the graph between a node and its nearest neighbor) can be constructed. Figure 1 The graph is then converted into a graph whose edge direction is opposite to that of the corresponding edge in the current nearest neighbor graph. This reverse graph is then merged into the current nearest neighbor graph. Subsequently, based on the number of unfavorable paths from each node to its neighbors in the merged nearest neighbor graph, the neighbors of each node are reordered and filtered to obtain a nearest neighbor graph that satisfies the connectivity objective, and the graph data in the cache is updated (i.e., the neighbor ID part of each node is updated).

[0027] In some embodiments, for any node A and its neighbor node B, if there exists another path from node A to node B that includes an intermediate node C, and the sum of the distance from node A to node C and the distance from node C to node B is less than the distance from node A to node B, then this path including the intermediate node C is an unfavorable path from node A to node B. When reordering the neighbors of each node in the merged nearest neighbor graph, the neighbors of the corresponding node i are reordered in descending order of the number of unfavorable paths from the corresponding node i to each of its neighbors, and the top d neighbors are retained based on a preset out-degree number d. The more unfavorable paths a node i has to its neighbors, the more likely this neighbor node is to connect to remote nodes that are far from other nodes in the nearest neighbor graph. Placing this neighbor node at the top ensures that subsequent approximate nearest neighbor search algorithms will prioritize processing this neighbor node, allowing traversal from this neighbor node to remote nodes, preventing the ANNS algorithm from ignoring such remote nodes, thereby improving the accuracy of ANNS.

[0028] This concludes the nearest neighbor graph construction phase, allowing for the subsequent approximate nearest neighbor search phase. In the approximate nearest neighbor search phase, a query vector from the user is received, and an approximate nearest neighbor search is performed on the nearest neighbor graph based on this query vector to obtain the multiple nodes most similar to the query vector.

[0029] Here, graph-based approximate nearest neighbor search algorithms typically involve multiple rounds of graph query processes. To improve the parallelism of the approximate nearest neighbor search stage and thus enhance the efficiency of vector retrieval, this embodiment of the invention performs parallel computation for each round of graph query processes. Specifically, as follows... Figure 3 As shown, each round of graph query process includes the following steps: Step S1: Randomly select multiple starting nodes from the nearest neighbor graph, and calculate the distance between the query vector and the starting nodes based on the vector calculation unit; Step S2: Add the starting node to the candidate list. After initializing the top-M list, select the M nodes with the smallest distance from the query vector from the candidate list and place them into the top-M list. Step S3: Obtain the node that is added to the top-M list for the first time, and add all the nodes' neighbors to the candidate list; Step S4: Determine the node added to the candidate list for the first time and mark it as a node to be calculated; construct a vector calculation task for all nodes to be calculated, and call the vector calculation unit or matrix calculation unit to execute the vector calculation task to obtain the distance between all nodes to be calculated and the query vector; Step S5: Sort all nodes in the candidate list in ascending order of distance from the query vector, and compare the sorting result with the top-M list; if the top M nodes in the sorting result are inconsistent with the top-M list, update the top-M list and jump to step S3; otherwise, output the top-M list.

[0030] Unlike the conventional ANNS algorithm, this embodiment of the invention integrates all vector calculations in each iteration into step S4, and performs vector calculations only on nodes newly added to the candidate list. In step S4, nodes newly added to the candidate list are identified and marked as nodes to be calculated. After marking, a unified vector calculation task is constructed for all nodes to be calculated, and the vector calculation unit or matrix calculation unit is invoked to execute the vector calculation task, obtaining the distance between all nodes to be calculated and the query vector. In some embodiments, for this vector calculation task, the current access frequency, edge connection density, and current computational load data of each node in the nearest neighbor graph can be obtained. Then, using the access pattern model of the graph data caching unit, based on the current access frequency, edge connection density, and current computational load data of each node in the nearest neighbor graph, the vector calculation task is scheduled to the vector calculation unit or matrix calculation unit to execute the vector calculation task based on the scheduled calculation unit, obtaining the distance between all nodes to be calculated and the query vector.

[0031] In summary, the hardware accelerator provided in this embodiment of the invention includes a vector computation unit, a matrix computation unit, and a graph data caching unit. The vector computation unit calculates the similarity between vectors and includes a built-in configurable vector register. This register dynamically adjusts the vector processing width based on the sparsity of the vectors to be computed, enabling parallel multi-vector operations. The matrix computation unit organizes the vectors to be computed into a matrix and performs batch similarity calculations. It also includes a built-in hardware compressor that converts the matrix into a block-compressed format before computation when the matrix is ​​detected to be sparse, improving the efficiency of vector computation. The graph data caching unit stores a cache of the nearest neighbor graph formed by each data point in the vector database and includes an access mode model. This model schedules the current vector computation task to either the vector computation unit or the matrix computation unit based on the access frequency of each node in the nearest neighbor graph, the edge connection density, and the computational load data of the matrix and vector computation units. This effectively solves the problem of idle computation units during the execution of the ANNS algorithm and significantly improves resource utilization.

[0032] Furthermore, the efficient vector retrieval method proposed based on the aforementioned hardware accelerator utilizes highly accessible nearest neighbor graph construction techniques, including nearest neighbor graph generation, reverse graph edge fusion, and edge rearrangement and filtering based on the number of unfavorable paths. This results in a strongly connected graph structure with controllable out-degree, reducing dynamic memory allocation and making memory access patterns more regular and predictable. Combined with the intelligent switching and load balancing of computing units (i.e., vector computing units and matrix computing units) in the hardware accelerator, resource utilization is optimized, achieving a more uniform computing load and thus improving the memory utilization of multi-core processors.

[0033] This invention provides a high-efficiency vector retrieval device, comprising: The nearest neighbor graph construction unit is used to read each data point in the vector database, organize the data points into a vector data matrix, and perform matrix multiplication on the vector data matrix and its transpose based on the matrix calculation unit to obtain the vector distance between each pair of data points. For each data point, the vector calculation unit sorts other data points in ascending order of vector distance, selects the K points with the smallest vector distance as the neighbors of the corresponding data point, and constructs a nearest neighbor graph based on each data point and its neighbors. The nearest neighbor graph is then stored in the cache of the graph data caching unit. Specifically, for each node in the nearest neighbor graph, the node ID, vector, cache offset, and neighbor ID of the corresponding node are stored together in the cache. An approximate nearest neighbor search unit is used to receive a query vector from a user, perform an approximate nearest neighbor search on the nearest neighbor graph based on the query vector, and obtain multiple nodes that are most similar to the query vector.

[0034] Based on any of the above embodiments, after storing the nearest neighbor graph in the cache of the graph data caching unit, the nearest neighbor graph construction unit is further configured to: Construct a reverse graph of the nearest neighbor graph and merge the reverse graph into the nearest neighbor graph. Based on the number of unfavorable paths from each node to its neighbors in the merged nearest neighbor graph, reorder and filter the neighbors of each node to obtain a nearest neighbor graph that satisfies the connectivity objective and update the cache. Specifically, for any node A and its neighbor node B, if there exists another path from node A to node B that includes an intermediate node C, and the sum of the distance from node A to node C and the distance from node C to node B is less than the distance from node A to node B, then the path including the intermediate node C is an unfavorable path from node A to node B.

[0035] Based on any of the above embodiments, the reordering and filtering of the neighbors of each node based on the number of unfavorable paths of each node in the merged nearest neighbor graph specifically includes: For each node in the merged nearest neighbor graph, the neighbors of the corresponding node are reordered in descending order of the number of unfavorable paths from the corresponding node to each of its neighbors, and the neighbors at the top are retained based on the preset out-degree number.

[0036] Based on any of the above embodiments, the approximate nearest neighbor search includes a multi-round graph query process, wherein any round graph query process includes: Step S1: Randomly select multiple starting nodes from the nearest neighbor graph, and calculate the distance between the query vector and the starting nodes based on the vector calculation unit; Step S2: Add the starting node to the candidate list. After initializing the top-M list, select the M nodes with the smallest distance to the query vector from the candidate list and place them in the top-M list. Step S3: Obtain the node that is added to the top-M list for the first time, and add all the neighbors of the node to the candidate list; Step S4: Determine the node that is added to the candidate list for the first time and mark it as a node to be calculated; construct a vector calculation task for all nodes to be calculated, and call the vector calculation unit or the matrix calculation unit to execute the vector calculation task to obtain the distance between all nodes to be calculated and the query vector; Step S5: Sort all nodes in the candidate list in ascending order of distance from the query vector, and compare the sorting result with the top-M list; if the top M nodes in the sorting result are inconsistent with the top-M list, update the top-M list and jump to step S3; otherwise, output the top-M list.

[0037] Based on any of the above embodiments, the vector calculation unit or the matrix calculation unit is invoked to execute the vector calculation task to obtain the distances between all nodes to be calculated and the query vector, specifically including: Obtain the current access frequency and edge connection density of each node in the nearest neighbor graph, as well as the current computational load data of the matrix calculation unit and the vector calculation unit; Using the access pattern model of the graph data caching unit, based on the current access frequency and edge connection density of each node in the nearest neighbor graph, as well as the current computational load data of the matrix computation unit and the vector computation unit, the vector computation task is scheduled to the vector computation unit or the matrix computation unit, so that the vector computation task is executed based on the scheduled computation unit, and the distance between all nodes to be computed and the query vector is obtained.

[0038] This invention provides an electronic device, including: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any of the above embodiments.

[0039] This invention provides a computer-readable storage medium storing computer instructions that cause a processor to perform the method described in any of the above embodiments.

[0040] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0041] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A hardware accelerator for efficient vector retrieval, characterized in that, include: Vector computation unit, matrix computation unit, and graph data caching unit; The vector calculation unit is used to calculate the similarity between vectors, and the vector calculation unit has a built-in configurable vector register, which is used to dynamically adjust the vector processing width according to the sparsity of the vector to be calculated. The matrix calculation unit is used to organize the vectors to be calculated into a matrix form and then perform batch similarity calculation. The matrix calculation unit has a built-in hardware compressor. When the matrix obtained by organizing the vectors to be calculated is detected to be a sparse matrix, the hardware compressor is used to convert the matrix into a block compression format before calculation. The graph data caching unit includes a cache for storing the nearest neighbor graph formed by each data point in the vector database, and has a built-in access mode model. The access mode model is used to schedule the current vector computing task to the vector computing unit or the matrix computing unit based on the access frequency and edge connection density of each node in the nearest neighbor graph formed by each data point in the vector database, as well as the computing load data of the matrix computing unit and the vector computing unit.

2. The hardware accelerator as described in claim 1, characterized in that, When the sparsity of the vector to be calculated is higher than a preset threshold, the configurable vector register sets the vector processing width to a first vector width; otherwise, the configurable vector register sets the vector processing width to a second vector width; wherein the first vector width is greater than the second vector width.

3. A high-efficiency vector retrieval method based on the hardware accelerator as described in claim 1 or 2, characterized in that, include: Read each data point from the vector database, organize each data point into a vector data matrix, and perform matrix multiplication on the vector data matrix and the transpose of the vector data matrix based on the matrix calculation unit to obtain the vector distance between each pair of data points. For each data point, the vector calculation unit sorts other data points in ascending order of vector distance, selects the K points with the smallest vector distance as the neighbors of the corresponding data point, and constructs a nearest neighbor graph based on each data point and its neighbors. The nearest neighbor graph is then stored in the cache of the graph data cache unit. Specifically, for each node in the nearest neighbor graph, the node ID, vector, cache offset, and neighbor ID of the corresponding node are stored together in the cache. Receive a query vector from the user, and perform an approximate nearest neighbor search on the nearest neighbor graph based on the query vector to obtain multiple nodes that are most similar to the query vector.

4. The efficient vector retrieval method as described in claim 3, characterized in that, The nearest neighbor graph is stored in the cache of the graph data cache unit, and then the process further includes: Construct a reverse graph of the nearest neighbor graph and merge the reverse graph into the nearest neighbor graph. Based on the number of unfavorable paths from each node to its neighbors in the merged nearest neighbor graph, reorder and filter the neighbors of each node to obtain a nearest neighbor graph that satisfies the connectivity objective and update the cache. Specifically, for any node A and its neighbor node B, if there exists another path from node A to node B that includes an intermediate node C, and the sum of the distance from node A to node C and the distance from node C to node B is less than the distance from node A to node B, then the path including the intermediate node C is an unfavorable path from node A to node B.

5. The efficient vector retrieval method as described in claim 4, characterized in that, The reordering and filtering of neighbors for each node based on the number of unfavorable paths in the merged nearest neighbor graph specifically includes: For each node in the merged nearest neighbor graph, the neighbors of the corresponding node are reordered in descending order of the number of unfavorable paths from the corresponding node to each of its neighbors, and the neighbors at the top are retained based on the preset out-degree number.

6. The efficient vector retrieval method as described in any one of claims 3 to 5, characterized in that, The approximate nearest neighbor search includes a multi-round graph query process, where any round of graph query process includes: Step S1: Randomly select multiple starting nodes from the nearest neighbor graph, and calculate the distance between the query vector and the starting nodes based on the vector calculation unit; Step S2: Add the starting node to the candidate list. After initializing the top-M list, select the M nodes with the smallest distance to the query vector from the candidate list and place them in the top-M list. Step S3: Obtain the node that is added to the top-M list for the first time, and add all the neighbors of the node to the candidate list; Step S4: Determine the node that is added to the candidate list for the first time and mark it as a node to be calculated; construct a vector calculation task for all nodes to be calculated, and call the vector calculation unit or the matrix calculation unit to execute the vector calculation task to obtain the distance between all nodes to be calculated and the query vector; Step S5: Sort all nodes in the candidate list in ascending order of distance from the query vector, and compare the sorting result with the top-M list; if the top M nodes in the sorting result are inconsistent with the top-M list, update the top-M list and jump to step S3; otherwise, output the top-M list.

7. The efficient vector retrieval method as described in claim 6, characterized in that, The vector calculation task is executed by invoking the vector calculation unit or the matrix calculation unit to obtain the distances between all nodes to be calculated and the query vector, specifically including: Obtain the current access frequency and edge connection density of each node in the nearest neighbor graph, as well as the current computational load data of the matrix calculation unit and the vector calculation unit; Using the access pattern model of the graph data caching unit, based on the current access frequency and edge connection density of each node in the nearest neighbor graph, as well as the current computational load data of the matrix computation unit and the vector computation unit, the vector computation task is scheduled to the vector computation unit or the matrix computation unit, so that the vector computation task is executed based on the scheduled computation unit, and the distance between all nodes to be computed and the query vector is obtained.

8. A high-efficiency vector retrieval device, characterized in that, include: The nearest neighbor graph construction unit is used to read each data point in the vector database, organize the data points into a vector data matrix, and perform matrix multiplication on the vector data matrix and its transpose based on the matrix calculation unit to obtain the vector distance between each pair of data points. For each data point, the vector calculation unit sorts other data points in ascending order of vector distance, selects the K points with the smallest vector distance as the neighbors of the corresponding data point, and constructs a nearest neighbor graph based on each data point and its neighbors. The nearest neighbor graph is then stored in the cache of the graph data caching unit. Specifically, for each node in the nearest neighbor graph, the node ID, vector, cache offset, and neighbor ID of the corresponding node are stored together in the cache. An approximate nearest neighbor search unit is used to receive a query vector from a user, perform an approximate nearest neighbor search on the nearest neighbor graph based on the query vector, and obtain multiple nodes that are most similar to the query vector.

9. An electronic device, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any one of claims 3-7.

Images