Neighbor vector query method of vector database and related product

Abstract

The invention provides a neighbor vector query method of a vector database and a related product, the method comprises the steps of establishing an index and performing query based on the index, and the step of establishing the index comprises the following steps: constructing a bottom layer graph so as to set the adjacency relationship among vectors in the bottom layer graph; clustering each vector to obtain a plurality of clusters, and obtaining a centroid vector of each cluster; the step of querying based on the index comprises the following steps: after a query vector is obtained, calculating the distance between each centroid vector and the query vector; and taking the centroid vector with the minimum distance from the query vector as an entry vector of the underlying graph so as to query a neighbor vector of the query vector from the underlying graph. According to the technical scheme, the advantages of the HNSW algorithm and the IVF algorithm can be integrated, and the accuracy of the query result is improved under the condition that the query efficiency is improved.

Description

Technical Field

[0001] The present invention relates to database technology, and in particular to a nearest neighbor vector query method for a vector database and related products. Background Art

[0002] The working principle of the vector database is to convert unstructured data such as images and texts into structured vector data, and store the vector data in the database. After receiving the user's image or text query instruction, the image or text to be queried will also be converted into vector data, and the vector data closest to the vector data will be found in the database and returned to the user, thereby improving the efficiency of the database in executing data queries.

[0003] In the process of searching for nearest neighbor vector data in a vector database in the prior art, commonly used query methods include query methods based on the HNSW (Hierarchical Navigable Small World) algorithm and query methods based on the IVF (Inverted File System) algorithm. The above-mentioned HNSW algorithm is an algorithm for efficient approximate nearest neighbor search in high-dimensional space. The algorithm first constructs a hierarchical graph structure, and then searches for the vector closest to the query vector from the top graph to the bottom graph in sequence, and uses the vector closest to the query vector in each layer of the graph as the entry point of the next layer of the graph, and uses the vector closest to the query vector in the bottom graph as the nearest neighbor vector of the query vector. Although the HNSW algorithm can accurately find the nearest neighbor vector of the query vector, the algorithm not only needs to establish a multi-layer graph, but also needs to connect each vector with its neighbor vector in each layer of the graph, so it needs to occupy a large amount of memory. And the HNSW algorithm needs to start from the top layer and search for the vector closest to the query vector layer by layer, so the query process consumes more time.

[0004] The IVF algorithm divides each vector into multiple categories. During the query process, the distance between the execution of each category and the query vector is first calculated. Then, from the category with the smallest distance between the centroid and the query vector, the vector closest to the query vector is found. This vector is the nearest neighbor vector of the query vector. The IVF algorithm does not need to build a multi-layer graph, so it can reduce the memory usage. However, the algorithm has an edge effect, that is, the query vector should belong to the first cluster, but because it is at the edge of the first cluster, the distance between the query vector and the centroid of the first cluster is greater than the distance between the query vector and the centroid of the second cluster, which leads to the search for the nearest neighbor vector of the query vector in the second cluster, resulting in poor accuracy of the query results. Summary of the invention

[0005] The purpose of the present invention is to provide a nearest neighbor vector query method and related products for a vector database, so as to improve the accuracy of nearest neighbor vector query.

[0006] In particular, in a first aspect, the present invention provides a method for querying neighbor vectors in a vector database, comprising:

[0007] The steps of establishing an index and performing a query based on the index, wherein

[0008] The steps of establishing an index include:

[0009] Constructing an underlying graph to set adjacency relations between vectors in the underlying graph;

[0010] Clustering the vectors to obtain a plurality of clusters, and obtaining a centroid vector of each cluster;

[0011] The step of performing a query based on the index comprises:

[0012] After obtaining the query vector, calculating the distance between each of the centroid vectors and the query vector;

[0013] The centroid vector with the smallest distance from the query vector is used as the entry vector of the underlying graph, so as to query the neighboring vectors of the query vector from the underlying graph.

[0014] Furthermore, the step of clustering the vectors to obtain a plurality of clusters includes:

[0015] Obtaining the initial centroid vector of each cluster;

[0016] The distances between each of the vectors and each of the initial centroid vectors are calculated respectively, and each of the vectors is assigned to a cluster corresponding to the initial centroid vector with the shortest distance.

[0017] Furthermore, the step of obtaining the centroid vector of each cluster comprises:

[0018] A preset cost function is used to calculate the cost function value of each of the vectors as the centroid vector of the cluster to which it belongs, so as to obtain the centroid vector of each of the clusters when the cost function value is minimized.

[0019] Furthermore, the step of constructing the underlying graph includes:

[0020] Acquire vectors to be added, and add the vectors to be added to the graph under construction according to a preset adding rule, and obtain the underlying graph after all the vectors to be added are added to the graph under construction;

[0021] The preset joining rules include:

[0022] Acquire a first set number of target vectors that are closest to the vector to be added from the map under construction;

[0023] Obtaining a neighbor vector of each of the target vectors from the map under construction, and obtaining a neighbor vector of the vector to be added in the map under construction according to each of the target vectors and its respective neighbor vectors;

[0024] The vector to be added is added to the graph under construction, and an adjacency relationship between the vector to be added and a corresponding neighbor vector is established.

[0025] Furthermore, the step of obtaining the neighbor vectors of the vector to be added in the graph under construction according to each of the target vectors and their respective neighbor vectors includes:

[0026] Adding a second set number of vectors, which are closest to the vector to be added among each of the target vectors and their respective neighbor vectors, to a result queue to initialize the result queue, wherein the result queue is used to store the neighbor vectors of the vector to be added in the graph under construction;

[0027] Storing each of the target vectors and their respective neighbor vectors in a candidate element queue corresponding to the target vector;

[0028] Obtain candidate vectors from the candidate element queue in sequence, and after obtaining the candidate vector each time:

[0029] Calculating the distance between the vector to be added and the candidate vector, and the maximum distance between each vector in the result queue and the candidate vector;

[0030] When the distance between the vector to be added and the candidate vector is less than the maximum distance, the candidate vector is added to the result queue.

[0031] Furthermore, before the step of acquiring a first set number of target vectors closest to the vector to be added from the graph under construction, the method further includes:

[0032] Detecting whether the user sets a value of the first set number;

[0033] If not, the value of the first set number is set as a default value.

[0034] Furthermore, the step of querying the neighbor vectors of the query vector from the underlying graph includes:

[0035] Initialize a query result set and add the entry vector to a candidate vector set and a visited vector set, wherein the query result set is used to store neighbor vectors of the query vector;

[0036] Obtaining the shortest distance between each vector in the candidate vector set and the query vector, and the shortest vector corresponding to the shortest distance; obtaining the farthest distance between each vector in the query result set and the query vector, and the farthest vector corresponding to the farthest distance;

[0037] When the closest distance is smaller than the farthest distance, the closest vector is deleted from the candidate vector set, and a neighbor vector of the closest vector in the underlying graph is obtained;

[0038] Determining whether the neighbor vector has been added to the visited vector set;

[0039] If not, the neighbor vector is added to the visited vector set, and the neighbor vector whose distance to the query vector is less than the farthest distance is added to the candidate vector set and the query result set.

[0040] Furthermore, before the step of adding the neighbor vectors whose distance to the query vector is less than the farthest distance to the candidate vector set and the query result set, the method further includes:

[0041] Determine whether the query result set is fully stored;

[0042] If so, the vector in the query result set that has the largest distance from the query vector is deleted.

[0043] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for querying a neighbor vector of a vector database described in any one of the above items are implemented.

[0044] In a third aspect, the present invention provides a computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the computer program implements the steps of the neighbor vector query method of the vector database described in any one of the above items.

[0045] The neighbor vector query method of the vector database provided by the present invention only constructs the underlying graph according to the distance between each vector in the process of constructing the index, so that the time of building the graph can be reduced, the work efficiency of building the graph can be improved, and the memory usage of the hierarchical graph structure can be reduced; in the process of obtaining the neighbor vector of the query vector according to the constructed index, the entry vector of the underlying graph is obtained from the centroid vector of each cluster cluster to obtain the neighbor vector of the query vector from the underlying graph, which can avoid the influence of the edge effect on the query result compared with the IVF algorithm of the prior art that obtains the neighbor vector of the query vector from the cluster cluster, and can improve the accuracy of obtaining the neighbor vector of the query vector. Therefore, the technical solution of the present invention can combine the advantages of the HNSW algorithm and the IVF algorithm, and improve the accuracy of the query result while improving the query efficiency.

[0046] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0048] Figure 1 is a schematic flow chart of a method for querying neighbor vectors in a vector database according to an embodiment of the present invention;

[0049] Figure 2 is a schematic flow chart of building an index in a neighbor vector query method according to an embodiment of the present invention;

[0050] Figure 3 is a schematic flow chart of obtaining a neighbor vector of a query vector in a neighbor vector query method according to an embodiment of the present invention;

[0051] Figure 4 is a schematic diagram of obtaining a neighbor vector of a query vector in a neighbor vector query method according to an embodiment of the present invention;

[0052] Figure 5 is a schematic flow chart of clustering each vector into a plurality of clusters in a neighbor vector query method according to an embodiment of the present invention;

[0053] Figure 6 is a schematic flow chart of obtaining the centroid vector of each cluster in a neighbor vector query method according to an embodiment of the present invention;

[0054] Figure 7is a schematic flow chart of adding a vector to be added to a graph under construction in a neighbor vector query method according to an embodiment of the present invention;

[0055] Figure 8 is a schematic flow chart of obtaining neighbor vectors of a vector to be added in the above-mentioned graph under construction in a neighbor vector query method according to an embodiment of the present invention;

[0056] Fig. 9 is a schematic flow chart of obtaining a first set number in a neighbor vector query method according to an embodiment of the present invention;

[0057] Fig.10 is a schematic flow chart of obtaining a neighbor vector of a query vector from an underlying graph in a neighbor vector query method according to an embodiment of the present invention;

[0058] Fig.11 is a schematic flow chart of obtaining the neighbor direction of a query vector from an underlying graph in a neighbor vector query method according to another embodiment of the present invention;

[0059] Fig.12 is a schematic diagram of a computer program product according to an embodiment of the present invention;

[0060] Fig.13 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Refer to the following Figures 1 to 13 To describe a neighbor vector query method and related products of a vector database in an embodiment of the present invention. In the description of this embodiment, it should be understood that in the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. When a feature "includes or contains" one or some of the features it covers, unless otherwise specifically described, this indicates that other features are not excluded and other features may be further included.

[0062] This embodiment provides a solution for a nearest neighbor vector query method of a vector database. The nearest neighbor vector query method combines the HNSW algorithm and the IVF algorithm, which can not only solve the problems of slow mapping speed, large memory usage and slow search speed of the HNSW algorithm, but also solve the problem of poor query result accuracy caused by the edge effect of the IVF algorithm, thereby improving the accuracy of the query results while improving the query efficiency.

[0063] See also Figure 1 , Figure 1 What is shown is a schematic flow chart of the nearest neighbor vector query method of the vector database of this embodiment. Figure 1In the process shown, the neighbor vector query method of this embodiment may generally include the following steps:

[0064] Step S102: construct an index;

[0065] Step S104: based on the constructed index, obtain the neighbor vector of the query vector.

[0066] In this embodiment, the method for constructing the index in step S102 is as follows: Figure 2 As shown, the following steps are included:

[0067] Step S112: adding each vector to the underlying graph, and setting the adjacency relationship between each vector in the underlying graph;

[0068] Step S114: clustering each vector to obtain a plurality of clusters, and obtaining the centroid vector of each cluster.

[0069] In the prior art, the query method based on the HNSW algorithm needs to construct a hierarchical graph structure, which includes multiple layers of graph structures, wherein the graph structure at the bottom layer is the bottom layer graph of HNSW, and the bottom layer graph contains the most vectors.

[0070] In the above step S112, the bottom-level graph construction method of HNSW can be used to construct only the bottom-level graph according to the distance between each vector, thereby not only reducing the time for building the graph and improving the work efficiency of building the graph, but also reducing the memory usage of the graph structure.

[0071] In this embodiment, setting the adjacency relationship between each vector in the underlying graph refers to setting the neighbor vector of each vector in the underlying graph, where the neighbor vector of a vector refers to the vector closest to the vector, and the distance can be the Euclidean distance, cosine distance or negative inner product distance between the query vector and the centroid vector of each cluster, or other distance defined by the user.

[0072] In the above step S114, clustering calculations may be performed on each vector using a preset clustering algorithm to cluster each vector into a plurality of clusters; and then calculations are performed on each cluster to obtain a centroid vector of each cluster.

[0073] In the above step S104, the method for obtaining the neighbor vector of the query vector is as follows: Figure 3 As shown, the following steps are included:

[0074] Step S122: after obtaining the query vector, calculating the distance between the centroid vector of each cluster and the query vector;

[0075] Step S124: The centroid vector with the smallest distance to the query vector is used as the entry vector of the underlying graph, and the neighbor vector of the query vector is obtained from the underlying graph according to the entry vector.

[0076] In the above step S122, a preset distance algorithm can be obtained to calculate the distance between the query vector and the centroid vector of each cluster. The distance can be the Euclidean distance, cosine distance or negative inner product distance between the query vector and the centroid vector of each cluster, or other distance defined by the user.

[0077] In the above step S124, after the entry vector of the bottom layer graph is obtained, the bottom layer graph query method in the HNSW algorithm may be used to query the bottom layer graph starting from the entry vector to obtain the neighbor vector of the query vector.

[0078] by Figure 4 Taking the application scenario shown as an example, after clustering each vector, two clusters are obtained, where the centroid vector of the first cluster is vector P1, and the centroid vector of the second cluster is vector P2; after obtaining the query vector p, the distance between vector P1 and the query vector p is calculated to be L1, and the distance between vector P2 and the query vector p is L2, and L1 is less than L2, so vector P1 is used as the entry vector to find the neighbor vector of the query vector p in the underlying graph.

[0079] According to the above content, the technical solution of this embodiment only constructs the underlying graph according to the distance between each vector in the process of constructing the index, thereby reducing the time of building the graph, improving the work efficiency of building the graph, and reducing the memory usage of the hierarchical graph structure; in the process of obtaining the neighbor vector of the query vector according to the constructed index, the entry vector of the underlying graph is obtained from the centroid vector of each cluster cluster to obtain the neighbor vector of the query vector from the underlying graph. Compared with the IVF algorithm obtaining the neighbor vector of the query vector from the cluster cluster, the influence of the edge effect on the query result can be avoided, and the accuracy of obtaining the neighbor vector of the query vector can be improved. Therefore, the neighbor vector query method of this embodiment can combine the advantages of the HNSW algorithm and the IVF algorithm, and improve the accuracy of the query result while improving the query efficiency.

[0080] In some embodiments of the present invention, in the above step S114, the method of clustering each vector into a plurality of clusters is as follows: Figure 5 As shown, the following steps are included:

[0081] Step S202: Obtaining the initial centroid vector of each cluster;

[0082] Step S204: Calculate the distance between each vector and each initial centroid respectively, so as to obtain the initial centroid vector to which each vector is closest;

[0083] Step S206: assign each vector to the cluster corresponding to the initial centroid vector that is closest to it.

[0084] In the above step S202, the user can set the number of clusters according to actual needs; after the number of clusters is obtained according to the user's setting data, the number of vectors is randomly selected from each vector, and the selected vectors are used as the initial centroid vectors of each cluster.

[0085] In the above step S204, a preset distance algorithm may be used to calculate the distance between each vector and each initial centroid vector.

[0086] Taking one of the vectors as an example, after using the preset distance algorithm to calculate the distance between the vector and each initial centroid vector, through comparison, it is found that the distance between the vector and the initial centroid vector of the nth cluster is the smallest, then the initial centroid vector of the nth cluster is used as the initial centroid vector closest to the vector, and in the above step S206, the vector is assigned to the nth cluster.

[0087] And so on, until each vector is assigned to a cluster, thus achieving clustering of each vector.

[0088] Through the technical solution of this embodiment, after obtaining the initial centroid vector of each cluster, each vector can be quickly clustered according to the distance between each vector and each initial centroid vector, so as to improve the working efficiency of clustering each vector.

[0089] In some embodiments of the present invention, in the above step S114, the method for obtaining the centroid vector of each cluster is as follows: Figure 6 As shown, the following steps are included:

[0090] Step S212: using a preset cost function, calculating the cost function value of each vector as the centroid vector of the cluster to which it belongs;

[0091] Step S214: Obtain the centroid vector of each cluster when the cost function value is minimum.

[0092] Taking an application scenario as an example, let the centroid vector of the i-th cluster be C i , Q i is the Qth cluster in the i-th cluster i vector, the cost function value is c, then

[0093]

[0094] Where N is the total number of clusters, |QC i | represents vector C i With vector Qi The distance between.

[0095] Then, the cost function values ​​of the vectors as the centroid vector of the cluster are compared to obtain the vector with the smallest cost function value, and the vector is used as the centroid vector of the cluster.

[0096] The method for obtaining the centroid vector of a cluster in the prior art is to take the average value of each vector in the cluster as the centroid vector, but the centroid vector may not be an existing vector and may not exist in the underlying graph. Therefore, the centroid vector obtained by the centroid vector acquisition method in the prior art cannot be used as the entry vector of the underlying graph.

[0097] In the technical solution of this embodiment, a centroid vector is selected from the existing vectors in each cluster according to a preset cost function, instead of taking the average value of each vector in each cluster as the centroid vector. Therefore, in the process of obtaining the neighbor vector of the query vector, the centroid vector closest to the query vector can be used as the entry vector of the underlying graph, so as to achieve the purpose of improving the reliability of obtaining the neighbor vector of the query vector.

[0098] In some embodiments of the present invention, the method of adding each vector to the underlying graph in step S112 includes:

[0099] Get a vector to be added, and add it to the graph under construction according to the preset adding rules, until all vectors are added to the graph under construction, and the construction of the underlying graph is completed.

[0100] In this embodiment, the vector to be added refers to a vector that has not been added but will be added to the map being built. The method of adding the vector to be added to the map being built according to the preset adding rule is as follows: Figure 7 As shown, the following steps are included:

[0101] Step S222: acquiring a first set number of target vectors closest to the vector to be added from the map under construction;

[0102] Step S224: Obtain the neighbor vectors of each target vector from the map under construction, and obtain the neighbor vectors of the vector to be added in the map under construction according to each target vector and the neighbor vectors of the target vector.

[0103] In the above step S222, a preset distance algorithm may be used to calculate the distance between each vector in the map and the vector to be added, so as to obtain a first set number of vectors in the map with the smallest distance to the vector to be added, and the vector is the target vector.

[0104] Since the neighbor vectors of each vector have been set in the map construction, the neighbor vectors of each target vector can be directly obtained from the map construction in the above step S224, and the neighbor vectors of the vector to be added exist in each target vector and the neighbor vectors of each target vector. Therefore, in this embodiment, the neighbor vectors of the vector to be added can be obtained from each target vector and the neighbor vectors of each target vector; then the vector to be added is added to the map construction, and the vector to be added is connected to the neighbor nodes of the vector to be added to set the adjacency relationship between the vector to be added and other vectors in the above map construction.

[0105] After all the vectors to be added are added to the graph under construction according to the above preset adding rules, the underlying graph containing all the vectors and the adjacency relationship between the vectors can be obtained.

[0106] Through the technical solution of this embodiment, the neighbor vectors of the vector to be added in the map can be obtained from the first set number of target vectors in the map that are closest to the vector to be added, and the neighbor vectors of each target vector, thereby improving the accuracy of obtaining the neighbor vectors of the vector to be added in the map. After the vector to be added is added to the map to be built, the vector to be added and its neighbor vectors can be connected to improve the reliability of constructing the underlying map.

[0107] In some embodiments of the present invention, the method for obtaining the neighbor vectors of the vector to be added in the above-mentioned graph under construction in the above-mentioned step S224 is as follows: Figure 8 As shown, the following steps are included:

[0108] Step S232: Initialize a result queue, which is used to store neighbor vectors of the vector to be added in the above-mentioned graph under construction;

[0109] Step S234: establishing a candidate element queue corresponding to each target vector, and storing each target vector and the neighbor vector of each target vector in the corresponding candidate element queue respectively;

[0110] Step S236: sequentially obtain candidate vectors from each candidate element queue, and if the obtained candidate vectors meet a preset condition, add the obtained candidate vectors to the result queue.

[0111] In the above step S232, a second set number of vectors closest to the vector to be added among the target vector and the neighboring vectors of the target vector may be stored in the result queue to initialize the result queue.

[0112] According to user needs, one or more vectors can be obtained from the graph under construction and added to the result queue to initialize the result queue.

[0113] In the above step S234, a first set number of candidate element queues may be set, and each candidate element queue corresponds to a target vector. After obtaining each target vector, each target vector is stored in the corresponding candidate element queue; similarly, after obtaining the neighbor vectors of each target vector, each neighbor vector is added to the corresponding candidate element queue.

[0114] In the above step S236, the method for determining whether the obtained candidate vector meets the preset condition is as follows, including:

[0115] Calculate the distance between the vector to be added and the candidate vector, as well as the maximum distance between each vector in the result queue and the candidate vector;

[0116] It is determined whether the distance between the vector to be added and the candidate vector is less than the maximum distance. If so, the candidate vector is used as a neighbor vector of the vector to be added.

[0117] Through the technical solution of this embodiment, a candidate element queue can be used to store each target vector and the neighbor vector of each target vector, so that the neighbor vector of the vector to be added can be selected from each candidate queue to improve the convenience of obtaining the neighbor vector of the vector to be added.

[0118] In some embodiments of the present invention, before obtaining a first set number of target vectors closest to the vector to be added from the map under construction in the above step S222, a step of obtaining the first set number is also included.

[0119] In this embodiment, the method for obtaining the first set quantity is as follows: Fig. 9 As shown, the following steps are included:

[0120] Step S242: Detect whether the user has set a first set number of values;

[0121] If not, execute step S244;

[0122] Step S244: Setting the value of the first set number as a default value.

[0123] In this embodiment, the first set number may be a configurable parameter of the vector database, and the user may set the value of the first set number according to the requirement. The vector database may obtain and store the value of the first set number set by the user.

[0124] In the above step S242, it can be detected whether a value of the first set number is stored to determine whether the first set number set by the user is received, that is, if the value of the stored first set number is detected, it is determined that the user has set the value of the first set number, and the target vector is obtained according to the value of the first set number in the above step S222.

[0125] On the contrary, if the stored value of the first set number is not detected, it is determined that the user has not set the first set number, and the value of the first set number is set to a default value, and the target vector is obtained according to the value of the first set number in the above step S222. In this embodiment, the default value is preferably 64.

[0126] Through the technical solution of this embodiment, the first set number can be set as a configurable parameter of the vector database to improve the flexibility of constructing the underlying graph. In addition, a default value of the first set number is also set in this embodiment, so that when the user does not set the first set number, the first set number uses the default value to obtain the target vector to be added to the vector, thereby improving the reliability of constructing the underlying graph.

[0127] In some embodiments of the present invention, the method for obtaining the neighbor vector of the query vector from the underlying graph in the above step S124 is as follows: Fig.10 As shown, the following steps are included:

[0128] Step S302: Initialize the query result set, and add the entry vector to the candidate vector set and the visited vector set;

[0129] Step S304: obtaining a nearest vector in the candidate vector set that is closest to the query vector, and a shortest distance between the nearest vector and the query vector;

[0130] Step S306: obtaining the farthest vector in the query result set that is farthest from the query vector, and the farthest distance between the farthest vector and the query result;

[0131] Step S308: determining whether the shortest distance is greater than the farthest distance;

[0132] If yes, execute step S322; if no, execute step S310;

[0133] Step S310: deleting the nearest vector from the candidate vector set, and obtaining the neighbor vectors of the nearest vector in the underlying graph;

[0134] Step S312: judging whether there is an unvisited vector in each neighboring vector of the bottom graph of the above-mentioned nearest vector according to the visited vector set;

[0135] If yes, execute step S314; if no, execute step S320;

[0136] Step S314: taking the vector of the nearest vector that has not been visited in each neighboring vector of the bottom graph as an unvisited vector, and adding the unvisited vector to the visited vector set;

[0137] Step S316: Calculate the distance between each unvisited vector and the query vector;

[0138] Step S318: adding the vectors whose distance to the query vector is less than the maximum distance among the unvisited vectors to the candidate vector set and the query result set;

[0139] Step S320: determining whether the candidate vector set is empty;

[0140] If yes, execute step S322; if no, return to step S304;

[0141] Step S322: End the query and use the vectors in the query result set as neighbor vectors of the query vector.

[0142] In the technical solution of this embodiment, vectors that may be neighbor vectors of the query vector are stored in a candidate vector set, and then the nearest vector in the candidate vector set that is closest to the query vector is obtained; if the distance between the nearest vector and the query vector is less than the farthest distance between the nearest vector and the query vector in the query result set, or the candidate vector set is empty, it is determined that all neighbor vectors of the query vector have been added to the query result set, and thus the query is terminated.

[0143] In this embodiment, the visited vector set is used to store visited vectors. Therefore, in the above step S312, the neighbor vectors of the nearest vector in the underlying graph can be obtained one by one, and after each neighbor vector is obtained, if the obtained neighbor vector has been added to the visited vector set, it is determined that the neighbor vector is an visited vector; conversely, if the obtained neighbor vector has not been added to the visited vector set, it is determined that the neighbor vector is an unvisited vector.

[0144] The technical solution provided in this embodiment, on the one hand, screens out the vectors closest to the query vector as the neighbor vectors of the query vector according to the distances between each vector in the candidate vector set and the query result set and the query vector, thereby improving the accuracy of querying the query vector; on the other hand, in this embodiment, by setting the visited vector set, it is determined whether there are unvisited vectors in each neighbor vector of the nearest vector in the underlying graph, thereby preventing the vectors in the underlying graph from being repeatedly visited, thereby improving the work efficiency of querying the query vector.

[0145] In some embodiments of the present invention, the method for obtaining the neighbor vector of the query vector from the underlying graph in the above step S124 is as follows: Fig.11 As shown, before the above step S318, it also includes:

[0146] Step S402: determine whether the result vector set is fully stored;

[0147] If yes, execute step S404; if no, execute step S318;

[0148] Step S404: Delete the vector in the result vector set that is farthest from the query, and then continue to execute step S318.

[0149] In this embodiment, an upper threshold of the result vector set can be set in advance; in the above step S402, if the vector stored in the result vector set is less than the upper threshold, it is determined that the result vector set is not yet full; conversely, if the vector stored in the result vector set is equal to or greater than the upper threshold, it is determined that the result vector set is full.

[0150] To ensure that the number of neighbor vectors of the query vector is not greater than the upper threshold, this embodiment deletes the vector farthest from the query vector in the result vector set in step S404, thereby ensuring that the distance between the vectors in the result vector set and the query vector is the shortest.

[0151] Through the technical solution of this embodiment, it can be ensured that the neighbor vectors of the query vector obtained are closest to the query vector and their number is not greater than the upper threshold of the result vector set, thereby ensuring the accuracy and reliability of query vector queries.

[0152] The flow chart provided by the present embodiment is not intended to indicate that the operation of the method will be performed in any particular order, or that all operations of the method are included in all every case. In addition, the method may include additional operations. Within the scope of the technical ideas provided by the present embodiment method, additional changes can be made to the above method.

[0153] It should be understood that in some embodiments, each part can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0154] This embodiment also provides a computer program product 10 and a computer readable storage medium 20 . Fig.12 is a schematic diagram of a computer program product 10 according to an embodiment of the present invention, Fig.13 is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which, when executed by a processor 32, implements the steps of any of the above-mentioned neighbor vector query methods for a vector database. The computer-readable storage medium 20 stores the above-mentioned computer program 11 thereon, which, when executed by a processor 32, implements the steps of any of the above-mentioned neighbor vector query methods for a vector database.

[0155] The computer program 11 for performing the operation of the present invention may be an assembly instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-related instruction, a microcode, a firmware instruction, a state setting data, a configuration data of an integrated circuit, or a source code or an object code written in any combination of one or more programming languages ​​and process programming languages. The computer program 11 may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider via the Internet). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0156] In the description of this embodiment, the computer program product 10 is a related product including the computer program 11 .

[0157] For the purposes of the description of this embodiment, the computer-readable storage medium 20 is a tangible device capable of retaining and storing the computer program 11, which may be any device that can contain, store, communicate, propagate or transmit the program 11 for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples (a non-exhaustive list) of the computer-readable storage medium 20 include the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination of the foregoing.

[0158] At this point, those skilled in the art should recognize that, although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived based on the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A method for querying neighbor vectors in a vector database, characterized in that: include: The steps of establishing an index and performing a query based on the index, wherein The steps of establishing an index include: Constructing an underlying graph to set adjacency relations between vectors in the underlying graph; Clustering the vectors to obtain a plurality of clusters, and obtaining a centroid vector of each cluster; The step of performing a query based on the index comprises: After obtaining the query vector, calculating the distance between each of the centroid vectors and the query vector; The centroid vector with the smallest distance from the query vector is used as the entry vector of the underlying graph, so as to query the neighboring vectors of the query vector from the underlying graph.

2. The neighbor vector query method according to claim 1, characterized in that: The step of clustering the vectors to obtain a plurality of clusters comprises: Obtaining an initial centroid vector of each of the clusters; The distances between each of the vectors and each of the initial centroid vectors are calculated respectively, and each of the vectors is assigned to a cluster corresponding to the initial centroid vector with the shortest distance.

3. The neighbor vector query method according to claim 1 or 2, characterized in that: The step of obtaining the centroid vector of each cluster comprises: A preset cost function is used to calculate the cost function value of each of the vectors as the centroid vector of the cluster to which it belongs, so as to obtain the centroid vector of each of the clusters when the cost function value is minimized.

4. The neighbor vector query method according to claim 1, characterized in that: The step of constructing the underlying graph includes: Acquire vectors to be added, and add the vectors to be added to the graph under construction according to a preset adding rule, and obtain the underlying graph after all the vectors to be added are added to the graph under construction; The preset joining rules include: Acquire a first set number of target vectors that are closest to the vector to be added from the map under construction; Obtaining a neighbor vector of each of the target vectors from the map under construction, and obtaining a neighbor vector of the vector to be added in the map under construction according to each of the target vectors and its respective neighbor vectors; The vector to be added is added to the graph under construction, and an adjacency relationship between the vector to be added and a corresponding neighbor vector is established.

5. The method for querying neighbor vectors according to claim 4, characterized in that: The step of obtaining the neighbor vectors of the vector to be added in the graph under construction according to each of the target vectors and their respective neighbor vectors includes: Adding a second set number of vectors, which are closest to the vector to be added among each of the target vectors and their respective neighbor vectors, to a result queue to initialize the result queue, wherein the result queue is used to store the neighbor vectors of the vector to be added in the graph under construction; Storing each of the target vectors and their respective neighbor vectors in a candidate element queue corresponding to the target vector; Obtain candidate vectors from the candidate element queue in sequence, and after obtaining the candidate vector each time: Calculating the distance between the vector to be added and the candidate vector, and the maximum distance between each vector in the result queue and the candidate vector; When the distance between the vector to be added and the candidate vector is less than the maximum distance, the candidate vector is added to the result queue.

6. The neighbor vector query method according to claim 4, characterized in that: Before the step of acquiring a first set number of target vectors closest to the vector to be added from the graph under construction, the method further includes: Detecting whether the user sets a value of the first set number; If not, the value of the first set number is set as a default value.

7. The method for querying neighbor vectors according to claim 1, characterized in that: The step of querying the neighbor vectors of the query vector from the underlying graph includes: Initialize a query result set and add the entry vector to a candidate vector set and a visited vector set, wherein the query result set is used to store neighbor vectors of the query vector; Obtaining the shortest distance between each vector in the candidate vector set and the query vector, and the shortest vector corresponding to the shortest distance; obtaining the farthest distance between each vector in the query result set and the query vector, and the farthest vector corresponding to the farthest distance; When the closest distance is less than or equal to the farthest distance, the closest vector is deleted from the candidate vector set, and a neighbor vector of the closest vector in the underlying graph is obtained; Acquire unvisited vectors in the neighbor vector according to the visited vector set, and calculate the distance between each of the unvisited vectors and the query vector; The vectors whose distances from the query vector in the unvisited vectors are smaller than the farthest distance are added to the candidate vector set and the query result set.

8. The neighbor vector query method according to claim 7, characterized in that: Before the step of adding the neighbor vectors whose distance to the query vector is less than the farthest distance to the candidate vector set and the query result set, the step further includes: Determine whether the query result set is fully stored; If so, the vector in the query result set that has the largest distance from the query vector is deleted.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for querying neighbor vectors in a vector database as described in any one of claims 1 to 8 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for querying neighbor vectors in a vector database according to any one of claims 1 to 8 are implemented.

Images