Method and apparatus for querying for similar vectors in a candidate vector set

Abstract

The application discloses a method for querying candidate vectors similar to a to-be-queried vector in a candidate vector set, comprising: obtaining a group of to-be-queried vectors; for each to-be-queried vector in the group of to-be-queried vectors, querying a first predetermined number of candidate vector subsets closest to the to-be-queried vector; generating a plurality of common calculation results based on a set of center vector parts of candidate vectors in the first predetermined number of candidate vector subsets and a set of residual vector parts, and storing the plurality of common calculation results; generating a pre-calculation result based on the group of to-be-queried vectors and the set of residual vector parts, and storing the pre-calculation result; and for each to-be-queried vector in the group of to-be-queried vectors, determining a second predetermined number of candidate vectors similar to the to-be-queried vector from candidate vectors in the corresponding first predetermined number of candidate vector subsets based on the stored pre-calculation result and common calculation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and more particularly, to a method and device for querying similar vectors in a candidate vector set. BACKGROUND

[0002] Similarity query of vectors needs to be performed by high-performance computing devices, and therefore is currently mainly completed by data processing devices with strong computing capability such as central processing units and graphic processing units. However, central processing units and graphic processing units are usually designed to meet the requirements of general computing, and have poor speciality for this specific application of vector similarity query, and no additional hardware optimization is performed for vector similarity query.

[0003] Therefore, it is necessary to provide a device and method capable of efficiently performing similarity query of vectors. SUMMARY

[0004] One object of the present application is to solve the technical problem of low processing efficiency caused by poor speciality of existing devices for similarity vector query.

[0005] In one aspect of the present application, a method for querying candidate vectors similar to a to-be-queried vector in a candidate vector set is provided, the candidate vector set includes a plurality of candidate vectors, each candidate vector is processed via quantization and represented as including a center vector part and a residual vector part, and the candidate vector set includes a plurality of candidate vector subsets, the method includes: obtaining a group of to-be-queried vectors; for each to-be-queried vector in the group of to-be-queried vectors, querying a first predetermined number of candidate vector subsets closest to the to-be-queried vector; generating a plurality of common calculation results based on a set of center vector parts and a set of residual vector parts of candidate vectors in the first predetermined number of candidate vector sets, and storing the plurality of common calculation results; generating a pre-computation result based on the group of to-be-queried vectors and the set of residual vector parts, and storing the pre-computation result; and for each to-be-queried vector in the group of to-be-queried vectors, determining a second predetermined number of candidate vectors similar to the to-be-queried vector in the first predetermined number of candidate vector subsets based on the stored pre-computation result and common calculation results.

[0006] In some embodiments, the generation of the plurality of common calculation results is performed offline, and the generation of the pre-computation result is performed online.

[0007] In some embodiments, the pre-computation result is stored as a pre-computation result table in the form of a lookup table.

[0008] In some embodiments, the step of determining the second predetermined number of candidate vectors similar to the to-be-queried vector from the corresponding first predetermined number of candidate vector subsets based on the stored pre-computed results, the public computed results comprises: calculating the distance between the to-be-queried vector and each corresponding candidate vector for each to-be-queried vector; and sorting the candidate vectors based on the calculated distances to select the second predetermined number of candidate vectors closest to the to-be-queried vector as the second predetermined number of candidate vectors similar to the to-be-queried vector.

[0009] In some embodiments, the center vector part, the residual vector part and the to-be-queried vector are each divided into M segments based on the IVF-PQ algorithm, where M is a natural number greater than 1, and the public computed results are represented as: (xq-C i ) 2 + 2 * (C 2 + 2 * (C i | pq_centroids(k, l)), where xq represents the to-be-queried vector, pq_centroids(k, l) represents the residual vector part, l represents the lth segment of the residual vector part, l = 1...M, and k represents the quantization value of the lth segment; Ci represents the ith center vector part associated with the candidate vector, and i is a natural number.

[0010] In some embodiments, the pre-computed results are represented as: -2(xq|pq_centroids(k, l)).

[0011] In another aspect of the present application, a similarity searching device is also provided for a method of searching for a candidate vector similar to a query vector from a candidate vector set, the candidate vector set comprising a plurality of candidate vectors, each candidate vector being processed via a quantization process and represented as comprising a center vector part and a residual vector part, and the candidate vector set comprising a plurality of candidate vector subsets, the similarity searching device comprising: a computing module for computing distances between the query vector and the candidate vector subsets respectively, and computing similarity between the candidate vectors and the query vector; and a sorting module for sorting the computed distances between the query vector and the candidate vector subsets, and the computed similarity between the candidate vectors and the query vector; wherein the computing module computes the distances and the similarity by: obtaining a set of query vectors; for each query vector in the set of query vectors, computing distances between each candidate vector subset thereof and providing to the sorting module; generating a plurality of common computation results based on a set of center vector parts of candidate vectors in the first predetermined number of candidate vector sets, and a set of residual vector parts, and storing the plurality of common computation results; generating a pre-computed result based on the set of query vectors and the set of residual vector parts, and storing the pre-computed result; and for each query vector in the set of query vectors, determining a second predetermined number of candidate vectors similar to the query vector from the first predetermined number of candidate vector subsets based on the stored pre-computed result, common computation results.

[0012] For the query method and device of similar vectors of the embodiments of the present application, when computing the similarity between each candidate vector and the query vector, only the pre-computed common computation results and pre-computed result need to be queried, and no repeated computation is needed, which simplifies the complex vector computation into a query operation, thereby effectively improving the computation efficiency.

[0013] The above is a summary of the present application, which may have simplified, generalized and omitted details, and therefore those skilled in the art should recognize that this section is only illustrative and is not intended to limit the scope of the present application in any way. This summary section is neither intended nor used to identify key or essential features of the claimed subject matter, nor used to build a base for determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0014] The above and other features of the present application will become more fully understood from the following detailed description and the accompanying drawings, wherein: It is to be understood that the drawings are only schematic and are therefore not to scale, and that they are merely intended to conceptual illustrate the application and to provide a conceptual understanding of the application. The combination of features shown in the drawings is intended to explain the principles of the application and the application is not limited to the combination shown in the drawings. In the drawings:

[0015] Figure 1 An apparatus 100 for querying similar vectors to a to-be-queried vector in a candidate vector set is shown according to one embodiment of the present application;

[0016] Figure 2 An example of the decoding calculator 114 in the querying apparatus 100 is shown. Figure 1 An example of the decoding calculator 114 in the querying apparatus 100 is shown.

[0017] Figure 3 An example of the ranking module 108 in the querying apparatus 100 is shown. Figure 1 An example of the ranking module 108 in the querying apparatus 100 is shown.

[0018] Figure 4 A method 200 for querying similar vectors to a to-be-queried vector in a candidate vector set is shown according to one embodiment of the present application. DETAILED DESCRIPTION

[0019] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments can be utilized, and other changes can be made, without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.

[0020] Faiss is a technique for processing large-scale multi-dimensional vector similarity retrieval (or proximity retrieval), which can train a plurality of original vectors and establish an associated index file. Generally speaking, after the index file is established, the similar vectors of a to-be-queried vector can be retrieved based on the index file, which can effectively improve the retrieval efficiency.

[0021] IVF (Inverted File System)-PQ (Product Quantizer) is a specific Faiss algorithm that can convert each original vector into a value represented by a centroid. In some examples, the IVF-PQ algorithm can also segment the original vectors and convert each segment into a value represented by a centroid. Taking a vector set containing multiple 128-dimensional vectors as an example, each 128-dimensional original vector can be divided into M segments (e.g., 4 segments) respectively as needed, and then the first segment of all original vectors can be clustered to obtain L centroids (e.g., 256 centroids), the second segment of all original vectors can also be similarly clustered to obtain L centroids, and so on until the Mth segment also obtains L centroids; for each segment of each original vector, its corresponding nearest centroid can be found. In this way, each original vector can be quantized from the original 128-dimensional vector into M numbers and M residuals, and each pair of numbers and residuals represents the number of the nearest centroid of a segment in the original vector and the distance between the segment in the original vector and the nearest centroid. In subsequent calculations, only the numbers of the encoded vectors need to be transmitted, so the amount of data to be transmitted is reduced. In some embodiments, the residuals can also be further quantized similarly.

[0022] It can be understood that after converting the original vectors into values represented by centroids, the vector set can be divided into multiple vector subsets based on different centroids, where each vector subset includes all vectors represented by a centroid, and more specifically, the vectors can be represented by a centroid and respective residuals, and the difference between the vectors lies in the values of the residuals.

[0023] In other cases, the original vectors can not be divided, but all original vectors can be directly clustered and quantized, and each quantized original vector can be represented as a pair of centroid and residual.

[0024] In the process of similarity retrieval, the similarity between a candidate vector in the vector set and the query vector needs to be calculated, i.e. the distance between the two needs to be calculated. In order to reduce the calculation amount of the similarity calculation between the query vector and the candidate vector in the vector set, the IVF-PQ algorithm can first calculate the top N cluster centers closest to the query vector. Since each cluster center in the top N cluster centers can represent a certain number of vectors (i.e. some vectors closest to the cluster center), these vectors respectively constitute a vector subset related to the corresponding cluster center (each vector subset can be represented by the corresponding cluster center and the respective residual), thereby a total of top N vector subsets are constituted. In this way, the similarity between the query vector and the vectors represented by the top N cluster centers can be calculated afterwards, i.e. the similarity between the query vector and the candidate vectors in the top N vector subsets represented by the top N cluster centers can be calculated. It can be seen that this retrieval method greatly reduces the calculation amount of the similarity calculation. In actual applications, the number N of cluster centers to be determined depends on the data used in the application, and those skilled in the art can adjust it according to the data situation, type of the vector set, and processing capacity of the hardware device, etc.

[0025] The inventors of the present application found that in the above-mentioned IVF-PQ algorithm, the distance calculation between the query vector and the cluster center can be represented by equation (1):

[0026] d(xq,xb[ib])≈d(xq,C i +pq_centroids(k,l))=(xq-C i -pq_centroids(k,l)) 2 =(xq-C i ) 2 +(pq_centroids(k,l)) 2 +2*(C i |pq_centroids(k,l))-2(xq|pq_centroids(k,l))equation (1) where xq represents the query vector, xb[ib] represents a candidate vector in the converted candidate vector set, the function d(a,b) represents the distance (i.e. similarity) between vectors a and b, C i represents the i-th cluster center (or center vector) associated with the candidate vector xb[ib], i.e. the cluster center corresponding to the vector subset to which the candidate vector belongs; and pq_centroids(k,l) represents the candidate vector xb[ib] and the cluster center C iwherein the residual vector Res and the query vector xq are each optionally divided into M segments (based on the IVF-PQ algorithm), M is a natural number and is able to divide the dimension of the candidate vector (i.e. so that each segment has an equal dimension), thus l represents the lth segment of the residual vector, and k represents the quantized value of the lth segment. The calculation of the expression (xq|pq_centroids(k, l)) represents that the M segments of the query vector will be multiplied with the M segments of the residual vector and then accumulated.

[0027] However, the two expressions (pq_centroids(k, l)) in the above equation (1) 2 +2*(C i |pq_centroids(k, l)) are independent of the query vector xq. In other words, the calculation results of these two expressions are the same for different query vectors, and the difference is only in the calculation results of the other two terms of equation (1). In practical applications, the query vectors are usually input in batches, that is, a plurality of query vectors are input in a batch, and then a part of the candidate vectors similar to these query vectors are queried in the candidate vector set, and after the query of this batch is completed, the next batch of query vectors is input and the same similarity calculation process is performed. Therefore, when calculating the similarity of each batch of query vectors, the first expression in equation (1) is usually also determined.

[0028] Although the last expression -2(xq|pq_centroids(k, l)) in equation (1) is related to the query vector xq, as described above, since each candidate vector is quantized to include a part represented by the residual vector, and the number of quantized residual vectors is usually limited, therefore, when calculating the similarity of each batch of query vectors, the calculation in this expression is usually repeated multiple times. Therefore, the result of -2(xq|pq_centroids(k, l)) can be calculated and stored in advance, and then the results are queried, so that the repeated calculation of this expression can be greatly reduced, which is beneficial to improve the processing efficiency in the entire query process of similar vectors.

[0029] Based on the above findings, the inventors of the present application designed a method for querying vectors similar to a query vector in a candidate vector set, and a device for implementing the method. The method and device can generate a plurality of public calculation results in advance based on each batch of input query vectors, and the center vector part C i and the residual vector part Res of the quantized vectors in the candidate vector set, wherein each public calculation result can include the expression represented by equation (2).

[0030] (xq-Ci ) 2 +(Res) 2 +2*(C i |Res) Equation (2)

[0031] In some embodiments, these pre-computed common computation results can be stored. Preferably, these common computation results can be pre-computed offline. In this way, in the subsequent query process of similar vectors, these stored common computation results can be called, so that only the last term expression -2(xq|pq_centroids(k,l)) (or represented as -2(xq|Res)) in Equation (1) related to the vector to be queried needs to be calculated.

[0032] As mentioned above, since the residual vector Res obtained after quantization of the candidate vector is also relatively determined, the last term expression in Equation (1) is also usually called multiple times. Accordingly, after receiving the vector to be queried, this term expression can also be calculated in advance as a pre-computed result, and stored for calling in the subsequent data processing process. It can be understood that, unlike the common computation part represented by Equation (2), the expression -2(xq|Res) is related to the vector to be queried, so it needs to be calculated online. In this way, when calculating the similarity between each candidate vector and the vector to be queried, only the pre-computed common computation result and the pre-computed result need to be queried to obtain, without repeated calculation, which simplifies the complex vector calculation to a query operation, thereby effectively improving the calculation efficiency.

[0033] Based on the above findings, the present application provides a method and device for querying similar vectors to a vector to be queried in a candidate vector set.

[0034] Figure 1 A device 100 for querying similar vectors to a vector to be queried in a candidate vector set according to an embodiment of the present application is shown.

[0035] As shown in Figure 1 , the query device 100 comprises a controller 102 for controlling the operation of other modules of the query device 100, including the flow of data between different modules, to realize the query calculation process of vector similarity.

[0036] The query device 100 further comprises one or more computing modules 104 configured to compute similarity between each candidate vector in the candidate vector set and the query vector, and output the similarity computation results. In some embodiments, the computing module 104 can also perform other similarity computation related computations, such as computing the distance between the query vector and each cluster center in the candidate vector set, or computing one or more expressions in equation (1). The computation results based on these expressions are part of the similarity computation results, such as common computation results or pre-computation results. In some cases, these common computation results and pre-computation results can be output by the computing module 104, such as to the ranking module 108.

[0037] In the IVF-PQ algorithm, the distance between each cluster center and the query vector can be computed first, and the distance computation results can be output to the ranking module 108, so that the ranking module 108 determines the top N cluster centers (i.e., the top N cluster centers) closest to the query vector by ranking.

[0038] Table 1. An example of distance between the query vector and the candidate vector set

[0039]

[0040] As shown in Table 1, it is assumed that the candidate vector set can be divided into four candidate vector subsets 1 to 4, each of which is associated with one of the cluster centers C1 to C4 and can have a different number of candidate vectors. For example, the candidate vector subset 1 has a cluster center C1 and includes three candidate vectors v11 to v13; the candidate vector subset 2 has a cluster center C2 and includes three candidate vectors v21 to v23; and so on. It can be understood that in this example, the number of candidate vector subsets included in the candidate vector set and the number of candidate vectors included in each subset are exemplary and can have other different values in actual applications. Although not necessary, in general, the number of candidate vectors included in each candidate vector subset is often significantly more than the number of most similar vectors to be determined subsequently, and the number of candidate vector subsets (cluster centers) varies depending on the application. For example, when it is necessary to query the most similar picture from a pet picture data set consisting of, for example, 500,000 pet pictures, different types of pets can be divided into different candidate vector subsets, and the number of candidate vector subsets depends on the types of pets and can include only a few, a dozen to several dozen; for another example, when it is necessary to query the most similar goods from a data set consisting of, for example, 50 million candidate goods, it can be necessary to divide the 50 million candidate goods into several hundred, several thousand or even tens of thousands of candidate vector subsets, which mainly depends on the number of combinations of various characteristics such as type, price, seller, etc.

[0041] Still referencing Figure 1 Each computation module 104 can be coupled to a storage module 106 to receive candidate vectors and query vectors from the storage module 106. In some embodiments, the storage module 106 may include two or more parts: a first part for caching candidate vectors and query vectors for provision to the computation module 104; and a second part for providing a cache region during computation by the computation module 104. It is understood that in some embodiments, the first and second parts of the storage module 106 can be dynamically defined; that is, in a first time period, the first part is used to cache vectors while the second part is used to provide a computation cache region; in a second time period after the computation of the first time period is completed, the first part can be used as a computation cache region while the second part is used to cache vectors; and so on, alternating between the two. This processing method can improve the overall processing efficiency of the device 100.

[0042] As mentioned above, the calculation module 104 is also coupled to the sorting module 108 to output similarity calculation results or other intermediate calculation results. In some embodiments, multiple calculation modules 104 can be coupled in parallel between the storage module 106 and the sorting module 108 to process data in parallel, which helps to improve the calculation and processing speed of the entire query device 100. Specifically, using multiple parallel calculation modules 104 helps to improve the ability of parallel computing. If the data input to the calculation module 104 can be reused each time, and multiple calculation modules 104 can share the data received from the sorting module 108, then unnecessary repetitive data input / output can be reduced. In other words, the parallel calculation modules 104 allow the similarity calculation task to be divided into several smaller subtasks, and each subtask can be processed by a different calculation module 104. In this way, the query device 100 can run multiple similarity calculation tasks required by the upper-level software simultaneously, which helps to improve the operating efficiency of the query device 100.

[0043] In some embodiments, the sorting module 108 can be a scalable sorting module, meaning it can be configured to operate in different sorting modes. Specifically, "scalable" means that the sorting module includes multiple identical or similar sub-modules, and the number of sub-modules participating in the sorting can be varied by modifying the serial / parallel coupling relationship between these sub-modules, thereby achieving sorting processing of different scales. For example, in a sorting application for implementing the IVF-PQ algorithm, the working mode of the sorting module 108 can be configured as top2k sorting, top4k sorting, top8k sorting, etc., that is, it can sort and determine the first 2k vectors, the first 4k vectors, or the first 8k vectors. In some embodiments, to achieve scalability, the sorting module 108 and the calculation module 104 can be configured to perform bidirectional data interaction. For example, the calculation result of the calculation module 104, after being sorted by the sorting module 108, can be selectively returned to the calculation module 104 for further calculation processing. The sorting mode can be set by, for example, a toggle switch, a data selector, or a similar switching module. The specific settings and switching of the sorting mode will be described in detail below.

[0044] Next, taking the similarity calculation shown in equation (1) as an example, and referring to Table 1, we will analyze... Figure 1 The process of the query device 100 executing the IVF-PQ algorithm will be further explained.

[0045] As mentioned earlier, query vectors are typically received in batches, and each batch may include multiple query vectors, such as the first batch of 5 query vectors xq_1, xq_2, ..., xq_5. For each query vector, the query device 100 can traverse each cluster center in the candidate vector set, and its calculation module 104 can calculate the first term of equation (1) (xq-C). i ) 2 This means calculating the distance between the query vector and each cluster center. After traversing all cluster centers, the calculation module 104 can provide the calculation result to the sorting module 108, which sorts and determines the N cluster centers (or 2N or other numbers, depending on the sorting mode of the sorting module 108) that are closest to the query vector.

[0046] Referring to Table 1, in a batch processing, 5 query vectors xq_1 to xq_5 are inputted, and the distances between the 5 query vectors and different cluster centers are different: the 2 cluster centers closest to the query vector xq_1 are C1 and C2 (i.e. topN is top2); the 2 cluster centers closest to the query vector xq_2 are C1 and C3; the 2 cluster centers closest to the query vector xq_3 are C2 and C3; the 2 cluster centers closest to the query vector xq_4 are C1 and C2; and the 2 cluster centers closest to the query vector xq_5 are C2 and C3. Therefore, when performing the subsequent similarity calculation, only the similarity between the candidate vectors included in the candidate vector subset corresponding to the 2 closest cluster centers and the query vector needs to be calculated. It can be understood that the distance relationship between the aforementioned cluster centers and the query vector is only exemplary, and in actual processing, the N (or other number) of cluster centers closest to different query vectors can be different, and the number of candidate vectors included in the vector subset corresponding to each cluster center can also be different.

[0047] The sorting result (e.g. the N or other number of closest cluster centers) sorted by the sorting module 108 can be stored in the register of the sorting module 108, and then provided to the control module 102. In some embodiments, the control module 102 can determine the query vector related to each candidate vector subset according to the sorting result cached in the register of the sorting module 108, and then perform the similarity calculation on the query vector according to the candidate vector subset. It can be understood that the query vector closest to each cluster center can also be different. For example, in the example shown in Table 1, the query vectors closest to the cluster center C1 are the query vectors xq_1, xq_2 and xq_4, the query vectors closest to the cluster center C2 are the query vectors xq_1, xq_4 and xq_5, the query vector closest to the cluster center C3 is the query vector xq_3, and the cluster center C4 has no query vector close to it. Performing the similarity calculation according to the candidate vector subset instead of the query vector can share the pre-computed result, which is beneficial to improve the calculation efficiency.

[0048] After determining the closest cluster centers (candidate vector subset), the device 100 can calculate the similarity between each candidate vector included in the candidate vector subset and the query vector received in the same batch according to Equation (1) pair by pair.

[0049] Specifically, as aforementioned, in the process of calculating the similarity, the common calculation result corresponding to the first, second and third terms of Equation (1) (i.e. Equation (2)) can be pre-calculated by the calculation module 104, and can be transferred and stored in the cache via the sorting module processing (e.g. Figure 2The computing module 104 can store all the pre-computed result tables for later processing calls. In some embodiments, the computing module 104 can store the pre-computed result tables in an internal cache 118 of the computing module 104.

[0050] Thus, the common computation results associated with the first three terms of the expression in Equation (1) and the pre-computed results associated with the last term of the expression have been pre-computed and stored. The pre-computed results associated with the last term of the expression can be stored as a look-up table for later calls.

[0051] The computing module 104 can then use the pre-stored computation results to compute the similarity between each pair of query vectors and candidate vectors. Specifically, the computing module 104 can query the pre-computed result tables for the corresponding pre-computed results based on the query vectors determined by the control module 102 for each subset of candidate vectors, and then accumulate the queried pre-computed results and the corresponding common computation results to compute the corresponding similarity computation results. Still referring to the example shown in Table 1, for the subset of candidate vectors 1, each of the three candidate vectors v11 to v13 has a corresponding residual vector portion; in computing the similarity between the query vector xq_1 and the candidate vector v11, the value of the last term of the expression in Equation (1) can be looked up in the pre-computed look-up table for the query vector xq_1 and all the residual vectors in the subset of candidate vectors 1 based on the residual vector portion of the candidate vector v11, and then the value is accumulated with the other common computation results to obtain the similarity between the query vector xq_1 and the candidate vector v11; similarly, in computing the similarity between the query vector xq_1 and the candidate vector v12, the value of the last term of the expression in Equation (1) can be looked up in the pre-computed look-up table for the query vector xq_1 and the residual vectors in the subset of candidate vectors 1 based on the residual vector portion of the candidate vector v12, and then the value is accumulated with the other common computation results to obtain the similarity between the query vector xq_1 and the candidate vector v12; and so on.

[0052] It can be seen that, by using the pre-stored common computation results and pre-computed results, the vector multiplication computation which originally consumes a large amount of resources is converted to look-up table computation, which can reduce a large amount of repeated computation operations and effectively improve the query speed.

[0053] Still referring to Figure 1As shown, the computing module 104 can include a data interface 110 that serves as an interface for the computing module 104 to interact with the storage module 106. In some embodiments, the data interface 110 can be designed to match the interface type of the storage module 106. Optionally, the data interface 110 can also be coupled to the ranking module 108 for receiving data, such as various ranking results, from the ranking module 108.

[0054] The computing module 104 includes a computing processor 112 that is configured to perform computation on data provided by the data interface 110. For example, the computing processor 112 can include one or more adders, multipliers, and accumulators for performing various computation processes. In some embodiments, the computing processor 112 can include a plurality of parallel coupled computing threads, each of which can include a series coupled adder, multiplier, and accumulator. The computing processor 112 can also include one or more buffers for buffering data from the data interface 110. The computing processor 112 can also include a register 116 that can store configuration parameters for configuring the computation process, state variables, or for storing common computation results or pre-computation results. The computation results generated by the computing processor 112 can be stored in a buffer 118 and subsequently provided to the ranking module 108.

[0055] The computing module 104 also includes a decoding calculator 114 that can be configured to include one or more lookup tables. For example, the pre-computation results generated by the computing processor 112 can be provided to the decoding calculator 114 and stored in the form of a lookup table. Upon query, the decoding calculator 114 can provide the selected decoding results to the ranking module 108. One advantage of using the decoding calculator 114 (lookup table performing computation) over using a computing processor similar to the computing processor 112 to perform addition, multiplication, and accumulation operations is that the results of generating the lookup table can result in substantial compression of data. In other words, after encoding by the lookup table, the data is assigned a corresponding identifier or number, and the subsequent computation process only needs to transmit the encoded identifier or number, which results in less data needing to be transmitted within the computing module 104, which greatly saves the time spent on data transmission and also saves the storage resources needed within the computing module 104 to store such data. It is also noted that although the decoding calculator 114 also needs processing time to perform the decompression operation, in the application scenario of vector speed computation, the overall time and resources are saved.

[0056] As described earlier, the input candidate vector and the query vector, after being encoded by the IVF-PQ algorithm or a similar algorithm, become a corresponding shorter vector. Each element in the vector is also compressed from a longer data format to a shorter data format (e.g., from 32-bit floating-point data format to 8-bit character data format). This data format conversion makes each element in the compressed vector correspond to a feature, and these features also exist in other vectors. In this way, the same features of different vectors can have the same results in subsequent calculations. In other words, many vector calculations in the subsequent calculation process have shared parts, that is, many operations only need to be performed once, for example, they are stored in a lookup table in advance for subsequent multiple calls. Therefore, the function of the decoding calculator 114 is not only to restore the data, but also to perform the corresponding calculations.

[0057] Figure 2 It shows Figure 1 An example of the decoding calculator 114 in the query device 100 shown.

[0058] like Figure 2 As shown, the decoding calculator 114 may include multiple parallel-coupled decoding threads 142_1, 142_2…142_n, where each decoding thread can respond to a decoding query request to provide the corresponding pre-calculated result, and the parallel coupling of multiple decoding threads can effectively improve decoding efficiency. In some embodiments, the lookup table in each decoding thread may include the same pre-calculated result table, so that different decoding threads can process decoding query requests in parallel. This can improve query and data processing speed. For example, for query vectors to be processed in the same batch, a set of pre-calculated result tables corresponding to these query vectors can be pre-generated, and each set of pre-calculated result tables will be stored in each decoding calculator 114. In this way, during subsequent query processing, different candidate vectors can be provided to the decoding calculators in different calculation modules 104, thereby comparing these candidate vectors with multiple query vectors in parallel.

[0059] In some embodiments, each decoding thread may further include a buffer 144 and an addition tree 146, wherein the buffer is used to cache data, such as the values ​​of the first three terms in equation (1); and the addition tree 146 is used to calculate the search results output by multiple lookup tables and their sum with the first three terms. In this way, the addition tree 146 can directly output the similarity calculation results to the sorting module coupled to it.

[0060] Figure 3 It shows Figure 1 An example of the sorting module 108 in the query device 100 shown.

[0061] like Figure 1 and3 As shown, the sorting module 108 can include a register 182, a buffer 184, a plurality of data queues 186, a plurality of sorting networks 188, a plurality of merging networks 190, and a plurality of data selectors 192. The register 182 is configured to store configuration parameters, data, and partial computation results. The sorting network 188 is configured to sort M data received thereby, while the merging network is configured to receive two or more groups of M data and sort them together. In some embodiments, the register 182 can be coupled to the controller 102 to receive configuration parameters therefrom and perform sorting under the control of the controller 102. The buffer 184 is coupled to the computation module 104, e.g., to the data interface 110 and the decoding calculator 114 and the buffer 118 of the computation module 104, such that via the buffer 184, the sorting module 108 can provide sorting results to the data interface 114 of the computation module 104 and receive similarity computation results output by the decoding calculator 114 and the computation processor 112 (via the buffer 118).

[0062] Each data queue 186, sorting network 188, and merging network 190 of the sorting module 108 is coupled in series to sort a plurality of similarity computation results received thereby. In some embodiments, the data queue 186 can be configured as a first-in-first-out queue. In practice, the sorting network 188 is typically a hardware device with specific sorting capability, e.g., the sorting network 188 can be a top2k sorting network that can sort vectors and determine the top 2k vectors; accordingly, the merging network 190 and the data selector 192 coupled between the partial sorting network 188 and the corresponding merging network 190 can be utilized to merge the sorting results of the vectors, such that the top2k sorting network is extended to a top4k sorting network, a top8k sorting network, or a larger scale sorting network. In other words, the sorting mode of the sorting module 108 can be configured or adjusted. For example, for a 4-group top2k sorting network, the data flow thereof can be as shown in FIG. 2A, where each data queue 186, sorting network 188, and merging network 190 processes one group of M data, and the sorting results of the 4 groups of M data are output to the buffer 184. Figure 3 As shown, the sorting module 108 can be configured as a 4-group top2k sorting network, the data flow thereof can be as shown in FIG. 2A, where each data queue 186, sorting network 188, and merging network 190 processes one group of M data, and the sorting results of the 4 groups of M data are output to the buffer 184. Figure 3 As shown, each row of data queue 186, sorting network 188, multiplexer 192 (which can be omitted in the first row), and merging network 190 processes one group of M data, and the sorting results of the 4 groups of M data are output to the buffer 184. The multiplexer 192 can also be replaced by a switch or similar selection / switching module. Alternatively, the sorting module 108 can also be configured as a 2-group top4k sorting network, the data flow thereof can be as shown in FIG. 2B, where each data queue 186, sorting network 188, and merging network 190 processes one group of 2M data, and the sorting results of the 2 groups of 2M data are output to the buffer 184. Figure 3The first row of data queue 186, the sorting network 188 and the merging network 190 shown are processed by the merging network 190 of the first row to provide a multiplexer 192 of the second row, so that the multiplexer 192 can selectively output the results received from the merging network 190 of the first row and the results output by the sorting network 188 of the second row, and continue to be processed by the merging network 190 of the second row to merge the two sets of results, thereby achieving the sorting processing of top4k; alternatively, the sorting module 108 can also be configured as a group of top8k sorting networks, and the data flow thereof is to merge the sorting results of the first four rows. Those skilled in the art can understand that in some embodiments, the sorting module can be configured in other sorting modes according to actual application conditions, for example, it can be configured as a plurality of groups of top2k sorting networks, a plurality of groups of top4k sorting networks, or a combination of sorting networks of other sizes.

[0063] In some other embodiments, one or more of the sorting network 188 or the merging network 190 can also be bypassed, so as to only sort or only merge. For example, the merging network 190 can be bypassed, so that the sorting network 188 can directly output the sorting results to the buffer 184 and output by the buffer 184; for another example, the sorting network 188 can be bypassed, so that the data queue 186 can directly provide data to the merging network 190 and be merged by the merging network 190.

[0064] In some embodiments, the output end of each merging network 190 can also be coupled to the input end thereof, so that the operation results of the merging network can be selectively provided to the input end thereof.

[0065] The reason for using the merging network in the sorting module 108 is that for the sorting of two ordered sequences, the merging network occupies less resources and has shorter processing time. For example, in order to sort two sequences with a length of 10, a sorting network with an input and output of 10 can be used, but such a sorting network with a size of 10 is larger than a merging network with an input and output of 10, has more processing stages and longer operation time. Therefore, using the merging network is beneficial to improve the sorting processing speed, and even the processing speed of the entire similar vector query.

[0066] Figure 4 A method 200 for querying vectors similar to a to-be-queried vector in a candidate vector set is shown according to an embodiment of the present application. The candidate vector set includes a plurality of candidate vectors, each of which is processed via quantization and represented as including a center vector part and a residual vector part, and the candidate vector set includes a plurality of candidate vector subsets.

[0067] As Figure 4As shown, the method 200 includes: S202, obtaining a set of query vectors; S204, for each query vector in the set of query vectors, querying a first predetermined number of candidate vector subsets that are closest to it; S206, generating multiple common calculation results based on the set of central vector parts and the set of residual vector parts of the candidate vectors in the first predetermined number of candidate vector subsets, and storing the multiple common calculation results; S208, generating pre-calculation results based on the set of query vectors and the set of residual vector parts, and storing the pre-calculation results; and S210, for each query vector in the set of query vectors, using the stored pre-calculation results and common calculation results to determine a second predetermined number of candidate vectors similar to the query vector in the corresponding first predetermined number of candidate vector subsets.

[0068] In some embodiments, the generation of the plurality of common computation results is performed offline, while the generation of the pre-computation results is performed online.

[0069] In some embodiments, the pre-calculation results are stored as a pre-calculation result table in the form of a lookup table.

[0070] In some embodiments, the step of calculating a second predetermined number of candidate vectors similar to each of the query vectors in the group includes: for each query vector, calculating the distance between the query vector and its corresponding candidate vectors; and sorting the candidate vectors based on the calculated distances, so as to select the second predetermined number of candidate vectors that are closest to the query vector as the second predetermined number of candidate vectors similar to the query vector.

[0071] In some embodiments, the center vector portion, the residual vector portion, and the query vector are all divided into M segments based on the IVF-PQ algorithm, where M is a natural number greater than 1, and the common calculation result is represented as: (xq-C i ) 2 +(pq_centroids(k,l)) 2 +2*(C i |pq_centroids(k,l)), where xq represents the query vector, pq_centroids(k,l) represents the residual vector part, l represents the l-th segment of the residual vector part, l=1…M, and k represents the quantization value of the l-th segment; Ci represents the i-th centroid vector part associated with the candidate vector, where i is a natural number.

[0072] In some embodiments, the pre-calculated result is represented as: -2(xq|pq_centroids(k,l)).

[0073] It should be noted that, although several modules or sub-modules, steps or sub-steps of the method and apparatus for querying a similar vector in a candidate vector set are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. Indeed, according to embodiments of the present application, the features and functionalities of two or more modules described above can be embodied in one module. Conversely, the features and functionalities of one module described above can be further divided into modules for embodiment.

[0074] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements and steps, and the word "a" or "an" does not exclude a plurality. One element of an "a" or "one" can perform the functions of several features recited in the claims. Reference signs in the claims shall not be construed that the reference signs relate to the same element or to several elements throughout the claims.

Claims

1. A method for querying a candidate vector set for a candidate vector similar to a vector to be queried, characterized in that, The candidate vector set includes a plurality of candidate vectors, each candidate vector being processed via quantization and represented as including a center vector part and a residual vector part, and the candidate vector set includes a plurality of candidate vector subsets, the method comprising: obtaining a set of query vectors; for each query vector in the set of query vectors, querying a first predetermined number of candidate vector subsets closest to the query vector; generating a plurality of common computation results based on a set of center vector parts and a set of residual vector parts of candidate vectors in the first predetermined number of candidate vector subsets, and storing the plurality of common computation results in a cache; generating a pre-computation result based on the set of query vectors and the set of residual vector parts, and storing the pre-computation result as a pre-computation result table in a form of a lookup table; and for each query vector in the set of query vectors, determining a second predetermined number of candidate vectors similar to the query vector from the first predetermined number of candidate vector subsets based on the stored pre-computation result and common computation results, wherein the generation of the plurality of common computation results is performed offline, and the generation of the pre-computation result is performed online; wherein the center vector part, the residual vector part and the vector to be queried are all divided into M segments based on the IVF-PQ algorithm, M is a natural number greater than 1, and the common calculation result is represented as: wherein, xq denotes the vector to be queried, denotes the residual vector part, l denotes the lth segment of the residual vector part, l = 1…M, and k denotes the quantized value of the lth segment; Ci denotes the ith center vector part associated with the candidate vector, i is a natural number. wherein the pre-computed result is represented as: .

2. The method of claim 1, wherein, The step of determining the second predetermined number of candidate vectors similar to the query vector from the first predetermined number of candidate vector subsets based on the stored pre-computation result and common computation results comprises: for each query vector, calculating distances between the query vector and each corresponding candidate vector; and sorting the candidate vectors based on the calculated distances to obtain the second predetermined number of candidate vectors closest to the query vector as the second predetermined number of candidate vectors similar to the query vector.

3. A similarity search device for searching for a candidate vector similar to a vector to be searched for among a set of candidate vectors, characterized by comprising: The candidate vector set includes a plurality of candidate vectors, each candidate vector being processed via quantization and represented as including a center vector part and a residual vector part, and the candidate vector set includes a plurality of candidate vector subsets, the similarity retrieval device comprising: a calculation module configured to calculate distances between query vectors and candidate vector subsets, and to calculate similarities between candidate vectors and query vectors; and a sorting module configured to sort the distances between query vectors and candidate vector subsets, and the similarities between candidate vectors and query vectors; wherein the calculation module calculates the distances and the similarities by the following process: obtaining a set of query vectors; for each query vector in the set of query vectors, calculating distances between the query vector and each corresponding candidate vector subset and providing the distances to the sorting module; generating a plurality of common computation results based on a set of center vector parts and a set of residual vector parts of candidate vectors in the first predetermined number of candidate vector subsets, and storing the plurality of common computation results in a cache; generating a pre-computation result based on the set of query vectors and the set of residual vector parts, and storing the pre-computation result as a pre-computation result table in a form of a lookup table; and for each query vector in the set of query vectors, determining a second predetermined number of candidate vectors similar to the query vector from the first predetermined number of candidate vector subsets based on the stored pre-computation result and common computation results, wherein the generation of the plurality of common computation results is performed offline, and the generation of the pre-computation result is performed online. For each of the query vectors, based on the stored pre-computed results and the common computed results, determine a second predetermined number of candidate vectors in the corresponding first predetermined number of candidate vector subsets that are similar to the query vector; Wherein, the generation of the plurality of common computed results is performed offline, and the generation of the pre-computed results is performed online; wherein the center vector part, the residual vector part and the vector to be queried are all divided into M segments based on the IVF-PQ algorithm, M is a natural number greater than 1, and the common calculation result is represented as: wherein, xq denotes the vector to be queried, denotes the residual vector part, l denotes the lth segment of the residual vector part, l = 1…M, and k denotes the quantized value of the lth segment; Ci denotes the ith center vector part associated with the candidate vector, i is a natural number. wherein the pre-computed result is represented as: .

4. The similarity search device according to claim 3, wherein The computing module comprises: a computing processor for computing processing data received by the computing module; and a decoding calculator for receiving and storing the pre-computed results calculated by the computing processor, and in response to a decoding query request, finding the corresponding pre-computed results, and based on the pre-computed results and the common computed results, calculating the distance between the corresponding candidate vectors and the query vector as the similarity calculation results of the candidate vectors.

5. The similarity search device according to claim 4, wherein The decoding calculator comprises one or more decoding threads; Wherein each decoding thread comprises: a buffer for buffering data; a plurality of lookup tables for storing pre-computed results corresponding to different query vectors; and a summation tree for summing data.

6. The similarity search device according to claim 3, wherein The sorting module comprises: a plurality of data queues, a plurality of sorting networks and a plurality of merging networks, wherein each data queue, sorting network and merging network is coupled in series for sorting the received similarity calculation results.

7. The similarity search device according to claim 6, wherein The sorting module further comprises: a plurality of data selectors, each data selector for selectively coupling the output of one merging network to the input of another merging network, thereby allowing the sorting of similarity calculation results of the two merging networks to be merged.

8. The similarity search device according to claim 7, wherein Each merging network is also coupled to the computing module to provide the computing module with the first predetermined number of candidate vector subsets closest to the query vector.

Images