An out-of-distribution vector retrieval method and device

Abstract

The present invention discloses an out-of-distribution vector retrieval method and device. The method first constructs a corresponding anchor vector for each base vector in the base vector set based on the query vector set, and constructs a fusion graph based on the base vector and the anchor vector. The fusion graph is used to search for the query vector to be retrieved, so as to obtain a result similar to the query vector to be retrieved. This method comprehensively considers the neighborhood relationship between the base vector set and the query vector set, and realizes adaptive balance through the fusion weight, which can effectively handle the out-of-distribution vector retrieval task and significantly improve the search accuracy and efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of vector retrieval, and particularly relates to an out-of-distribution vector retrieval method and device. Background Art

[0002] Vector retrieval is a fundamental module in many modern applications (e.g., search engines, large language models, recommendation systems). Vector retrieval algorithms build indexes that can describe the similarity relationships among the base vectors in the base vector set, providing a good trade-off between search latency and accuracy during the search phase.

[0003] However, the theory on which existing vector retrieval relies has an implicit premise, that is, the query vector set and the base vector set have the same distribution (i.e., the query vector set is in-distribution data). When the query vector set and the base vector set have different distributions (i.e., the query vector set is out-of-distribution data), the out-of-distribution query performance is worse than the in-distribution query by an order of magnitude or more. Taking the image library of a search engine as an example, when a user inputs an image as a query, it is an in-distribution query, and when the user inputs text as a query, it is an out-of-distribution query. The out-of-distribution query problem of vector retrieval causes users to take longer to obtain results and have inaccurate search results when searching for images by text. This is mainly because the neighbors of out-of-distribution queries lack clustering, that is, the neighbors of the neighbors of out-of-distribution queries are likely not to be neighbors. To solve the problems of slow speed and poor accuracy of searching for images by text in search engines, existing research has proposed out-of-distribution vector retrieval algorithms, such as RoarGraph and RobustVamana, which add edges between the nearest neighbors of a sampled query vector set (usually 10% of the size of the base vector set) to enhance the clustering of the graph index. However, this method can only optimize some nodes of the graph index, and most nodes still have not been optimized for out-of-distribution queries. Therefore, search engines based on this algorithm can only obtain limited performance improvement when searching for images by text. To address the above problems, the present invention comprehensively considers the neighbor relationships of each node in the graph index under the base vector set distribution and the query vector set distribution, and proposes a new vector retrieval method for out-of-distribution queries. Summary of the Invention

[0004] In view of the above existing problems, the present invention proposes an out-of-distribution vector retrieval method and device, which constructs a fusion graph based on the neighbor relationships of the base vector set and the query vector set that follow different distributions to implement the retrieval task. Compared with existing algorithms, it can better adapt to the out-of-distribution vector retrieval task and improve the search accuracy and search efficiency.

[0005] To achieve the above object, the technical solutions adopted by the present invention are as follows:

[0006] An out-of-distribution vector retrieval method, comprising the following steps:

[0007] Step (1): Obtain a set of basis vectors and a set of query vectors, where the set of query vectors is out-of-distribution data with respect to the set of basis vectors.

[0008] Step (2): For each basis vector in the set of basis vectors, find several nearest query vectors in the set of query vectors, and generate an anchor vector for each basis vector based on the found query vectors; all the anchor vectors form a set of anchor vectors.

[0009] Step (3): Construct a fusion graph based on the set of basis vectors and the set of anchor vectors.

[0010] Step (4): Obtain the query text input by the search engine user and convert it into a query vector to perform a search on the fusion graph to obtain basis vectors similar to the query vector, and return the obtained basis vectors as the query results.

[0011] Preferably, in step (1), the definition that the set of query vectors is out-of-distribution data with respect to the set of basis vectors is as follows:

[0012] Step (1-1): For obtaining a set of basis vectors containing independent and identically distributed basis vectors, , and a set of query vectors containing independent and identically distributed query vectors; The query vector is any element in the set of query vectors , and its nearest neighbor set in the set of basis vectors is , where is the -th nearest neighbor of , and is the nearest neighbor of . The nearest neighbor of the query vector in the set of basis vectors has a nearest neighbor set , where is the -th nearest neighbor of . The neighborhood alignment quality of the query vector with respect to the set of basis vectors is defined as:

[0013]

[0014] The neighborhood alignment quality of the set of query vectors with respect to the set of basis vectors is defined as:

[0015]

[0016] Step (1-2) If the neighborhood alignment quality is less than a preset constant , it indicates that the query vector set is out-of-distribution data with respect to the base vector set , and vice versa for in-distribution data.

[0017] Preferably, in the said step (2), the process of constructing the anchor vector set includes the following steps:

[0018] Step (2-1) For any base vector in the base vector set , obtain the nearest neighbor sets of in the query vector set as , where is the parameter of the number of query nearest neighbors required for constructing the anchor vector set;

[0019] Step (2-2) Input the nearest neighbor set corresponding to each base vector into the aggregation function to obtain an anchor vector . All the anchor vectors form the anchor vector set .

[0020] Preferably, in the said step (3), the process of constructing the fusion graph includes the following steps:

[0021] Step (3-1) Based on the base vector set construct a basic neighbor graph with an out-degree upper limit of L, where each node v in represents a base vector , represents the neighbor relationship between nodes. For any two nodes and in the basic neighbor graph evaluate the similarity between and using the distance , where is the base vector corresponding to node , and is the base vector corresponding to node

[0022] Step (3-2) For each node v in the basic neighbor graph , obtain the node v in The basic neighbors in are used as candidates and re - sorted;

[0023] The said basic neighbors are the set of endpoints of the directed edges starting from node v in

[0024] The said re - sorting includes: for node v and the basic neighbors in any node , during the re - sorting process, the fusion distance is used to evaluate and for similarity, where is the fusion weight parameter, and are the base vector and the anchor vector corresponding to node v, and are the base vector and the anchor vector corresponding to node ;

[0025] Step (3 - 3) sets the out - degree upper limit , for each node v, from the basic neighbors , similarity sorting is performed according to the fusion distance, and nodes are selected for edge connection from largest to smallest similarity. When the node meets the occlusion rule, edge connection is performed, and the number of edge connections stops when it is equal to , obtaining the fusion neighbors ;

[0026] When all nodes in have been edge - connected according to the occlusion rule, the fusion graph is obtained;

[0027] is the set of endpoints of the directed edges starting from node v in the fusion graph .

[0028] Preferably, in step (4), searching is performed on the fusion graph to obtain the base vector similar to the query vector, specifically including the following steps:

[0029] Step (4 - 1) selects nodes from the fusion graph as the entry points, and places the nodes into the candidate set ;

[0030] Step (4 - 2) selects the node closest to the query vector from the candidate set that has not been visited As the currently visited node, and the distance between is evaluated using distance and the fused neighbors of the currently visited node are placed into the candidate set where

[0031] is the basis vector corresponding to the currently visited node v; The to-be-query vector and the query vector set

[0032] are independently and identically distributed; Step (4-3): For any node v in the candidate set the similarity between v and is evaluated using distance and all nodes in the candidate set are sorted in descending order of similarity according to distance and nodes in the candidate set H whose ranking is greater than are deleted, where

[0033] is the basis vector corresponding to node v; Step (4-4): Repeat step (4-2) to step (4-3) until there are no unvisited nodes in the candidate set and select the first basis vectors as the result.

[0034] Preferably, the basis vector set in step (1) is composed of vectors transformed from pictures or videos in the search engine image library, and the query vector set is composed of vectors transformed from query texts input by search engine users;

[0035] Step (4) further includes the following steps:

[0036] Based on the basis vectors obtained in step (4), pictures or videos in the search engine image library corresponding to the basis vectors are output.

[0037] Preferably, in the distance and the fusion distance among them, is any one of the following three distance metrics: Euclidean distance, cosine similarity, and inner product.

[0038] The present invention also provides an out-of-distribution vector retrieval device for implementing the out-of-distribution vector retrieval method, including:

[0039] An input module for obtaining a basis vector set and a query vector set as inputs, where the query vector set is out-of-distribution data with respect to the basis vector set;

[0040] The anchor vector set construction module is used to find several nearest query vectors of each basis vector in the basis vector set in the query vector set, and generate an anchor vector for each basis vector based on the found query vectors; all the anchor vectors form an anchor vector set.

[0041] The fusion graph construction module is used to construct a fusion graph based on the basis vector set and the anchor vector set.

[0042] The search result output module is used to perform a search on the fusion graph to obtain basis vectors similar to the query vector to be searched, and return the obtained basis vectors as query results.

[0043] Preferably, the input module includes: a basis vector set input module for obtaining the basis vector set; a query vector set input module for obtaining the query vector set; and a query distribution difference discrimination module for determining whether the query vector set is out-of-distribution or in-distribution data.

[0044] An out-of-distribution vector retrieval method provided by the present invention constructs a fusion graph based on the neighbor relationship between a basis vector set and a query vector set that follow different distributions, comprehensively considers the similarity relationship between the basis vector set and the query vector set, so as to better adapt to the out-of-distribution vector retrieval task and improve the search accuracy and search efficiency.

[0045] After applying the present invention, taking the text-to-image search application of a search engine as an example, after the user inputs a text description, an image or video in the image library that is more appropriate to the text description can be obtained under the same search delay. For example, the recall rate of the k results obtained by the user in the search is higher; in addition, the search engine applying the present invention can effectively improve the throughput. For example, the search engine system has a higher queries per second rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a schematic flowchart of the present invention.

[0047] Figure 2 is a graph of the benchmark test experimental results of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0049] (1) First, obtain the original database and the original query set. In this embodiment, the original database is video data, and the original query set is text data.

[0050] (2) Generate a set of basis vectors and a set of query vectors, where the set of basis vectors is generated from the original database through the UniterVideo network model, and the set of query vectors is generated from the original query set through the Transformer network model. Specifically, the set of basis vectors contains independent and identically distributed basis vectors, and the set of query vectors contains independent and identically distributed query vectors. The set of nearest neighbors of the query vector in the set of basis vectors is , where is the -th nearest neighbor of , and is the nearest neighbor of . The nearest neighbor of the query vector in the set of basis vectors is the set of nearest neighbors, where is the -th nearest neighbor of and . The set of query vectors satisfies the following conditions for the set of basis vectors :

[0051]

[0052] where is defined as:

[0053]

[0054] Preferably, is set as follows: Obtain a data set with the same distribution as the set of basis vectors, calculate its NAQ for the set of basis vectors , and use the calculated NAQ as .

[0055] (3) For any basis vector in the set of basis vectors , obtain the set of query nearest neighbors of in , where is a parameter to be determined according to the given set of basis vectors and the set of query vectors. For the set of query nearest neighbors corresponding to each basis vector , input the aggregation function , and output an anchor vector​​ The set of anchor vectors forms an anchor vector set ;

[0056] (4) Preferably, the aggregation function

[0057] (5) Based on the set of basis vectors Construct a basic neighbor graph , Each node v in represents a basis vector , represents the neighbor relationship between nodes. For the basic neighbor graph any two nodes in and , use the distance as the similarity measure, where the out-degree upper limit of the basic neighbor graph is L, is the basis vector corresponding to node , is the basis vector corresponding to node ;

[0058] (6) For each node v in the basic neighbor graph , obtain the basic neighbor of node v in as a candidate and reorder it, where is the set of the endpoints of the directed edges starting from node v in is Specifically, for node v and any node in the basic neighbor , use the fusion distance as the new similarity measure to reorder, where is the fusion weight parameter, and are the basis vector and the anchor vector corresponding to node v, and are the basis vector and the anchor vector corresponding to node . The fusion distance enables each node in the graph index to comprehensively represent the neighbor relationship under the distribution of the basis vector set and the query vector set through the distance. For example, for two nodes that are close under the basis vector distribution, , at this time, the fusion distance supports further considering the neighbor relationship under the query vector set distribution to enhance the aggregation of query neighbors. Conversely, for two nodes that are close under the query vector set distribution, , thus ensuring that the nodes maintain the basic neighbor relationship;

[0059] (7) Preferably, the parameter and the fusion weight parameter Obtained by the following method: For a set of fusion vectors , where the fusion vectors are , . The parameter and the fusion weight parameter satisfy that the query vector set for the set of fusion vectors has the maximum neighborhood alignment quality ;

[0060] (8) Preferably, set the out-degree upper limit . For each node v, select the node with the smallest fusion distance from the basic neighbors for attempting to connect edges. When the node satisfies the occlusion rule of the neighborhood graph (such as the navigation extension graph NSG) algorithm: , Connect an edge to , update the fusion neighbors { , and update the candidate , where is the set of endpoints of the directed edges starting from the node v in the fusion graph . Re-connecting edges through the occlusion rule disperses the neighbors of each node in different directions, avoiding the situation where neighbors are too concentrated and causing the search process to not converge quickly.

[0061] (9) Repeat step (8) until the number of connected edges is equal to , then stop selecting connected edges to obtain the new neighbors , thereby generating the fusion graph ;

[0062] (10) Select nodes from the fusion graphas the entry points and place them into the candidate set . The size of the candidate set is ;

[0063] (11) Select the node in the candidate set that has not been visited and is closest to the vector to be queried as the currently visited node. The distance metric is , and place the fusion neighbors of the currently visited node into the candidate, where is the basis vector corresponding to the currently visited node v;

[0064] (12) The candidate set Sort in ascending order and delete the nodes in the candidate set whose sorting is greater than . During the sorting process, for any node v in the candidate set , use as the distance metric, where is the basis vector corresponding to node v;

[0065] (13) Repeat (11) to (12) until there are no unvisited nodes in the candidate set . Select the first basis vectors in the candidate set as the result.

[0066] The present invention also provides an out-of-distribution vector retrieval device for implementing the out-of-distribution vector retrieval method, including:

[0067] An input module for obtaining a basis vector set and a query vector set as inputs, where the query vector set is out-of-distribution data relative to the basis vector set;

[0068] An anchor vector set construction module for finding several nearest query vectors of each basis vector in the basis vector set within the query vector set, and generating an anchor vector for each basis vector based on the found query vectors; all the anchor vectors constitute an anchor vector set; A fusion graph construction module for constructing a fusion graph based on the basis vector set and the anchor vector set;

[0069] A search result output module for performing a search on the fusion graph to obtain basis vectors similar to the query vector to be searched, and returning the obtained basis vectors as query results.

[0070] Preferably, the input module includes: a basis vector set input module for obtaining the basis vector set; a query vector set input module for obtaining the query vector set; a query distribution difference discrimination module for determining whether the query vector set is out-of-distribution or in-distribution data.

[0071] (14) To illustrate the effectiveness and efficiency of the present invention, the results of the benchmark test experiment are as Figure 2 shown. The experimental data, baseline algorithms, experimental environment, and experimental results involved in the experiment are described as follows:

[0072] Experimental Data: In this experiment, five commonly used out-of-distribution datasets (Text-to-Image, LAION, WIT, WebVid, and CC3M) were used for benchmark testing. The specific introductions of the five out-of-distribution datasets are as follows: Text-to-Image is an out-of-distribution dataset from a visual search engine. The base vector set consists of image vectors generated by the Se-ResNext-101 model, and the query vector set consists of text vectors extracted from user-specified text queries using a variant of the DSSM model. The distance metric is the inner product; LAION is a widely used text-image dataset. Both the base vector set and the query vector set are embedded using CLIP-ViT-B / 3, and cosine similarity is used as the metric; WIT is a dataset derived from text-image applications. The text-image pairs are extracted from Wikipedia pages. The base vector set is encoded using CLIP-ViT-B / 32, the query vector set is generated using CLIP-ViT-B32-multilingual-v1, and cosine similarity is used as the metric; WebVid is a caption-video dataset from a stock website. The data is encoded using CLIP-ViT-B / 32, and the distance metric used is cosine similarity; CC3M is a text-image pair dataset designed specifically for caption-image systems. The base vectors are encoded using ViT-B / 16, and the additional text is converted into query vectors using BERT. Cosine similarity is used as the metric.

[0073] Baseline Algorithms: In the benchmark experiment, the present invention was compared with four advanced graph indexing algorithms, where HNSW and -MNG are in-distribution graph indexing algorithms, and RoarGraph and RobustVamana are out-of-distribution graph indexing algorithms.

[0074] Experimental Environment: All experiments were conducted on an Ubuntu 20.04 server equipped with an Intel(R) Xeon(R) Gold 5218 CPU (2.30GHz) and 125GB of memory. The Eigen library was used to enhance matrix operations. In addition, OpenMP with 20 threads was used for parallel index construction.

[0075] Explanation of Experimental Results: As Figure 2As shown, the experiments evaluate the search precision using recall@100 (lines 1 and 2) and recall@10 (lines 3 and 4), and evaluate the search efficiency using queries per second and the number of distance calculations. In the graphs showing recall@k versus queries per second (lines 1 and 3), the curve in the upper right indicates better performance. In contrast, in the graphs depicting recall@k versus the number of distance calculations (lines 2 and 4), the curve in the lower right indicates better performance. The experimental results show that the present invention outperforms all baseline algorithms on all datasets, with a better recall@k and queries per second / number of distance calculations trade-off, that is, under the same precision requirements, it has a shorter search latency and fewer calculations, and under the requirements of the same search latency or calculation number limit, the search results are more accurate.

Claims

1. An out-of-distribution vector retrieval method, characterized in that, It includes the following steps: Step (1): Obtain a set of base vectors and a set of query vectors, where the set of query vectors is out-of-distribution data with respect to the set of base vectors; Step (2): For each base vector in the set of base vectors, find several nearest query vectors in the set of query vectors, and generate an anchor vector for each base vector based on the found query vectors; all the anchor vectors form an anchor vector set; Step (3): Construct a fusion graph based on the set of base vectors and the anchor vector set; Step (4): Obtain the query text input by the search engine user and convert it into a vector to be queried. Perform a search on the fusion graph to obtain base vectors similar to the vector to be queried, and return the obtained base vectors as the query result.

2. The out-of-distribution vector retrieval method according to claim 1, wherein In step (2), the process of constructing the anchor vector set includes the following steps: Step (2-1) For a set of basis vectors X = {x1,..., x N} containing N independent and identically distributed basis vectors, and a set of query vectors Q = {q1,..., q M} containing M independent and identically distributed query vectors; For any basis vector x in the basis vector set X = {x1,..., x N}, obtain the set of C query nearest neighbors of x in the query vector set Q as where C is the parameter of the number of query nearest neighbors required to construct the anchor vector set; Step (2-2): Input the query neighbor set GT(x) corresponding to each base vector x into the aggregation function e(·) to obtain an anchor vector τ, and all the anchor vectors form the anchor vector set Τ.

3. The out-of-distribution vector retrieval method according to claim 2, wherein In step (3), the process of constructing the fusion graph includes the following steps: Step (3-1) constructs a basic neighbor graph G with an out-degree upper limit of L based on the basis vector set X * (V, E * ), where each node v in V represents a basis vector x, and E * represents the neighbor relationship between nodes. For the basic neighbor graph G * (V, E * ), for any two nodes and , the distance δ(x i , χ j ) is used to evaluate and 's similarity, where χ i is the basis vector corresponding to node , and χ j is the basis vector corresponding to node ; Step (3-2) for the basic neighbor graph G * (V,E * ) for each node v in G * (V,E * ) in the basic neighbor N * (v) as candidates and re-ranked; The basic neighbor N * (v) is G * (V, E * ) is the set of the endpoints of the directed edges starting from the node v; The reordering includes: for node v and its basic neighbors N * any node in During the rearrangement process, the fusion distance is used to evaluate the similarity between and v, where α is the fusion weight parameter, x i and τ i are the basis vector and the anchor vector corresponding to the node respectively; x and τ are the basis vector and the anchor vector corresponding to node v; Step (3-3) sets the out-degree upper limit R, where R < L. For each node v, from the basic neighbors N * (v), perform similarity sorting according to the fusion distance, and select nodes from largest to smallest similarity to attempt edge connection. When the node satisfies the occlusion rule, then perform edge connection, and stop selecting edge connections when the number of edge connections is equal to R to obtain the fused neighbors N(v); When G * (V, E * ) and all the nodes in it are connected according to the occlusion rules, the fused graph G(V, E) is obtained; N(v) is the set of the endpoints of the directed edges starting from the node v in the fusion graph G(V,E).

4. A method for retrieving out-of-distribution vectors according to claim 3, characterized in that In step (4), performing a search on the fusion graph to obtain base vectors similar to the vector to be queried specifically includes the following steps: Step (4-1): Select P nodes from the fusion graph G(V,E) as the entry points, and place the P nodes into the candidate set H; Step (4-2) selects an unvisited node v from the candidate set H that is closest to the query vector q * as the currently visited node. The similarity between v and q * is evaluated using the distance δ(x, q * ), and the fused neighbors N(v) of the currently visited node are placed into the candidate set H, where x is the basis vector corresponding to the currently visited node v; The query vector q to be queried * is independently and identically distributed with the query vector set Q; Step (4-3) takes any node v in the candidate set H and uses the distance δ(x, q * ) to evaluate the similarity between v and q * . For all nodes in the candidate set H, sort them in descending order of similarity according to the distance δ(x, q * ), and delete the nodes in the candidate set H whose ranking is greater than P, where x is the base vector corresponding to node v; Step (4-4): Repeat steps (4-2) to (4-3) until there are no unvisited nodes in the candidate set H, and select the top k base vectors in the candidate set H as the result.

5. An out-of-distribution vector retrieval method according to claim 4, characterized in that In step (1), the set of base vectors is composed of vectors obtained by converting pictures or videos in the search engine image library, and the set of query vectors is composed of vectors obtained by converting the query text input by the search engine user; Step (4) further includes the following steps: Based on the base vectors obtained in step (4), output the pictures or videos in the search engine image library corresponding to the base vectors.

6. An out-of-distribution vector retrieval method according to claim 5, characterized in that The distance δ(χ i , χ j ) and the fusion distance Among them, δ is any one of the following three distance metrics: Euclidean distance, cosine similarity, and inner product.

7. An out-of-distribution vector retrieval device for implementing the out-of-distribution vector retrieval method according to any one of claims 1 to 6, characterized in that, It includes: An input module, used to obtain a set of base vectors and a set of query vectors as inputs, where the set of query vectors is out-of-distribution data with respect to the set of base vectors; An anchor vector set construction module, used to find several nearest query vectors in the set of query vectors for each base vector in the set of base vectors, and generate an anchor vector for each base vector based on the found query vectors; all the anchor vectors form an anchor vector set; A fusion graph construction module, used to construct a fusion graph based on the set of base vectors and the anchor vector set; A search result output module, used to perform a search on the fusion graph to obtain base vectors similar to the vector to be queried, and return the obtained base vectors as the query result.

8. An out-of-distribution vector retrieval device according to claim 7, characterized in that The input module includes: a basis vector set input module for obtaining a basis vector set; a query vector set input module for obtaining a query vector set; and a query distribution difference discrimination module for determining whether the query vector set is out-of-distribution or in-distribution data.

Images