Vector retrieval method and device for balanced graph topological structure

Abstract

The invention discloses a vector retrieval method and device for a balanced graph topological structure, and relates to the field of vector retrieve.The method comprises the steps that firstly, a graph is initialized into a minimum directed complete graph with any vector meeting the target in-out degree, a gradual vector adding mode is adopted, a forward optimization multi-level edge selection strategy and an edge refining mode are combined, and the minimum directed complete graph with the target in-out degree is obtained; and finally, constructing a neighbor graph of a balanced graph topological structure, and then performing greedy approximate nearest neighbor retrieval on the neighbor graph for the query vector to obtain a query result. According to the method, the balance of the topological structure of the neighbor graph is realized, the problem of deflection of the topological structure of the neighbor graph caused by unbalanced vector distribution is eliminated, and the accuracy and efficiency of vector retrieval are effectively improved. Compared with a current mainstream neighbor graph composition method, the composition method does not depend on reverse edge adding operation to update neighbors for existing vectors in the graph, and composition complexity and time expenditure are reduced.

Description

Technical Field

[0001] This invention relates to the field of vector retrieval, specifically to a vector retrieval method and apparatus for a balanced graph topology. Background Technology

[0002] With the rise and development of large-scale artificial intelligence models, vector retrieval technology has been widely applied in information retrieval, recommender systems, and retrieval augmentation generation (RAG). In recent years, vector retrieval methods based on nearest neighbor graphs have become a research hotspot due to their high retrieval performance and good scalability.

[0003] However, vector distribution in vector space exhibits imbalances such as clustering and sparsity. This phenomenon significantly affects the neighbor selection tendency of current mainstream nearest neighbor graph construction methods, leading to large differences in the in-degree of vectors in the graph, and further resulting in two extreme cases: (1) extremely high in-degree vectors, which make the search prone to getting trapped in local optima; (2) extremely low in-degree vectors (even 0), which have extremely low reachability. Ultimately, this results in a topologically skewed nearest neighbor graph, severely affecting the accuracy and efficiency of vector retrieval.

[0004] Furthermore, current mainstream nearest neighbor graph construction methods typically rely on adding reverse edge operations to existing vectors to update their out-neighbors. This not only increases the complexity and time cost of graph construction, but may also further exacerbate the skewness of the graph topology and reduce retrieval performance.

[0005] Therefore, in order to address the above-mentioned technical problems, there is an urgent need for a vector retrieval method and device capable of constructing a balanced graph topology to improve the accuracy and efficiency of vector retrieval while reducing graph construction complexity and time overhead. Summary of the Invention

[0006] This invention proposes a vector retrieval method and apparatus for a balanced graph topology. First, the graph is initialized as a minimum directed complete graph where any vector satisfies the target in-degree and out-degree condition. A progressive vector addition approach, combined with a forward optimization multi-level edge selection strategy and edge refinement, ultimately constructs a nearest neighbor graph with a balanced topology. Then, a greedy approximate nearest neighbor retrieval is performed on this graph for the query vector to obtain the query result. This invention achieves the balance of the nearest neighbor graph topology, eliminating the problem of topological skew caused by uneven vector distribution, and effectively improving the accuracy and efficiency of vector retrieval. Compared to current mainstream nearest neighbor graph construction methods, this method does not rely on adding reverse edges to update neighbors for existing vectors in the graph, effectively reducing the complexity and time cost of graph construction.

[0007] To achieve the above objectives, this application provides the following solution:

[0008] In a first aspect, the present invention provides a vector retrieval method for balanced graph topology structures, comprising the following steps:

[0009] Step 1: Given a vector dataset V and a target in-degree R, select R+1 vectors from dataset V to initialize graph G as a minimum directed complete graph where any vector satisfies the target in-degree R. Simultaneously, construct its reverse graph G. ' ;

[0010] Step 2: Select vector w from the vector dataset V that has not yet been added to graph G. Perform a greedy approximate nearest neighbor search on the current graph G using vector w as the query vector. Use all vectors visited during the search process as the candidate set of out neighbors C for w.

[0011] Step 3: Traverse each candidate outgoing neighbor v in the candidate outgoing neighbor set C of w, according to the reverse graph G. ' Get the in-neighbor set N of v in (v) Then, a multi-level edge selection strategy is used to select R edges, resulting in an edge set P, where all edges have unique starting and ending points. Then, based on the edge set P, vector w is added to graph G through edge refinement to optimize the outgoing neighbors of existing vectors in the graph, and the reverse graph G is updated simultaneously. ' ;

[0012] Step 4: Repeat steps 2 and 3 above until all vectors in the vector dataset V have been added to graph G, finally constructing a near-neighbor graph G with a balanced graph topology. * ;

[0013] Step 5: Using the query vector q as input, in the nearest neighbor graph G * A greedy approximate nearest neighbor search is performed to obtain the K vectors with the smallest similarity distance to the query vector q and output them as the vector search results.

[0014] Preferably, as mentioned in step 1, based on the vector dataset V and given the target in-degree R, R+1 vectors are selected from the dataset V to initialize the graph G as a minimum directed complete graph where any vector satisfies the target in-degree R, and its reverse graph G is constructed simultaneously. ' The specific method is as follows:

[0015] Step 1.1: Calculate the mean vector v of the vector dataset V. means Then calculate the similarity distance between each vector and the mean vector;

[0016] Step 1.2: Select the vector with the smallest similarity distance to the mean vector as the first vector s of graph G, and also as the entry vector for greedy approximate nearest neighbor retrieval in subsequent steps;

[0017] Step 1.3: Randomly select R vectors (excluding s) from the dataset V. For each of these R+1 vectors (including s), establish a directed edge between each pair of different vectors. Use the resulting directed complete graph as the initial graph G, where the out-degree and in-degree of each vector in the graph are both R. Simultaneously, construct its reverse graph G. ' .

[0018] Preferably, the method mentioned in step 2, which uses vector w as the query vector to perform a greedy approximate nearest neighbor search on the current graph G, and uses all vectors visited during the search process as the candidate set C of outgoing neighbors of w, is as follows:

[0019] Step 2.1: Maintain a priority queue Q with a capacity of L, sorted in ascending order by similarity distance to vector w, and a set of visited vectors S;

[0020] Step 2.2: Calculate the similarity distance δ(w,s) between vector w and the entry vector s obtained in step 1.2, and add s to the priority queue Q and the set of visited vectors S;

[0021] Step 2.3: Select the vector u in priority queue Q that has the smallest similarity distance to vector w and has not been expanded. x , get u x The set of out neighbors N out (u x And mark u x It has been expanded;

[0022] Step 2.4: For u x Each outgoing neighbor v x ∈N out (u x If v x If the vector is not in the visited vector set S, then calculate the relationship between vectors w and v. x The similarity distance δ(w,v) between them x ) and v x Add the vectors to the priority queue Q and the set of visited vectors S, while keeping the top L vectors in priority queue Q that have the smallest similarity distance to vector w.

[0023] Step 2.5: Repeat steps 2.3 and 2.4 until all vectors in priority queue Q have been marked as expanded, and then use the set of visited vectors S as the candidate set of out neighbors C for vector w.

[0024] Preferably, in step 3, each candidate outgoing neighbor v in the candidate outgoing neighbor set C of w is traversed according to the inverse graph G. ' Get the in-neighbor set N of v in(v), and then a multi-level edge selection strategy is used to filter out R edges, resulting in an edge set P, where all edges have unique starting points and unique ending points. The specific method is as follows:

[0025] Step 3.1: Initialize the edge set P as an empty set and the starting point set C. in With the endpoint set C out For an empty set, first-level, second-level, and third-level candidate neighbor sets An empty set, the first-level candidate neighbor set Second-level neighbor candidate set Third-level neighbor candidate set These are used to store the vectors with the highest, second highest, and lowest priority during the filtering process, respectively.

[0026] Step 3.2: Sort the candidate outgoing neighbors v in the candidate outgoing neighbor set C in ascending order according to their similarity distance δ(w,v) with vector w;

[0027] Step 3.3: For each candidate outgoing neighbor v, use the reverse graph G ' Get the in-neighbor set N of v in (v) and clear

[0028] Step 3.4: Traverse each incoming neighbor u∈N of v in (v), if u is not in the starting set C in In the process, the priority of the incoming neighbor u is determined based on the similarity distance relationship among w, u, and v, and then the corresponding set is added. and middle;

[0029] Step 3.5: If the first-level incoming neighbor candidate set If the vector is not empty, then select the vector with the smallest similarity distance to vector w as the best in-neighbor u. * Otherwise, if the second-level neighbor candidate set If the vector is not empty, then select the vector with the smallest similarity distance to vector w as the best in-neighbor u. * ,like and If all sets are empty, skip step 3.6;

[0030] Step 3.6: Add edge (u) * v) Add to edge set P and add the best neighbor u * Add the candidate outgoing neighbor v to the starting set C respectively. in With the endpoint set C out ;

[0031] Step 3.7: Repeat steps 3.3 to 3.6 above until all candidate outgoing neighbors in the candidate outgoing neighbor set C have been traversed in order, or until the edge set P contains R edges.

[0032] Step 3.8: If the number of edges in the edge set P is less than R, then re-traverse the neighbor candidate set C in order and perform the following steps:

[0033] Step 3.9: For each candidate outgoing neighbor v in C, first clear the empty list. If v is not in the endpoint set C out In the middle, from the set of ingress neighbors N of v in Select all sets C that are not at the starting point from (v). in Adding neighbors to the set These incoming neighbors have the lowest priority.

[0034] Step 3.10: If set If the vector is not empty, then select the vector with the largest similarity distance to v as the best in-neighbor u. * , will the edge (u * v) Add to edge set P and add the best neighbor u * Add the candidate outgoing neighbor v to the starting set C respectively. in With the endpoint set C out ;

[0035] Step 3.11: Repeat steps 3.9 and 3.10 during the traversal until the edge set P contains R edges.

[0036] Preferably, as mentioned in step 3.4, the priority of the incoming neighbor u is determined based on the similarity distance relationship among w, u, and v, and then the corresponding set is added. and The specific methods are as follows:

[0037] Step 3.4.1: Calculate the similarity distances δ(w,u), δ(w,v), and δ(u,v);

[0038] Step 3.4.2: If the similarity distance satisfies δ(w,u)<δ(u,v) and δ(w,v)<δ(u,v), then u has the highest priority and is added to the first-level neighbor candidate set.

[0039] Step 3.4.3: If the similarity distance satisfies δ(w,u)<δ(u,v) and δ(w,v)≥δ(u,v), then u has the second highest priority and is added to the secondary neighbor candidate set.

[0040] Preferably, as mentioned in step 3, vector w is added to graph G by edge refinement based on edge set P to optimize the out-neighbors of existing vectors in the graph, and the reverse graph G is updated simultaneously. ' The specific method is as follows:

[0041] Step 3.12: For each edge (u) in the edge set P, * Remove vectors (w, v) ∈ P from graph G, and add an outgoing edge (w, v) to vector w, for u. * Add outgoing edges (u * ,w);

[0042] Step 3.13: In the reverse graph G ' Delete the reverse edge (v, u) of each edge in the edge set P. * Add a new reverse edge (v, w) and (w, u). * ).

[0043] Preferably, as mentioned in step 5, the query vector q is used as input in the nearest neighbor graph G. * A greedy approximate nearest neighbor search is performed to obtain the K vectors with the highest similarity distance to the query vector q and output them as vector search results, where K is the preset number of search results. The specific method is as follows:

[0044] Step 5.1: Maintain a priority queue Q with a capacity of L, sorted in ascending order by similarity distance to the query vector q, and a set of visited vectors S;

[0045] Step 5.2: Calculate the similarity distance δ(q,s) between the query vector q and the entry vector s obtained in step 1.2, and add s to the priority queue Q and the set of visited vectors S;

[0046] Step 5.3: Select the vector u in the priority queue Q that has the smallest similarity distance to the query vector q and has not been expanded. x , get u x The set of out neighbors N out (u x And mark u x It has been expanded;

[0047] Step 5.4: For u x Each outgoing neighbor v x ∈N out (u x If v x If the query vector q is not in the visited vector set S, then calculate the query vector q and v. x The similarity distance δ(q,v) between them x ) and v xAdd the vectors to the priority queue Q and the set of visited vectors S, while retaining the top L vectors in priority queue Q that have the smallest similarity distance to the query vector q;

[0048] Step 5.5: Repeat steps 5.3 and 5.4 until all vectors in priority queue Q have been expanded. Then, select the K vectors from priority queue Q that have the smallest similarity distance to the query vector q as the vector retrieval results output.

[0049] Secondly, the present invention provides a vector retrieval device for a balanced graph topology structure, the vector retrieval device for a balanced graph topology structure comprising:

[0050] Initialization module: Based on the vector dataset V and given the target in-degree R, select R+1 vectors from dataset V to initialize graph G as a minimum directed complete graph where any vector satisfies the target in-degree R. Simultaneously, construct its reverse graph G. ' ;

[0051] Candidate set retrieval module: Used to perform greedy approximate nearest neighbor retrieval on the current graph G with vector w as the query vector, and take all vectors visited during the search process as the candidate set C of out neighbors of vector w;

[0052] Edge filtering and update module: used to traverse each candidate outgoing neighbor v in the candidate outgoing neighbor set C of vector w, based on the inverse graph G. ' Get the in-neighbor set N of v in (v) Then, a multi-level edge selection strategy is used to select R edges to obtain the edge set P; then, based on the edge set P, the vector w is added to the graph G through edge refinement, and the reverse graph G is updated synchronously. ' ;

[0053] The graph construction module is used to repeatedly select vectors from the vector dataset V that have not yet been added to graph G, sequentially calling the candidate set retrieval module and the filtering and updating module, until all vectors in the vector dataset V have been added to graph G, ultimately constructing a nearest-neighbor graph G with a balanced graph topology. * ;

[0054] Vector retrieval module: used to retrieve data from the nearest neighbor graph G, taking the query vector q as input. * A greedy approximate nearest neighbor search is performed to obtain the approximate nearest neighbor result of the query vector.

[0055] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when executed by the processor, implements the vector retrieval method for the balanced graph topology structure.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] 1. This invention initializes the graph as a minimal directed complete graph with in-degree and out-degree R for any vector. It uses an edge refinement method to progressively add vectors, ensuring the balance and strong connectivity of the graph's topology. Finally, it constructs a nearest neighbor graph with a balanced graph topology. The graph construction method of this invention has low time complexity and computational overhead. It does not rely on adding reverse edges to update the neighbors of existing vectors in the graph, thus improving the efficiency of graph construction.

[0058] 2. This invention comprehensively considers both the incoming and outgoing neighbors of a vector when selecting its incoming and outgoing neighbors, aiming to optimize the outgoing neighbors of existing vectors in the graph. This improves the proximity and navigation capabilities of the graph, effectively addressing the performance issues of proximity graphs in high-dimensional spaces. Furthermore, this strategy does not rely on adding reverse edges to update the outgoing neighbors of existing vectors in the graph, reducing graph construction complexity and time overhead.

[0059] 3. This invention eliminates the problem of skewed topological structure of nearest neighbor graphs caused by uneven distribution of vectors in the dataset, so that the out-degree and in-degree of each vector in the graph are the given target R. It eliminates the problem of navigation difficulties or even unreachability caused by some vectors having too low or even zero in-degree, thereby improving the accuracy and efficiency of vector retrieval. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of a vector retrieval method for a balanced graph topology structure according to an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of a multi-level edge selection strategy in one embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] In one exemplary embodiment, such as Figure 1 As shown, a vector retrieval method for balanced graph topology is provided. The following is a detailed description of the vector retrieval method for balanced graph topology provided in this embodiment.

[0064] In cross-domain knowledge fusion and semantic association mining tasks (such as terminology association in finance and healthcare, and concept mapping in law and education), cross-domain word vector semantic association retrieval is a core supporting technology, which relies on similarity matching between word vector data from different domains. However, word vector data from different domains often exhibits a clustered distribution, and traditional nearest-neighbor graph construction methods struggle to effectively handle this uneven distribution, leading to a skewed graph topology and consequently reduced retrieval performance. Furthermore, cross-domain word vector datasets generally possess extremely high dimensionality, further exacerbating the complexity and computational cost of nearest-neighbor graph construction and retrieval. Moreover, the specificity of domain semantic systems easily generates severe "information silos," causing traditional methods to face low accessibility and navigation difficulties in cross-domain semantic association retrieval, severely impacting accuracy and efficiency. Therefore, this embodiment proposes a vector retrieval method with a balanced graph topology to improve the accuracy and efficiency of cross-domain word vector semantic association retrieval.

[0065] First, features are extracted from the original cross-domain word text dataset to generate a high-quality word vector dataset V. The feature extraction methods include, but are not limited to, using existing word embedding models such as Word2Vec, GloVe, or BERT, ensuring that the generated word vectors effectively capture cross-domain semantic information.

[0066] The similarity distance between vectors in dataset V is calculated using the Euclidean distance between vectors:

[0067]

[0068] Next, based on the vector dataset V, according to Figure 1 The process shown involves performing the following steps to construct a nearest-neighbor graph G with a balanced graph topology. * And perform vector retrieval, including the following steps:

[0069] Step 1: Given a vector dataset V and a target in-degree R, take R+1 vectors from dataset V and initialize graph G as a minimum directed complete graph where any vector satisfies the target in-degree R. Simultaneously, construct its reverse graph G. ′ The target in-degree R can be set according to the specific application scenario and dataset size, as follows:

[0070] Step 1.1: Calculate the mean vector of the vector dataset V. Then calculate the similarity distance between each vector and the mean vector;

[0071] Step 1.2: Select the vector with the smallest similarity distance to the mean vector as the first vector s of graph G, and also as the entry vector for greedy approximate nearest neighbor retrieval in subsequent steps;

[0072] Step 1.3: Randomly select R vectors (excluding s) from the dataset V. For each of these R+1 vectors (including s), establish a directed edge between each pair of different vectors. Use the resulting directed complete graph as the initial graph G, where the out-degree and in-degree of each vector in the graph are both R. Simultaneously, construct its reverse graph G. ' .

[0073] Step 2: Select vector w from the vector dataset V that has not yet been added to graph G. Perform a greedy approximate nearest neighbor search on the current graph G using vector w as the query vector. All vectors visited during the search process are used as the candidate set C of outgoing neighbors of w. The specific method is as follows:

[0074] Step 2.1: Maintain a priority queue Q with a capacity of L, sorted in ascending order by similarity distance to vector w, and a set of visited vectors S;

[0075] Step 2.2: Calculate the similarity distance δ(w,s) between vector w and the entry vector s obtained in step 1.2, and add s to the priority queue Q and the set of visited vectors S;

[0076] Step 2.3: Select the vector u in priority queue Q that has the smallest similarity distance to vector w and has not been expanded. x , get u x The set of out neighbors N out (u x And mark u x It has been expanded;

[0077] Step 2.4: For u x Each outgoing neighbor v x ∈N out (u x If v x If the vector is not in the visited vector set S, then calculate the relationship between vectors w and v. x The similarity distance δ(w,v) between them x ) and v x Add the vectors to the priority queue Q and the set of visited vectors S, while keeping the top L vectors in priority queue Q that have the smallest similarity distance to vector w.

[0078] Step 2.5: Repeat steps 2.3 and 2.4 until all vectors in priority queue Q are marked as expanded, and then use the set of visited vectors S as the candidate set of out neighbors C for vector w.

[0079] Step 3: Traverse each candidate outgoing neighbor v in the candidate outgoing neighbor set C of w, according to the reverse graph G. ' Get the in-neighbor set N of v in(v) Then, a multi-level edge selection strategy is used to select R edges, resulting in an edge set P, where all edges have unique starting and ending points. Then, based on the edge set P, vector w is added to graph G through edge refinement to optimize the outgoing neighbors of existing vectors in the graph, and the reverse graph G is updated simultaneously. ′ The specific method is as follows:

[0080] Step 3.1: Initialize the edge set P as an empty set and the starting point set C. in With the endpoint set C out Let C be an empty set, where C in C is used to record the starting vector of the selected edges. out Used to record the endpoint vectors of selected edges to ensure the uniqueness of the start and end points in the edge set P; initializes the first-level, second-level, and third-level candidate neighbor sets. An empty set, the first-level candidate neighbor set Second-level neighbor candidate set Third-level neighbor candidate set These are used to store the vectors with the highest, second highest, and lowest priority during the filtering process, respectively.

[0081] Step 3.2: Sort the candidate outgoing neighbors v in the candidate outgoing neighbor set C in ascending order according to their similarity distance δ(w,v) with vector w;

[0082] Step 3.3: For each candidate outgoing neighbor v, use the reverse graph G ′ Get the in-neighbor set N of v in (v and clear)

[0083] Step 3.4: Traverse each incoming neighbor u∈N of v in (v), if u is not in the starting set C in Then, based on the similarity distance relationship among w, u, and v, the priority of the incoming neighbor u is determined and added to the corresponding set. and The specific method is as follows:

[0084] Step 3.4.1: Calculate the similarity distances δ(w,u), δ(w,v), and δ(u,v);

[0085] Step 3.4.2: If the similarity distance satisfies δ(w,u)<δ(u,v) and δ(w,v)<δ(u,v), then u has the highest priority and is added to the first-level neighbor candidate set.

[0086] Step 3.4.3: If the similarity distance satisfies δ(w,u)<δ(u,v) and δ(w,v)≥δ(u,v), then u has the second highest priority and is added to the secondary neighbor candidate set.

[0087] Step 3.5: If the first-level incoming neighbor candidate set If the vector is not empty, then select the vector with the smallest similarity distance to vector w as the best in-neighbor u. * Otherwise, if the second-level neighbor candidate set If the vector is not empty, then select the vector with the smallest similarity distance to vector w as the best in-neighbor u. * ,like and If all sets are empty, skip step 3.6;

[0088] Step 3.6: Add edge (u) * v) Add to edge set P and add the best neighbor u * Add the candidate outgoing neighbor v to the starting set C respectively. in With the endpoint set C out ;

[0089] Step 3.7: Repeat steps 3.3 to 3.6 above until all candidate outgoing neighbors in the candidate outgoing neighbor set C have been traversed in order, or until the edge set P contains R edges.

[0090] Step 3.8: If the number of edges in the edge set P is less than R, then re-traverse the neighbor candidate set C in order and perform the following steps:

[0091] Step 3.9: For each candidate outgoing neighbor v in C, first clear the empty list. If v is not in the endpoint set C out In the middle, from the set of ingress neighbors N of v in Select all sets C that are not at the starting point from (v). in Adding neighbors to the set These incoming neighbors have the lowest priority.

[0092] Step 3.10: If set If the vector is not empty, then select the vector with the largest similarity distance to v as the best in-neighbor u. * , will the edge (u * v) Add to edge set P and add the best neighbor u * Add the candidate outgoing neighbor v to the starting set C respectively. in With the endpoint set C out ;

[0093] Step 3.11: Repeat steps 3.9 and 3.10 during the traversal until the edge set P contains R edges.

[0094] Step 3.12: For each edge (u) in the edge set P, *Remove vectors (w, v) ∈ P from graph G, add an outgoing edge (w, v) to vector w, and add an outgoing edge (u, v) to vector u. * ,w);

[0095] Step 3.13: In the reverse graph G ′ Delete the reverse edge (v, u) of each edge in the edge set P. * Add a new reverse edge (v, w) and (w, u). * ).

[0096] Figure 2 This diagram illustrates the multi-level edge selection strategy mentioned in step 3. It shows the similarity distance relationship between vector w and its candidate outgoing neighbor v and its incoming neighbor u, and indicates the optimal incoming neighbor u for the corresponding situation. * This strategy aims to optimize the outgoing neighbors of existing vectors in the graph, while comprehensively considering the incoming and outgoing neighbors of vector w, thereby effectively improving the proximity and navigation of the graph.

[0097] Step 4: Repeat steps 2 and 3 above until all vectors in the vector dataset V have been added to graph G, finally constructing a near-neighbor graph G with a balanced graph topology. * , where Figure G * In graph G, the out-degree and in-degree of each vector are both R, and graph G... * It is a strongly connected graph.

[0098] At this point, the nearest neighbor graph G of the balanced graph topology is complete. * The dataset has been built and is ready for subsequent vector retrieval tasks. The text of the words to be retrieved is converted into a query vector q using the same word embedding method as the generated vector dataset V.

[0099] Step 5: Using the query vector q as input, in the nearest neighbor graph G * A greedy approximate nearest neighbor search is performed to obtain the K vectors with the highest similarity distance to the query vector q and output them as vector search results, where K is the preset number of search results. The specific method is as follows:

[0100] Step 5.1: Maintain a priority queue Q with a capacity of L, sorted in ascending order by similarity distance to the query vector q, and a set of visited vectors S;

[0101] Step 5.2: Calculate the similarity distance δ(q,s) between the query vector q and the entry vector s obtained in step 1.2, and add s to the priority queue Q and the set of visited vectors S;

[0102] Step 5.3: Select the vector u in the priority queue Q that has the smallest similarity distance to the query vector q and has not been expanded. x , get u x The set of out neighbors Nout (u x And mark u x It has been expanded;

[0103] Step 5.4: For u x Each outgoing neighbor v x ∈N out (u x If v x If the query vector q is not in the visited vector set S, then calculate the query vector q and v. x The similarity distance δ(q,v) between them x ) and v x Add the vectors to the priority queue Q and the set of visited vectors S, while retaining the top L vectors in priority queue Q that have the smallest similarity distance to the query vector q;

[0104] Step 5.5: Repeat steps 5.3 and 5.4 until all vectors in priority queue Q have been expanded. Then, select the K vectors from priority queue Q that have the smallest similarity distance to the query vector q as the vector retrieval results output.

[0105] Finally, the K retrieved vectors are post-processed to improve the semantic relevance and usability of the search results. Post-processing methods include, but are not limited to, domain-knowledge-based filtering, semantic re-ranking, and result optimization based on contextual information. The result vectors are then mapped back to their corresponding word texts, outputting the final semantically related search results.

[0106] The above method eliminates the performance degradation of semantic association retrieval caused by the skewed topology of the nearest neighbor graph due to the uneven distribution of cross-domain word vector data. By constructing a near neighbor graph with a balanced topology where the out-degree and in-degree of any vector in the graph are both R and strongly connected, the accuracy and efficiency of cross-domain word vector semantic association retrieval are improved. The multi-level edge selection strategy comprehensively considers the in-neighbors and out-neighbors of vectors, and, together with edge refinement operations, optimizes the out-neighbors of existing vectors in the graph. Furthermore, it does not rely on adding reverse edges to update the out-neighbors of existing vectors, thus reducing the complexity of graph construction and time overhead.

[0107] It should be noted that the vector retrieval method and apparatus for balanced graph topology proposed in this invention are not limited to specific types of vector data. They are also applicable to high-dimensional vector datasets obtained by different feature extraction methods and exhibiting uneven distribution characteristics in the vector space.

[0108] In practical applications, in addition to the cross-domain word vector semantic association retrieval scenarios mentioned above, tasks such as image retrieval, video analysis, and multimedia content understanding also commonly suffer from imbalances in vector distribution, such as clustering and sparsity, due to the overall feature modeling method. Traditional nearest neighbor graph construction methods are also difficult to effectively handle this uneven distribution problem in such scenarios, thus affecting retrieval performance.

[0109] Therefore, to further verify the universality and effectiveness of the present invention under different vector types and application scenarios, another exemplary embodiment is given below in the field of video content retrieval to illustrate the application of the vector retrieval method of balanced graph topology proposed in the present invention in similarity retrieval based on the overall temporal semantics and action patterns of videos.

[0110] In the field of video content retrieval and understanding, similarity retrieval based on the overall temporal semantics and action patterns of videos is an important application. Unlike methods that rely solely on single-frame image features, video retrieval typically models the motion patterns, rhythm changes, and spatiotemporal structure of videos over time, generating high-dimensional temporal vectors to represent video content. Therefore, video clips with similar action primitives, temporal structures, or shooting patterns are often mapped and compressed into the same region in the vector space, resulting in significant clustering in the video vector dataset. Traditional nearest-neighbor graph construction methods struggle to maintain a stable graph structure and efficient search paths when faced with this imbalanced distribution caused by temporal redundancy and repetitive action patterns, leading to graph topology skew and consequently, decreased retrieval performance. Meanwhile, video vectors typically have higher dimensions and stronger temporal relevance, which significantly increases the cost of constructing nearest neighbor graphs. In addition, the differences in content structure and semantic expression among different video domains (such as sports, surveillance, short videos, or educational videos) can easily lead to the formation of fragmented subspace structures in the vector space. This results in insufficient accessibility and low navigation efficiency during cross-domain video semantic association retrieval, which seriously affects the accuracy and response efficiency of retrieval.

[0111] To address this, this embodiment proposes a vector retrieval method based on a balanced graph topology to improve the accuracy and efficiency of similarity retrieval based on the overall temporal semantics and action patterns of the video. For video content, a high-quality video vector dataset V can be generated by extracting spatiotemporal features from the video using a 3D convolutional neural network (3D-CNN) and performing global pooling.

[0112] Based on the vector dataset V, according to Figure 1 The process shown constructs a nearest neighbor graph G with a balanced graph topology. * Vector retrieval is then performed, and the specific steps are the same as steps 1 to 5 in the vector retrieval method for a balanced graph topology structure described above, so they will not be repeated here.

[0113] When returning results, post-processing can be performed by combining video metadata information (such as title, tags, description, etc.) to improve the relevance of the results and user experience. Then, the result vectors are mapped back to the corresponding video segments to form the final search output.

[0114] The method in this embodiment can effectively construct a nearest neighbor graph with a balanced graph topology when dealing with the problem of unbalanced vector distribution caused by the overall temporal feature extraction method for video content representation vectors. This improves the accuracy and efficiency of similarity retrieval based on the overall temporal semantics and action patterns of the video.

[0115] In one exemplary embodiment, a vector retrieval device for a balanced graph topology is provided, the device comprising:

[0116] The initialization module is configured to select R+1 vectors from the vector dataset V, given a target in-degree R, and initialize the graph G as a minimum directed complete graph where any vector satisfies the target in-degree R. Simultaneously, its reverse graph G is constructed. ′ ;

[0117] The candidate set retrieval module is configured to perform a greedy approximate nearest neighbor retrieval on the current graph G using vector w as the query vector, and to use all vectors visited during the search process as the candidate set of out neighbors C of w.

[0118] The edge filtering and updating module is configured to traverse each candidate outgoing neighbor v in the candidate outgoing neighbor set C of w, based on the inverse graph G. ′ Get the in-neighbor set N of v in (v) Then, a multi-level edge selection strategy is used to select R edges to obtain the edge set P; then, based on the edge set P, the vector w is added to the graph G through edge refinement, and the reverse graph G is updated synchronously. ' ;

[0119] The graph construction module is configured to repeatedly select vectors from the vector dataset V that have not yet been added to graph G, sequentially calling the candidate set retrieval module and the filtering and updating module, until all vectors in the vector dataset V have been added to graph G, ultimately constructing a nearest-neighbor graph G with a balanced graph topology. * ;

[0120] The vector retrieval module is configured to take the query vector q as input and retrieve data from the nearest neighbor graph G. * A greedy approximate nearest neighbor search is performed to obtain the approximate nearest neighbor result of the query vector.

[0121] In one exemplary embodiment, a carrier for implementing a vector retrieval device for a balanced graph topology is provided: a computer device including a memory, a processor, and a computer program stored in the memory.

[0122] In this invention, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the corresponding device.

[0123] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A vector retrieval method for a balanced graph topology, characterized in that, Includes the following steps: Step 1: Based on the vector dataset V and given the target in-degree R, select R+1 vectors from the dataset V, initialize the graph G as the minimum directed complete graph where any vector satisfies the target in-degree R, and at the same time construct its reverse graph G'. Step 2: Select vector w from the vector dataset V that has not yet been added to graph G. Perform a greedy approximate nearest neighbor search on the current graph G using vector w as the query vector. Take all the vectors visited during the search process as the candidate set of out neighbors C for vector w. Step 3: Traverse each candidate outgoing neighbor v in the candidate outgoing neighbor set C of vector w, and obtain the set N of incoming neighbors of outgoing neighbor v according to the reverse graph G'. in (v), and then a multi-level edge selection strategy is used to filter out R edges to obtain an edge set P, in which the starting point and ending point of all edges are unique; then according to the edge set P, the vector w is added to the graph G by edge refinement to optimize the out-neighbors of the existing vectors in the graph, and the reverse graph G' is updated synchronously. Step 4: Repeat steps 2 and 3 above until all vectors in the vector dataset V have been added to graph G, finally constructing a near-neighbor graph G with a balanced graph topology. * ; Step 5: Using the query vector q as input, in the nearest neighbor graph G * A greedy approximate nearest neighbor search is performed to obtain the K vectors with the smallest similarity distance to the query vector q and output them as the vector search results.

2. The vector retrieval method for a balanced graph topology structure according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step 1.1: Calculate the mean vector v of the vector dataset V. means Then calculate the similarity distance between each vector and the mean vector; Step 1.2: Select the vector with the smallest similarity distance to the mean vector as the first vector s of graph G, and also as the entry vector for greedy approximate nearest neighbor retrieval in subsequent steps; Step 1.3: Randomly select R vectors other than s from the vector dataset V. For each of the R+1 vectors including s, establish a directed edge between each pair of different vectors. Use the resulting directed complete graph as the initial graph G, where the out-degree and in-degree of each vector in the graph are both R. At the same time, construct its reverse graph G'.

3. The vector retrieval method for a balanced graph topology structure according to claim 2, characterized in that, Step 2 involves performing a greedy approximate nearest neighbor search on the current graph G using vector w as the query vector. All vectors visited during the search process are used as the candidate set C of outgoing neighbors of w. This includes the following steps: Step 2.1: Maintain a priority queue Q with a capacity of L, sorted in ascending order by similarity distance to vector w, and a set of visited vectors S; Step 2.2: Calculate the similarity distance δ(w,s) between vector w and the entry vector s obtained in step 1.2, and add the entry vector s to the priority queue Q and the set of visited vectors S; Step 2.3: Select the vector u in priority queue Q that has the smallest similarity distance to vector w and has not been expanded. x , get u x The set of out neighbors N out (u x And mark u x It has been expanded; Step 2.4: For u x Each outgoing neighbor v x ∈N out (u x If v x If the vector is not in the visited vector set S, then calculate the relationship between vectors w and v. x The similarity distance δ(w,v) between them x ) and v x Add the vectors to the priority queue Q and the set of visited vectors S, while keeping the top L vectors in priority queue Q that have the smallest similarity distance to vector w. Step 2.5: Repeat steps 2.3 and 2.4 until all vectors in priority queue Q are marked as expanded, and then use the set of visited vectors S as the candidate set of out neighbors C for vector w.

4. The vector retrieval method for a balanced graph topology structure according to claim 3, characterized in that: In step 3, each candidate outgoing neighbor v in the candidate outgoing neighbor set C of vector w is traversed, and the set of incoming neighbors N of outgoing neighbor v is obtained according to the reverse graph G'. in (v) Then, a multi-level edge selection strategy is used to filter out R edges, resulting in an edge set P, where all edges have unique starting points and unique ending points. The specific steps include: Step 3.1: Initialize the edge set P as an empty set and the starting point set C. in With the endpoint set C out For an empty set, first-level, second-level, and third-level candidate neighbor sets An empty set, the first-level candidate neighbor set Second-level neighbor candidate set Third-level neighbor candidate set These are used to store the vectors with the highest, second highest, and lowest priority during the filtering process, respectively. Step 3.2: Sort the candidate outgoing neighbors v in the candidate outgoing neighbor set C in ascending order according to their similarity distance δ(w,v) with vector w; Step 3.3: For each candidate outgoing neighbor v, use the reverse graph G ' Obtain the set of in-neighbors N of the out-neighbor v. in (v) and clear Step 3.4: Traverse each incoming neighbor u∈N of the outgoing neighbor v. in (v), if the incoming neighbor u is not in the starting set C in In the process, the priority of the in-neighbor u is determined based on the similarity distance relationship between vector w, in-neighbor u, and out-neighbor v, and then the in-neighbor u is added to the corresponding set. and middle; Step 3.5: If the first-level incoming neighbor candidate set If the vector is not empty, then select the vector with the smallest similarity distance to vector w as the best in-neighbor u. * Otherwise, if the second-level neighbor candidate set If the vector is not empty, then select the vector with the smallest similarity distance to vector w as the best in-neighbor u. * ,like and If all sets are empty, skip step 3.6; Step 3.6: Add edge (u) * v) Add to edge set P and add the best neighbor u * Add the candidate outgoing neighbor v to the starting set C respectively. in With the endpoint set C out ; Step 3.7: Repeat steps 3.3 to 3.6 above until all candidate outgoing neighbors in the candidate outgoing neighbor set C have been traversed in order, or until the edge set P contains R edges; Step 3.8: If the number of edges in the edge set P is less than R, then re-traverse the neighbor candidate set C in order and execute steps 3.9-3.11; Step 3.9: For each candidate outgoing neighbor v in the candidate outgoing neighbor set C, first clear the list. If the outgoing neighbor v is not in the endpoint set C out In the middle, the set of ingress neighbors N from the outgress neighbor v is... in Select all sets C that are not at the starting point from (v). in Adding neighbors to the set These incoming neighbors have the lowest priority. Step 3.10: If set If the vector is not empty, then select the vector with the largest similarity distance to the outgoing neighbor v as the best incoming neighbor u. * , will the edge (u * v) Add to edge set P and add the best neighbor u * Add the candidate outgoing neighbor v to the starting set C respectively. in With the endpoint set C out ; Step 3.11: Repeat steps 3.9 and 3.10 during the traversal until the edge set P contains R edges.

5. The vector retrieval method for a balanced graph topology structure according to claim 4, characterized in that, In step 3.4, the priority of the in-neighbor u is determined based on the similarity distance relationship between vector w, in-neighbor u, and out-neighbor v, and then added to the corresponding set. and Specifically, it includes the following steps: Step 3.4.1: Calculate the similarity distances δ(w,u), δ(w,v), and δ(u,v); Step 3.4.2: If the similarity distance satisfies δ(w,u)<δ(u,v) and δ(w,v)<δ(u,v), then u has the highest priority and is added to the first-level neighbor candidate set. Step 3.4.3: If the similarity distance satisfies δ(w,u)<δ(u,v) and δ(w,v)≥δ(u,v), then u has the second highest priority and is added to the secondary neighbor candidate set.

6. The vector retrieval method for a balanced graph topology according to claim 5, characterized in that, In step 3, vector w is added to graph G by edge refinement based on edge set P to optimize the outgoing neighbors of existing vectors in the graph, and the reverse graph G' is updated simultaneously. The specific steps include: Step 3.12: For each edge (u) in the edge set P, * Remove vectors (w, v) ∈ P from graph G, and add an outgoing edge (w, v) to vector w, for u. * Add outgoing edges (u * ,w); Step 3.13: Delete the reverse edge (v, u) of each edge in the edge set P in the reverse graph G'. * Add a new reverse edge (v, w) and (w, u). * ).

7. The vector retrieval method for a balanced graph topology according to claim 2, characterized in that, Step 5 specifically includes the following steps: Step 5.1: Maintain a priority queue Q with a capacity of L, sorted in ascending order by similarity distance to the query vector q, and a set of visited vectors S; Step 5.2: Calculate the similarity distance δ(q,s) between the query vector q and the entry vector s obtained in step 1.2, and add s to the priority queue Q and the set of visited vectors S; Step 5.3: Select the vector u in the priority queue Q that has the smallest similarity distance to the query vector q and has not been expanded. x , get u x The set of out neighbors N out (u x And mark u x It has been expanded; Step 5.4: For u x Each outgoing neighbor v x ∈N out (u x If v x If the query vector q is not in the visited vector set S, then calculate the query vector q and v. x The similarity distance δ(q,v) between them x ) and v x Add the vectors to the priority queue Q and the set of visited vectors S, while retaining the top L vectors in priority queue Q that have the smallest similarity distance to the query vector q; Step 5.5: Repeat steps 5.3 and 5.4 until all vectors in priority queue Q have been expanded. Then, select the K vectors from priority queue Q that have the smallest similarity distance to the query vector q as the vector retrieval results output.

8. A vector retrieval device for balanced graph topology, used to implement the vector retrieval method for balanced graph topology as described in any one of claims 1-7, characterized in that, The vector retrieval device for the balanced graph topology structure includes: Initialization module: Based on the vector dataset V and given the target in-degree R, select R+1 vectors from the dataset V to initialize the graph G as the minimum directed complete graph where any vector satisfies the target in-degree R, and at the same time construct its reverse graph G'. Candidate set retrieval module: Used to perform greedy approximate nearest neighbor retrieval on the current graph G with vector w as the query vector, and take all vectors visited during the search process as the candidate set C of out neighbors of vector w; Edge filtering and update module: used to traverse each candidate outgoing neighbor v in the candidate outgoing neighbor set C of vector w, and obtain the set N of incoming neighbors of v based on the reverse graph G'. in (v), and then a multi-level edge selection strategy is used to filter out R edges to obtain the edge set P; then according to the edge set P, the vector w is added to the graph G by edge refinement, and the reverse graph G' is updated synchronously. The graph construction module is used to repeatedly select vectors from the vector dataset V that have not yet been added to graph G, sequentially calling the candidate set retrieval module and the filtering and updating module, until all vectors in the vector dataset V have been added to graph G, ultimately constructing a nearest-neighbor graph G with a balanced graph topology. * ; Vector retrieval module: used to retrieve data from the nearest neighbor graph G, taking the query vector q as input. * A greedy approximate nearest neighbor search is performed to obtain the approximate nearest neighbor result of the query vector.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is executed by the processor, it implements the vector retrieval method for the balanced graph topology structure as described in any one of claims 1-7.

Images