Inner Product Delaunay Graph for Vector Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for Maximum Inner Product Search (MIPS) in natural language processing and machine learning face inefficiencies due to the lack of effective graph-based solutions that accurately approximate Delaunay Graphs for inner product spaces, leading to performance limitations in retrieving similar word embeddings.
Innovation Solution
The development of Inner Product Delaunay Graph (IPDG) methodology, which includes a new edge selection method and a two-rounds graph construction approach to efficiently approximate Delaunay Graphs in inner product spaces, reducing unnecessary edges and improving search efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional graph-based methods are used for MIPS, then retrieval speed is improved, but accuracy of Delaunay Graph approximation deteriorates
Solution Approach 1:
The patent changes the parameter of graph construction by introducing a two-round process with different edge selection criteria. In the first round, edges are selected based on inner product similarity to capture semantic relationships. In the second round, edges are refined based on geometric Delaunay criteria. This parameter change allows the graph to achieve both speed and accuracy by combining the strengths of both approaches.
Solution Approach 2:
The patent segments the graph construction process into two distinct rounds. The first round focuses on building initial connections based on semantic similarity, while the second round refines these connections based on geometric accuracy. This segmentation allows each round to optimize for its specific goal, ultimately achieving both speed and accuracy.
2Reliability
If more edges are included in the graph, then search completeness is improved, but computational complexity increases
Solution Approach 1:
The patent extracts and removes unnecessary edges from the graph through the two-round construction process. By selectively adding edges only when they meet specific criteria in each round, the method eliminates redundant connections that would increase computational complexity while maintaining search completeness through the refined edge selection.
Solution Approach 2:
The patent applies partial action by not including all possible edges in the graph, but only the most relevant ones identified through the two-round selection process. This partial graph construction maintains sufficient search completeness while significantly reducing computational complexity compared to using all possible edges.
Data Source
AI summary
Retrieval of relevant vectors produced by representation learning can critically influence the efficiency in Natural Language Processing (NLP) tasks. Presented herein are systems and methods for searching vectors via a typical nonmetric matching function: inner product. Embodiments, which construct an approximate Inner Product Delaunay Graph (IPDG) for top-1 Maximum Inner Product Search (MIPS), transform retrieving the most suitable latent vectors into a graph search problem with great benefits of efficiency. Experiments on data representations learned for different machine learning tasks verify the outperforming effectiveness and efficiency of IPDG embodiments.