Context-Supplemented Vector Index Search for Efficient Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vector-based search engines face inefficiencies in identifying relevant data items due to the need for repeated probing of indexes and additional computational operations to filter out non-matching results, leading to wasted time and resources.
Innovation Solution
The technique generates a source context-supplemented vector by combining a language-agnostic vector and a context vector, which includes parameters like natural language, geographic region, and user behavior, to directly identify matching target data items, reducing the need for repeated searches and improving resource efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a search engine uses distributed vectors to represent data items, then the quality of search output results is improved, but the efficiency of vector-based search operations deteriorates due to repeated probing and additional computational operations
Solution Approach 1:
The patent segments the search process into two distinct phases: (1) an initial probe phase that retrieves candidate items using distributed vectors, and (2) a filtering phase that efficiently eliminates non-matching items using context vectors. This segmentation allows the system to maintain high search quality while improving efficiency by processing only relevant candidates in detail.
Solution Approach 2:
The patent performs preliminary action by pre-computing and storing context vectors alongside distributed vectors in the index. This preliminary preparation enables the system to quickly filter candidates during search operations without performing computationally expensive operations on all items, thus improving search efficiency while maintaining result quality.
2Reliability
If a computing system repeatedly probes the index to locate sufficient matching target data items, then the completeness of search results is improved, but the time and computational resources are wasted
Solution Approach 1:
The system performs preliminary action by pre-computing context vectors for all data items and storing them in the index. This allows the search operation to quickly filter candidates using simple context vector comparisons rather than repeatedly probing with complex distributed vector operations, reducing search time while ensuring completeness.
Solution Approach 2:
The patent applies partial action by retrieving a superset of candidate items in the initial probe phase and then filtering to the exact number of matching items needed. This approach ensures completeness by initially retrieving more items than necessary, then efficiently reducing the set using context vector filtering, rather than repeatedly probing to find the exact number needed.
3Measurement precision
If a computing system filters a set of candidate target data items to identify a relevant subset, then the precision of search results is improved, but additional computational operations are incurred
Solution Approach 1:
The patent segments the filtering operation into two stages: (1) initial candidate retrieval using distributed vectors, and (2) precision filtering using context vectors. This segmentation reduces computational complexity by using simple context vector comparisons for filtering rather than complex distributed vector operations, improving precision while reducing the complexity of each filtering step.
Solution Approach 2:
The patent changes the parameter used for filtering from distributed vectors to context vectors. Context vectors contain simplified contextual information that can be compared more efficiently than full distributed vectors, thereby reducing computational complexity while maintaining or improving result precision through better contextual matching.
Data Source
AI summary
Technology is described herein for searching an index, including operations of: obtaining a source data item; generating a source context-supplemented vector based on the source data item; and searching the index to find one or more target context-supplemented vectors that are determined to match the source context-supplemented vector. Each context-supplemented vector, which is associated with a particular data item, is made up of two parts: a language-agnostic vector and a context vector. The language-agnostic vector expresses the meaning of the particular data item in a manner that is independent of a natural language that is used to express the particular data item, while the context vector expresses a context associated with the formation of the particular data item. More generally, the technology's use of context vectors allows it to perform index search operations in a more efficient manner, compared to a search engine that does not use context vectors.