Multi-layer Semantic Search Using Compressed Vector Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semantic search in semantic vector spaces poses a large computational burden due to the need for exhaustive searches to find true nearest neighbors, which is prohibitive in terms of latency and memory consumption, especially in web search applications requiring rapid responses with modest memory usage.
Innovation Solution
A method involving a semantic search machine with three layers: the routing layer identifies candidate clusters, the preview layer selects a refined set of candidate vectors using compressed answer vectors, and the full-view layer deterministically finds the best answer by retrieving and comparing uncompressed vectors, leveraging a multi-layer graph and product-quantization codes to reduce latency and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exhaustive search is performed to find true nearest neighbors in semantic vector space, then measurement precision is improved, but productivity deteriorates due to large computational burden
Solution Approach 1:
The patent divides the semantic vector space into multiple clusters, where each cluster contains a subset of vectors. The search process is segmented into two stages: first identifying candidate clusters using compressed representations, then performing detailed search only within selected clusters. This segmentation reduces the overall computational burden while maintaining search accuracy.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing compressed representations (codebooks) of semantic vectors before the actual search. During search, these pre-computed compressed forms are used to quickly identify candidate clusters, avoiding the need to process all full-precision vectors from scratch. This preliminary preparation significantly speeds up the search process.
2Measurement precision
If exhaustive search is performed to find true nearest neighbors, then measurement precision is improved, but loss of time increases due to prohibitive latency
Solution Approach 1:
The patent segments the search process into candidate cluster identification and detailed candidate evaluation. By using compressed vector representations for initial cluster selection, the system avoids time-consuming comparisons with all vectors in the database, thereby reducing search latency while maintaining accuracy through subsequent detailed evaluation of selected candidates.
Solution Approach 2:
The patent performs preliminary clustering and compressed representation creation before the actual search query is processed. This pre-computation allows the system to quickly narrow down candidate vectors during the search phase, significantly reducing the time required to find nearest neighbors while preserving measurement precision.
3Measurement precision
If full uncompressed vectors are stored in memory for search, then measurement precision is improved, but quantity of substance increases due to high memory consumption
Solution Approach 1:
The patent creates compressed copies (codebooks) of the semantic vectors that retain the essential semantic information in a more compact form. These compressed representations are stored instead of, or in addition to, the full uncompressed vectors. The compressed copies enable efficient candidate selection with minimal memory usage, while full vectors are only loaded into memory when needed for final candidate evaluation.
Solution Approach 2:
The patent segments the data storage into compressed representations stored in main memory and full uncompressed vectors stored in slower, higher-capacity storage. This segmentation allows the system to maintain high measurement precision by keeping full vectors available when needed, while reducing active memory consumption through the use of compressed forms for the majority of operations.
Data Source
AI summary
A method for semantic search includes receiving a query vector including a semantic feature value for each of a plurality of semantic feature dimensions. A cluster is selected from a plurality of different candidate clusters held in a relatively fast memory, each candidate cluster including a plurality of compressed answer vectors. A subset of the plurality of compressed answer vectors are promoted as candidate answers. For each of the candidate answers, a corresponding uncompressed answer vector is retrieved from a relatively slower memory. A selected answer is promoted from among the candidate answers.