Turbo Locally-Adaptive Vector Quantization for Faster Similarity Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing similarity search technologies face challenges with limited throughput and substantial memory footprint at large database sizes, leading to computational bottlenecks in end-to-end applications, particularly in graph-based methods, which struggle with random memory access patterns and inefficient vector decompression.
Innovation Solution
Implementing Turbo Locally-Adaptive Vector Quantization (Turbo LVQ) to permute the storage order of vector dimensions, enabling faster unpacking using SIMD instructions, reducing the number of assembly instructions required for decompression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If graph-based methods are used for similarity search, then search accuracy is improved, but throughput is limited and memory footprint increases
Solution Approach 1:
The patent segments the high-dimensional vector space by organizing vectors into clusters and sub-clusters, creating a hierarchical structure. This segmentation allows the search to proceed in stages, first identifying relevant clusters and then searching within those clusters, thereby improving throughput without sacrificing search accuracy.
Solution Approach 2:
The patent introduces a hierarchical dimension to the vector storage structure, organizing vectors into multiple levels of clusters. This additional organizational dimension enables more efficient memory access patterns and reduces the computational burden of searching through all vectors, thus improving throughput while maintaining accuracy.
2Measurement precision
If graph-based methods are used for similarity search, then search accuracy is improved, but memory footprint increases
Solution Approach 1:
By segmenting the vector database into hierarchical clusters, the patent reduces the number of vectors that need to be loaded into memory simultaneously. Each cluster contains a subset of vectors, allowing the system to work with smaller memory footprints while maintaining search accuracy through systematic cluster traversal.
Solution Approach 2:
The patent performs preliminary organization of vectors into hierarchical clusters before the actual search operation. This preliminary action creates an efficient memory layout that reduces the memory footprint during search operations, as only relevant clusters need to be accessed and loaded into memory.
3Device complexity
If standard vector storage order is used, then memory layout is simple, but decompression efficiency is low
Solution Approach 1:
The patent reorganizes vector dimensions into a transposed memory layout, where dimensions are stored contiguously across different vectors rather than vectors being stored contiguously. This dimensional reorganization enables SIMD instructions to operate on multiple dimension values simultaneously, dramatically improving decompression efficiency while maintaining reasonable memory layout complexity.
Solution Approach 2:
The patent changes the storage parameter from row-major order (vectors stored sequentially) to column-major order (dimensions stored sequentially). This parameter change in memory organization enables parallel processing of dimension values using SIMD instructions, improving decompression efficiency without significantly increasing memory layout complexity.
4Quantity of substance
If quantization is applied to reduce memory footprint, then memory efficiency is improved, but distance computation accuracy decreases
Solution Approach 1:
The patent applies different precision levels to different parts of the vector data structure. By organizing vectors into hierarchical clusters, the system can use lower precision representations for cluster identification while maintaining higher precision for actual distance computations within relevant clusters, thus reducing overall memory footprint while preserving distance computation accuracy.
Solution Approach 2:
The patent performs preliminary quantization and clustering operations to reduce memory footprint before the actual similarity search. By pre-organizing vectors into clusters with quantized representations, the system reduces memory requirements while maintaining the ability to perform accurate distance computations on the full-precision vectors when needed for final similarity calculations.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Systems, apparatuses and methods may provide for technology that conducts, in accordance with a first instruction, a load of a block of data into a register, wherein the block of data is to include a plurality of lanes, conducts, in accordance with a second instruction, a first bitwise mask application to each lane in the plurality of lanes, and extracts a set of vector dimensions from the block of data based on the first bitwise mask application.