Dual-Stage Vector Search With Quantized Retrieval Re-Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Vector databases in AI applications experience significant storage bloat due to large embedding vectors and indexing structures, which increases storage space requirements and slows down retrieval processes.
Innovation Solution
A dual-stage vector search process involving quantized vectors during training and re-ranking of base vectors during inference to reduce storage space and maintain accuracy, utilizing binary quantization and GPUs for faster searches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large base embedding vectors are stored in the vector database, then retrieval accuracy is improved, but storage space requirements increase significantly
Solution Approach 1:
The patent segments the vector search process into two distinct stages: a first stage using quantized vectors for initial retrieval, and a second stage using base vectors for re-ranking. This segmentation allows the system to store compact quantized vectors while maintaining accurate base vectors temporarily during inference, thus resolving the contradiction between storage efficiency and retrieval accuracy.
Solution Approach 2:
The patent applies parameter changes by transforming base vectors into quantized vectors with reduced precision (e.g., from 32-bit floats to lower precision representations). This parameter transformation enables significant storage compression while preserving sufficient information for effective retrieval, and the system compensates for precision loss through the two-stage re-ranking process.
2Measurement precision
If high-dimensional embedding vectors are used, then search accuracy is improved, but indexing data structure size increases
Solution Approach 1:
The patent segments the vector representation into quantized form for indexing purposes, reducing the dimensional complexity stored in the indexing data structures. The full-precision base vectors are generated temporarily during inference for accurate comparison and re-ranking, thus reducing indexing structure size while maintaining search accuracy.
3Quantity of substance
If quantized vectors are used to reduce storage size, then storage efficiency is improved, but information loss occurs
Solution Approach 1:
The patent performs preliminary quantization of vectors during the training phase, storing the quantized representations in the vector database. This preliminary action enables efficient storage and retrieval, while the system prepares to compensate for information loss by generating full-precision base vectors during the inference stage for accurate re-ranking of retrieved results.
Solution Approach 2:
The two-stage search process implements a feedback mechanism where the initial retrieval using quantized vectors feeds into a second stage that uses base vectors for re-ranking. This feedback loop compensates for the information loss in quantization by allowing the system to refine results with higher precision representations, thus recovering lost information.
Data Source
AI summary
The disclosure describes system, devices, and methods for dual-stage vector search. In an example implementation, a method for operating a computer-implemented service is provided. The method includes receiving a context request for content with which to augment a prompt, generating a base vector based on input data in the context request and quantizing the base vector to produce a quantized vector. The method also includes searching a vector database to identify content items based at least on the quantized vector and obtaining the content items and generating base vectors for the content items. The method further includes selecting a subset of the content items based on at least on the base vector generated for the input data and the base vectors for the content items.