Vector Retrieval Acceleration via Parallel Cluster Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing vector retrieval algorithms face performance bottlenecks due to the need to repeatedly load the entire center vector database during the coarse search, leading to increased retrieval time and reduced processing speed as the database size grows.
Innovation Solution
The method and system divide query and center vectors into blocks, performing matrix multiplication and sorting in parallel using calculation and sorting acceleration modules, allowing the center vector database to be loaded only once during the coarse search.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire center vector database is repeatedly loaded from DDR for each query vector during coarse search, then distance calculation can be performed, but the loading time increases rapidly and processing speed decreases as database size grows
Solution Approach 1:
The patent divides the center vector database into multiple clusters, where each cluster is assigned to a dedicated calculation module. This segmentation allows parallel processing of distance calculations across multiple clusters simultaneously, eliminating the need to repeatedly load the entire database for each query vector while maintaining calculation accuracy.
Solution Approach 2:
The patent performs preliminary loading of cluster center vectors into local memory of calculation modules before query processing begins. By pre-loading and caching the center vectors in local memory, the system avoids repeated DDR loading during query execution, significantly improving processing speed while maintaining the ability to perform accurate distance calculations.
2Speed
If the entire center vector database is loaded into local memory for each query, then fast distance calculation is achieved, but the time required to load the database increases rapidly with database size
Solution Approach 1:
The patent segments the large center vector database into smaller cluster-specific subsets, each fitting in local memory of dedicated calculation modules. This allows each module to load only its required cluster data once, rather than repeatedly loading the entire database, reducing loading time while maintaining fast calculation speed for local operations.
Solution Approach 2:
The patent creates a parallel architecture where multiple calculation modules can simultaneously process multiple query vectors against their respective clusters using the same local memory resources. This multi-functional setup allows the system to handle many queries in parallel without repeated loading, achieving both fast calculation and reduced loading time.
3Device complexity
If sequential processing of query vectors is used with RISC-V VPU, then simple hardware architecture is maintained, but the processing speed of the hardware computing system decreases
Solution Approach 1:
The patent divides the processing system into multiple independent calculation modules, each handling a specific cluster. This segmentation enables parallel processing of multiple query vectors across different clusters simultaneously, dramatically improving system throughput while keeping each individual module relatively simple in design.
Solution Approach 2:
The patent transitions from sequential single-threaded processing to parallel multi-threaded processing by introducing multiple calculation modules operating simultaneously. This dimensional change from 1D sequential to 2D/3D parallel architecture increases processing capacity without significantly complicating the fundamental module design.
Data Source
AI summary
A method and system for vector retrieval acceleration includes at least one calculation module and at least one sorting acceleration module. Each calculation module includes a first receiving end and a second receiving end for receiving query vector blocks and center vector blocks respectively, and performs operations on the received query vector blocks and center vector blocks, so as to obtain distance calculation results between each query vector and all center vectors. sorting acceleration module. The at least one calculation module corresponds one-to-one with the at least one sorting acceleration module. Each sorting acceleration module receives the distance calculation results output by the corresponding calculation module, sorts the distance calculation results, so as to obtain a sorting intermediate state, and then stores sorting intermediate states into a system memory.