Vector Search Request Pipelining for Multiple Outstanding Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vector database systems experience idle time and reduced hardware resource utilization due to dependency on vector index hosts during vector search processing, leading to inefficiencies in handling multiple outstanding vector requests.
Innovation Solution
Implementing a method that allows the vector index host to transmit multiple search requests to vector search hardware concurrently, reducing dependency on the vector index host and eliminating idle time by scheduling and batching search tasks efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the vector search hardware waits for response from vector index host before processing next search request, then the dependency on vector index host is maintained, but idle time increases and hardware resource utilization decreases
Solution Approach 1:
The vector search hardware performs preliminary actions by pre-processing search requests and maintaining an outstanding request queue before receiving responses from the vector index host. This allows the hardware to be ready to process subsequent requests immediately, eliminating idle waiting time and improving resource utilization while maintaining proper dependency management through the queue structure.
2Device complexity
If the vector index host processes requests sequentially, then dependency control is simplified, but idle time increases and search efficiency decreases
Solution Approach 1:
The system implements continuous useful action by maintaining a pipeline where the vector index host continuously sends search requests to the vector search hardware without idle gaps. The hardware processes requests continuously from the outstanding queue, ensuring that both components remain actively engaged in productive work, thereby eliminating idle time while managing dependency through the queue mechanism.
3Productivity
If multiple search requests are transmitted concurrently to vector search hardware, then hardware resource utilization improves, but complexity of managing outstanding requests increases
Solution Approach 1:
The system segments the request management process by dividing outstanding requests into discrete, manageable units in a queue structure. Each search request is treated as an independent segment that can be tracked, processed, and responded to individually. This segmentation reduces the perceived complexity by breaking down the management of multiple concurrent requests into smaller, organized units that are easier to handle systematically.
Data Source
AI summary
A method of processing multiple outstanding vector requests in a vector search system includes transmitting, by an apparatus for executing a vector index host, one or more queries to an apparatus for executing a vector search upon receiving the one or more queries; creating, by the apparatus for executing a vector index host, a candidate node list for searching the queries, for each of the queries; confirming, by the apparatus for executing a vector index host, a multiple-search counting number, selecting one or more search target nodes from the candidate node list, and transmitting a search request with respect to neighbor nodes of the search target node to the apparatus for executing the vector search; and transmitting, by the apparatus for executing a vector search, a search result for the search request to the apparatus for executing a vector index host.