Virtual Node Graph Index for Isolated Vector Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional graph-based indexes for nearest neighbor search in high-dimensional vector data often suffer from reduced search accuracy due to graph structures that can isolate nodes, making it difficult to retrieve similar data when the target node lacks neighboring nodes in its neighborhood.
Innovation Solution
The introduction of a virtual node in the graph-based index system to enhance search paths and improve accuracy by adding virtual nodes to the graph, allowing for easier retrieval of isolated or difficult-to-reach nodes, thereby improving the search accuracy by creating new search paths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a graph-based index is used for nearest neighbor search in high-dimensional vector data, then retrieval speed is improved, but search accuracy deteriorates when nodes are isolated in the graph
Solution Approach 1:
The patent introduces virtual nodes as intermediary elements between isolated nodes and the rest of the graph. These virtual nodes act as mediators that enable search paths to reach isolated nodes without requiring direct connections in the original graph structure, thus maintaining retrieval speed while improving search accuracy.
Solution Approach 2:
The patent extends the graph structure by adding virtual nodes that create additional pathways in the search space. This dimensional expansion allows the search algorithm to access isolated nodes through new routes, effectively resolving the accuracy-speed tradeoff by utilizing the virtual node layer.
2Productivity
If the graph structure is used to connect nodes for similarity search, then retrieval efficiency is improved, but accessibility to isolated nodes deteriorates
Solution Approach 1:
Virtual nodes serve as intermediaries that bridge isolated nodes to the main graph structure. This allows efficient retrieval operations to access isolated nodes through the virtual node pathways, maintaining high productivity while improving ease of access to previously unreachable nodes.
Solution Approach 2:
The virtual nodes are pre-configured in the graph structure during indexing, establishing preliminary pathways to isolated nodes before search operations occur. This preliminary setup ensures that when searches need to access isolated nodes, the pathways are already in place, maintaining efficiency without sacrificing accessibility.
Data Source
AI summary
An information processing apparatus according to the application concerned includes an obtaining unit and a generating unit. The obtaining unit obtains a first-type graph including a plurality of node corresponding to a plurality of search target. The generating unit generates, based on placement of the plurality of nodes in the first-type graph, a second-type graph that includes a virtual node representing a node not associated to a search target by adding the virtual node to the first-type graph.