Two-Dimensional Graph Indexing for Low Latency Search

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional search engines using one-dimensional posting lists are inefficient for searching data graphs, as they struggle to effectively search nodes and relationships, leading to high latency and poor query performance.

Innovation Solution

A two-dimensional index system is introduced, utilizing triples to represent entities and relationships, with additional triples for text searches, chain-path, and converge-path posting lists for efficient querying, and bucketed posting lists for range queries, enabling quick access to result sets and minimizing index space.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If traditional one-dimensional posting lists are used for indexing, then the index structure is simple and easy to implement, but the search latency is high and query performance is poor when searching data graphs

Engineering Contradiction:

Improvesearch latencyVSAvoidindex structure complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The patent transforms the traditional one-dimensional posting list into a two-dimensional index structure where the first dimension represents posting list values (entities) and the second dimension represents intersection identifiers (relationships). This dimensional transformation enables efficient graph traversal by allowing simultaneous access to entities and their relationships, thereby reducing search latency while maintaining manageable complexity through structured organization.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The index is segmented into multiple posting lists, each dedicated to a specific entity type or relationship category. This segmentation allows the system to process different types of graph queries efficiently by routing them to appropriate posting lists, improving search speed while keeping each individual posting list relatively simple in structure.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If comprehensive indexing of all triples is performed, then query coverage is complete, but index space consumption increases significantly

Engineering Contradiction:

Improvequery coverageVSAvoidindex space

Core Design Contradiction:

Adaptability or versatilityVSVolume of stationary object

Solution Approach 1:

The system pre-computes and stores intersection results between posting lists during the indexing phase. By calculating and caching these intersections in advance, the system avoids performing expensive join operations at query time, thereby maintaining complete query coverage while reducing the actual storage requirements for the index structure.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Different posting lists are optimized with different levels of detail and pre-computation based on their specific characteristics and query frequencies. Frequently accessed relationships receive more aggressive pre-computation and optimization, while less critical relationships maintain simpler structures, achieving versatile query coverage with efficient space utilization.

Inventive Principle:

Principle #3Local quality

3Loss of time

If pre-computed intersections are stored for all posting list pairs, then query response time is minimized, but index space and maintenance overhead increase

Engineering Contradiction:

Improvequery response timeVSAvoidindex space

Core Design Contradiction:

Loss of timeVSVolume of stationary object

Solution Approach 1:

The system performs pre-computation of intersections only for posting list pairs that are likely to be queried together, rather than all possible pairs. This partial pre-computation approach significantly reduces query response time for common graph patterns while keeping index space consumption manageable by avoiding redundant pre-computed intersections.

Inventive Principle:

Principle #16Partial or excessive action

4Productivity

If the index structure is optimized for fast retrieval, then query performance improves, but update operations become more complex and slower

Engineering Contradiction:

Improvequery performanceVSAvoidupdate operation simplicity

Core Design Contradiction:

ProductivityVSEase of manufacture

Solution Approach 1:

The index structure is designed to be dynamically adaptable, allowing it to reconfigure itself during update operations. When new triples are added or existing ones modified, the system can efficiently update only the affected posting lists and their intersections, rather than requiring complete re-indexing. This dynamic approach maintains fast query performance while simplifying update operations through localized modifications.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10102268B1Efficient index for low latency search of large graphs

Publication Date: 2018.10.16 GOOGLE LLC

AI summary

A system for efficiently responding to proximity queries may include a memory storing an index for searching a graph-based data store, the index including posting lists for one or more proximity ranges compatible with a space. A posting list can include one or more entities of a type compatible with the space, each entity having a location within the space, the location being a basic unit in a location hierarchy for the space and, for each entity, at least one node in the location hierarchy that falls within the proximity range of the posting list with reference to the location of the entity. The system may also include a memory storing instructions that cause the system to use the index to respond to a query that includes a query proximity range for the space. The space can be a geographic space or a time space.