Distributed In-Memory Spatial Data Store for K-Nearest Neighbor Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in handling heavy write loads and scaling complex index techniques for real-time K-nearest neighbor searches in ride-hailing services, particularly due to high I/O costs and the need for efficient indexing of moving objects.
Innovation Solution
A scalable in-memory spatial data store with decentralized data storage across multiple nodes, using a two-level index hierarchy and consistent hashing to reduce remote calls and optimize K-nearest neighbor searches by distributing data based on geographical relationships, allowing for fast and efficient querying of nearest moving objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If complex index techniques (R-tree, TPR-tree, TB-tree) are used for K-nearest neighbour search on moving objects, then search performance is improved, but system complexity and difficulty of scaling increase
Solution Approach 1:
The patent divides the geographical space into multiple shards, each stored on a separate node in the distributed system. This segmentation allows the system to handle large-scale data by distributing it across multiple nodes, reducing the complexity of maintaining a single global index while preserving efficient search performance through localized indexing within each shard.
Solution Approach 2:
The patent introduces a new dimension of organization by using consistent hashing to map geographical locations to storage nodes. This hashing function creates a virtual dimension that transforms physical geographical proximity into a manageable data distribution pattern, enabling efficient K-nearest neighbour searches without requiring complex traditional spatial indexes.
2Reliability
If traditional databases are used to track moving objects, then data persistence is ensured, but I/O costs increase significantly under heavy write loads
Solution Approach 1:
The patent replaces the mechanical disk-based storage system with an in-memory data store. By moving data from disk to RAM, the system eliminates the high I/O costs associated with traditional databases while maintaining data persistence through the distributed memory architecture. This substitution fundamentally changes the storage mechanism to achieve both speed and reliability.
3Stability of the object's composition
If centralized data storage is used for K-nearest neighbour search, then data consistency is maintained, but scalability and load distribution are limited
Solution Approach 1:
The patent segments the centralized data storage into distributed nodes, each responsible for specific geographical shards. This segmentation enables scalability by allowing the system to add more nodes as data volume increases, while data consistency is maintained through the consistent hashing mechanism that ensures proper data distribution and retrieval across the distributed architecture.
Data Source
AI summary
A database system is configured to enable fast searching for neighbours nearest to a mobile object located in a geographical space made up of plural spatially distinct subspaces, each being made up of plural cells. The database system has an operating system controlling storage of object data amongst the plural storage nodes, to represent one or more spatially distinct subspaces, in a respective single one of the storage nodes. Location data of each object is used to index that object with respect to cells making up each spatially distinct subspace in each node.