Bloom Filter Data Location in Distributed Store
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed databases, the name server often becomes a bottleneck due to the need for clients to query it for data location information, leading to reduced throughput and performance.
Innovation Solution
The use of bloom filters to identify probable data locations, allowing clients to access data directly without relying on the name server, by generating and transmitting bloom filters that contain information about data item locations, enabling clients to determine and access data locations independently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If clients query the name server to determine data locations, then clients can access data in the distributed data store, but the name server becomes a bottleneck that reduces throughput
Solution Approach 1:
The system pre-computes and stores bloom filter data structures in the distributed data store that encode information about data locations. When a client needs to locate data, it queries these pre-computed bloom filters directly from the data store nodes, bypassing the need to contact the name server for location information. This preliminary preparation of location data eliminates the bottleneck at the name server while maintaining accurate data location determination.
2Productivity
If clients directly access data without using the name server, then throughput improves, but clients need additional information structures to determine data locations
Solution Approach 1:
The system introduces bloom filters as an intermediary data structure that resides in the distributed data store and mediates between clients and data locations. These bloom filters encode location information in a compact form that clients can query efficiently. The bloom filters act as a self-service mechanism embedded in the data store, allowing clients to determine data locations independently without complex client-side logic or name server involvement.
Data Source
AI summary
A computing device identifies a location of a data item in a distributed data store. The computing device generates at least one probabilistic data structure that contains information to identify a probable location of the data item. The computing device transmits the at least one probabilistic data structure to one or more client computing devices, wherein the at least one probabilistic data structure is usable by the one or more client computing devices to determine the probable location of the data item.