Partitioned Database Data Leakage Detection via Sentinel Value
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-tenant data storage systems, ensuring data isolation between customers is challenging due to the risk of data leakage when queries miss the identifier predicate, leading to potential exposure of data from one customer to others.
Innovation Solution
Introducing a dummy or sentinel value in the customer identifier range, which is never used for valid data storage but helps in query verification by setting one partition as offline, causing queries without a predicate to fail, thus preventing data leakage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate databases are used for each customer, then data isolation and security are improved, but system complexity and maintenance costs increase
Solution Approach 1:
The patent applies segmentation by introducing a partitioning scheme that divides a single shared database into logical partitions using a partition key column. Each customer's data is segmented into specific partitions based on their identifier, providing data isolation similar to separate databases while maintaining a unified physical storage structure. This resolves the contradiction by achieving segmentation benefits without the complexity of multiple separate database systems.
2Device complexity
If data consolidation is implemented, then costs are reduced, but data leakage risk increases
Solution Approach 1:
The patent applies preliminary action by pre-defining partition ranges and establishing partition boundaries before data leakage can occur. The partitioning scheme proactively organizes data into customer-specific partitions with explicit boundary definitions, preventing data leakage before it can happen. This resolves the contradiction by consolidating databases for cost efficiency while preemptively establishing security boundaries.
Solution Approach 2:
The patent introduces a partition key column and partition management mechanism as an intermediary between the consolidated database structure and customer data. This intermediary layer ensures that even though data is consolidated, access is mediated through partition boundaries that prevent unauthorized cross-customer data access. This resolves the contradiction by enabling consolidation while maintaining isolation through the intermediary partitioning layer.
3Ease of operation
If identifier predicate enforcement is manual, then query flexibility is maintained, but correctness verification becomes painstaking and error-prone
Solution Approach 1:
The patent applies self-service by enabling the system to automatically verify predicate correctness through the partitioning scheme. The partition key column and partition boundaries provide self-verifying mechanisms that automatically ensure queries are restricted to appropriate customer partitions without requiring manual verification. This resolves the contradiction by maintaining query flexibility while enabling automatic correctness verification through the self-service partitioning structure.
Data Source
AI summary
Embodiments relate to a process for identifying data leakage in a data storage system. A table is created with multiple units. Each unit in the table has a unique identifier as a leading key in a schema. Two partitions are set in the table, and one of the partitions is set as unavailable. One or more queries are run on the table. Any queries that attempt to access the unavailable partition are identified through an error message or other alert.