Hierarchical Data Partitioning for Multi-Tenant Database Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for data management in large database systems, such as multi-tenant systems, require copying entire databases, leading to inefficient data partitioning, high storage costs, and prolonged processing times, which can introduce inconsistencies and are not easily manageable.
Innovation Solution
Implementing a data partitioning method that uses defined categorization levels and associated partition keys to segment data hierarchically, allowing for efficient and intuitive segmentation of large data blocks, enabling partial copying based on user-defined criteria and maintaining data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire database is copied row by row, then complete data copy is achieved, but storage cost increases enormously and processing time is prolonged
Solution Approach 1:
The patent segments the database into multiple partitions based on partition keys (e.g., organization_id, tenant_id). Instead of copying the entire database, only specific partitions are copied to the sandbox environment. This segmentation allows selective data copying, reducing storage costs while maintaining data completeness for the required scope.
Solution Approach 2:
The patent extracts only the necessary partitions from the source database based on user-defined criteria and partition keys. By taking out only the required data subsets rather than copying everything, the system achieves complete data copy for the needed scope while significantly reducing storage requirements.
2Reliability
If entire database is copied row by row, then complete data copy is achieved, but processing time is prolonged and maintenance window increases
Solution Approach 1:
The database is divided into partitions that can be copied independently and in parallel. This segmentation enables the copy operation to process multiple partitions simultaneously, significantly reducing the total maintenance window while ensuring complete data copy for each required partition.
Solution Approach 2:
The patent implements partial copying by selecting only the necessary partitions based on user criteria rather than copying the entire database. This partial action approach reduces processing time and maintenance window while achieving sufficient data completeness for the sandbox environment's needs.
3Reliability
If large amount of data is copied, then data completeness is achieved, but data inconsistencies are introduced during extended copy process
Solution Approach 1:
By segmenting the database into smaller partitions with well-defined partition keys, the system can copy each partition atomically and independently. This reduces the window for data changes during copying, minimizing inconsistencies while maintaining completeness of each partition's data.
Solution Approach 2:
The patent copies only the necessary partitions rather than the entire database, reducing the total copy time and the window during which data inconsistencies could occur. This partial copying approach achieves sufficient data completeness while maintaining consistency within the copied scope.
4Quantity of substance
If partition keys are defined for data segmentation, then storage efficiency improves, but system complexity increases
Solution Approach 1:
The patent introduces partition keys as a systematic way to segment data, which improves storage efficiency by enabling selective copying. While this adds some complexity, the use of standard database partitioning concepts and automated tools manages this complexity, achieving net benefit through improved storage efficiency.
Data Source
AI summary
A method for data partitioning method includes defining a set of category levels associated with a plurality of entities stored within a first database, wherein the set of category levels is hierarchical (e.g., proceeding from higher to lower levels). Each of the plurality of entities is assigned to a category level within the set of category levels. One or more partition keys are defined for at least one of the category levels. The plurality of entities are then copied to a second database based on the set of category levels and the one or more partition keys.