Dynamic Database Partitioning via Automatic Creation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage partitioning mechanisms require manual intervention by database administrators to create new partitions, which is inefficient and resource-intensive, as administrators must anticipate future needs and manage empty partitions, leading to potential delays and resource wastage.
Innovation Solution
Implementing an automatic partition creation mechanism within the database system, where a values-to-partition mapping is maintained to identify new data items that don't fit existing partitions, allowing for the dynamic creation of new partitions based on user-specified intervals or time-based triggers, ensuring data items are stored efficiently without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual partition creation by database administrators is used, then partition control and planning is possible, but administrative burden and resource wastage increase
Solution Approach 1:
The database system automatically creates partitions based on incoming data characteristics without requiring manual intervention from database administrators. The system monitors data insertion, detects when new partitions are needed based on predefined criteria, and creates appropriate partitions autonomously, eliminating the need for administrative time investment in partition management.
Solution Approach 2:
The system pre-establishes partitioning criteria and thresholds before data insertion begins. When data arrives that falls outside existing partition ranges, the system automatically creates new partitions in advance of any potential performance issues, ensuring continuous efficient data storage without waiting for administrative intervention.
2Reliability
If all partitions are created in advance, then future data storage needs are covered, but resource wastage occurs with empty partitions
Solution Approach 1:
The partitioning scheme transitions from a static pre-defined structure to a dynamic adaptive structure. Partitions are created on-demand based on actual data characteristics and insertion patterns. The system continuously monitors data distribution and automatically adjusts the partition structure by creating new partitions only when necessary, optimizing storage resource utilization while maintaining reliability.
3Device complexity
If manual partition creation is used, then partition structure can be controlled, but productivity decreases due to administrative tasks
Solution Approach 1:
The database system performs self-management of partition creation and optimization. It automatically analyzes incoming data, determines when new partitions are required, creates appropriate partitions, and maintains optimal data distribution without requiring administrative involvement. This self-service capability eliminates productivity losses associated with manual partition management while keeping the system relatively simple.
Data Source
AI summary
A method and apparatus for managing data in a partitioned data storage space is provided. Data referencing a new data item to be stored in the data storage space is received. It is determined that the data item does not pertain to any previously created partition. In response to the determination, a new partition is automatically created in the data storage space. The partition is mapped to a new set of values, such as to a next unpartitioned interval of dates or numbers.