Dynamic Partition Allocation for Data Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face inefficiencies due to uneven data distribution across partitions, leading to underutilization and saturation issues, resulting in costly and inefficient storage solutions.
Innovation Solution
A data storage system with a system manager, key manager, and allocation manager dynamically allocates storage space by assigning data subsets to partitions based on frequency and capacity, using a key space to manage data values and periodically readjusting storage space to optimize usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the storage medium is augmented to have enough storage to store every data value of the dataset, then the storage capacity is sufficient to handle all possible data values, but the cost increases and the storage space is inefficiently utilized
Solution Approach 1:
The patent implements dynamic partition allocation where partition sizes are not fixed but are continuously adjusted based on actual data distribution patterns. The system monitors data arrival rates and partition utilization in real-time, dynamically resizing partitions to match actual needs rather than provisioning for worst-case scenarios, thereby eliminating wasted storage capacity while maintaining sufficient space for all data values.
Solution Approach 2:
The system changes the parameter of partition sizes from static predetermined values to dynamic values that adapt based on observed data characteristics. By continuously monitoring data arrival patterns and adjusting partition capacities accordingly, the system optimizes storage utilization without compromising the ability to store all necessary data values.
2Device complexity
If partitions are predetermined with fixed sizes, then the storage structure is simple and easy to manage, but data distribution becomes uneven leading to saturation and underutilization
Solution Approach 1:
The patent transforms the static partition structure into a dynamic one where partition sizes automatically adjust based on data distribution. The system incorporates continuous monitoring mechanisms that track data arrival rates and partition utilization, triggering automatic resizing operations to maintain balanced utilization across all partitions, thereby eliminating both saturation and underutilization without significantly increasing operational complexity.
Solution Approach 2:
The system implements feedback loops where partition performance metrics (utilization levels, data arrival rates) are continuously monitored and fed back to the partition management logic. This feedback mechanism enables automatic adjustments to partition sizes based on actual data patterns, ensuring optimal storage utilization while maintaining manageable system complexity through automated control.
3Adaptability or versatility
If the system receives different types of data at different rates, then the data distribution reflects actual data source behavior, but certain partitions become saturated while others remain under-utilized
Solution Approach 1:
The patent implements dynamic partition allocation that automatically adapts to varying data arrival rates. The system continuously monitors the rate at which different data types are received and dynamically adjusts partition sizes to match these patterns, ensuring that partitions handling high-rate data streams are enlarged while those handling low-rate streams are reduced, thereby optimizing storage space distribution across all partitions.
Solution Approach 2:
The system changes partition capacity parameters in response to observed data rate variations. By monitoring data arrival patterns and adjusting partition sizes accordingly, the system optimizes storage allocation to match actual data distribution patterns, preventing both saturation of high-rate partitions and underutilization of low-rate partitions.
4Stability of the object's composition
If data storage space is not dynamically adjusted, then the system operation is simple and stable, but storage resources are either wasted or insufficient
Solution Approach 1:
The patent introduces dynamic adjustment mechanisms that maintain system stability through controlled, incremental changes rather than abrupt modifications. The partition resizing operations are performed systematically based on monitored performance thresholds, ensuring that the system adapts to changing data patterns while maintaining operational stability. The dynamic nature of the system allows it to respond to actual data distribution without compromising stability.
Solution Approach 2:
The system implements controlled parameter changes to partition sizes based on observed data patterns. Rather than making arbitrary adjustments, the system modifies partition capacity parameters systematically in response to measured performance metrics, thereby optimizing storage resource allocation while maintaining stable and predictable system operation.
Data Source
AI summary
The disclosed embodiments include data storage systems and methods to store data. In one embodiment, a computer-implemented method for storing data is disclosed. The method includes receiving a data value of a dataset. The method also includes assigning a plurality of keys of a key space to a plurality of data values of the dataset. The method further includes determining whether to readjust storage space of at least one partition of a plurality of partitions based on data values stored on the plurality of partitions. In response to a determination to readjust the at least one partition, the method further includes dynamically re-mapping the key space to readjust a number of keys of the plurality of keys that are assigned to data values stored on the plurality of partitions based on a number of data values that are stored on the plurality of partitions.