Storage Container Management for Data Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face challenges in balancing scalability and availability while being cumbersome to query, often requiring excessive resources and being inefficient in optimizing data storage and retrieval processes.
Innovation Solution
A data storage system that dynamically manages storage containers by creating new containers when existing ones reach capacity, either by 'spilling' records into new containers in sequential spaces or 'splitting' records into subspaces in finite spaces, optimizing query responses and distributing data across multiple assets for high availability and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional data storage systems are configured to optimize scalability and availability, then the number of data storage assets increases, but the system becomes overly complex and cumbersome to query
Solution Approach 1:
The patent segments data storage into multiple containers of fixed size, where each container holds a specific number of data records. This segmentation allows the system to manage large volumes of data through standardized, manageable units rather than treating storage as a monolithic structure, thereby reducing complexity while maintaining scalability and availability.
2Adaptability or versatility
If conventional data storage systems increase the number of data storage assets to improve scalability, then storage capacity increases, but query efficiency deteriorates
Solution Approach 1:
The patent implements preliminary organization of data records into containers with fixed capacities before queries are executed. By pre-structuring data into manageable containers and maintaining metadata about container contents, the system enables efficient query routing and execution without requiring complex real-time analysis across all storage assets, thus preserving query efficiency while achieving scalability.
3Quantity of substance
If data storage containers are filled to capacity, then storage utilization improves, but the system requires excessive resources to manage data distribution
Solution Approach 1:
The patent changes the parameter of container capacity from variable to fixed, where each container has a predetermined maximum number of data records. This parameter change simplifies resource management because the system can precisely track and allocate storage resources without complex dynamic adjustments. When containers reach their fixed capacity, the system efficiently handles data redistribution through structured operations rather than resource-intensive dynamic reallocation, thus improving storage utilization while controlling resource consumption.
Data Source
AI summary
Various implementations of the invention create storage containers in a data storage system. A computing processor receives a new data record to be stored in a data storage container which is configured to store a fixed number of stored data records. The computing processor determines whether a number of the plurality of stored data records in the data storage container is within a certain threshold of the fixed number of stored data records for the data storage container. For data storage containers residing in a sequential data space, when the number of records is within the certain threshold, the computing processor opens a new data storage container, stores the new data record in the new data storage container, and closes the data storage container to new data records. For the data storage containers residing in a finite data space, when the number of records is within the certain threshold, the computing processor opens a new data storage container, splits the plurality of stored data records in the data storage container into a first plurality of stored data records and a second plurality of stored data records, where each of the first plurality of stored data records has a data value within a first range, and where each of the second plurality of stored data records has the data value within a second range. The computing processor stores the first plurality of stored data records in the data storage container, stores the second plurality of the stored data records in the new data storage container. The computing processors stores the new data record in either the data storage container or the new data storage container based on whether the data value in the new data record corresponds to the first range or the second range.