Hybrid Storage File Segmentation for Power and Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage methods for less-frequently used digital information are cost-inefficient due to power consumption by on-premise storage devices, and managing digital information across cloud-based storage faces challenges like data integrity and accessibility issues during file migrations and system crashes.
Innovation Solution
Implementing a hybrid storage system that divides files into portions, storing metadata locally and file portions on cloud-based storage, with each portion named using file metadata, allowing for seamless integration of local and cloud-based storage, enabling cost-effective storage and ensuring data integrity through consistent file management across devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If on-premise storage devices are used to store less-frequently used digital information, then data accessibility and reliability are improved, but power consumption and storage costs increase
Solution Approach 1:
The storage system is segmented into two distinct parts: local storage devices for frequently accessed data and cloud-based storage for less frequently used data. This segmentation allows the system to maintain high reliability for critical data while avoiding continuous power consumption for archival data, as cloud storage can be accessed on-demand without maintaining constant local presence.
Solution Approach 2:
A hybrid storage management system acts as an intermediary between local storage devices and cloud-based storage. This mediator intelligently manages data placement, migration, and retrieval, ensuring that data accessibility is maintained through automated workflows while minimizing local storage usage for cold data, thereby reducing power consumption.
2Quantity of substance
If data is migrated to cloud-based storage, then storage costs are reduced, but data integrity and accessibility during system crashes deteriorate
Solution Approach 1:
The system implements beforehand cushioning by maintaining local storage buffers and caching mechanisms that hold critical data and metadata before and during cloud migrations. This ensures that even if cloud storage experiences issues during system crashes, the local buffers provide a safety net that preserves data integrity and enables recovery.
Solution Approach 2:
Different quality levels are applied to different data portions based on their access patterns and criticality. Frequently accessed and critical data maintains high local quality with redundancy and caching, while less critical archival data uses standard cloud storage quality. This differentiated approach ensures data integrity for important information while achieving cost savings for non-critical data.
3Quantity of substance
If hybrid storage management is implemented, then storage cost efficiency is improved, but system complexity increases
Solution Approach 1:
The hybrid storage management system operates with a high degree of self-service through automated decision-making algorithms that independently determine data placement, migration timing, and retrieval priorities based on predefined policies and real-time conditions. This automation eliminates the need for complex manual management procedures, reducing operational complexity while maintaining cost efficiency.
Solution Approach 2:
The system manages complexity by dynamically changing key parameters such as data placement thresholds, migration triggers, and caching strategies based on workload patterns and storage conditions. These parameter adjustments allow the system to adapt to changing requirements without requiring fundamental architectural changes, thereby controlling system complexity while optimizing storage costs.
Data Source
AI summary
The disclosed computer-implemented method for storing information within hybrid storage with local and cloud-based storage devices may include (1) dividing, if required, at the computing device, a file into multiple portions, (2) storing a first portion of the multiple portions as at least one first respective separate object on a local volume stored on the local storage device, (3) storing a second portion of the multiple portions as at least one second respective separate object on a cloud-based volume stored on a cloud-based storage device, and (4) naming each stored portion of the file with a respective object name comprising file metadata describing respective characteristics of the respective portion of the file. Various other methods, systems, and computer-readable media are also disclosed.