Cloud Storage Buffering for Local Capacity Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limitations in data storage capacity and management arise from the need to manually manage data transfers between local storage devices and cloud storage systems, which can be cumbersome due to differences in application processes and throughput issues, leading to reduced storage capacity and unreliable data access.
Innovation Solution
A method and system that utilizes a local storage drive to temporarily store data before transferring it to cloud storage, allowing automatic upload and deletion of files, creating 'ghost files' for user accessibility, and managing storage areas for local and cloud-based data, enabling seamless data management and expansion of storage capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is manually transferred between local storage and cloud storage, then data can be stored remotely, but the process becomes cumbersome and time-consuming
Solution Approach 1:
The system performs preliminary actions by automatically initiating data upload processes when files are saved to the drive, eliminating the need for manual intervention. The background transfer mechanism pre-configures the data movement process, allowing uploads to occur automatically in the background without user involvement.
Solution Approach 2:
The drive operates autonomously by self-managing the data transfer process between local and cloud storage. The system monitors its own storage capacity and automatically initiates uploads when the local drive approaches capacity limits, without requiring external control or manual user actions.
2Quantity of substance
If large amounts of data are uploaded to cloud storage, then storage capacity is expanded, but the transfer process may fail and require multiple attempts
Solution Approach 1:
The system divides large data transfer operations into smaller, manageable segments or batches. By processing data in chunks rather than as a single large operation, the system reduces the likelihood of complete transfer failures and enables more granular error handling and recovery for individual segments.
Solution Approach 2:
The system implements error buffering and recovery mechanisms that prepare for potential transfer failures in advance. Checksum verification, temporary staging areas, and automatic retry logic are built into the transfer process to cushion against failures and ensure data integrity without requiring manual intervention.
3Speed
If files are stored locally to maintain fast access, then data retrieval is quick, but the device's storage capacity is limited and reduced
Solution Approach 1:
The system introduces a cloud storage intermediary that acts as an extension of the local drive. The drive presents a unified interface to the user, automatically managing which files are cached locally for fast access and which are stored in the cloud, mediating between the user's local file system and remote cloud storage without requiring user awareness of the distinction.
Solution Approach 2:
The system adds a cloud storage dimension to the traditional local storage model. Instead of only storing data physically present in the device, the system creates a hybrid storage architecture that combines local and remote storage spaces into a single logical volume, effectively expanding capacity without sacrificing local access speed for frequently used files.
4Quantity of substance
If cloud storage is used to expand capacity, then unlimited storage is available, but the throughput and transfer speed between local system and cloud are limited
Solution Approach 1:
The system employs periodic background synchronization rather than continuous or on-demand large transfers. Data is uploaded and synced in periodic batches during idle system periods or off-peak hours, maximizing throughput by utilizing available bandwidth efficiently without interfering with primary system operations or user experience.
Data Source
AI summary
In some embodiments, a computer system may buffer, in a local storage buffer, data to be stored at the cloud and upload the data to the cloud storage from the local storage buffer. Upon receipt of an acknowledgement, from the cloud storage, that the data was successfully uploaded, the computer system may delete the data from the local storage buffer. The computer system may provide a ghost file representation of the uploaded data through the computer system, through which the data uploaded to the cloud storage may be accessed. In some embodiments, the computer system may access multiple cloud storage accounts, each corresponding to a file folder on the computer system, allowing the user to easily access different cloud storage accounts. The computer system may recover data that was deleted from either local persistent storage or the cloud storage.