Shredded Store Blob Aggregation for Remote Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote data storage systems face challenges with high costs and user-perceptible delays due to large data sizes overwhelming networks, especially when handling incremental updates and historical versions of files.
Innovation Solution
The implementation of a 'shredded store' configuration that aggregates data into smaller units (blobs) based on client-side and server-side parameters, allowing for efficient data transfer and storage by grouping changes rather than entire files, reducing write costs and disk space utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is stored remotely on another computer, then data accessibility across multiple devices is improved, but network delays and storage costs increase
Solution Approach 1:
The patent segments files into smaller chunks or blocks, allowing selective transmission and storage of only modified portions rather than entire files. This reduces network delay by transmitting smaller data units and enables parallel processing of multiple chunks.
Solution Approach 2:
The patent extracts only the changed portions (deltas) of files for transmission and storage, rather than transferring complete files. This extraction of essential changes reduces network bandwidth consumption and storage requirements while maintaining data accessibility.
2Reliability
If large amounts of data are transmitted over the network, then complete file updates are achieved, but network bandwidth is overwhelmed causing user-perceptible delays
Solution Approach 1:
The system extracts and transmits only the modified portions of files (change deltas) rather than complete files. This maintains data completeness for updated sections while dramatically reducing network bandwidth consumption and transfer time.
Solution Approach 2:
The patent implements partial action by transmitting only the necessary portion of data (changes rather than complete files). This partial transmission approach maintains sufficient data completeness for updates while improving transfer speed.
3Reliability
If entire files are stored and updated remotely, then data integrity is maintained, but storage costs and disk space utilization increase
Solution Approach 1:
The patent segments files into chunks and stores only the modified chunks rather than entire files. This segmentation maintains data integrity for updated portions while reducing overall storage space requirements by eliminating redundant unchanged data.
Solution Approach 2:
The system discards redundant unchanged portions of files and recovers complete files by reconstructing them from stored chunks and local cached portions. This approach maintains data integrity while reducing storage space utilization.
Data Source
AI summary
The concepts relate to data handling, and more specifically to data handling scenarios where data is revised on one computer and stored on another computer. One example can obtain a set of blobs relating to revisions of a file. The example can determine a target size of datastore blobs. In an instance where a total size of the set of blobs is less than the target size, this example can aggregate the set of blobs into an individual datastore blob. Otherwise, the example can identify new or edited individual blobs of the set and aggregate the new or edited individual blobs into first datastore blobs. The example can also aggregate other individual blobs of the set into second datastore blobs.