Cloud Storage Object Generation from Access Patterns
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Legacy techniques for generating off-premises storage objects in distributed storage systems are inefficient, leading to increased object access latency, higher computing resource usage, and decreased user satisfaction due to the lack of logical data association information, resulting in multi-object access during operations.
Innovation Solution
The techniques capture data access patterns to group sequentially accessed data blocks into objects for storage in off-premises network-accessible object storage facilities, reducing the demand for computer memory, processing power, and inter-component communication by using access pattern capture trigger signals and metadata to map data blocks to objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data blocks are divided into equally sized chunks for storage in object storage facility, then storage simplicity is improved, but object access latency increases due to multi-object access requirements
Solution Approach 1:
The system performs preliminary analysis of data access patterns before storing data blocks in the object storage facility. By capturing and analyzing access patterns in advance, the system can pre-group data blocks that are likely to be accessed together into single objects, thereby avoiding multi-object access latency when actual data access occurs.
Solution Approach 2:
The system introduces a new dimension of organization by grouping data blocks based on access pattern similarity rather than just sequential ordering. This creates a logical dimension of access affinity that transcends the physical storage dimension, allowing related data blocks to be co-located in the same object even if they are not sequentially adjacent.
2Ease of manufacture
If data blocks are divided into equally sized chunks, then storage organization is simplified, but computing resource usage increases due to multiple object accesses
Solution Approach 1:
The system merges data blocks that exhibit similar access patterns into single storage objects. By combining multiple data blocks that are frequently accessed together into one object, the system reduces the number of separate object access operations required, thereby decreasing computing resource usage and energy consumption associated with multiple network I/O operations.
3Adaptability or versatility
If data blocks are grouped without logical associations, then storage flexibility is improved, but user satisfaction decreases due to increased access times
Solution Approach 1:
The system implements a feedback mechanism by continuously capturing data access patterns and using this information to inform future data block grouping decisions. The access pattern capture module monitors how data blocks are actually accessed and feeds this information back to the object generation process, allowing the system to adapt its grouping strategy to match real-world usage patterns and improve user satisfaction.
Data Source
AI summary
A method embodiment commences upon identifying an on-premises storage area such as a file or virtual disk that is composed of a set of data blocks. A history of data access patterns over the data blocks is determined by observing accesses to the data blocks over time. When offsite storage is demanded (e.g., such as to a cloud-based storage facility), one or more storage objects are formed such that the blocks in a first one of the one or more storage objects comprise the blocks that have been measured to have the highest access rates (e.g., most recently accessed or most frequently accessed). Metadata that serves as a map is stored with the storage objects. A process or thread can access the storage objects in expectation that the most recently accessed or most frequently accessed blocks will be available in the first ones of the one or more storage objects.