Metadata Aggregation and Compression for Access-Efficient Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Metadata attributes in object storage consume significant data storage capacity and can be inefficiently compressed, especially when individual attributes are short, while access and modification patterns impact overall system efficiency.
Innovation Solution
Adaptive aggregation and compression of metadata attributes based on access patterns and attribute characteristics, where aggregation and compression decisions are made dynamically based on individual and combined metadata attributes' predictions and weights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If metadata attributes are stored in detail without aggregation, then metadata information completeness is maintained, but data storage capacity is significantly consumed
Solution Approach 1:
The patent merges multiple individual metadata attributes into a single aggregated metadata structure. Instead of storing each attribute separately, the system combines them into one consolidated metadata object that preserves all original information while reducing overall storage requirements through efficient encoding and compression techniques.
Solution Approach 2:
The patent segments the metadata storage into hierarchical levels: the aggregated metadata structure is divided into individual attribute components that can be selectively accessed. This segmentation allows the system to store compact aggregated forms while maintaining the ability to retrieve and process individual attributes when needed, balancing storage efficiency with information accessibility.
2Quantity of substance
If individual metadata attributes are compressed separately, then some space is saved, but compression efficiency is low especially for short attributes
Solution Approach 1:
The patent combines multiple short metadata attributes into a single aggregated structure before compression. This merging allows compression algorithms to work on larger data blocks with higher redundancy, significantly improving compression ratios compared to compressing each short attribute individually. The aggregated structure enables more effective application of compression techniques across the combined data.
3Quantity of substance
If metadata attributes are aggregated and compressed, then storage capacity is reduced, but access and modification patterns may be impacted
Solution Approach 1:
The patent segments the aggregated metadata into individual accessible attributes while maintaining the compressed aggregated form for storage. This allows the system to efficiently store compact aggregated metadata while enabling selective access to individual attributes when needed, preserving operational ease without sacrificing storage efficiency.
Solution Approach 2:
The patent implements dynamic metadata management where the system can adapt between aggregated and individual attribute access modes based on operational needs. The aggregated structure provides efficient bulk operations and storage, while individual attribute access can be performed selectively, allowing the system to optimize performance based on specific access patterns and requirements.
Data Source
AI summary
Systems, methods, and non-transitory machine-readable medium storing a program are disclosed for aggregating and compressing a plurality of metadata attributes related to an object. For example, a system including a memory and a processor in communication with the memory, where the processor is configured to: measure an access pattern for the plurality of metadata attributes, apply an assigned weight to each of the metadata attributes, calculate an individual prediction for each of the metadata attributes, compare the individual prediction for each of the metadata attributes to a threshold, form a combined attribute with aggregated metadata attributes having individual predictions above the threshold, and compress the combined attribute.