Granular Object Storage Cost Allocation via Metadata Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based SaaS providers face challenges in allocating storage costs among customers and determining usage at a granular level due to the lack of mechanisms for detailed cost allocation in existing object storage services, which are typically only capable of providing high-level cost information at the bucket level.
Innovation Solution
An automated method is implemented to retrieve and aggregate metadata from object storage services, allowing for the calculation of total storage costs and sizes based on specific criteria, using a program or script that scans S3 buckets, evaluates object metadata, and stores the information in a separate data store for efficient querying and cost allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If objects are stored in shared containers for efficient resource utilization, then storage resource utilization is improved, but cost allocation granularity deteriorates
Solution Approach 1:
The patent segments cost allocation by introducing hierarchical levels (container-level metadata tags identifying customer/application, object-level metadata for detailed tracking). This allows shared containers to maintain resource efficiency while enabling granular cost breakdowns through multi-level segmentation of cost data.
Solution Approach 2:
The patent introduces metadata as an intermediary layer between physical storage and cost allocation. Metadata tags on containers and objects serve as mediators that link shared storage resources to specific customers and applications, enabling precise cost tracking without requiring physical separation of storage resources.
2Ease of manufacture
If bucket-level cost tracking is used, then implementation simplicity is improved, but cost allocation granularity deteriorates
Solution Approach 1:
The patent segments cost tracking into multiple hierarchical levels: container-level metadata tags for customer/application identification, object-level metadata for detailed object tracking, and automated aggregation logic. This segmentation enables granular cost allocation while maintaining implementation simplicity through structured, modular processing at each level.
Solution Approach 2:
The patent applies preliminary action by pre-tagging containers and objects with relevant metadata (customer ID, application name, cost center) before cost calculation. This preliminary classification enables automated, granular cost allocation without complex real-time computation, maintaining simplicity while achieving fine-grained precision.
3Measurement precision
If granular metadata collection is implemented, then cost allocation precision is improved, but data processing complexity increases
Solution Approach 1:
The patent segments data processing into distinct, manageable components: metadata collection from objects, metadata aggregation at container level, cost calculation based on aggregated data, and result presentation. This segmentation reduces processing complexity by breaking down the complex task into standardized, reusable modules operating at different hierarchical levels.
Solution Approach 2:
The patent creates universal metadata structures and processing logic that can be applied across all containers and objects consistently. The same metadata tags and aggregation rules work universally for different customers, applications, and storage scenarios, simplifying processing complexity through standardized multi-functional logic.
Data Source
AI summary
Systems and method for automated determination of granular object storage service metrics are provided. In one example, metadata regarding a subset of objects stored in one or more shared containers of an object storage service is retrieved. One or more meta properties (e.g., a storage tier with which the object is associated, a size of the object, an object type, a file type, user-defined tags, and a full path of the object, including, for example, information indicative of the application and the tenant) may be obtained for each object of the subset of objects. The subset of objects satisfy a set of one or more criteria involving the metadata and/or the meta properties. A total storage cost and/or a total size of the subset of objects is determined by aggregating a storage cost and/or a size of each object of the subset of objects.