User-Level Deduplication via Unique Salts for Accurate Billing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication systems struggle to accurately attribute and bill users for deduplication services, as they are not designed to distinguish between individual users, leading to inefficient resource usage and potential exploitation, where users who receive less value subsidize those who receive more, and lack precise measurement of backend storage consumption.
Innovation Solution
Implementing user-level deduplication by assigning a unique salt to each user, which is used to hash their data, allowing for user-specific deduplication and accurate measurement of storage consumption, enabling fine-grained deduplication at the user object level and preventing sharing of storage space between users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If global deduplication systems are used to share storage space among multiple users, then storage efficiency is improved, but the ability to accurately attribute and bill services to individual users deteriorates
Solution Approach 1:
The patent segments the global deduplication system into user-specific deduplication domains by introducing unique identifiers (such as user IDs, device IDs, or container IDs) that associate data chunks with specific users. This segmentation enables the system to maintain shared storage efficiency while accurately tracking and attributing consumption to individual users through user-specific metadata and accounting mechanisms.
2Device complexity
If global deduplication systems treat all data sets equally, then system complexity is reduced, but the ability to distinguish between different users and their consumption patterns deteriorates
Solution Approach 1:
The patent introduces intermediary mechanisms such as user-specific metadata, accounting databases, and identification tags that act as mediators between the global deduplication system and individual users. These intermediaries enable the system to track user-specific consumption patterns and maintain user identification information without significantly increasing the core deduplication algorithm's complexity.
3Ease of manufacture
If pre-compressed capacity is used to measure user consumption, then ease of implementation is improved, but the accuracy of deduplication effectiveness measurement deteriorates
Solution Approach 1:
The patent implements feedback mechanisms that continuously monitor and measure actual backend storage consumption through user-specific accounting systems. These systems provide real-time feedback on deduplication effectiveness by comparing original data size with actual stored size, enabling accurate measurement and adjustment of deduplication performance while maintaining ease of implementation through automated tracking.
Data Source
AI summary
In one example, a method includes receiving, at a cloud storage site, chunks that each take the form of a hash of a combination that includes two or more salts and a file object, and one of the salts is a retention salt shared by the chunks, monitoring a time period associated with the retention salt, when the time period has expired, removing the chunks that include the retention salt, and depositing the removed chunks in a deleted items cloud store.