Tenant-Aware Variable-Length Data Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data deduplication methods are inefficient in cloud computing environments, particularly when dealing with large sets of documents from multiple tenants, as they fail to effectively segregate data and often result in unnecessary data amalgamation, consuming significant computing resources and time.
Innovation Solution
A computer-implemented method for tenant-aware, variable-length deduplication that splits data items into blocks using block markers, restricts deduplication to data items associated with the same tenant, and combines unique contiguous blocks to reduce storage requirements while maintaining data segregation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional fixed-width chunk deduplication is used, then the deduplication process is simple, but large amounts of duplicate information are overlooked because sequences may begin at different offsets
Solution Approach 1:
The patent segments data items into variable-width chunks based on content boundaries rather than fixed positions. This is achieved by identifying delimiters or structural markers within the data that indicate natural segmentation points, allowing duplicate sequences at arbitrary offsets to be properly recognized and deduplicated.
Solution Approach 2:
The patent employs dynamic chunk sizing where the width of each chunk varies based on the content structure and duplicate patterns detected. Instead of static fixed-width boundaries, the system adapts chunk boundaries to match actual data repetitions, improving deduplication effectiveness while managing complexity through adaptive algorithms.
2Productivity
If variable-width chunks are used to find duplicate sequences at arbitrary offsets, then deduplication effectiveness improves, but millions or billions of calculations are required for each file consuming computing resources and time
Solution Approach 1:
The patent performs preliminary indexing of data items by creating a map of potential duplicate sequences before full comparison. This pre-processing step identifies candidate chunks and their locations, allowing the system to focus computational effort only on promising matches rather than exhaustively comparing all possible variable-width chunks across the entire dataset.
Solution Approach 2:
The patent introduces an intermediary indexing structure that acts as a mediator between raw data and the deduplication comparison process. This index layer pre-organizes data by potential duplicate patterns and offsets, enabling faster lookup and comparison without requiring exhaustive calculations across all possible variable-width chunk combinations.
3Quantity of substance
If data deduplication is performed across all tenants, then storage space is maximized, but tenant data segregation is compromised resulting in undesirable data amalgamation
Solution Approach 1:
The patent segments the deduplication process by tenant, creating separate deduplication contexts for each tenant's data. This tenant-aware segmentation allows each tenant's data to be deduplicated independently against their own data set, maximizing storage efficiency within each tenant's data while maintaining strict segregation boundaries that prevent cross-tenant data amalgamation.
Data Source
AI summary
A computer-implemented method for providing tenant aware, variable length, deduplication of data stored on a non-transitory computer readable storage medium is described here. The method is performed at least in part by circuitry and the data comprises a plurality of data items. Each of the plurality of data items is associated with a particular tenant of a group of tenants that store data on the storage medium. In addition, a data deduplication data storage medium made up of one or more tenant storage areas and a deduplicated block database stored on the data deduplication data storage medium is described. The tenant storage areas comprise a plurality of data items, where each data item comprises a plurality of item blocks and each item block comprises a pointer to a storage location in a deduplicated block database. The deduplicated block database comprises a plurality of the storage locations and a plurality of data blocks, where the data blocks are unique for an individual tenant storage area, and all unique, contiguous data blocks for the individual tenant storage area are stored as a single combined block.