Data Object Grouping for Storage I/O Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data deduplication systems face inefficiencies in managing references to data constructs, particularly when dealing with large numbers of files, changes, and shared access, leading to increased storage unit accesses and fragmentation, which complicates backup and restoration processes.
Innovation Solution
The implementation of data object groups with change tracking and deduplication management operations at the group level, using fingerprinting algorithms to identify unique data segments and manage references efficiently, reducing the number of storage I/O operations and improving data storage organization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data deduplication systems manage references to individual data constructs separately, then data storage organization is simplified, but the number of storage I/O operations increases and storage space efficiency decreases
Solution Approach 1:
The patent merges multiple individual data construct references into a single data object group reference. Instead of managing references to each individual file or data construct separately, the system groups multiple data constructs under a single reference structure, thereby reducing the total number of storage I/O operations required while maintaining organized access to all grouped data objects.
2Loss of substance
If data objects are managed individually without grouping, then data object tracking is simpler, but duplicate data transfers increase and storage space usage is inefficient
Solution Approach 1:
The system combines multiple data objects into data object groups, allowing duplicate data transfers to be eliminated at the group level rather than individually for each data object. When a data object group is transferred or deduplicated, all member data objects benefit from this operation, significantly reducing redundant transfers and improving storage efficiency.
Solution Approach 2:
The data object group reference structure serves multiple functions simultaneously: it tracks multiple data objects, enables bulk deduplication operations, manages storage space allocation, and facilitates efficient data transfer. This multi-functional approach eliminates the need for separate management mechanisms for each individual data object.
3Measurement precision
If fingerprinting algorithms are applied to each data object separately, then data uniqueness identification is more accurate, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary fingerprinting at the data object group level before processing individual data objects. By first identifying unique data object groups through fingerprinting, the system can then process only the necessary individual objects within those groups, significantly reducing total processing time while maintaining accurate uniqueness identification.
Data Source
AI summary
Methods, computer program products, and computer systems for the management of data references in an efficient and effective manner are disclosed. Such methods, computer program products, and computer systems include receiving a change tracking stream at the computer system, identifying a data object group, and performing a deduplication management operation on the data object group. The change tracking stream is received from a client computing system. The change tracking stream identifies one or more changes made to a plurality of data objects of the client computing system. The identifying is based, at least in part, on at least a portion of the change tracking stream. The data object group represents the plurality of data objects.