Policy-Based Data Deduplication Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data deduplication techniques are inefficient as they fail to consider storage policies and relationships between data owners, leading to unnecessary data replication and increased storage demands.
Innovation Solution
Implement policy-based and rule-based reconciliation techniques that analyze storage relationships and policies to determine whether to replicate or not replicate data, optimizing storage decisions based on specific owner requirements and storage capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deduplication logic is based merely on fingerprint or uniqueness characteristic, then data storage decisions are simple and fast, but storage efficiency deteriorates due to unnecessary data replication
Solution Approach 1:
The patent segments the deduplication decision process into multiple independent evaluation dimensions: fingerprint matching, storage policy assessment, relationship analysis, and capability verification. Each dimension is evaluated separately through structured queries, allowing the system to make comprehensive decisions without sacrificing speed. The segmentation enables parallel processing of different criteria while maintaining overall decision efficiency.
Solution Approach 2:
The patent changes the parameters used for deduplication decisions from单一的 fingerprint matching to a multi-parameter evaluation system. It introduces policy parameters (storage tier requirements, retention periods), relationship parameters (data owner associations, access patterns), and capability parameters (storage system abilities). This parameter expansion allows the system to optimize storage efficiency by considering multiple factors simultaneously while maintaining decision speed through efficient query structures.
2Quantity of substance
If multiple storage policies and relationships are considered for each data item, then storage efficiency improves, but processing power and system complexity increase
Solution Approach 1:
The patent implements preliminary action by pre-establishing storage policies, data relationships, and system capabilities in structured databases before deduplication operations. Policies defining storage tiers, retention requirements, and access patterns are predetermined and stored. Data relationships between owners and items are pre-mapped. This preliminary structuring eliminates the need for complex real-time analysis during deduplication, reducing system complexity while maintaining high storage efficiency.
Solution Approach 2:
The patent introduces an intermediary layer consisting of structured policy databases and relationship repositories that mediate between raw data and deduplication logic. Instead of directly analyzing complex policies and relationships during data processing, the system uses these intermediary structures to pre-process and organize information. This intermediary layer simplifies the deduplication decision process by providing pre-organized policy and relationship data, reducing system complexity while enabling comprehensive storage efficiency optimization.
3Use of energy by moving object
If data replication is prevented based solely on fingerprint matching, then network bandwidth and processing power are reduced, but storage demands increase due to lack of policy-based optimization
Solution Approach 1:
The patent implements dynamic decision-making where the deduplication logic adapts to different storage policies and data characteristics. Instead of a static fingerprint-only approach, the system dynamically evaluates multiple criteria including policy requirements, relationship contexts, and system capabilities. This dynamic approach allows the system to optimize network bandwidth usage by making intelligent replication decisions based on current policy requirements, preventing unnecessary data transfer while ensuring storage demands are met through policy-compliant data placement.
Data Source
AI summary
A data storage site receives data from different data producer sites. Each of the data producer sites has a particular relationship to the data storage site, and each particular relationship carries corresponding data storage policies, constraints and commitments. When a data storage site receives a data storage request from a data producer, and that particular data is already present from a prior storage operation at the data storage site, the characteristics of the policies, constraints and commitments that were applied when that data was saved by the prior storage operation are reconciled with the policies, constraints and commitments of the requesting data producer. Deduplication logic reconciles different sets of policies, constraints and commitments such that the data can be effectively deduplicated by saving data-producer-specific metadata. Alternatively, the data can be effectively deduplicated by promoting the storage of the data so it covers a broader set of policies, constraints and commitments.