Deduplication System Data Entity Deletion Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplication systems face challenges in efficiently managing data deletion, particularly in prioritizing and handling requests from legitimate and malicious sources, which can lead to data loss and storage capacity issues.
Innovation Solution
Implementing a prioritized deletion method within the deduplication system that uses a housekeeping queue with priority values and total unshared chunk sizes to determine the order of data entity deletion, deferring high-priority data entities and allowing for restoration opportunities while ensuring storage capacity is maintained by deleting low-priority entities first and then high-priority ones based on available space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If data entities are deleted from the deduplication store, then storage capacity is freed up, but data loss occurs and system reliability deteriorates
Solution Approach 1:
The system performs preliminary actions by identifying and marking data entities for deletion before actual deletion occurs. The housekeeping queue stores deletion requests with priority values, allowing the system to prepare and evaluate deletion candidates in advance, ensuring that high-priority data is protected while low-priority data can be safely removed to free storage capacity.
Solution Approach 2:
The system changes the parameter of data entities by assigning priority values (e.g., high, medium, low) to different data entities. This parameter change enables differentiated deletion behavior where low-priority data entities are deleted first to free storage capacity, while high-priority entities are preserved, thus resolving the contradiction between freeing storage and maintaining data protection.
2Device complexity
If all data entities are treated equally during deletion, then deletion process is simple, but storage capacity management becomes inefficient
Solution Approach 1:
The system applies local quality by assigning different priority values to different data entities based on their importance. Instead of treating all data entities uniformly, the system creates local distinctions in the deletion process where high-priority entities are protected and low-priority entities are deleted first, improving storage capacity management efficiency without requiring complex overall system redesign.
3Volume of stationary object
If high-priority data entities are deleted first, then storage capacity is freed quickly, but data loss increases and system reliability worsens
Solution Approach 1:
The system inverts the conventional deletion approach by deleting low-priority data entities first instead of high-priority ones. This inversion ensures that storage capacity is freed while minimizing data loss, as the deleted data represents less critical information. The housekeeping queue processes deletion requests in reverse priority order, preserving important data while achieving storage reclamation.
4Speed
If data deletion is performed without prioritization, then processing speed is fast, but data loss increases
Solution Approach 1:
The system segments the deletion process into priority-based groups using the housekeeping queue. Data entities are divided into high-priority, medium-priority, and low-priority segments, with deletion processing occurring in a specific order. This segmentation maintains processing speed by using efficient queue structures while preventing data loss through ordered deletion that preserves critical data entities.
Data Source
AI summary
Examples described herein relate to deletion of data entities in a deduplication system. Examples may maintain entries in a housekeeping queue, each entry including a priority value and a total unshared chunk size of a data entity to be deleted from the deduplication system. Examples may delete the data entities corresponding to the entries including a low priority value from the deduplication system. Examples may determine whether an available storage capacity of the deduplication system is sufficient after deleting the data entities corresponding to the entries including the low priority value. Examples may delete a data entity corresponding to an entry including a high priority value and a largest total unshared chunk size if the available storage capacity is insufficient.