Soft Delete Volume Compaction in Data Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional data storage systems face challenges in efficiently managing the deletion of volumes, particularly in virtualized environments, where accidental deletion and legal compliance requirements complicate the process of permanently removing data, leading to issues with storage space reclamation and regulatory compliance.

Innovation Solution

A method and system that implement a reversible soft delete mechanism, allowing data volumes to be marked for deletion, compacted, and then either restored or permanently deleted, with an optional time-to-live (TTL) counter for ensuring timely hard deletion, integrated with deduplication and garbage collection processes to optimize storage efficiency and compliance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is permanently deleted immediately to free up storage capacity, then storage efficiency is improved, but the ability to recover accidentally deleted data is lost

Engineering Contradiction:

Improvestorage efficiencyVSAvoiddata recovery capability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary actions by marking data for deletion without immediately removing it from physical storage. The soft delete mechanism prepares the deletion process by updating metadata and setting deletion flags, but delays the actual data removal until a hard delete is explicitly triggered, allowing recovery operations during the intermediate period.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary state between data existence and complete deletion. The soft deleted data resides in a transitional state where it is marked for deletion but physically preserved, allowing the system to mediate between storage efficiency requirements and data recovery needs. This intermediary state enables recovery operations while still progressing toward eventual deletion.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If data is preserved after deletion to enable recovery, then data reliability is improved, but storage space cannot be reclaimed

Engineering Contradiction:

Improvedata recovery capabilityVSAvoidstorage space utilization

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically transitions data between different states: from active to soft deleted to hard deleted. The deletion process is not static but evolves through stages, with the ability to intervene at the soft delete stage to prevent complete deletion. This dynamic approach allows the system to adapt between preserving data for recovery and reclaiming storage space based on actual needs.

Inventive Principle:

Principle #15Dynamics

3Reliability

If a two-stage deletion process is implemented to allow recovery, then data protection is improved, but system complexity increases

Engineering Contradiction:

Improvedata protectionVSAvoiddeletion process complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The deletion process is segmented into distinct stages: soft delete and hard delete. Each stage has specific functions and requirements, allowing the system to manage complexity through modular design. The soft delete stage handles recovery-related operations, while the hard delete stage handles actual data removal, separating concerns and making the overall process more manageable.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The soft delete mechanism serves multiple functions: it marks data for deletion, preserves data for potential recovery, updates metadata, and enables recovery operations. This multi-functional approach consolidates several operations into a single mechanism, reducing overall system complexity despite the two-stage process.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Reliability

If data is marked for deletion but not physically removed, then recovery capability is maintained, but storage capacity cannot be freed

Engineering Contradiction:

Improvedata recovery capabilityVSAvoidavailable storage capacity

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11321007B2Deletion of volumes in data storage systems

Publication Date: 2022.05.03 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Storing data volumes in virtual and physical address spaces such that the data units are contiguous in virtual address space but fragmented in physical address space. The mapping between virtual and physical address space is managed by a storage controller that is configured to implement deletes reversibly with a so-called soft delete, the soft delete being reversible up to a later permanent or hard delete. A soft delete triggers a compaction in which the data units of the to-be-deleted volume are gathered together in physical address space. During the time between compaction and hard delete (or restore), the soft deleted volume is thus stored in a space efficient manner. Moreover, the subsequent hard delete can be performed more quickly than if the soft deleted volume were still fragmented across physical address space freeing up space quicker.