Garbage Collection Cold Storage Data Recovery

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In distributed storage systems, especially in cloud environments, the garbage collection process often inadvertently deletes critical data due to race conditions between new references and garbage collection processing, leading to errors when data is requested after deletion.

Innovation Solution

Implementing a method that moves unreferenced data objects from the active tier to a cold tier during garbage collection, updating storage location indices, and applying lifecycle policies to manage these objects, allowing for recovery and deletion based on set policies to prevent data loss and optimize storage space.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If garbage collection process deletes unreferenced data objects, then storage space is reclaimed, but critical data may be inadvertently deleted due to race conditions

Engineering Contradiction:

Improvestorage spaceVSAvoiddata availability

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The system performs preliminary actions by checking for new references to data objects before deleting them during garbage collection. The method examines reference counts and verifies whether new references have been created before finalizing deletion, preventing race condition errors where critical data might be inadvertently removed

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The garbage collection process implements feedback mechanisms by continuously monitoring reference counts and data object status. When a deletion is initiated, the system checks back to verify no new references have been created, and can cancel or postpone deletion if references are detected, ensuring data integrity while managing storage space

Inventive Principle:

Principle #23Feedback

2Adaptability or versatility

If data objects are shared between multiple storage volumes and snapshots, then data accessibility is improved, but garbage collection becomes complex and error-prone

Engineering Contradiction:

Improvedata sharing capabilityVSAvoidgarbage collection process

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system introduces an intermediary reference counting mechanism that tracks references to shared data objects across multiple storage volumes and snapshots. This intermediary layer manages the complexity of tracking and deleting shared objects by providing a centralized reference management system that coordinates deletions across different storage locations

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The garbage collection method implements a universal reference counting system that works across multiple storage volumes, snapshots, and data sharing scenarios. The same reference checking and deletion logic applies universally regardless of how many locations reference a data object, simplifying the complexity of managing shared data

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11269539B2Methods for managing deletion of data objects by utilizing cold storage and devices thereof

Publication Date: 2022.03.08 NETAPP INC

AI summary

Methods, non-transitory computer readable media, and computing devices that manages deletion of data objects by utilizing cold storage includes identifying unreferenced data objects within a set of identified data objects in a storage device. The identified unreferenced data objects are moved from an active tier portion of the storage device to a cold tier portion of the storage devices. The moved unreferenced data objects in the cold tier portion of the storage device manage based on a lifecycle policy.