Orphan File Detection in Content Addressed Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data migration systems face inefficiencies due to the formation of missing parent and orphan files, which disrupt the relationships between primary and secondary storage devices, leading to inconsistencies and storage issues.

Innovation Solution

A policy engine server manages the migration of files between primary and secondary storage devices, identifying and updating placeholder files and secondary files to maintain valid references, and marking orphan files for deletion, ensuring data integrity and storage efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If files are migrated between primary and secondary storage devices, then storage efficiency is improved, but file reference relationships may become invalid creating orphan files

Engineering Contradiction:

Improvestorage efficiencyVSAvoidfile reference relationship

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system implements a feedback mechanism where the policy engine server continuously monitors and validates file references between primary and secondary storage devices. After migration operations, the system checks whether placeholder files on the primary storage device still have valid corresponding files on the secondary storage device, and vice versa. This feedback loop detects and corrects broken references, ensuring data integrity while maintaining storage efficiency.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary validation checks before and during file migration operations. The policy engine server validates file references proactively to prevent orphan file creation, rather than waiting for problems to manifest. This preliminary action ensures that migration operations maintain reference integrity.

Inventive Principle:

Principle #10Preliminary action

2Stability of the object's composition

If placeholder files are updated to maintain references, then data consistency is improved, but system complexity increases

Engineering Contradiction:

Improvedata consistencyVSAvoidsystem complexity

Core Design Contradiction:

Stability of the object's compositionVSDevice complexity

Solution Approach 1:

The policy engine server acts as an intermediary between the primary and secondary storage devices, managing placeholder file updates and reference validation. This centralized mediator handles the complexity of maintaining data consistency across distributed storage locations, simplifying the overall system architecture while ensuring data integrity.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system implements self-service mechanisms where the policy engine server automatically detects and corrects broken references without manual intervention. The automated validation and update processes maintain data consistency through self-correction, reducing the need for complex manual management procedures.

Inventive Principle:

Principle #25Self-service

3Reliability

If orphan files are detected and eliminated, then storage inconsistencies are reduced, but detection time increases

Engineering Contradiction:

Improvestorage consistencyVSAvoiddetection time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system implements periodic validation of file references between primary and secondary storage devices. The policy engine server schedules regular checks to detect orphan files and broken references, balancing the need for storage consistency with acceptable detection time intervals. This periodic action ensures data integrity while minimizing the time impact on system operations.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS7640406B1Detecting and managing orphan files between primary and secondary data stores for content addressed storage

Publication Date: 2009.12.29 EMC IP HLDG CO LLC

AI summary

A method and system for locating and eliminating orphan files within a secondary storage device. The method includes identifying a secondary file on a secondary storage device, the secondary file being associated with file identification data, wherein the file identification data includes a content address, identifying a placeholder file on a primary storage device, the placeholder file being associated with an offline reference, and determining if the offline reference of the placeholder file validly references the content address of the secondary file.