Policy-Based File Deletion in Distributed Systems

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In cloud computing environments, managing file deletion in Distributed File Systems (DFS) is challenging due to the large number of files and distributed nature, making it difficult to implement file deletion policies efficiently, which can lead to memory issues and impact system performance.

Innovation Solution

The implementation of a policy-based file deletion system using a Hadoop Map-Reduce Framework, where a batch server initiates deletion processes based on defined policies, distributing tasks among master and slave computing systems to automate the deletion of files that meet specific criteria, such as age or type, thereby freeing up memory.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If manual file deletion is used in Distributed File Systems, then simplicity is maintained, but productivity is low and memory management is inefficient

Engineering Contradiction:
Improvefile deletion efficiencyVSAvoiddeletion system complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The file deletion system is segmented into multiple independent components: a policy management module, a task distribution module, and slave computing systems that execute deletions locally. This segmentation allows each component to handle specific aspects of file deletion independently, improving overall productivity while keeping individual components relatively simple.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system implements self-service through automated policy-based deletion where the master computing system automatically identifies files meeting deletion criteria and distributes deletion tasks to slave systems without manual intervention. The slave computing systems autonomously execute deletions based on received tasks, eliminating the need for manual file management while maintaining system simplicity through automation.

Inventive Principle:
Principle #25Self-service

2Quantity of substance

If no file deletion policy is implemented, then system simplicity is maintained, but memory space becomes insufficient impacting performance

Engineering Contradiction:
Improvememory space availabilityVSAvoidpolicy implementation complexity
Core Design Contradiction:
Quantity of substanceVSDevice complexity

Solution Approach 1:

The system implements preliminary action by establishing deletion policies in advance that define criteria for file removal (such as age, size, or type). These pre-defined policies enable the system to automatically identify and delete unnecessary files before memory space becomes critical, proactively managing memory capacity without requiring complex real-time decision-making algorithms.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system manages memory space by changing parameters related to file retention policies. Different deletion policies can be configured with varying parameters (retention periods, size thresholds, file type filters), allowing the system to adapt memory management strategies based on current needs while maintaining implementation simplicity through parameter-based control rather than complex logic.

Inventive Principle:
Principle #35Parameter changes

3Productivity

If automated file deletion is implemented, then productivity improves, but reliability may be impacted due to potential deletion errors

Engineering Contradiction:
Improveautomated deletion speedVSAvoiddeletion accuracy
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The system incorporates feedback mechanisms where the master computing system monitors the execution of deletion tasks by slave systems and can verify deletion completion. This feedback loop ensures that automated deletions are executed correctly and provides the ability to correct errors if files were mistakenly identified for deletion, thereby maintaining reliability while achieving high productivity through automation.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system applies partial action by executing deletion operations in controlled batches rather than attempting to delete all files at once. This batch processing approach with controlled scope reduces the risk of errors in any single deletion operation while maintaining high overall productivity. The system can selectively apply deletion policies to specific file sets, ensuring accuracy through targeted rather than blanket deletion actions.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS10909070B2Memory efficient policy-based file deletion system
Publication Date: 2021.02.02 SALESFORCE INC
  • US10909070B2 patent drawing
  • US10909070B2 patent drawing
  • US10909070B2 patent drawing

AI summary

Disclosed are some implementations of systems, apparatus, methods and computer program products for facilitating policy-based file deletion. Policy-based file deletion is implemented via a tiered system that includes a master computing system and a plurality of slave computing systems. The master computing system distributes policies among the slave computing systems, which each applies assigned policies to cause deletion of files that satisfy those policies.