Retention Policy Data Deletion for Accurate Large-Scale Cleanup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data deletion methods are inefficient and risky, often leading to accidental deletion of important data and excessive computational demands, which can cause system failure.
Innovation Solution
A method involving a retention policy expressed in policy expression language code to selectively delete data elements, using dataset and transaction selectors, with mechanisms for parallel processing and idempotent deletion attempts to manage computational load and ensure data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual data deletion or ad-hoc scripts are used, then data can be deleted, but the risk of deleting important data increases substantially
Solution Approach 1:
The system enables self-service data deletion through retention policies that automatically identify and delete data based on predefined criteria. The retention policy engine evaluates datasets against retention policies without requiring manual intervention, allowing the system to manage its own data lifecycle autonomously and reduce human error.
Solution Approach 2:
The system implements feedback mechanisms where the retention policy engine continuously monitors and evaluates data against retention policies. This feedback loop ensures that only data meeting the deletion criteria is removed, while important data is preserved, thereby maintaining high accuracy in data deletion operations.
2Quantity of substance
If large quantities of data are deleted using manual methods, then data storage is reduced, but substantial demands are made on computational resources potentially resulting in system failure
Solution Approach 1:
The system segments the data deletion process into multiple manageable components: data identification, policy evaluation, and deletion execution. This segmentation allows the system to handle large quantities of data through structured processing stages, reducing the computational burden on any single operation and preventing system failure.
Solution Approach 2:
The system performs preliminary actions by pre-evaluating data against retention policies before actual deletion occurs. This preliminary evaluation stage identifies and categorizes data for deletion, allowing the system to prepare deletion operations in advance and execute them efficiently without overwhelming computational resources during the actual deletion process.
Data Source
AI summary
Apparatus and methods receive input descriptive of a retention policy; evaluate one or more datasets against the retention policy to determine one or more deletable data elements in the one or more datasets; and delete the one or more deletable data elements from a data store.