Data Retention Handling for Object Stores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud platforms with data object stores face computational inefficiencies and latency when handling data retention and deletion in file-based systems, particularly due to the high cost of accessing and processing large datasets in non-relational database schemas.
Innovation Solution
Implementing a data retention system that aggregates data retention actions into consolidated work items, allowing work processors to efficiently process and delete or export data by identifying datasets once and performing all associated actions, while utilizing retention rule evaluators to manage and schedule these processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data retention actions are processed individually in a file-based system, then data retention functionality can be provided, but processing overhead and latency increase significantly
Solution Approach 1:
The patent merges multiple individual data retention actions into a single consolidated work item. Instead of processing each deletion or export action separately, the system aggregates actions targeting the same dataset into one unified work item, which is then processed in a single operation. This combining approach maintains complete data retention functionality while dramatically reducing processing latency by eliminating repeated dataset accesses.
2Adaptability or versatility
If data retention actions are processed individually in a non-relational database schema, then data manipulation flexibility is maintained, but computational cost increases
Solution Approach 1:
The patent applies merging by consolidating multiple data retention actions into aggregate actions that operate on datasets as unified entities. This approach preserves the flexibility of non-relational database schemas for data storage and retrieval while reducing computational cost by minimizing the number of separate access and manipulation operations required to execute retention policies.
3Reliability
If multiple data retention actions are performed on the same dataset, then comprehensive retention policy enforcement is achieved, but processing efficiency decreases
Solution Approach 1:
The patent merges multiple data retention actions targeting the same dataset into a single aggregate action within a consolidated work item. This ensures that comprehensive retention policies are fully enforced by including all necessary actions, while simultaneously improving processing efficiency by executing all actions in one operation rather than multiple separate operations.
Solution Approach 2:
The consolidated work item structure provides multi-functionality by enabling a single work item to encompass multiple different types of data retention actions (deletion, export, etc.) targeting the same dataset. This universal approach allows the system to enforce comprehensive retention policies while maintaining high processing efficiency through unified execution.
Data Source
AI summary
Methods, systems, and devices for data retention handling are described. In some data storage systems, data objects are stored in a non-relational database schema. The system may support configurable data retention policies for different tenants, users, or applications. For example, a data store may receive retention requests, where the retention requests may specify deletion or exportation actions to perform on records contained within data objects. The data store may determine retention rules based on these retention requests, and may periodically or aperiodically evaluate the rules to determine active actions to perform. To improve the efficiency of the system, the data store may aggregate the active actions (e.g., according to the dataset to perform the actions on), and may generate work items corresponding to the aggregate actions. A work processor may retrieve these work items and may efficiently perform the data retention actions on datasets stored in the data object store.