Distributed File System Data Deletion via Latest Record Consolidation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed file systems face challenges in efficiently deleting data, leading to poor performance due to the accumulation of deletion records and increased processor and I/O resource usage.
Innovation Solution
A data operation method that involves obtaining operation records, determining the latest deletion record for target data, and directly deleting the corresponding versions of the target data, thereby reducing the number of deletion records and improving deletion performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional deletion methods are used in distributed file systems, then data can be deleted, but deletion performance deteriorates due to accumulation of deletion records and increased processor and I/O resource usage
Solution Approach 1:
The patent merges multiple deletion operations into a single deletion record by tracking the latest deletion timestamp. Instead of creating separate deletion records for each version deletion, the system consolidates them by comparing timestamps and keeping only the most recent deletion record, thereby reducing the quantity of deletion records while maintaining deletion functionality.
Solution Approach 2:
The deletion record is designed to serve multiple functions: it tracks deletion status, stores deletion timestamp for version comparison, and enables efficient identification of the latest deletion. This multi-functional design allows a single record to replace multiple specialized records, reducing overall record quantity while maintaining comprehensive deletion tracking capability.
2Productivity
If traditional deletion methods are used in distributed file systems, then data can be deleted, but processor resource usage increases due to complex deletion record management
Solution Approach 1:
The system performs preliminary actions by maintaining deletion records with timestamps in a structured format that facilitates efficient comparison. Before actual deletion processing, the system has already organized deletion information chronologically, allowing quick identification of the latest deletion record without requiring complex real-time analysis during deletion operations.
Solution Approach 2:
Instead of processing all deletion records equally, the system applies partial action by focusing only on the latest deletion record (the one with the maximum timestamp). This selective approach avoids the excessive processing of outdated deletion records, significantly reducing processor resource usage while maintaining correct deletion behavior.
3Productivity
If traditional deletion methods are used in distributed file systems, then data can be deleted, but I/O resource usage increases due to frequent disk access for deletion record management
Solution Approach 1:
The patent merges multiple I/O operations into fewer operations by consolidating deletion record management. Instead of writing separate deletion records for each version deletion to disk, the system combines them into a unified deletion tracking mechanism that requires fewer disk I/O operations, thereby reducing I/O resource usage while maintaining deletion functionality.
Solution Approach 2:
The system applies partial I/O action by only persisting the latest deletion record to disk rather than every deletion record. This selective I/O approach reduces the frequency of disk access for deletion record management, lowering I/O resource usage while ensuring that the most relevant deletion information is preserved.
Data Source
AI summary
The present disclosure provides a data operation method, apparatus, device, and storage medium, which relates to the technical fields of distributed file system, in particular, to the technical fields of multi-version concurrency control and log-structured merge tree. The specific implementation scheme is as follows: obtaining a plurality of operation records on at least one piece of data in a file system; determining a target operation record of target data in the plurality of operation records, where the target operation record is a deletion record, and the target operation record is a latest operation record of the target data; and deleting at least one version of the target data in the file system according to the target operation record.