Criticality-Based Immutable Backups for Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in managing data storage and retrieval, particularly in flash-based systems, due to redundant operations performed by both higher and lower level processes, leading to increased latency and reduced reliability.
Innovation Solution
Implementing a direct-mapped flash storage system where higher level processes, such as the operating system, manage data blocks directly without address translation by lower level storage controllers, and utilizing non-volatile RAM for buffering data before writing to persistent storage, thereby offloading device management responsibilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If lower level storage controllers perform address translation and data movement, then data retrieval is enabled, but write operations become redundant and latency increases
Solution Approach 1:
The patent extracts the address translation function from the storage controller and relocates it to the operating system. The storage controller no longer performs address translation or data movement,而是专注于底层存储管理。This eliminates redundant write operations where data was being moved back and forth between controller buffers and host memory, thereby reducing latency and improving write performance.
2Ease of operation
If storage controllers manage device responsibilities, then device control is maintained, but system complexity increases and efficiency decreases
Solution Approach 1:
The patent removes device management responsibilities from the storage controller and assigns them to the operating system. The storage controller becomes a simpler device that only handles raw storage operations, while the OS manages higher-level tasks such as address translation, data buffering, and coordination across multiple flash drives. This division reduces controller complexity and improves overall system efficiency.
Solution Approach 2:
The operating system assumes multiple functions previously handled by the storage controller, including address translation, data buffering, and device management. This multi-functionality approach consolidates control in a single system component, reducing the need for complex controller logic and enabling more flexible resource management across the storage system.
3Productivity
If both higher and lower level processes perform storage operations, then data access is enabled, but redundant operations increase and efficiency decreases
Solution Approach 1:
The patent extracts the data movement function from the storage controller and assigns it exclusively to the operating system. This eliminates redundant data copying operations where data was being moved from host memory to controller buffers and back. The OS directly manages data in host memory and coordinates writes to flash drives, eliminating unnecessary energy-consuming transfer operations.
Data Source
AI summary
An example method for automated backups in a distributed storage system comprises accessing, by a storage controller of a storage system, a user-defined set of criteria indicating a criticality of data stored by the storage system; detecting, by the storage controller, a change in the data; and generating, by the storage controller, based on the detecting and on the criticality, a point-in-time immutable version of the change in the data.