OS-Driven Storage Controller Extraction for Write Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage systems face inefficiencies in data management and reliability due to redundant write operations and lack of direct control over storage processes, leading to increased latency and potential data loss.
Innovation Solution
Implementing a storage system where the operating system directly initiates and controls data processes, such as writing and erasing, across multiple flash drives without involving lower-level storage controllers, using non-volatile random access memory (NVRAM) for quick data buffering and energy reserves for power persistence, and employing erasure coding and mirroring for redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional storage controllers manage data writing across multiple flash drives, then data redundancy and reliability are maintained, but write operations become redundant and latency increases
Solution Approach 1:
The patent extracts the storage controller from the data path by allowing the operating system to directly manage flash drive operations. The OS reads identifiers, determines mapping relationships, and directly controls writing and erasing operations, eliminating the traditional storage controller intermediary. This reduces redundant write operations and latency while maintaining reliability through OS-level coordination of erasure coding and mirroring operations.
Solution Approach 2:
The patent introduces NVRAM as an intermediary buffer between the OS and flash drives. The OS writes data to NVRAM first, which then manages the actual transfer to flash drives according to erasure coding schemes. This mediator enables efficient batch operations and reduces direct OS-flash drive communication overhead while maintaining data integrity through coordinated redundancy operations.
2Ease of operation
If storage controllers are involved in data management processes, then control and monitoring are simplified, but direct control over storage processes is lost leading to potential data loss
Solution Approach 1:
The patent segments control responsibilities between the OS and storage system. The OS handles high-level data management, identifier reading, and mapping determination, while the storage system executes specific writing, erasing, and redundancy operations. This segmentation enables direct OS control over critical decisions while maintaining simple automated execution at the storage level, improving both control and reliability.
Solution Approach 2:
The patent implements feedback mechanisms where the storage system continuously monitors write operations, erasure coding status, and mirror synchronization. The OS receives status information and can adjust operations in real-time, ensuring data persistence while maintaining operational simplicity through automated monitoring and reporting of data integrity status.
3Reliability
If redundant write operations are performed for data safety, then data integrity is improved, but write efficiency and productivity decrease
Solution Approach 1:
The patent performs preliminary actions by pre-calculating erasure coding schemes and mirror mappings before actual data writing. The OS determines the distribution of data across multiple flash drives in advance, preparing the redundancy structure beforehand. This preliminary planning enables efficient parallel write operations without real-time computation overhead, maintaining data integrity while improving write efficiency.
Solution Approach 2:
The patent implements periodic redundancy operations where mirroring and erasure coding are performed in scheduled batches rather than continuously for every write operation. NVRAM buffers data and consolidates redundancy operations periodically, reducing the frequency of redundant writes while maintaining data integrity through regular synchronization cycles.
Data Source
AI summary
An illustrative method includes a storage management system detecting an event within a storage system, determining, based on the event, an operation related to a compliance ruleset associated with a compliance policy, and providing a notification of the operation to an orchestration system configured to manage an execution of the operation by a computing system associated with the storage system.