Bypassing OS for Direct Storage Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage filers require client read messages to be suspended in the operating system while data is retrieved from the physical storage system, leading to inefficiencies and additional processing steps after data retrieval.
Innovation Solution
The physical storage system sends data directly to the client, bypassing the operating system, allowing the RAID system to send data retrieved from disk directly to the client after standard verification procedures, eliminating the need for the data to pass through the operating system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is retrieved through the operating system's buffer cache, then data validity is guaranteed, but additional processing steps and latency are introduced
Solution Approach 1:
The patent segments the data access path into two separate channels: a fast path that bypasses the filesystem for performance-critical operations, and a traditional path through the buffer cache for reliability. The data access manager independently manages a separate cache structure that parallelizes access with the filesystem, allowing selective use of each path based on requirements.
Solution Approach 2:
The data access manager acts as an intermediary layer between the application and the storage system. It provides alternative data access routes that can bypass the traditional filesystem buffer cache, introducing a new mediator component that manages its own caching mechanism and validation logic independently.
2Productivity
If the data access manager provisions buffers separately from the buffer cache, then parallel data access is enabled, but memory resources are reduced for the filesystem
Solution Approach 1:
The patent introduces a new dimensional space for buffer management by creating a separate buffer provisioning system outside the traditional filesystem hierarchy. Instead of expanding vertically within the existing buffer cache structure, the data access manager establishes a parallel buffer management dimension with its own allocation and deallocation mechanisms.
3Reliability
If the operating system processes read requests through suspension and resumption, then data integrity checks can be performed, but system efficiency decreases
Solution Approach 1:
The data access manager performs preliminary data retrieval and validation actions before the traditional OS processing cycle completes. By pre-fetching data through alternative paths and performing initial validation in parallel with OS operations, it eliminates the need for suspending and resuming read requests, maintaining integrity checks while improving throughput.
Data Source
AI summary
Conventional storage filers utilize a data reading process that requires client read request messages to be suspended in the operating system while the data is retrieved from the physical storage system. Then, once the operating system retrieves the data from the physical storage system, the operating system must restart the suspended read message in order to forward the retrieved data to the client. Accordingly, the inventors have developed a system and method that allows the physical storage system's server to send the data directly to the client rather than routing back through the operating system.