DPU-Mapped Flash Storage for Lower-Latency AI Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solid-state drives are designed to conform to hard disk drive standards, limiting their ability to leverage unique aspects of flash and other solid-state memory, and are inflexible in connection and communication paths within storage arrays.
Innovation Solution
Implementing a storage system with direct-mapped flash storage that addresses data blocks without translation by storage controllers, utilizing non-volatile RAM for quick data buffering, and offloading device management from storage drives to controllers, enabling higher-level processes to manage operations like zone management and data relocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If solid-state drives conform to hard disk drive standards for compatibility, then compatibility is improved, but the ability to leverage unique aspects of flash memory and flexibility in connection paths deteriorates
Solution Approach 1:
The patent inverts the traditional storage architecture by allowing storage drives to directly initiate communication with data processing units without requiring storage controllers as intermediaries. This reversal enables flash memory's unique features to be leveraged while simplifying the connection paths, as drives can directly connect to and communicate with data processing units through standardized interfaces.
Solution Approach 2:
Storage drives are empowered to autonomously manage their own communication and data transfer operations without relying on storage controllers. Each drive can independently initiate read/write operations, manage its own connection paths, and directly interface with data processing units, eliminating the need for complex controller-mediated communication protocols.
2Productivity
If storage controllers translate data blocks in flash storage, then compatibility with standard interfaces is maintained, but write operations become unnecessary and latency increases
Solution Approach 1:
The patent extracts the data translation function from storage controllers and relocates it to the flash storage medium itself. The translation layer is embedded within the storage drive, allowing data blocks to be directly translated at the source without requiring intermediary processing by storage controllers, thereby eliminating translation latency and unnecessary write operations.
Solution Approach 2:
The patent introduces a direct communication pathway that acts as an intermediary between storage drives and data processing units, bypassing the traditional storage controller. This direct interface enables efficient data transfer and translation without the overhead of controller-mediated operations, reducing latency while maintaining compatibility.
3Device complexity
If storage drives manage their own device management functions, then controller complexity is reduced, but the need for higher-level management processes increases
Solution Approach 1:
The patent segments device management functions by distributing them across multiple autonomous storage drives rather than centralizing them in a single storage controller. Each drive independently manages its own operations, while higher-level management processes coordinate between drives through standardized interfaces, reducing overall system complexity through functional decomposition.
Data Source
AI summary
A mapping of portions of a dataset that designates one or more managed flash storage devices of a storage system that store corresponding portions of the dataset are transmitted by one or more storage controllers to one or more data processing units (DPUs). A request for accessing a portion of the dataset stored at a particular managed flash storage device indicated by the mapping of the portions of the dataset is received from a DPU of the one or more DPUs.