Distributed Storage Path Management Using External CXL Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As storage device capacity increases, the burden on managing large amounts of data efficiently while minimizing the host's workload becomes significant, particularly due to limitations in buffer memory capacity for storing and managing mapping data.
Innovation Solution
A computing system that utilizes a separate, high-capacity CXL memory outside the storage device to manage mapping data, allowing for efficient data path management between storage devices and the host through a unified interface, reducing the need for additional buffer memory within the storage device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If storage device capacity increases, then data storage capability is improved, but management burden on host increases
Solution Approach 1:
A memory device is introduced as an intermediary component between the host and distributed storage devices. The memory device stores map data that associates logical addresses with physical locations across multiple storage devices, allowing the host to access stored data without directly managing the complexity of distributed storage operations. This intermediary layer handles the burden of tracking data locations across the storage network.
Solution Approach 2:
The management functionality for distributed storage is extracted from the host system and relocated to a dedicated memory device. By storing map data externally in the memory device rather than maintaining it in the host, the system separates the data management burden from the host, allowing the host to focus on primary computational tasks while the memory device handles storage coordination.
2Quantity of substance
If buffer memory capacity in storage device is increased, then mapping data management is improved, but device cost and complexity increase
Solution Approach 1:
The map data storage function is extracted from individual storage devices and consolidated in a separate memory device. Instead of each storage device maintaining its own buffer memory for map data, the system uses an external memory device to store the mapping information, reducing the buffer memory requirements within each storage device while improving overall system efficiency.
Solution Approach 2:
The memory device serves multiple functions: it stores map data for multiple distributed storage devices, provides a unified interface for host access, and coordinates data retrieval operations across the storage network. This multi-functional component replaces what would otherwise require separate buffer memories in each storage device plus additional host management logic.
3Quantity of substance
If distributed storage devices are used, then storage capacity is improved, but data path management complexity increases
Solution Approach 1:
The memory device acts as an intermediary that simplifies data path management across distributed storage devices. By storing map data that contains location information for data across the distributed network, the memory device provides a centralized coordination point that reduces the complexity of routing and managing data paths between the host and multiple storage devices.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of operating storage devices, a memory device, a host device, and a switch, is provided. The method includes: receiving, by the memory device, a first request corresponding to target user data from the host device; generating, by the memory device, first input/output (I/O) stream information based on telemetry information corresponding to the storage devices and map data in a buffer memory of the memory device based on the first request, wherein the first I/O stream information indicates a data path between a first storage device of the storage devices and the host device; providing, by the memory device, a first redirection request including the first request and the first I/O stream information to the host device or the first storage device through the switch; and processing the target user data according to the first I/O stream information in the first redirection request.