Modular Storage Device Scalability via Distributed I/O Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage devices are limited in scalability and flexibility, requiring different designs for various use scenarios, which increases costs and limits the reuse of existing storage resources, and are often based on specific hardware, restricting their adaptability.
Innovation Solution
A method and apparatus for constructing a scalable storage system using modularized building blocks with an IO processing flow that allows for a non-centralized, high-performance system, enabling flexible configuration and improved performance by forming clusters and distributing data using a consistent Hash algorithm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional storage devices are designed for predefined optimal use scenarios with specific hardware, then the device can achieve optimized performance for that specific scenario, but the scalability and flexibility are limited and different designs are required for different use scenarios
Solution Approach 1:
The patent implements a universal storage device design that can adapt to multiple use scenarios through software configuration rather than hardware specialization. The storage device uses a standardized hardware platform with configurable firmware and software layers that allow the same physical device to be optimized for different workloads (e.g., cloud storage, enterprise storage, consumer storage) without requiring redesign, thus achieving both performance optimization and versatility.
Solution Approach 2:
The storage device architecture is segmented into distinct layers: hardware layer, firmware layer, and software layer. This segmentation allows independent optimization of each layer - the hardware provides standardized high-performance storage interfaces, while the firmware and software can be configured and optimized for specific use scenarios, enabling the system to achieve scenario-specific performance without sacrificing scalability.
2Productivity
If different designs are required for different use scenarios, then optimal performance can be achieved for each scenario, but the cost increases and existing storage resources cannot be reused
Solution Approach 1:
By designing a universal hardware platform that can be software-configured for different scenarios, the patent eliminates the need to manufacture separate hardware designs for each use case. This reduces manufacturing costs, simplifies supply chain management, and allows existing storage resources to be reused across different applications by simply reconfiguring the software layer.
Solution Approach 2:
The patent utilizes parameter changes in the software and firmware configuration to adapt the storage device to different scenarios without changing the physical hardware. By modifying software parameters, caching strategies, data organization methods, and interface protocols, the same hardware can be optimized for different workloads, avoiding the costs associated with manufacturing multiple hardware variants.
3Productivity
If storage devices are based on specific hardware, then hardware optimization can be achieved, but the flexibility of constructing storage products is limited
Solution Approach 1:
The patent segments the storage system into hardware components and software components, allowing the hardware to be optimized for performance while the software provides flexibility. The hardware layer includes standardized high-performance interfaces (SAS, SATA, NVLink), while the software layer includes configurable firmware and management software that can be adapted to different construction scenarios, enabling both hardware optimization and construction flexibility.
Solution Approach 2:
The patent introduces a software intermediary layer (firmware and management software) between the hardware and the storage applications. This intermediary layer translates diverse hardware capabilities into standardized storage interfaces, allowing the hardware to be optimized for specific functions while maintaining flexibility in how storage products are constructed and deployed across different scenarios.
4Device complexity
If a centralized architecture is used for storage processing, then simplified control can be achieved, but the system performance and scalability are limited
Solution Approach 1:
The patent segments the storage processing architecture into distributed processing units rather than a single centralized controller. Each processing unit can independently handle storage operations, and multiple units work in parallel through standardized communication protocols. This segmentation maintains control simplicity through standardized interfaces while dramatically improving performance through parallel processing and eliminating single points of failure.
Data Source
AI summary
Embodiments of the present disclosure relate to a method an apparatus and a computer program product for constructing a scalable storage device by constructing the scalable storage device by combining a plurality of modularized building blocks; wherein each modularized building block in the plurality of modularized building blocks comprises a magnetic disk enclosure; and at least one modularized building block in the plurality of modularized building blocks comprises a storage processor, the storage processor comprising an input output processing unit; forming a cluster using the input output processing unit in the at least one modularized building block; and processing, using the cluster, an input or output (I/O) request from a host and metadata service.