Automatic Storage Discovery via DHCP in Hyperscale Datacenters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In hyperscale datacenters, managing the discovery and availability of thousands of storage devices across multiple racks and performance-optimized datacenters is challenging, especially when some devices are down for maintenance, requiring an efficient method to allocate and schedule workloads.
Innovation Solution
A distributed storage system utilizing a dynamic host configuration protocol (DHCP) client on each storage device to provide discovery log entries to a discovery server, which updates and maintains discovery information, allowing a host computer to establish connections with target storage devices through a discovery log page containing necessary address information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual discovery information collection is used for thousands of storage devices, then completeness of discovery information can be maintained, but the complexity of management and time consumption increase significantly
Solution Approach 1:
The storage devices automatically send discovery information to the host computer without requiring manual collection. Each storage device acts as a self-service node that proactively provides its own discovery information (including namespace information, address, and status) through standardized protocols, eliminating the need for manual discovery processes in hyperscale environments with thousands of devices.
2Adaptability or versatility
If automatic discovery protocols are implemented across hundreds or thousands of storage devices, then the scalability and ease of operation improve, but the reliability of maintaining accurate discovery information becomes challenging
Solution Approach 1:
The system implements bidirectional communication where the host computer can query storage devices for their current status and discovery information. Storage devices respond with updated information about their namespace, address, and operational state. This feedback mechanism ensures that discovery information remains accurate and synchronized even as devices are added, removed, or updated in the hyperscale environment.
3Measurement precision
If discovery information is manually updated for each storage device, then accuracy can be maintained, but the time consumption and productivity are severely reduced
Solution Approach 1:
Storage devices pre-populate and maintain their own discovery information in readiness, including namespace details, address information, and status indicators. This preliminary preparation allows the host computer to immediately access accurate discovery information when workload allocation is needed, eliminating manual update steps and enabling rapid productivity in dynamic hyperscale environments.
Data Source
AI summary
A storage system includes: a plurality of storage devices, each of which being configured to run a DHCP client and a discovery log updater; a host computer configured to provide I/O or compute workloads to the plurality of storage devices; and a discovery server configured to provide a discovery log page to the host computer. The DHCP client and the discovery log updater are running on each of the plurality of storage devices provides a discovery log entry of the corresponding storage device to the discovery server. The discovery log page contains discovery information of the plurality of storage devices including one or more discovery log entries, and each of the one or more discovery log entries includes an address of the corresponding storage device that is assigned to establish communication between the host computer and the corresponding storage device. The host computer establishes a connection with a target storage device using the discovery log entry of the target storage device contained in the discovery log page and provides I/O or compute workloads to the target storage device.