SMR Hard Drive Cluster Buffer Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage devices with internal buffers face delays and reduced performance due to the need to stop processing incoming data write operations while transferring data from buffers to permanent storage, especially when buffer capacity is low, which can counteract the performance improvements offered by these buffers.
Innovation Solution
Implementing a system where data is written to internal buffers of some storage devices while others transfer data to their permanent storage, switching to available devices with sufficient buffer capacity, and grouping storage devices into clusters based on attributes like buffer size and performance to optimize data write operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is written to internal buffers of storage devices, then write performance is improved, but the storage devices must stop processing incoming data write operations while transferring data from buffers to permanent storage, causing delays
Solution Approach 1:
The system divides storage devices into multiple clusters and segments data write operations across these clusters. When one cluster is performing buffer-to-storage transfer, another cluster can simultaneously accept new write operations, thereby segmenting the workload to avoid delays.
Solution Approach 2:
The system ensures continuous data write operations by switching between multiple clusters. When a cluster completes its buffer transfer or has available buffer capacity, it can immediately accept new write operations, maintaining continuous useful action without idle delays.
2Quantity of substance
If storage devices use SMR format to increase storage capacity, then storage capacity is improved, but data read and write access performance decreases
Solution Approach 1:
The system segments storage devices into clusters and further divides each device into buffer regions and permanent storage regions. By writing to buffer regions first (which can use faster PMR format) and then transferring to permanent storage (SMR format), the system achieves both high capacity and good performance.
Solution Approach 2:
The buffer region acts as an intermediary between the host and permanent storage. Data is first written to the buffer (using faster access), then transferred to permanent storage asynchronously. This intermediary layer decouples the performance requirements, allowing SMR to provide high capacity while buffers provide fast access.
3Productivity
If storage devices use internal buffers to mitigate lower write performance, then write performance is improved, but buffer capacity is limited and devices must stop accepting new writes when buffers are full
Solution Approach 1:
The system segments the buffer resource across multiple clusters of storage devices. Each cluster has its own buffer capacity, but the system can switch between clusters based on buffer availability. This segmentation effectively multiplies the total available buffer capacity and prevents single-point bottlenecks.
Solution Approach 2:
The system dynamically selects which cluster to write to based on real-time buffer capacity status. When one cluster's buffer is full, the system switches to another cluster with available buffer space. This dynamic adaptation ensures continuous write capability and maximizes the utilization of total buffer capacity across all clusters.
Data Source
AI summary
Technologies are provided for storing data by alternating the performance of data write operations using multiple clusters of storage devices. Data is written to internal buffers of storage devices in one cluster while data stored in buffers of storage devices in another cluster is transferred to the storage devices' permanent storages. When available buffer capacity in a cluster falls below a specified threshold, data write commands are no longer sent the cluster and the storage devices in the cluster transfer data stored in their buffers to their permanent storages. While the data is being transferred, data write commands are transmitted to other clusters. When the data transfer is complete, the storage devices in the cluster can be scheduled to receive data write commands again. A cluster can be selected for performing a given data write request by matching the attributes of the cluster to parameters of the data write request.