Buffer Node Data Migration for Cluster Storage Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance computer clusters face inefficiencies in data migration to disk drives due to excessive 'seek' time and power consumption, as disk drives are often underutilized and powered continuously, even during short I/O cycles, leading to suboptimal duty cycles and bandwidth utilization.
Innovation Solution
An intermediate data storage layer coupled to the computer cluster enables high-speed random access storage, allowing data to be written independently of the I/O cycle, with a data storage management system that schedules sequential data transfers to minimize disk seeking time and deactivate unused disk drives, thereby optimizing power usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data is transferred directly from computer cluster to disk drives during I/O cycles, then data migration is accomplished, but seek time increases and disk drives must remain powered continuously
Solution Approach 1:
The patent introduces an intermediate data storage layer (buffer storage) between the computer cluster and disk drives. This intermediary buffer absorbs data during I/O cycles and releases it during idle periods, eliminating the need for disk drives to remain powered continuously while reducing seek time by batching transfers.
Solution Approach 2:
The system performs preliminary data staging in the intermediate buffer during I/O cycles, preparing data for subsequent sequential transfer to disk drives during idle periods. This preliminary action separates the random write operations from sequential read operations, optimizing both phases.
2Productivity
If disk drives are powered continuously to handle random data transfers, then data migration is ensured, but power consumption increases and duty cycle decreases
Solution Approach 1:
The patent implements periodic action by transferring data in batches during scheduled idle periods rather than continuously. Disk drives are powered only when needed to transfer data from the intermediate buffer, creating periodic operation cycles that reduce overall power consumption while maintaining productivity.
Solution Approach 2:
The intermediate buffer maintains continuous data availability to the computer cluster while enabling discontinuous, optimized transfer to disk drives. This decoupling allows the cluster to maintain high duty cycle without interruption for data migration.
3Quantity of substance
If random data transfers are performed to disk drives, then data is stored, but excessive number of disk drives are needed increasing power consumption
Solution Approach 1:
The patent merges multiple random write operations into a single sequential transfer operation by buffering data first. This consolidation reduces the number of active disk drives needed at any given time, as data is transferred in organized batches rather than simultaneous random writes to multiple drives.
Data Source
AI summary
A system for data migration between a compute cluster and disk drives by inclusion of a buffer node coupled to the compute cluster to store data received therefrom in a random fashion. The buffer node signals the computer nodes to promptly return from the I/O cycle to the computing state to improve the duty cycle of the device. The system further includes a storage controller which is coupled between the buffer node and the disk drives to schedule data transfer activity between them. The data transfers are actuated in the sequence determined based on minimization of seeking time and tier usage, and harvest priority, when the buffer node either reaches a predetermined storage space minimal level or a predetermined time has elapsed since the previous I/O cycle. The storage controller deactivates the disk drives which are not needed for the data transfer.