Hybrid Storage Data Transfer for Hot Block Contiguity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hybrid storage systems face challenges in efficiently transferring data from low-performance, high-capacity storage devices to high-performance storage devices, leading to prolonged data access times and poor user experience.
Innovation Solution
A method that determines physically contiguous data blocks with high access frequency in a low-performance storage device and transfers them to a high-performance storage device for contiguous storage, thereby optimizing data access speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in low-performance high-capacity storage devices to balance cost and storage capacity, then storage cost is reduced, but data access time increases
Solution Approach 1:
The patent segments data blocks based on their access patterns (hot vs. cold data) and stores them in different storage devices. By dividing the storage system into multiple tiers with different performance characteristics and dynamically allocating data blocks accordingly, the system achieves both cost efficiency and fast access for frequently accessed data.
Solution Approach 2:
The patent performs preliminary identification of hot data blocks before transfer decisions are made. By pre-processing data to identify access patterns and marking hot data blocks in advance, the system can quickly make transfer decisions without delaying the actual data movement process.
2Speed
If hot data blocks are transferred individually from low-performance to high-performance storage devices, then data access speed improves, but transfer overhead and system complexity increase
Solution Approach 1:
The patent merges multiple small transfer operations into larger batch transfers. By grouping hot data blocks that need to be transferred and moving them together in batch operations, the system reduces the overhead associated with individual transfer operations while maintaining the performance benefits of storing hot data in high-speed storage.
Solution Approach 2:
The patent implements self-service mechanisms where the storage system automatically identifies hot data blocks, makes transfer decisions, and executes transfers without requiring external intervention. This automation reduces the operational complexity of managing data transfers between storage devices.
3Loss of time
If all data blocks are transferred to high-performance storage devices, then data access time is minimized, but storage cost increases significantly
Solution Approach 1:
The patent applies local quality by providing different storage performance levels for different data blocks based on their specific access requirements. Hot data blocks that require fast access are stored in high-performance storage devices, while cold data blocks are stored in low-performance devices, optimizing the overall cost-performance ratio of the storage system.
Data Source
AI summary
Techniques for transferring data involve: determining a first physical block in a first storage device, the first physical block having a plurality of data blocks stored thereon. The techniques further involve: determining, based on a first data block of the plurality of data blocks, whether a set of logically contiguous data blocks that comprise the first data block exist in the first physical block, the set of data blocks having a target heat greater than a threshold heat. The techniques further involve: transferring, in response to the existence of the set of data blocks in the first physical block, the set of data blocks into a second storage device for contiguous storage in a second physical block of the second storage device. Accordingly, relatively high heat data blocks are transferred to device with higher access speed, thereby reducing data access time, increasing data processing efficiency, and improving user experience.