Garbage Collection Timing for Tiered Hard Disk Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing scenarios face challenges in meeting both fast read/write and large-capacity storage needs due to the limitations of existing physical hard disks, with fast disks lacking capacity and large-capacity disks being slow.
Innovation Solution
A data processing method that utilizes a high-speed physical hard disk for fast read/write and migrates data to a capacity hard disk when necessary, accompanied by a garbage collection operation at an optimized time to prevent bandwidth interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a high-speed physical hard disk is used to store user data, then fast read/write speed is achieved, but storage capacity is insufficient
Solution Approach 1:
The storage system is segmented into two distinct physical hard disks: a high-speed disk for fast read/write operations and a large-capacity disk for bulk storage. This segmentation allows each disk to specialize in its strength, resolving the contradiction between speed and capacity by distributing data across both devices rather than relying on a single disk to provide both attributes.
Solution Approach 2:
The solution introduces a dimensional shift from using a single storage device to a two-dimensional storage architecture combining speed-oriented and capacity-oriented disks. This dimensional change in the storage system architecture enables simultaneous satisfaction of both performance and capacity requirements that cannot be achieved with a single disk type.
2Quantity of substance
If a large-capacity physical hard disk is used to store user data, then storage capacity is achieved, but read/write speed is slow
Solution Approach 1:
The storage system is segmented into two distinct physical hard disks: a high-speed disk for fast read/write operations and a large-capacity disk for bulk storage. This segmentation allows each disk to specialize in its strength, resolving the contradiction between speed and capacity by distributing data across both devices rather than relying on a single disk to provide both attributes.
Solution Approach 2:
The high-speed physical hard disk acts as an intermediary between user operations and the large-capacity slow disk. Frequently accessed data is cached on the high-speed disk, which mediates access requests and reduces direct reads from the slow large-capacity disk, thereby improving perceived read/write speed while maintaining access to large storage capacity.
3Quantity of substance
If garbage collection operation is performed on duplicated data in the capacity physical hard disk, then storage space is optimized, but bandwidth may be preempted and data loss may occur
Solution Approach 1:
The system performs preliminary actions by continuously monitoring the high-speed disk for migration conditions and proactively migrating data before the capacity disk runs out of space. The garbage collection is scheduled at predetermined time instants rather than being triggered immediately when space is needed, allowing advance preparation and preventing bandwidth preemption during critical operations.
Solution Approach 2:
The system implements feedback mechanisms by monitoring storage space levels, garbage ratios, and IO quantity in real-time. Based on this feedback, the system dynamically adjusts garbage collection thresholds and schedules operations at optimal time instants, preventing bandwidth preemption and ensuring data safety while maintaining efficient space utilization.
Data Source
AI summary
Provided in the present application are a data processing method and apparatus, and a device. The method includes: acquiring target decision information corresponding to a capacity physical hard disk; determining a target time instant corresponding to a garbage collection operation based on the target decision information; performing the garbage collection operation on duplicated data in the capacity physical hard disk at the target time instant. Through the technical solution of the present application, the garbage collection operation can be performed at an appropriate time, which prevents the garbage collection operation from preempting a bandwidth of a data read operation, so that the capacity physical hard disk can meet a fast read/write need.