Distributed storage system data balancing method and related device

Abstract

The invention discloses a distributed storage system data balancing method and a related device, and the method comprises the steps: selecting a data block for data reorganization from a current diskof a target server as a target data block; according to the use condition of each disk of the target server and the for data reorganization concurrent data, screening the disks meeting a preset use condition and a preset data reorganization concurrent condition as target disks; and performing data reorganization on the target data block, and writing the data after data reorganization into the target disks. A disk of which the use condition and the data reorganization concurrent data accord with a preset use condition and a preset data reorganization concurrent condition is preferentially selected on the aspect of writing the data after data reorganization into a target selection; by the adoption of the mode, through existing data reorganization processing, the using conditions of differentdisks are dynamically adjusted, data balance between different disks in a single server in a server cluster is achieved, no extra disk IO is added, and the writing bottleneck of the server cluster isavoided.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular to a method and a related device for data balancing in a distributed storage system. Background technique [0002] With the rapid growth of data, distributed storage systems are widely used, for example, distributed object storage systems. Distributed storage systems usually use a cyclic strategy to automatically disperse and store data on different disks of different servers. When a new server is expanded, because the new server has not stored data yet, the distributed storage system uses the free space strategy to write data first. Import new servers, so that the utilization rate of each server in the server cluster tends to be balanced. [0003] Disk failures often occur in distributed storage systems. When disks are repaired and rejoined to the server, the free space policy is usually adopted or the active data balance policy is used to achieve data balanc...

AI Technical Summary

Problems solved by technology

However, the available space strategy is not effective for data balancing among different disks in a single server, and it is prone to write bottlenecks in server clusters; the active data balancing strategy causes additional IO amplification for server clusters with throughput pressure

Images