Method for data fragmentation of distributed storage system

Abstract

The invention discloses a method for data fragmentation of a distributed storage system. The method comprises the steps that a physical node state is monitored through a cluster controller and the correctness of data transmission is guaranteed; physical spaces of storage nodes are managed in a unified mode through the consistent Hash algorithm, so that a logic storage space is formed; the reliability of the system is guaranteed in multi-transcript fragment mode. According to the method, the storage capacity is lager and flexible and the storage capacity virtualized by a host computer depends on resources of a native machine. Due to the limitation on hardware design of the host computer, the storage capacity cannot be infinitely increased; the storage capacity cannot be limited to the storage capacity of the host computer after the distributed system is adopted, storage resources can be increased or reduced at will according to the requirement of a client, and the storage capacity and the flexibility of the system are improved to the greatest extent.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a data fragmentation method of a distributed storage system. Background technique [0002] A complete storage system must have at least a storage node and a control node. The storage node is responsible for storing files, and the control node is used as a file index, and is responsible for monitoring the capacity and load balance between storage nodes. Together, these two parts form a storage system. [0003] Traditional storage systems mostly use a serial architecture, where storage nodes and control nodes are placed on one device, and storage capacity allocation is limited by physical hard disks. When we want to deploy new storage devices for capacity expansion. We must be very clear about the type, interface and transmission protocol of the storage device. We must clearly know how many disks there are in the storage system, what type and capacity they are, and what kind o...

AI Technical Summary

Problems solved by technology

[0004] Traditional storage systems have problems such as high procurement costs, closed systems, poor maintainability, low scalability, and limited availability

For example, when it is necessary to expand the system or replace the hardware, it is usually necessary to stop the service and perform a series of complex operations to succeed; at the same time, due to the limitations of the architecture, there are still several important technical problems to be solved in traditional storage, such as single point Fault issues, brain split issues, etc., put customer data at risk of loss or service interruption

[0005] The above shortcomings are mainly due to the limitations of the traditional storage system architecture. It deploys storage nodes and control nodes on the same device, that is, software and hardware devices are bundled to provide storage services. The software is separated from the inherent hardware and cannot be used on other storage devices. Running on the Internet, it depends heavily on hardware devices, and does not separate software and hardware

Images