Distributed storage system, cluster nodes and range management method of cluster nodes

Abstract

Disclosed are a distributed storage system, cluster node and interval management method thereof. The method comprises: the cluster node segments the interval corresponding to a first routing table item in a local routing table into at least two sub-intervals according to configuration parameters, wherein the first routing table item refers to the routing table item of which the duplicate information records the information representing the cluster node; the cluster node creates a log queue for each sub-interval respectively; and the cluster node determines a corresponding sub-interval on the basis of a key field of a data operation request from a client, executes the corresponding data read / write operation for the determined sub-interval, and updates the corresponding log queue accordingly. By segmenting a continuous data storage interval into at least two sub-intervals on a cluster node, and creating and maintaining a log queue for each sub-interval, the serial operation for the whole interval previously on the cluster node can be changed into parallel operations for at least two sub-intervals, and the overall throughput of the system can be improved effectively.

Description

technical field [0001] The present invention relates to a distributed storage system, in particular to a cluster node in the distributed storage system and its section management method. Background technique [0002] The network topology of the distributed storage system can be as follows: figure 1 As shown, it mainly includes a client (client), a management (manager) node and several cluster (cluster) nodes (nodes). Generally speaking, a node is a physical machine. Cluster nodes are collectively referred to as master nodes and slave nodes. Each master node corresponds to one or more slave nodes, and each node communicates through the network. . [0003] for figure 1 In the distributed storage system shown, in order to improve the throughput and overall performance of data processing, data is generally stored in pieces and queried in pieces. Among them, the commonly used sharding is range partition, that is, a large data storage range is divided into several small ranges...

AI Technical Summary

Problems solved by technology

[0006] This makes, in the case of a certain number of cluster nodes, if the number of intervals maintained on a node is small, such as the database Spinnake that performs interval segmentation based on the primary key, there may be multiple threads competing for the same interval. The permission of data operation, especially for some hot partitions, is easy to cause serial execution phenomenon

This will inevitably lead to the extension of access time and the reduction of data read and write efficiency and backup efficiency, thus affecting the data processing throughput of the system

[0007] On the other hand, in the case of a certain number of cluster nodes, if a node maintains a large number of intervals, such as HyperDex, a distributed key-value database that performs interval segmentation based on both primary and non-primary keys, the routing interval The number of is relatively large, which will increase the routing overhead of the client

Images