A concurrent storage system construction method for convenient expansion of storage node quantity

Abstract

This invention discloses a structuring method for a parallel storage system convenient for expanding numbers of stored nodes, which uses a data distribution mode of selecting in order to structure a storage system, when stored nodes are increased, it' s only necessary to move data from the original stored node to newly increased nodes, and it' s not necessary to move data among the original stored nodes to secure balance of the volume of stored nodes and consistency of the sequence selection mode.

Description

technical field [0001] The invention relates to the field of computer application technology, and provides a parallel storage system construction method that facilitates the expansion of the number of storage nodes. Background technique [0002] Parallel storage systems are often faced with adding new storage nodes to meet the ever-increasing demands of applications on storage space and storage bandwidth. However, the existing parallel storage systems mostly store data in a round-robin data distribution mode. The data is divided into data blocks according to a certain size and numbered sequentially from zero. The data blocks are stored according to the calculation results of the modulo function. Nodes and sequence numbers within nodes. In this way, when new storage nodes need to be added, data rebalancing operations must be performed to maintain the storage capacity balance between storage nodes, and also to ensure the consistency of the modulo function before and after the...

AI Technical Summary

Problems solved by technology

However, due to the nature of the modulo function, when the number of new storage nodes is relatively prime to the number of original nodes, almost all data blocks in the system need to be moved, and most of the data blocks are moved between the original storage nodes , this move is an ineffective move for the capacity balance among nodes in the new system

This invalid move is to ensure the consistency of the modulo function, but for a parallel storage system with a large amount of data, the rebalancing operation will consume a lot of CPU processing power and bandwidth due to moving a large amount of data, and will Cause a longer system service pause

[0003] In response to this situation, some parallel storage systems require that the number of new storage nodes be a multiple of the original number of nodes, which can eliminate invalid data block movement, but for larger parallel storage systems, due to the large number of storage nodes, Adding new nodes in multiples will lead to a huge cost of system expansion, and at the same time lose the flexibility of system expansion, which cannot fully meet the needs of users

[0004] In other fields, some studies focus on data / object distribution methods. For example, the application of Consistent Hashing in web cache can achieve zero invalid mobility when the amount of node data changes, but its implementation is based on uncontrollable hash functions, storage Only probabilistic capacity balance can be achieved between nodes, and the data parallelism is very low

Images