Online capacity expansion method for mass object storage

Abstract

The invention discloses an online capacity expansion method for massive object storage, which comprises the following steps that: S1, a user creates a bucket through a resetful API, a rear end synchronously creates one or more index objects for storing the object list under the bucket; and S11, when the user inquires the bucket object list through the resetful API, the rear end outputs the objectsin the bucket by using merging sorting, the sorting algorithm increases the calculated amount and the memory along with the increase of the object scale, and the data IO performance is seriously influenced. According to the on-line capacity expansion method for mass object storage, after capacity expansion, an original user S3 interface is kept unchanged, the user can realize elastic expansion ofa storage pool without perception, and after capacity expansion, traversal search of an original S3 interface query storage bucket object mode is changed into hash calculation, so that the routing number and addressing efficiency are greatly reduced, and the loads of multiple clusters are flexibly adjusted.

Description

technical field [0001] The invention relates to the technical field of storage, in particular to an online expansion method for massive object storage. Background technique [0002] With the development of new technologies such as cloud native, big data, cloud computing, Internet of Things, and 5G, and the rapid development of applications in industries such as telecommunications, the Internet, government and enterprise, and medical care, the explosive growth of massive data has caused many challenges to traditional storage systems. , the current main usage scenarios include massive data archiving and backup, content distribution and data sharing, cloud-native applications and multi-cloud application scenarios. [0003] Compared with the traditional file system storage, object storage abandons the complex semantics and directory design of the file system, and stores data through the flat Key-Value method, which greatly simplifies the complexity of metadata management, and th...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to provide an online capacity expansion method for massive object storage to solve the problems in the application scenarios of big data, log data, database backup, data collection in the Internet of Things industry, audio and video files in the monitoring industry, and images in the medical industry proposed by the above-mentioned background technology. , In scenarios such as enterprise organization government archives, traditional storage solutions are not only difficult to maintain, but also relatively expensive

Images