A method of recording object storage bucket statistical counts

Abstract

The invention discloses a method for recording statistics and counting of object storage buckets, belonging to the field of data storage, comprising: after the object storage gateway RGW completes the write operation, synchronously updating the value of the key of the storage bucket statistics and counting to which the target object belongs; and counting the target storage bucket The key of the target key is split into several sub-keys, and the entire key space is divided into sub-spaces with a specified number of slices; the task thread reads the value of the target key from its cache, re-stores and persists it to the sub-key of the target key after modification. In TIKV; after RGW accepts the request to read the statistics of the target bucket, combined with the bucket corresponding to the request, it splits it into processing requests for multiple subkeys and sends it to the backend TIKV; S5, TIKV statistics on the target bucket The counted subkeys are traversed and merged in the TIKV node, and then the merged result is returned to RGW for secondary merging. Avoid unnecessary waiting time due to request queuing in high-concurrency scenarios, as well as write conflicts and write hotspots caused by modifying the same Key.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a method for recording statistical counts of object storage buckets. Background technique [0002] Object storage, or object-based storage, is a general term used to describe methods of addressing and processing discrete units called objects. Like files, objects contain data, but unlike files, objects are never hierarchical within a hierarchy. Every object is in the same level of a flat address space called a storage pool, and one object does not belong to another level below another object. [0003] Both files and objects have metadata about the data they contain, but objects are characterized by extended metadata. Each object is assigned a unique identifier, allowing a server or end user to retrieve the object without knowing the physical location of the data. This approach is helpful for automating and simplifying data storage in cloud computing environments. Object s...

AI Technical Summary

Problems solved by technology

In view of this, the present invention proposes a method for recording object storage bucket statistics and counting, aiming at solving unnecessary waiting caused by request queuing in high concurrency scenarios by replacing back-end storage components on the basis of synchronous statistics flow Time-consuming, and avoid writing conflicts and writing hotspots caused by modifying the same Key in high-concurrency scenarios

Images