Object storage platform, object aggregation method and device and server

Abstract

The embodiment of the invention discloses an object storage platform, an object aggregation method and device and a server. The platform comprises an object storage gateway, a first storage pool and asecond storage pool, the first storage pool stores a first class of objects which are not aggregated to the second class of objects and an aggregation mapping relationship between the first class ofobjects aggregated to the second class of objects and the aggregated second class of objects, and the second storage pool stores the second class of objects aggregated by the first class of objects; and the object storage gateway periodically aggregates the first type of objects which are not aggregated to the second type of objects in the first storage pool to the second type of objects in the second storage pool, and replaces the first type of objects which are aggregated this time with an aggregation mapping relationship between the first type of objects and the aggregated second type of objects in the first storage pool. According to the technical scheme provided by the embodiment of the invention, the storage structure of the first type of objects is optimized, and the reading performance of the first type of objects which are not aggregated to the second type of objects is improved.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of data processing, and in particular, relate to an object storage platform, an object aggregation method, device, and server. Background technique [0002] With the gradual increase of multimedia resources (such as pictures, audio and video, etc.), the current open source Ceph file system will have corresponding storage and operation requirements for file objects of different sizes, and the Ceph file system will allocate a minimum for data storage. Space unit, even if the data volume of the small object is smaller than the minimum space unit, it will all occupy the minimum space unit, resulting in a great waste of storage space; at the same time, when the Ceph file system expands or hardware fails, it usually supports the minimum operation granularity The file objects stored on it are transferred as a unit. At this time, for small objects whose data volume is lower than the minimum ope...

AI Technical Summary

Problems solved by technology

[0002] With the gradual increase of multimedia resources (such as pictures, audio and video, etc.), the current open source Ceph file system will have corresponding storage and operation requirements for file objects of different sizes, and the Ceph file system will allocate a minimum for data storage. Space unit, even if the amount of data of small objects is smaller than the minimum space unit, it will all occupy the minimum space unit, resulting in a great waste of storage space; at the same time, when the Ceph file system expands or hardware fails, it usually supports the minimum operation granularity The file objects stored on it are transferred as a unit. At this time, for small objects whose data volume is lower than the minimum operation granularity, the Ceph file system may have the risk of data loss when performing conversion operations, and the transfer of a large number of small objects will Greatly increase the data read and write efficiency of the Ceph file system

[0003] At present, for small objects whose data volume is lower than the minimum operation granularity, the Ceph file system will use a third-party storage system to additionally store the custom object name of each small object and the name of the large object when merging multiple small objects into one large object The mapping relationship between the merging locations in , at this time, with the help of the function of the third-party storage system, the complexity and maintenance difficulty of object storage are increased, and the operation performance of the merged object is poor.

Images