A method and system for synchronizing aggregate objects in an object storage system

Abstract

The embodiment of the present invention provides a method and system for synchronizing aggregated objects in an object storage system, specifically including: sending an http request from a site to obtain bucket sharding information in the data_log of the master site; the slave site analyzes whether the object needs to be synchronized through the index information of the object in the bucket sharding information; if synchronization is required, sending a get request from the site to obtain the object of the master site; the master site reads the data information of the object according to the aggregation characteristics of the object, and returns the data information of the object to the slave site; the slave site synchronizes the data information of the object to the storage cluster of the slave site according to the preset storage rules. The method and system for synchronizing aggregated objects provided by the embodiment of the present invention solves the problem of incompatibility between synchronization characteristics and aggregation characteristics during synchronization between sites in an object storage system, improves the applicability of storage products, and improves the competitiveness of object storage.

Description

Technical Field

[0001] The present invention relates to the technical field of storage systems, and in particular to a method and system for synchronizing aggregate objects in an object storage system. Background Art

[0002] With the development of technologies such as cloud computing and mobile Internet, the objects in object storage systems are showing an explosive growth trend, with a large number of objects and frequent operations.

[0003] In an object storage system, an aggregate object aggregates multiple small objects smaller than 512k into a large object. A small object only stores the metadata of the small object, and the data of the small object is stored in the aggregated large object. The aggregation feature modifies the distribution rules of objects in the storage pool.

[0004] When synchronizing between sites, the slave site reads the object metadata information and data information of the master site, and then writes it to the local site (slave site); due to the aggregation feature, the master site aggregates the data information in the small object header into the aggregated large object, resulting in failure to read the object data information during synchronization between sites, causing incompatibility between the synchronization feature and the aggregation feature. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for synchronizing aggregate objects in an object storage system to solve the problem of incompatibility between synchronization characteristics and aggregation characteristics during synchronization between sites in the object storage system.

[0006] The embodiment of the present invention discloses the following technical solution:

[0007] A first aspect of the present invention provides a method for synchronizing aggregate objects in an object storage system, the method comprising:

[0008] Send an http request from the site to obtain the bucket shard information in the data_log of the main site;

[0009] The slave site analyzes whether the object needs to be synchronized based on the index information of the object in the bucket shard information;

[0010] If synchronization is required, a get request is sent from the site to obtain the object from the master site;

[0011] The master site reads the data information of the object according to the aggregation characteristics of the object, and returns the data information of the object to the slave site;

[0012] The slave site synchronizes the data information of the object to the storage cluster of the slave site according to the preset storage rule.

[0013] Furthermore, the index information of the object is distributed in the bucket shard information according to a hash rule.

[0014] Furthermore, the primary site reads the data information of the object according to the aggregation characteristics of the object specifically as follows:

[0015] If the object is marked with an aggregation feature, data information of the object is read from the SSD storage pool of the primary site;

[0016] If the object is not marked with an aggregation feature, metadata information of the object is read from the SSD storage pool of the primary site, and data information and an offset of the object are obtained from the HDD storage pool of the primary site through the metadata information of the object.

[0017] Furthermore, the preset storage rules include a grading characteristic and an aggregation characteristic;

[0018] The tiering feature is used to store objects in the SSD storage pool of the slave site and tag the objects with aggregation features;

[0019] The aggregation characteristic is used to aggregate data information of a plurality of objects marked with the aggregation characteristic into one aggregate object, and store the aggregate object in the HDD storage pool of the site.

[0020] Furthermore, the grading characteristics specifically include:

[0021] The metadata information and data information of the object are stored in the SSD storage pool of the slave site;

[0022] If the capacity value of the object is less than a first preset value, marking the object with an aggregation characteristic;

[0023] If the capacity value of the object is greater than or equal to a first preset value, the aggregation characteristic is not marked for the object.

[0024] Furthermore, the polymerization characteristics specifically include:

[0025] According to a preset period, data information of a plurality of objects in the SSD storage pool, the number of which is equal to a second preset value and which are marked with an aggregation characteristic, is read out;

[0026] Aggregate the data information of the plurality of objects into an aggregate object;

[0027] Storing the data information and offset of the aggregate object in the HDD storage pool of the slave site;

[0028] Delete data information of the plurality of objects in the SSD storage pool.

[0029] A second aspect of the present invention provides a synchronization system for aggregated objects in an object storage system, which is implemented based on the method, and the system includes:

[0030] Request sending module, used to send http requests and get requests;

[0031] The main site is used to store the objects of the main site and the bucket shard information in data_log;

[0032] A synchronization judgment module is used to judge whether the object on the primary site needs to be synchronized;

[0033] The slave site is used to store data information of objects that need to be synchronized to the slave site;

[0034] A preset storage rule module is used to preset storage rules of objects in the master site and the slave site;

[0035] The object analysis module reads the objects in the master site and the bucket shard information in the data_log through the request sending module, analyzes the index information in the bucket shard information, and writes the objects to be synchronized to the slave site.

[0036] Furthermore, the preset storage rule module includes:

[0037] A hierarchical feature unit, used to store objects in the SSD storage pool of the slave site and tag the objects with aggregate features;

[0038] The aggregation characteristic unit is used to aggregate the data information of the multiple objects marked with the aggregation characteristic into an aggregate object, and store the aggregate object in the HDD storage pool of the slave site.

[0039] Furthermore, the object analysis module includes:

[0040] An object reading unit, which reads metadata information of the object in the primary site, data information of the object, and bucket fragmentation information in data_log according to the aggregation characteristics of the object;

[0041] The object writing unit writes the objects that need to be synchronized to the slave site;

[0042] The object analysis unit analyzes the index information in the bucket shard information.

[0043] Furthermore, the process of the object reading unit reading the object is specifically as follows:

[0044] If the object is marked with an aggregation feature, data information of the object is read from the SSD storage pool of the primary site;

[0045] If the object is not marked with an aggregation feature, metadata information of the object is read from the SSD storage pool of the primary site, and data information and an offset of the object are obtained from the HDD storage pool of the primary site through the metadata information of the object.

[0046] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0047] The method for synchronizing aggregated objects in an object storage system provided by the present invention obtains the bucket sharding information in the data_log of the master site from the slave site; and analyzes whether the object needs to be synchronized through the index information of the object in the bucket sharding information; if synchronization is required, the master site reads the data information of the object according to the aggregation characteristics of the object, and if the object is marked with the aggregation characteristics, the data information of the object is read from the SSD storage pool of the master site; if the object is not marked with the aggregation characteristics, the metadata information of the object is read from the SSD storage pool of the master site, and the data information and offset of the object are obtained from the HDD storage pool of the master site through the metadata information of the object; the master site returns the data information of the object to the slave site, and the slave site synchronizes the data information of the object to the storage cluster of the slave site according to the preset storage rules. Therefore, the present invention solves the problem of incompatibility between synchronization characteristics and aggregation characteristics during synchronization between sites in the object storage system, improves the applicability of storage products, and improves the competitiveness of object storage.

[0048] The object storage system provided by the present invention provides a synchronization system for aggregated objects. The request sending module sends an http request and a get request. The master site stores the bucket sharding information in the objects and data_log of the master site. The synchronization judgment module judges whether the objects on the master site need to be synchronized. The slave site stores the data information of the objects that need to be synchronized to the slave site. The preset storage rule module presets the storage rules of the objects in the master site and the slave site. The object analysis module requests the sending module to read the bucket sharding information in the objects and data_log in the master site, analyzes the index information in the bucket sharding information, and writes the objects that need to be synchronized to the slave site. If synchronization is required, the master site reads the data information of the object according to the aggregation characteristics of the object. If the object is marked with the aggregation characteristics, the data information of the object is read from the SSD storage pool of the master site. If the object is not marked with the aggregation characteristics, the metadata information of the object is read from the SSD storage pool of the master site, and the data information and offset of the object are obtained from the HDD storage pool of the master site through the metadata information of the object. The master site returns the data information of the object to the slave site, and the slave site synchronizes the data information of the object to the storage cluster of the slave site according to the preset storage rules. The system of the present invention solves the problem of incompatibility between synchronization characteristics and aggregation characteristics during synchronization between sites in an object storage system, increases multi-site and aggregation compatibility, improves the applicability of storage products, and improves the competitiveness of object storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 is a flow chart of the method of the present invention;

[0051] Figure 2 This is a diagram of the bucket sharding information structure in the data_log method of the present invention;

[0052] Figure 3 A storage structure diagram of objects in the site of the method of the present invention;

[0053] Figure 4 This is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0054] In order to clearly illustrate the technical features of the present solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The disclosure below provides many different embodiments or examples for realizing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the accompanying drawings are not necessarily drawn to scale. The present invention omits the description of known components and processing techniques and processes to avoid unnecessary limitations on the present invention.

[0055] Memory system refers to the system in the computer that is composed of various storage devices for storing programs and data, control components, and equipment (hardware) and algorithms (software) for managing information scheduling. The main memory of a computer cannot simultaneously meet the requirements of fast access speed, large storage capacity, and low cost. There must be multi-level hierarchical memory in the computer with speeds ranging from slow to fast and capacities ranging from large to small, with the optimal control scheduling algorithm and reasonable cost to form a storage system with acceptable performance. The performance of the storage system is becoming increasingly important in computers. The quality of storage management and organization affects the efficiency of the entire machine. Modern information processing such as image processing, databases, and knowledge bases have high requirements for storage systems.

[0056] As scientific computing and data processing place higher and higher demands on storage systems, it is necessary to continuously improve existing storage technologies, research new storage media, and improve the structure and management of storage systems. Large-scale integrated circuits and disks are still the main storage media.

[0057] Synchronization refers to data synchronization redundancy between two storage clusters. One of the two storage clusters is used normally, and the other is used for redundant backup, so that one site can immediately take over the other site after a disaster. The storage cluster used daily is the primary site, and the storage cluster used as a backup is the secondary site.

[0058] Bucket shards are index information of objects in a bucket. When bucket shards mode is enabled, a bucket corresponds to one or more rados objects (distributed objects).

[0059] data_log: After the multi-site function is enabled, each time an object is uploaded or deleted, the information of the bucket shard where the object is located will be recorded in data_log.

[0060] Embodiment 1:

[0061] like Figure 1 As shown, a method for synchronizing aggregate objects in an object storage system provided by an embodiment of the present invention includes:

[0062] Send an http request from the site to obtain the bucket shard information in the data_log of the main site;

[0063] The slave site uses the object index information in the bucket shard information to analyze whether the object needs to be synchronized.

[0064] If synchronization is required, send a get request from the site to obtain the object of the main site;

[0065] The master site reads the data information of the object according to the aggregation characteristics of the object and returns the data information of the object to the slave site;

[0066] The slave site synchronizes the object's data information to the storage cluster of the slave site according to the preset storage rules.

[0067] like Figure 2 As shown, this is a diagram of the bucket shard information structure in the data_log method of the present invention. In order to improve the read and write performance of object storage, the index information of the objects in the bucket is evenly distributed on 128 bucket shards according to the hash rule, so the bucket shard stores the index information of the object.

[0068] The primary site reads the data information of the object according to the aggregation characteristics of the object:

[0069] If the object is marked with the aggregation feature, the object's data information is read from the SSD storage pool at the primary site;

[0070] If the object is not marked with aggregation characteristics, the metadata information of the object is read from the SSD storage pool of the primary site, and the data information and offset of the object are obtained from the HDD storage pool of the primary site through the metadata information of the object (the offset is the starting position where the data information of the object is stored in the HDD storage pool).

[0071] The preset storage rules include hierarchical features and aggregation features;

[0072] The tiering feature is used to store objects in the SSD storage pool of the slave site and tag the objects with aggregation features;

[0073] The aggregation feature is used to aggregate data information of multiple objects marked with the aggregation feature into an aggregate object, and store the aggregate object in the HDD storage pool of the site.

[0074] In the object storage system method without the tiering feature, the files in the massive storage data are in the billions, and tens of billions of massive small files are stored in the same storage cluster in the same way as ordinary file data storage. This storage method will have tens of billions of underlying objects. When the underlying storage objects are high, it will affect the performance of the file system, data recovery in failure scenarios, and disk utilization.

[0075] The hierarchical characteristics specifically include: storing the metadata information and data information of the object in the SSD storage pool of the slave site; if the capacity value of the object is less than the first preset value, the object is marked as an aggregation feature; if the capacity value of the object is greater than or equal to the first preset value, the object is not marked as an aggregation feature; the data information of the object not marked with the aggregation feature is read from the SSD storage pool and written into the HDD storage pool of the site. In this embodiment, the first preset value is set to 512k.

[0076] In this embodiment, after the tiering feature is enabled, when the storage cluster stores objects, objects smaller than 512k are stored in the SSD storage pool, and objects larger than 512k are stored in the HDD storage pool.

[0077] The aggregation feature specifically includes: reading data information of multiple objects in the SSD storage pool whose number is equal to the second preset value and marked with the aggregation feature according to a preset period; aggregating the data information of multiple objects into an aggregate object; storing the data information and offset of the aggregate object in the HDD storage pool of the slave site; and deleting the data information of multiple objects in the SSD storage pool. In this embodiment, the second preset value is set to 1024.

[0078] The aggregation feature depends on the tiering feature. After the tiering aggregation feature is enabled, the storage cluster reads the data information of the objects stored in the SSD storage pool, 1024 objects form an aggregate object, writes the aggregate object to the HDD storage pool, and then deletes the data information of the 1024 objects stored in the SSD storage pool.

[0079] like Figure 3 As shown, it is a storage structure diagram of objects in the method site of the present invention. Each object contains two parts of information, data information data and metadata information xattr. xattr records the metadata information of the object, such as creation time, size, aggregation characteristics and other custom metadata information.

[0080] After the hierarchical aggregation feature is enabled, the process of storing objects with a capacity value greater than or equal to 512k at a site is as follows:

[0081] The metadata information and data information of the object are stored in the SSD storage pool of the site, and the aggregation characteristics are not marked for the object; the data information of the object not marked with the aggregation characteristics is read from the SSD storage pool and written to the HDD storage pool of the site.

[0082] After the hierarchical aggregation feature is enabled, the process of storing objects with a capacity value less than 512k at the site is as follows:

[0083] The metadata and data information of the object are stored in the SSD storage pool of the site, and the object is marked with aggregation characteristics;

[0084] According to a preset cycle, the data information of 1024 objects marked with aggregation characteristics in the SSD storage pool is read out; the data information of the 1024 objects is aggregated into an aggregate object; the data information and offset of the aggregate object are stored in the HDD storage pool of the slave site; and the data information of multiple objects in the SSD storage pool is deleted.

[0085] Therefore, in the method of the present invention, the storage cluster of the slave site includes an SSD storage pool and an HDD storage pool, and the storage cluster of the master site also includes an SSD storage pool and an HDD storage pool.

[0086] The method for synchronizing aggregated objects in an object storage system provided by the present invention obtains the bucket sharding information in the data_log of the master site from the slave site; and analyzes whether the object needs to be synchronized through the index information of the object in the bucket sharding information; if synchronization is required, the master site reads the data information of the object according to the aggregation characteristics of the object, and if the object is marked with the aggregation characteristics, the data information of the object is read from the SSD storage pool of the master site; if the object is not marked with the aggregation characteristics, the metadata information of the object is read from the SSD storage pool of the master site, and the data information and offset of the object are obtained from the HDD storage pool of the master site through the metadata information of the object; the master site returns the data information of the object to the slave site, and the slave site synchronizes the data information of the object to the storage cluster of the slave site according to the preset storage rules. Therefore, the method of the present invention solves the problem of incompatibility between synchronization characteristics and aggregation characteristics during synchronization between sites in the object storage system. The present invention proposes a method for synchronizing aggregated objects in an object storage system, which increases multi-site and aggregation compatibility, improves the applicability of storage products, and improves the competitiveness of object storage.

[0087] Embodiment 2:

[0088] like Figure 2 As shown, the synchronization system provided by the present invention is implemented based on the method, and the system includes:

[0089] Request sending module, used to send http requests and get requests;

[0090] The main site is used to store the objects of the main site and the bucket shard information in data_log;

[0091] A synchronization judgment module is used to judge whether the object on the primary site needs to be synchronized;

[0092] The slave site is used to store data information of objects that need to be synchronized to the slave site;

[0093] A preset storage rule module is used to preset storage rules of objects in the master site and the slave site;

[0094] The object analysis module reads the objects in the master site and the bucket shard information in the data_log through the request sending module, analyzes the index information in the bucket shard information, and writes the objects that need to be synchronized to the slave site.

[0095] In order to improve the read and write performance of object storage, the index information of objects in the bucket is evenly distributed on 128 bucket shards according to the hash rule, so the bucket shards store the index information of the objects.

[0096] The preset storage rule modules include:

[0097] A hierarchical feature unit, used to store objects in the SSD storage pool of the slave site and tag the objects with aggregate features;

[0098] The aggregation characteristic unit is used to aggregate data information of multiple objects marked with the aggregation characteristic into an aggregate object, and store the aggregate object in the HDD storage pool of the slave site.

[0099] In an object storage system without tiering, the files in the massive storage data are in the billions, and tens of billions of small files are stored in the same storage cluster in the same way as ordinary file data. This storage method will have tens of billions of underlying objects. When the number of underlying storage objects is high, it will affect the performance of the file system, data recovery in failure scenarios, and disk utilization.

[0100] The hierarchical characteristic unit is used to define hierarchical characteristics, and the hierarchical characteristics specifically include: storing metadata information and data information of the object in the SSD storage pool of the slave site;

[0101] If the capacity value of the object is less than the first preset value, the object is marked with an aggregation feature; if the capacity value of the object is greater than or equal to the first preset value, the object is not marked with an aggregation feature; the data information of the object not marked with an aggregation feature is read from the SSD storage pool and written to the HDD storage pool of the site. In this embodiment, the first preset value is set to 512k.

[0102] In this embodiment, after the tiering feature is enabled, when the storage cluster stores objects, objects smaller than 512k are stored in the SSD storage pool, and objects larger than 512k are stored in the HDD storage pool.

[0103] The aggregation characteristic unit is used to define the aggregation characteristic, which specifically includes: reading out data information of multiple objects in the SSD storage pool whose number is equal to the second preset value and marked with the aggregation characteristic according to a preset period; aggregating the data information of multiple objects into an aggregate object; storing the data information and offset of the aggregate object in the HDD storage pool of the slave site; deleting the data information of multiple objects in the SSD storage pool. In this embodiment, the second preset value is set to 1024.

[0104] The aggregation feature depends on the tiered feature. After the tiered aggregation feature is enabled, the working process of the preset storage rule module is as follows: the storage cluster reads the data information of the objects stored in the SSD storage pool, 1024 objects form an aggregate object, and then writes the aggregate object to the HDD storage pool, and then deletes the data information of the 1024 objects stored in the SSD storage pool.

[0105] After the hierarchical aggregation feature is enabled, the preset storage rule module stores objects with a capacity value greater than or equal to 512k at the site as follows: the metadata information and data information of the object are stored in the site's SSD storage pool, and the aggregation feature is not marked for the object; the data information of the object not marked with the aggregation feature is read from the SSD storage pool and written to the site's HDD storage pool.

[0106] After the hierarchical aggregation feature is enabled, the preset storage rule module stores objects with a capacity value less than 512k at the site as follows: the metadata information and data information of the object are stored in the SSD storage pool of the site, and the aggregation feature is marked for the object; according to the preset period, the data information of 1024 objects marked with the aggregation feature in the SSD storage pool is read out; the data information of the 1024 objects is aggregated into an aggregate object; the data information and offset of the aggregate object are stored in the HDD storage pool of the slave site; and the data information of multiple objects in the SSD storage pool is deleted.

[0107] The object analysis module includes:

[0108] The object reading unit reads the metadata information of the object in the primary site, the data information of the object, and the bucket sharding information in the data_log according to the aggregation characteristics of the object;

[0109] The object writing unit writes the objects that need to be synchronized to the slave site;

[0110] The object analysis unit analyzes the index information in the bucket shard information.

[0111] The object reading unit reads the object in the following process:

[0112] If the object is marked with the aggregation feature, the object's data information is read from the SSD storage pool at the primary site;

[0113] If the object is not marked with aggregation characteristics, the metadata information of the object is read from the SSD storage pool of the primary site, and the data information and offset of the object are obtained from the HDD storage pool of the primary site through the metadata information of the object (the offset is the starting position where the data information of the object is stored in the HDD storage pool).

[0114] The specific process of the object analysis module is as follows:

[0115] When the slave site sends an http request to the master site through the request sending module, the object reading unit obtains the bucket sharding information in the data_log of the master site;

[0116] The object analysis unit analyzes the index information in the bucket shard information and sends it to the synchronization judgment module, and the synchronization judgment module judges whether the object on the primary site needs to be synchronized;

[0117] If synchronization is required, the slave site sends a get request to the master site through the request sending module to obtain the object of the master site. The object reading unit reads the metadata information and data information of the object in the master site according to the aggregation characteristics of the object;

[0118] The object writing unit writes the object to be synchronized read by the object reading unit into the storage cluster of the slave site according to the grading characteristics and aggregation characteristics defined by the pre-set storage rule module.

[0119] Therefore, in the system of the present invention, the storage cluster of the slave site includes an SSD storage pool and an HDD storage pool, and the storage cluster of the master site also includes an SSD storage pool and an HDD storage pool.

[0120] The object storage system provided by the present invention provides a synchronization system for aggregated objects. The request sending module sends an http request and a get request. The master site stores the bucket sharding information in the objects and data_log of the master site. The synchronization judgment module judges whether the objects on the master site need to be synchronized. The slave site stores the data information of the objects that need to be synchronized to the slave site. The preset storage rule module presets the storage rules of the objects in the master site and the slave site. The object analysis module requests the sending module to read the bucket sharding information in the objects and data_log in the master site, analyzes the index information in the bucket sharding information, and writes the objects that need to be synchronized to the slave site. If synchronization is required, the master site reads the data information of the object according to the aggregation characteristics of the object. If the object is marked with the aggregation characteristics, the data information of the object is read from the SSD storage pool of the master site. If the object is not marked with the aggregation characteristics, the metadata information of the object is read from the SSD storage pool of the master site, and the data information and offset of the object are obtained from the HDD storage pool of the master site through the metadata information of the object. The master site returns the data information of the object to the slave site, and the slave site synchronizes the data information of the object to the storage cluster of the slave site according to the preset storage rules. The system of the present invention solves the problem of incompatibility between synchronization characteristics and aggregation characteristics during synchronization between sites in an object storage system, increases multi-site and aggregation compatibility, improves the applicability of storage products, and improves the competitiveness of object storage.

[0121] The above description is only a preferred embodiment of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for synchronizing aggregate objects in an object storage system, characterized in that: The method comprises: Send an http request from the site to obtain the bucket shard information in the data_log of the main site; The slave site analyzes whether the object needs to be synchronized based on the index information of the object in the bucket shard information; If synchronization is required, a get request is sent from the site to obtain the object from the master site; The master site reads the data information of the object according to the aggregation characteristics of the object, and returns the data information of the object to the slave site; The slave site synchronizes the data information of the object to the storage cluster of the slave site according to the preset storage rules; The primary site reads the data information of the object according to the aggregation characteristics of the object as follows: If the object is marked with an aggregation feature, data information of the object is read from the SSD storage pool of the primary site; If the object is not marked with an aggregation feature, metadata information of the object is read from the SSD storage pool of the primary site, and data information and offset of the object are obtained from the HDD storage pool of the primary site through the metadata information of the object; The preset storage rules include classification characteristics and aggregation characteristics; The tiering feature is used to store objects in the SSD storage pool of the slave site and tag the objects with aggregation features; An aggregation feature, used to aggregate data information of a plurality of objects marked with the aggregation feature into one aggregate object, and store the aggregate object in the HDD storage pool of the site; The classification characteristics specifically include: The metadata information and data information of the object are stored in the SSD storage pool of the slave site; If the capacity value of the object is less than a first preset value, marking the object with an aggregation characteristic; If the capacity value of the object is greater than or equal to a first preset value, then the aggregation characteristic is not marked for the object; The polymerization characteristics specifically include: According to a preset period, data information of a plurality of objects in the SSD storage pool, the number of which is equal to a second preset value and which are marked with an aggregation characteristic, is read out; Aggregate the data information of the plurality of objects into an aggregate object; Storing the data information and offset of the aggregate object in the HDD storage pool of the slave site; Delete data information of the plurality of objects in the SSD storage pool.

2. A method for synchronizing aggregate objects in an object storage system according to claim 1, characterized in that: The index information of the object is distributed in the bucket shard information according to the hash rule.

3. A synchronization system for aggregated objects in an object storage system, implemented based on the method according to any one of claims 1 to 2, characterized in that: The system comprises: Request sending module, used to send http requests and get requests; The main site is used to store the objects of the main site and the bucket shard information in data_log; A synchronization judgment module is used to judge whether the object on the primary site needs to be synchronized; The slave site is used to store data information of objects that need to be synchronized to the slave site; A preset storage rule module is used to preset storage rules of objects in the master site and the slave site; The object analysis module reads the objects in the master site and the bucket shard information in the data_log through the request sending module, analyzes the index information in the bucket shard information, and writes the objects to be synchronized to the slave site.

4. The synchronization system for aggregated objects in an object storage system according to claim 3, characterized in that: The preset storage rule module includes: A hierarchical feature unit, used to store objects in the SSD storage pool of the slave site and tag the objects with aggregate features; The aggregation characteristic unit is used to aggregate the data information of the multiple objects marked with the aggregation characteristic into an aggregate object, and store the aggregate object in the HDD storage pool of the slave site.

5. The synchronization system for aggregated objects in an object storage system according to claim 4, characterized in that: The object analysis module comprises: An object reading unit, which reads metadata information of the object in the primary site, data information of the object, and bucket fragmentation information in data_log according to the aggregation characteristics of the object; The object writing unit writes the objects that need to be synchronized to the slave site; The object analysis unit analyzes the index information in the bucket shard information.

6. The synchronization system for aggregated objects in an object storage system according to claim 5, characterized in that: The process of the object reading unit reading the object is specifically as follows: If the object is marked with an aggregation feature, data information of the object is read from the SSD storage pool of the primary site; If the object is not marked with an aggregation feature, metadata information of the object is read from the SSD storage pool of the primary site, and data information and an offset of the object are obtained from the HDD storage pool of the primary site through the metadata information of the object.

Images