Object metadata storage method, device, electronic device and storage medium

Abstract

The present application provides an object metadata storage method, device, electronic device and storage medium. In an object storage system, at least two storage buckets correspond to the same storage space, the storage space is divided into N storage partitions, and each storage partition corresponds to at least one storage medium. Through the method provided by the present application, multiple object metadata to be stored in a storage bucket can be evenly stored in the storage space (i.e., storage medium) corresponding to each storage partition, which enables the object metadata in the same storage bucket to be stored across storage media, and is no longer limited to the storage capacity of a single storage medium, thereby increasing the object metadata storage capacity of a single bucket.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to an object metadata storage method, device, electronic device, and storage medium. Background Art

[0002] With the explosive growth of unstructured data, object storage has emerged. Objects are the fundamental unit of object storage, encompassing a file's data (i.e., object data) and its associated attribute information (i.e., object metadata). Buckets are the fundamental entity in object storage, serving as containers for objects and object metadata.

[0003] To facilitate object management, object data and object metadata are usually stored separately. Currently, the object metadata of a single bucket is usually stored on a single storage medium (such as a disk). Due to the capacity of a single storage medium, the storage capacity of the object metadata in a single bucket is also limited. Summary of the Invention

[0004] In view of this, the present application provides an object metadata storage method, apparatus, electronic device, and storage medium to increase the object metadata storage capacity of a single bucket.

[0005] An embodiment of the present application provides an object metadata storage method, wherein a storage space is divided into N storage partitions, each storage partition having a corresponding object range; the method includes:

[0006] Determine a reference storage partition for the target object metadata to be stored in the target bucket; the target object is within the object range corresponding to the reference storage partition;

[0007] If it is determined that the reference storage partition is in the first state, the reference storage partition is determined as the target storage partition; any storage partition being in the first state means that under the current object scope, the object metadata stored in the storage space corresponding to the storage partition has not been migrated to the storage space corresponding to other storage partitions;

[0008] If it is determined that the reference storage partition is in the second state, the target storage partition is found from each reference storage partition according to the load balancing principle; if any storage partition is in the second state, it means that under the current object range, the object metadata stored in the storage space corresponding to the storage partition is migrated to the storage space corresponding to other storage partitions;

[0009] Store the target object metadata in the storage space corresponding to the target storage partition.

[0010] An embodiment of the present application further provides an object metadata storage device, which is applied to an object storage system. In the object storage system, at least two storage buckets correspond to the same storage space, and the storage space is divided into N storage partitions, each storage partition having a corresponding object range. The device includes:

[0011] A first determining module is configured to determine a reference storage partition for metadata of a target object to be stored in a target bucket; the target object is within an object range corresponding to the reference storage partition;

[0012] A second determining module is configured to determine the reference storage partition as the target storage partition if it is determined that the reference storage partition is in the first state; if any storage partition is in the first state, it means that, under the current object scope, object metadata stored in the storage space corresponding to the storage partition has not been migrated to the storage space corresponding to other storage partitions;

[0013] A search module is configured to search for a target storage partition from each reference storage partition based on a load balancing principle if it is determined that the reference storage partition is in the second state; if any storage partition is in the second state, the object metadata stored in the storage space corresponding to the storage partition is migrated to the storage space corresponding to other storage partitions under the current object scope;

[0014] The storage module is used to store the target object metadata in the storage space corresponding to the target storage partition.

[0015] An embodiment of the present application further provides an electronic device, comprising: a processor and a computer-readable storage medium for storing computer program instructions, wherein the computer program instructions, when executed by the computer-readable storage medium, enable the processor to execute the steps of the above method.

[0016] An embodiment of the present application further provides a machine-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the steps of the above method can be implemented.

[0017] It can be seen from the above technical solution that in this embodiment, at least two storage buckets in the object storage system correspond to the same storage space, the storage space is divided into N storage partitions, and each storage partition corresponds to at least one storage medium. After obtaining the target object metadata to be stored in the target bucket, the object name of the object to which the target object metadata belongs is first matched with the object range configured in each storage partition to obtain a reference storage partition. Afterwards, if it is determined that the reference storage partition is in the first state, the reference storage partition is determined as the target storage partition. If it is determined that the reference storage partition is in the second state, the target storage partition is found from each reference storage partition according to the load balancing principle, and the target object metadata is stored in the storage space corresponding to the target storage partition. In this way, the metadata of multiple objects to be stored in a storage bucket can be evenly stored in the storage space (i.e., storage medium) corresponding to each storage partition. This enables the object metadata in the same storage bucket to be stored across storage media, and is no longer limited by the storage capacity of a single storage medium, thereby increasing the object metadata storage capacity of a single bucket. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of the object metadata storage method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of a segmentation mark provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a process for finding a target storage partition provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of changing a segmentation mark provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of segmentation identifiers and storage partitions provided in an embodiment of the present application;

[0023] Figure 6 A schematic diagram of the structure of the device provided in the embodiment of the present application;

[0024] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0026] In order to implement the solution provided by the embodiment of the present application, it is necessary to improve the object storage system. Here, at least two storage buckets in the object storage system correspond to the same storage space, which is divided into N storage partitions. The storage space corresponding to each storage partition can be a storage medium (such as a storage disk), or K storage media set according to the copy backup principle (for example, a storage partition corresponds to three storage disks according to the 3-copy principle), which makes each storage bucket correspond to more than one storage medium. On this basis, a corresponding object range is configured for each storage partition, and then the method provided by this embodiment is implemented, so that the metadata of multiple objects to be stored in a storage bucket can be evenly stored in the storage space corresponding to each storage partition, that is, the metadata of multiple objects to be stored in a storage bucket can be evenly stored in the storage medium (for example, a storage disk) corresponding to each storage partition. This realizes that the object metadata in the same storage bucket can be stored across storage media, and is no longer limited by the storage capacity of a single storage medium. Therefore, the storage capacity of object metadata in a single bucket can be increased.

[0027] The following is a detailed description of the object metadata storage method provided in the embodiment of the present application:

[0028] See also Figure 1 , Figure 1 A flowchart of the object metadata storage method provided in an embodiment of the present application.

[0029] like Figure 1 As shown, the process may include the following steps:

[0030] S101, determining a reference storage partition for target object metadata to be stored in a target bucket; the target object is within an object range corresponding to the reference storage partition.

[0031] In this embodiment, each storage partition is configured with a corresponding object range. As an example, the object ranges configured for each partition are divided by split identifiers. That is, the object ranges of N storage partitions are divided by N-1 split identifiers. By setting split identifiers in this way, the storage space of the entire target bucket is logically divided into multiple consecutive storage partitions. When object metadata is to be stored, it is mapped to the storage partition of the corresponding range based on the object name.

[0032] As an embodiment, the form of the segmentation identifier is the same as the form of the object name of the object to which the object metadata belongs. The form mentioned here may be an encoded form. For example, the object name to which the metadata belongs is composed of characters from a specified dictionary (such as an ASCII dictionary), then the N-1 segmentation identifiers are the N-1 characters in the specified dictionary.

[0033] For example, if Figure 2As shown, the four storage partitions of the target bucket are divided by the specified characters F, K, and Q. If the name of the object to which the target object metadata to be stored in the target bucket belongs is S, and according to the sorting size in the specified dictionary corresponding to the established lexicographic order, S is greater than Q, then storage partition 4 is determined to be the reference storage partition. If the name of the object to which the target object metadata to be stored in the target bucket belongs is H, and H is between F and Q, then storage partition 2 is determined to be the reference storage partition.

[0034] It should be noted that when setting N-1 partition identifiers, it is necessary to ensure that the object metadata to be subsequently stored in each storage partition complies with the load balancing principle. Furthermore, the designated dictionary is not limited to the above-mentioned dictionary and can also be other dictionaries. It can be set based on actual operating conditions and is not specifically limited in this embodiment of the application.

[0035] S102: If it is determined that the reference storage partition is in the first state, the reference storage partition is determined as the target storage partition.

[0036] In this embodiment, any storage partition being in the first state means that, under the current object scope, object metadata stored in the storage space corresponding to the storage partition has not been migrated to the storage space corresponding to other storage partitions.

[0037] S103: If it is determined that the reference storage partition is in the second state, the target storage partition is found from the reference storage partitions according to the load balancing principle.

[0038] In this embodiment, any storage partition being in the second state means that, under the current object scope, object metadata stored in the storage space corresponding to the storage partition is migrated to storage spaces corresponding to other storage partitions.

[0039] It should be noted that after configuring the object range for each storage partition, the initial state of each storage partition is the first state. As for when to switch to the second state, it will be explained in the following embodiments and will not be repeated here.

[0040] In this embodiment, the storage partition can distinguish whether it is in the first state or the second state by an indicator tag. For example, after configuring the object range for each storage partition, each storage partition carries a first indicator tag. When it is determined that the storage partition has changed to the second state, the first indicator tag is changed to a second indicator tag. For example, the first indicator tag is 0 and the second indicator tag is 1.

[0041] In this embodiment, the step S103 of searching for the target storage partition from the reference storage partitions based on the load balancing principle will be described later and will not be repeated here.

[0042] S104: Store the target object metadata into the storage space corresponding to the target storage partition.

[0043] In this embodiment, optionally, as an embodiment, a storage partition corresponds to a storage medium, such as a storage disk. After the target storage partition is determined, the target object metadata is stored in the storage disk corresponding to the target storage partition.

[0044] So far, completed Figure 1 The process shown.

[0045] pass Figure 1 The process shown in this embodiment is based on the premise that at least two storage buckets in the object storage system correspond to the same storage space, the storage space is divided into N storage partitions, and each storage partition corresponds to at least one storage medium. After obtaining the target object metadata to be stored in the target bucket, the object name of the object to which the target object metadata belongs is first matched with the object range configured in each storage partition to obtain a reference storage partition. Afterwards, if it is determined that the reference storage partition is in the first state, the reference storage partition is determined as the target storage partition. If it is determined that the reference storage partition is in the second state, the target storage partition is found from each reference storage partition according to the load balancing principle, and the target object metadata is stored in the storage space corresponding to the target storage partition. In this way, the metadata of multiple objects to be stored in a storage bucket can be evenly stored in the storage space (i.e., storage medium) corresponding to each storage partition. This enables the object metadata in the same storage bucket to be stored across storage media, and is no longer limited by the storage capacity of a single storage medium, thereby increasing the object metadata storage capacity of a single bucket.

[0046] The following describes how to find the target storage partition from each reference storage partition based on the load balancing principle:

[0047] See also Figure 3 , Figure 3 A schematic diagram of the process of finding the target storage partition provided in an embodiment of the present application.

[0048] like Figure 3 As shown, the process may include the following steps:

[0049] S301, setting the number of virtual nodes for each storage partition based on a hash consistency algorithm.

[0050] In this embodiment, a virtual layer is established between the storage bucket and each partition. This virtual layer consists of several virtual nodes. The target storage partition can be found by determining the virtual node of the target object metadata. Therefore, it is necessary to set the number of virtual nodes for each storage partition based on the hash consistency algorithm.

[0051] As an embodiment, by balancing the relationship between the number of virtual nodes and uniformity, the total number of virtual nodes in the virtual layer is 0 to 2 ^16 , that is, 65536 virtual nodes. If there are 8 storage partitions, the number of virtual nodes for each storage partition should be 8192, which is 65536 divided by 8.

[0052] The distribution of virtual nodes in each storage partition is as follows:

[0053] Storage partition 1 - (0 to 8191 virtual nodes)

[0054] Storage partition 2 - (8192 to 16383 virtual nodes)

[0055] Storage partition 3 - (16384 to 24575 virtual nodes)

[0056] And so on until storage partition 8.

[0057] S302 , performing a hash operation on the object name of the target object to which the target object metadata belongs, the bucket name of the storage bucket corresponding to the target object, and the version number of the target object to obtain a corresponding hash value.

[0058] In this embodiment, the hash value can be expressed by the following formula:

[0059] H=hash(BucketName+ObjectName+VersionId)

[0060] BucketName represents the bucket name; ObjectName represents the object name; hash represents the hash operation; VersionId represents the version number; and H represents the hash value.

[0061] It should be noted that if the bucket does not have multiple versions enabled, the default VersionId is null.

[0062] S303: Calculate the modulus of the obtained hash value by the total number of virtual nodes of all storage partitions, and determine the target storage partition to which the target object metadata belongs based on the obtained modulus result and the number of virtual nodes set for each storage partition.

[0063] Specifically, it can be expressed as:

[0064] V=H%K

[0065] Where K represents the total number of virtual nodes; V is the remainder result.

[0066] For example, if the region result is 2, reference partition 2 is determined to be the target storage partition.

[0067] By adopting the above approach, when it is determined that the reference storage partition is in the second state, the object metadata to be stored can be distributed and stored in each partition as evenly as possible to ensure load balancing of the object storage system.

[0068] The above describes in detail how to find the target storage partition from various storage partitions based on the load balancing principle.

[0069] The switching between the first state and the second state is described in detail below:

[0070] It is known that after each storage partition is configured with the object range, each storage partition is in the first state. Figure 2 If the target bucket stores a large amount of object metadata whose names begin with A, the object metadata stored in storage partition 1 will be much larger than that in other partitions. In this case, the object metadata in storage partition 1 needs to be migrated to achieve load balancing.

[0071] As an embodiment, when a set migration period arrives, for each storage partition in the first state, the occupancy rate of the storage space corresponding to the storage partition is obtained. If the occupancy rate of the storage space corresponding to the storage partition is greater than a set occupancy rate threshold, at least a portion of the object metadata stored in the storage space corresponding to the storage partition is migrated to a destination storage partition different from the current storage partition, and the state of the storage partition is changed from the first state to the second state.

[0072] As an embodiment, the migration of object metadata stored in the storage space corresponding to any storage partition is achieved by:

[0073] Traverse each stored object metadata in the storage space corresponding to the storage partition, and use the traversed stored object metadata as the current object metadata. Perform a hash operation on the object name of the object to which the current object metadata belongs, the bucket name of the storage bucket corresponding to the object, and the version number of the object to obtain the corresponding hash value. Take the remainder of the obtained hash value and the total number of virtual nodes of all storage partitions, and determine the reference migration storage partition corresponding to the current object metadata based on the obtained remainder result and the number of virtual nodes set for each storage partition (the specific implementation method is the same as above). Figure 3 (The implementation method is similar and will not be repeated here.) If the reference migration storage partition is different from the current storage partition, the reference migration storage partition is determined as the destination storage partition. If the reference migration storage partition is the same as the current storage partition, no migration is performed.

[0074] Based on the above description, it can be seen that for storage partitions in the second state, another hash operation is required when storing object metadata, but hash operations still take time. To optimize system performance, when the object storage system is in a set idle state, the object range corresponding to at least one storage partition can be changed according to the load balancing principle. In this case, the state of each storage partition is changed from the second state to the first state. This allows subsequent object metadata storage to simply match the object name with the configured object range of each storage partition, eliminating the need for another hash operation. This optimizes the performance of the object storage system during subsequent storage. Furthermore, changing the object range according to the load balancing principle ensures that the storage space corresponding to each storage partition also complies with load balancing during subsequent storage.

[0075] There are many ways to implement the change of the object range corresponding to at least one storage partition. For example, as an embodiment, Figure 4 As shown in the figure, the split identifier between two adjacent storage partitions is changed. The specific change principle must follow the load balancing principle.

[0076] It should be noted that after the object range corresponding to each storage partition is changed, the object metadata stored in each storage partition needs to be re-migrated. The re-migration can be implemented in the following specific ways:

[0077] For each storage partition, traverse each stored object metadata in the storage space corresponding to the storage partition, and use the traversed stored object metadata as the current object metadata. Compare the object name of the object to which the current object metadata belongs with the new object range corresponding to each partition, and determine the reference migration storage partition corresponding to the current object metadata (the specific implementation method is similar to the implementation method of step 101 above, which will not be repeated here). If the reference migration storage partition is different from the current storage partition, then determine the reference migration storage partition as the destination storage partition. If the reference migration storage partition is the same as the current storage partition, no further migration is performed.

[0078] After the above re-migration is completed, the state of the storage partition currently in the second state is changed from the second state to the first state.

[0079] The switching between the first state and the second state is described in detail above.

[0080] In order to explain the method provided by the present application in more detail, the solution provided by the present application is described in more detail below in the form of specific embodiments.

[0081] In this embodiment, the pre-configuration stage includes the following steps:

[0082] 1. Please refer to Figure 5As shown in the figure, suppose there are four nodes (0001, 0002, 0003, and 0004). Each node has two disks (A and B) for storing object metadata. Each disk of each node forms a storage partition. That is, 0001A, 0002A, 0003A, 0004A, 0001B, 0002B, 0003B, and 0004B correspond to the eight storage partitions 1 through 8, respectively. The partition identifiers of the eight storage partitions are D, G, J, M, P, S, and U.

[0083] 2. The initial indicator tag of each storage partition is 0. When the disk occupancy rate of a storage partition exceeds the set occupancy rate threshold, the indicator tag is changed to 1.

[0084] 3. Based on the hash consistency algorithm, virtual nodes are allocated to the eight storage partitions, storage partition 1 to storage partition 8.

[0085] Specifically, storage partition 1-(0 to 8191 virtual nodes)

[0086] Storage partition 2 - (8192 to 16383 virtual nodes)

[0087] Storage partition 3 - (16384 to 24575 virtual nodes)

[0088] And so on until storage partition 8.

[0089] Subsequent object metadata storage includes the following steps:

[0090] 1. When it is determined that the target object metadata is to be written, the reference storage partition corresponding to the target object metadata is determined by comparing the object name of the target object metadata with each segmentation identifier.

[0091] 2. If the identifier of the reference storage partition is 0, the reference storage partition is determined as the target storage partition.

[0092] 3. If the reference storage partition is identified as 1, use the following formula:

[0093] H=hash(BucketName+ObjectName+VersionId) performs a hash operation to obtain a hash value, and uses the following formula:

[0094] V=H%K is modulo to obtain a modulo result, and the target storage partition is determined according to the modulo result. For example, if the modulo result is 2, the storage partition 2 is determined as the target storage partition.

[0095] 4. Store the target object metadata in the storage space corresponding to the target storage partition.

[0096] This completes the description of the method provided in this embodiment. The following describes the device provided in this embodiment:

[0097] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an apparatus provided in an embodiment of the present application. The apparatus is applied to an object storage system, where at least two buckets correspond to the same storage space. The storage space is divided into N storage partitions, each of which has a corresponding object range. The apparatus 600 includes a first determination module 601, a second determination module 602, a search module 603, and a storage module 604.

[0098] The first determining module 601 is configured to determine a reference storage partition for metadata of a target object to be stored in a target bucket; the target object is within an object range corresponding to the reference storage partition;

[0099] A second determining module 602 is configured to determine the reference storage partition as the target storage partition if it is determined that the reference storage partition is in the first state; if any storage partition is in the first state, it means that, under the current object scope, object metadata stored in the storage space corresponding to the storage partition has not been migrated to the storage space corresponding to another storage partition;

[0100] Search module 603 is configured to search for a target storage partition from each reference storage partition based on a load balancing principle if the reference storage partition is determined to be in the second state; if any storage partition is in the second state, the object metadata stored in the storage space corresponding to the storage partition is migrated to the storage space corresponding to another storage partition within the current object scope;

[0101] The storage module 604 is configured to store the target object metadata in the storage space corresponding to the target storage partition.

[0102] As an embodiment, there is a split identifier between two adjacent storage partitions, and the split identifier limits the object range corresponding to each of the two adjacent storage partitions; wherein the format of the split identifier is the same as the format of the object name of the object to which the object metadata belongs.

[0103] As an embodiment, searching for a target storage partition from each reference storage partition according to a load balancing principle includes:

[0104] Set the number of virtual nodes for each storage partition based on the hash consistency algorithm;

[0105] Perform a hash operation on the object name of the target object to which the target object metadata belongs, the bucket name of the storage bucket corresponding to the target object, and the version number of the target object to obtain a corresponding hash value;

[0106] The obtained hash value is modulo the total number of virtual nodes in all storage partitions, and the number of virtual nodes is set for each storage partition based on the obtained modulo result to determine the target storage partition to which the target object metadata belongs.

[0107] As an embodiment, when a set migration cycle arrives, for each storage partition in the first state, an occupancy rate of the storage space corresponding to the storage partition is obtained; wherein the initial state of any storage partition in the current object scope is the first state;

[0108] If the occupancy rate of the storage space corresponding to the storage partition is greater than the set occupancy rate threshold, at least part of the object metadata stored in the storage space corresponding to the storage partition is migrated to a destination storage partition different from the current storage partition, and the state of the storage partition is changed from the first state to the second state.

[0109] As an embodiment, the destination storage partition is determined by the following steps:

[0110] For each current object metadata stored in the storage space corresponding to the storage partition, a hash operation is performed on the object name of the object to which the current object metadata belongs, the bucket name of the storage bucket corresponding to the object, and the version number of the object to obtain a corresponding hash value;

[0111] The obtained hash value is modulo the total number of virtual nodes in all storage partitions. Based on the obtained modulo result and the number of virtual nodes set for each storage partition, the reference migration storage partition corresponding to the current object metadata is determined.

[0112] If the reference migration storage partition is different from the current storage partition, the reference migration storage partition is determined as the destination storage partition.

[0113] As an embodiment, the device further includes:

[0114] The update module is used to change the object range corresponding to at least one storage partition according to the load balancing principle when the state of the object storage system is a set idle state, so that the storage space corresponding to each storage partition complies with load balancing in subsequent storage; and change the state of each storage partition currently in the second state from the second state to the first state.

[0115] So far, completed Figure 6 Structural description of the device shown.

[0116] See Figure 7 , Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 7As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0117] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0118] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0119] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for storing object metadata, characterized in that: The method is applied to an object storage system, wherein at least two storage buckets in the object storage system correspond to the same storage space, the storage space is divided into N storage partitions, and each storage partition has a corresponding object range; the method comprises: Determine a reference storage partition for target object metadata to be stored in a target bucket; the target object is within an object range corresponding to the reference storage partition; If it is determined that the reference storage partition is in the first state, the reference storage partition is determined as the target storage partition; any storage partition being in the first state means that under the current object scope, the object metadata stored in the storage space corresponding to the storage partition has not been migrated to the storage space corresponding to other storage partitions; If it is determined that the reference storage partition is in the second state, the target storage partition is found from each reference storage partition according to the load balancing principle; if any storage partition is in the second state, it means that under the current object range, the object metadata stored in the storage space corresponding to the storage partition is migrated to the storage space corresponding to other storage partitions; Storing the target object metadata in the storage space corresponding to the target storage partition; When the set migration period arrives, for each storage partition in the first state, obtaining the occupancy rate of the storage space corresponding to the storage partition; wherein the initial state of any storage partition in the current object scope is the first state; If the occupancy rate of the storage space corresponding to the storage partition is greater than the set occupancy rate threshold, at least part of the object metadata stored in the storage space corresponding to the storage partition is migrated to a destination storage partition different from the current storage partition, and the state of the storage partition is changed from the first state to the second state.

2. The method according to claim 1, characterized in that There is a split identifier for two adjacent storage partitions, and the split identifier limits the object range corresponding to each of the two adjacent storage partitions; wherein the format of the split identifier is the same as the format of the object name of the object to which the object metadata belongs.

3. The method according to claim 1, characterized in that The step of finding the target storage partition from the reference storage partitions according to the load balancing principle includes: Set the number of virtual nodes for each storage partition based on the hash consistency algorithm; Perform a hash operation on the object name of the target object to which the target object metadata belongs, the bucket name of the storage bucket corresponding to the target object, and the version number of the target object to obtain a corresponding hash value; The obtained hash value is modulo the total number of virtual nodes of all storage partitions, and the number of virtual nodes set for each storage partition is determined based on the obtained modulo result to determine the target storage partition to which the target object metadata belongs.

4. The method according to claim 1, wherein The destination storage partition is determined by the following steps: For each current object metadata stored in the storage space corresponding to the storage partition, a hash operation is performed on the object name of the object to which the current object metadata belongs, the bucket name of the storage bucket corresponding to the object, and the version number of the object to obtain a corresponding hash value; Taking the remainder of the obtained hash value and the total number of virtual nodes of all storage partitions, and determining the reference migration storage partition corresponding to the current object metadata based on the remainder and the number of virtual nodes set for each storage partition; If the reference migration storage partition is different from the current storage partition, the reference migration storage partition is determined as the destination storage partition.

5. The method according to claim 1, wherein The method further comprises: When the state of the object storage system is a set idle state, the object range corresponding to at least one storage partition is changed according to the load balancing principle so that the storage space corresponding to each storage partition complies with load balancing in subsequent storage; and the state of each storage partition currently in the second state is changed from the second state to the first state.

6. An object metadata storage device, characterized in that: The device is applied to an object storage system, wherein at least two storage buckets in the object storage system correspond to the same storage space, the storage space is divided into N storage partitions, and each storage partition has a corresponding object range; the device comprises: A first determining module is configured to determine a reference storage partition for metadata of a target object to be stored in a target bucket; the target object is within an object range corresponding to the reference storage partition; a second determining module, configured to determine the reference storage partition as a target storage partition if it is determined that the reference storage partition is in a first state; if any storage partition is in the first state, it means that, under a current object scope, object metadata stored in a storage space corresponding to the storage partition has not been migrated to a storage space corresponding to another storage partition; a search module configured to search for a target storage partition from each reference storage partition based on a load balancing principle if it is determined that the reference storage partition is in the second state; if any storage partition is in the second state, the object metadata stored in the storage space corresponding to the storage partition is migrated to the storage space corresponding to other storage partitions under the current object scope; A storage module, configured to store the target object metadata in a storage space corresponding to a target storage partition; When the set migration period arrives, for each storage partition in the first state, obtaining the occupancy rate of the storage space corresponding to the storage partition; wherein the initial state of any storage partition in the current object scope is the first state; If the occupancy rate of the storage space corresponding to the storage partition is greater than the set occupancy rate threshold, at least part of the object metadata stored in the storage space corresponding to the storage partition is migrated to a destination storage partition different from the current storage partition, and the state of the storage partition is changed from the first state to the second state.

7. The device according to claim 6, characterized in that There is a split identifier for two adjacent storage partitions, and the split identifier limits the object range corresponding to each of the two adjacent storage partitions; wherein the format of the split identifier is the same as the format of the object name of the object to which the object metadata belongs; And / or, searching for the target storage partition from each reference storage partition according to the load balancing principle includes: Set the number of virtual nodes for each storage partition based on the hash consistency algorithm; Perform a hash operation on the object name of the target object to which the target object metadata belongs, the bucket name of the storage bucket corresponding to the target object, and the version number of the target object to obtain a corresponding hash value; Taking the remainder of the obtained hash value and the total number of virtual nodes of all storage partitions, and determining the target storage partition to which the target object metadata belongs based on the remainder obtained and the number of virtual nodes set for each storage partition; and / or, The destination storage partition is determined by the following steps: For each current object metadata stored in the storage space corresponding to the storage partition, a hash operation is performed on the object name of the object to which the current object metadata belongs, the bucket name of the storage bucket corresponding to the object, and the version number of the object to obtain a corresponding hash value; Taking the remainder of the obtained hash value and the total number of virtual nodes of all storage partitions, and determining the reference migration storage partition corresponding to the current object metadata based on the remainder and the number of virtual nodes set for each storage partition; If the reference migration storage partition is different from the current storage partition, determining the reference migration storage partition as the destination storage partition; And / or, the device further comprises: An update module is used to change the object range corresponding to at least one storage partition according to the load balancing principle when the state of the object storage system is a set idle state, so that the storage space corresponding to each storage partition complies with load balancing in subsequent storage; and change the state of each storage partition currently in the second state from the second state to the first state.

8. An electronic device, characterized in that: The electronic device includes: processor; and A computer-readable storage medium having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the method according to any one of claims 1 to 5.

Images