A quota correction method and related components

Abstract

The application discloses a quota correction method and related components, and relates to the technical field of distributed storage. A total control server respectively issues corresponding correction identification information to each object storage gateway server. The correction identification information includes a first identification of a target bucket and shard information. The number of bucket index shards corresponding to the shard information is equal to the total number of bucket index shards corresponding to the target bucket. Therefore, the pressure on each object storage gateway server during quota correction is relatively average, and a single object storage gateway server does not bear excessive processing pressure. According to the correction reference information fed back by the object storage gateway server, the historical quota information corresponding to the target bucket is updated. The scheme is beneficial to accurately measuring the storage cluster resources, beneficial to reasonably and accurately planning the storage cluster resources, and improves the market influence of the distributed storage cluster product.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and in particular to a quota correction method and related components. Background Technology

[0002] Distributed storage systems are clustered storage systems with advantages such as high performance, high scalability, and high reliability. Object storage, from a cluster perspective, distributes data to be stored across multiple storage nodes and is now widely used. As market businesses become increasingly reliant on storage systems, distributed storage systems may perform multiple object operations during actual storage processes. This means that for a specific bucket, various operations such as object upload, object deletion, object modification, cancellation of shard upload tasks, and garbage collection may be performed. Taking the deletion or recycling of a stored object as an example, it is possible that although the object has been deleted or recycled, the quota corresponding to that bucket may not have been modified, leading to inaccurate quota records. Since the quota corresponding to a bucket is essentially a statistical measure of the resources within that bucket, errors in this record will result in a poor user experience and are detrimental to practical applications.

[0003] Therefore, how to provide a solution to the above-mentioned technical problems is a problem that those skilled in the art need to solve at present. Summary of the Invention

[0004] The purpose of this invention is to provide a quota adjustment method and related components. The quota adjustment scheme provided by this invention for distributed storage clusters facilitates the accurate measurement of storage cluster resources, which in turn facilitates the reasonable and accurate planning of storage cluster resources and enhances the market influence of distributed storage clusters.

[0005] To address the aforementioned technical problems, this invention provides a quota adjustment method applied to a central control server in a distributed storage cluster. The central control server is connected to multiple object storage gateway servers within the distributed storage cluster. The quota adjustment method includes:

[0006] The corresponding correction identification information is sent to each of the object storage gateway servers. The correction identification information includes the first identifier of the target bucket and the shard information of the bucket index shard corresponding to the target bucket. The sum of the number of bucket index shards corresponding to each of the shard information is equal to the total number of bucket index shards corresponding to the target bucket.

[0007] Based on the calibration baseline information received from the object storage gateway server, the historical quota information corresponding to the target bucket is updated. The calibration baseline information represents the current actual storage status of each object in the corresponding bucket index shard.

[0008] Furthermore, the step of determining the correction identifier information includes:

[0009] Group all bucket index shards corresponding to the target bucket to obtain X basic correction groups, where X is the first total number of the object storage gateway servers. The X-1 basic correction groups include S bucket index shards, and the remaining basic correction groups include A bucket index shards, satisfying (X-1).

[0010] *S+A=M, where M is the total number of bucket index shards, and S is the result of dividing M by X and rounding down, where X is an integer greater than 1 and S is an integer greater than 1.

[0011] The segment information corresponding to the basic correction group and the first identifier of the target bucket are determined as a set of correction identification information to obtain multiple sets of correction identification information.

[0012] Furthermore, the calibration reference information includes the second identifier and actual storage information of each object in the corresponding bucket index shard;

[0013] Based on the calibration baseline information received from the object storage gateway server, update the historical quota information corresponding to the target bucket, including:

[0014] Based on the second identifier of the target object fed back by the current object storage gateway server as the search benchmark, determine the quota information to be corrected corresponding to the target object in the historical quota information of the target bucket;

[0015] The quota information to be corrected is updated using the actual storage information of the target object.

[0016] Furthermore, after receiving the calibration benchmark information fed back by the object storage gateway server, the process also includes:

[0017] Based on the quota correction identifier corresponding to the target bucket, determine whether the target bucket is undergoing its first correction;

[0018] If so, clear the historical storage information corresponding to each object in the target bucket.

[0019] Furthermore, it also includes:

[0020] Determine whether all objects in the bucket index shard corresponding to the target object have completed quota information correction;

[0021] If quota information correction has been completed, change the correction status of the bucket index shard to the corrected status.

[0022] Determine whether each bucket index fragment corresponding to the target bucket is in a corrected state;

[0023] If all are in a calibrated state, the quota calibration of the target bucket is considered complete.

[0024] Furthermore, corresponding correction identification information is sent to each of the object storage gateway servers, including:

[0025] When the quota correction conditions are met, corresponding correction identification information is sent to each of the object storage gateway servers, wherein the quota correction conditions are reaching a preset correction period and / or receiving a trigger signal indicating that quota correction is to be performed.

[0026] Furthermore, the step of the object storage gateway server determining the calibration reference information includes:

[0027] The object storage gateway server determines whether the object still exists in the target bucket based on the first identifier of the target bucket and the second identifier of each object in the sharding information of the bucket index shard.

[0028] If not, determine that the current storage state of the object is cleared and the corresponding current storage capacity is 0;

[0029] If so, determine that the current storage state of the object is normal storage;

[0030] Determine the current storage capacity of the object;

[0031] The second identifier of the object, the current storage state of the object, and the current storage capacity are used as the correction reference information corresponding to the object.

[0032] To address the aforementioned technical problems, the present invention also provides a quota adjustment system applied to a central control server in a distributed storage cluster. The central control server is connected to multiple object storage gateway servers in the distributed storage cluster. The quota adjustment system includes:

[0033] The information delivery unit is used to send corresponding correction identification information to each of the object storage gateway servers. The correction identification information includes the first identifier of the target bucket and the shard information of the bucket index shard corresponding to the target bucket. The sum of the number of bucket index shards corresponding to each of the shard information is equal to the total number of bucket index shards corresponding to the target bucket.

[0034] The update unit is used to update the historical quota information corresponding to the target bucket according to the correction benchmark information fed back by the object storage gateway server. The correction benchmark information represents the current actual storage status of each object in the corresponding bucket index shard.

[0035] To address the aforementioned technical problems, the present invention also provides a server, comprising:

[0036] Memory, used to store computer programs;

[0037] A processor for executing the computer program to implement the steps of the quota correction method as described above.

[0038] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium, comprising:

[0039] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the quota correction method as described above.

[0040] This application provides a quota correction method and related components. The central control server sends corresponding correction identification information to each object storage gateway server. This correction identification information includes the first identifier of the target bucket and the shard information of the bucket index shards corresponding to the target bucket. The sum of the number of bucket index shards corresponding to the shard information equals the total number of bucket index shards corresponding to the target bucket. This ensures a more even distribution of pressure on each object storage gateway server during quota correction, avoiding excessive processing load on any single object storage gateway server and reducing the impact of quota correction on the actual business processing of each object storage gateway server. Based on the correction baseline information received from the object storage gateway servers, the historical quota information corresponding to the target bucket is updated. This correction baseline information represents the current actual storage status of each object in the corresponding bucket index shard. Therefore, this quota correction scheme for distributed storage clusters facilitates accurate measurement of storage cluster resources, enabling subsequent reasonable and precise planning of storage cluster resources and enhancing the market influence of distributed storage cluster products. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the relevant technologies and the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a quota adjustment method provided by the present invention;

[0043] Figure 2 This is a schematic diagram illustrating the distribution of corresponding correction identification information to the object storage gateway server, as provided by the present invention.

[0044] Figure 3 This is a schematic diagram of the structure of a quota correction system provided by the present invention;

[0045] Figure 4A schematic diagram of the structure of a server provided by the present invention;

[0046] Figure 5 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0047] The core of this invention is to provide a quota correction method and related components. The quota correction scheme provided by this invention for distributed storage clusters is conducive to the accurate measurement of storage cluster resources, which is beneficial for subsequent reasonable and accurate planning of storage cluster resources, thereby enhancing the market influence of distributed storage cluster products.

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Please refer to Figure 1 , Figure 1 A flowchart of a quota correction method provided by the present invention.

[0050] In this embodiment, considering that distributed storage systems may perform multiple object operations during actual storage, such as object uploading, object deletion, and object modification, it is possible that the corresponding operation has been performed on the stored object, but the quota corresponding to the bucket has not been modified, resulting in inaccurate quota records. Since the quota corresponding to a bucket is essentially a statistical measure of the resources under that bucket, errors in this record will lead to a poor user experience and affect the user's planning and allocation of resources in the distributed storage system. To solve the above technical problems, this application provides a quota correction scheme for distributed storage clusters, which facilitates the accurate measurement of storage cluster resources.

[0051] This quota adjustment method is applied to a central control server in a distributed storage cluster. The central control server is connected to multiple object storage gateway servers in the distributed storage cluster. The quota adjustment method includes:

[0052] S11: Send the corresponding correction identification information to each object storage gateway server respectively. The correction identification information includes the first identifier of the target bucket and the shard information of the bucket index shard corresponding to the target bucket. The sum of the number of bucket index shards corresponding to each shard information is equal to the total number of bucket index shards corresponding to the target bucket.

[0053] Specifically, the distributed storage cluster includes multiple RGWs (RADOS gateways), i.e., object storage gateway servers, which can provide object storage services to the outside world; the target bucket is any one of the multiple buckets in the distributed storage cluster, and the target bucket includes multiple objects. Each target bucket corresponds to multiple bucket index shards. The bucket index shard is used to store the index information of multiple objects. It maintains a key-val omap structure, where the key is the name of the object and the val is some metadata information of the object.

[0054] Considering that assigning quota correction tasks to a single object storage gateway server would overload that server and hinder business operations, a corresponding correction identifier is assigned to each object storage gateway server and distributed to all other object storage gateway servers. Each object storage gateway server processes its assigned quota correction task independently, and the processing procedure is identical across all servers. Please refer to [reference needed]. Figure 2 , Figure 2 This is a schematic diagram illustrating the distribution of corresponding correction identification information to an object storage gateway server, provided by the present invention. The example uses X object storage gateway servers. Specifically, the correction identification information may include a first identifier and shard information. The first identifier includes, but is not limited to, the name of the target bucket (each bucket name is unique). The shard information is used to identify which bucket index shards under the target bucket will be distributed to the object storage gateway server. For example, assuming the total number of bucket index shards is 1024, the sum of the number of bucket index shards corresponding to the shard information of each object storage gateway server is 1024, so that quota correction for all bucket index shards in the target bucket can be subsequently implemented.

[0055] S12: Based on the calibration baseline information received from the object storage gateway server, update the historical quota information corresponding to the target bucket. The calibration baseline information represents the current actual storage status of each object in the corresponding bucket index shard.

[0056] Specifically, for any object storage gateway server, upon receiving the correction identification information, it determines which target bucket and which bucket index shards under that target bucket it needs to be responsible for in terms of quota correction. The shard information corresponding to each bucket index shard records the second identifiers of multiple objects. These second identifiers include, but are not limited to, the names of the objects. Based on this, the object storage gateway server can determine the correction benchmark information representing the current actual storage status of each object from the storage pool and feed it back to the central control server. The central control server records the historical quota information corresponding to the target bucket. This historical quota information records the total number of objects included in the target bucket, as well as the second identifier and historical storage capacity of each object. By updating the historical quota information according to the received correction benchmark information, the quota correction for the target bucket can be achieved.

[0057] In summary, this application provides a quota correction method in which the central control server sends corresponding correction identification information to each object storage gateway server, thereby distributing the pressure on each object storage gateway server more evenly during quota correction. This avoids excessive processing pressure on any single object storage gateway server and reduces the impact of quota correction on the actual business processing of each object storage gateway server. Furthermore, as a quota correction scheme for distributed storage clusters, this approach facilitates accurate measurement of storage cluster resources, enabling reasonable and precise planning of these resources in the future. This enhances the market influence of distributed storage clusters and provides users with a better storage experience.

[0058] Based on the above embodiments:

[0059] In some embodiments, the step of determining the corrected identification information includes:

[0060] Group all bucket index shards corresponding to the target bucket to obtain X basic correction groups, where X is the first total number of object storage gateway servers, X-1 basic correction groups include S bucket index shards and the remaining basic correction groups include A bucket index shards, and satisfy (X-1)*S+A=M, where M is the total number of bucket index shards, S is obtained by dividing M by X and rounding down, where X is an integer greater than 1 and S is an integer greater than 1;

[0061] The segment information corresponding to the basic calibration group and the first identifier of the target bucket are determined as a set of calibration identifier information to obtain multiple sets of calibration identifier information.

[0062] This embodiment provides the steps for determining each correction identifier information, as detailed in the previous embodiment, and will not be repeated here. For example, taking a target bucket containing 1024 bucket index shards (M = 1024) and a total of 5 object storage gateway servers (X = 5), the steps are further explained: M divided by X and rounded down yields S, which equals 204. Therefore, the four basic correction groups contain 204 bucket index shards, and the remaining basic correction groups contain 208 bucket index shards (A = 208), to distribute the bucket index shards as evenly as possible to each object storage gateway server. The specific number of bucket index shards selected in each basic correction group is not specifically limited here and can be flexibly allocated according to actual business needs. Only the number of bucket index shards included in the basic correction group is limited here. Each bucket index shard in any basic correction group corresponds to shard information. Together with the first identifier, they form a set of correction identifier information, and finally multiple sets of correction identifier information are obtained. It can be understood that the total number of correction identifier information sets is X.

[0063] It should also be noted that in practical applications, based on the business pressure of each object storage gateway server, a portion of the object storage gateway servers can be selected as target object storage gateway servers to perform quota correction tasks. No special restrictions are imposed here; it can be determined according to actual needs. It should be noted that in this case, X will be the second total number of target object storage gateway servers.

[0064] In some embodiments, the calibration baseline information includes the second identifier and actual storage information of each object in the corresponding bucket index shard;

[0065] Based on the calibration baseline information received from the object storage gateway server, update the historical quota information corresponding to the target bucket, including:

[0066] Based on the second identifier of the target object fed back by the current object storage gateway server, the quota information to be corrected corresponding to the target object in the historical quota information of the target bucket is determined.

[0067] Update the quota information to be corrected using the actual storage information of the target object.

[0068] In this embodiment, the correction benchmark information fed back by any object storage gateway server may include the second identifier and actual storage information of each object in the bucket index shard it is responsible for. The second identifier includes, but is not limited to, the name of the object, and the actual storage information includes, but is not limited to, the current storage status and current storage capacity of the object. The target object is any one of the objects in the bucket index shard. Therefore, the update of historical quota information can be done by overwriting. That is, the historical quota information of the target bucket includes the identity identifier of the target object and its historical storage capacity. Therefore, the second identifier can be used as a search benchmark to determine the quota information to be corrected corresponding to the target object. Thus, the quota information to be corrected is updated by using the actual storage information of the target object in the form of overwriting. For example, when the storage capacity of the target object changes (e.g., the current storage status of the target object is normal storage and there is a non-zero current storage capacity), the historical storage capacity is updated according to the latest current storage capacity. When the target object is deleted (e.g., the current storage status of the target object is cleared and the current storage capacity is 0), the relevant information of the target object is directly deleted, and the number of objects included in the target bucket is reduced by 1.

[0069] In some embodiments, after receiving the calibration baseline information fed back by the object storage gateway server, the method further includes:

[0070] Based on the quota correction identifier corresponding to the target bucket, determine whether the target bucket is undergoing its first correction;

[0071] If so, clear the historical storage information corresponding to each object in the target bucket.

[0072] In this embodiment, considering the possible differences in the order in which each object storage gateway server feeds back the correction baseline information, a more practical operation method is provided. That is, a quota correction flag is set for the target bucket. The quota correction flag indicates whether the target bucket is undergoing its first correction. The first correction refers to the first time the correction baseline information fed back by the object storage gateway server is received and quota correction begins. If it is the first correction, the historical storage information corresponding to each object in the target bucket is cleared to facilitate the writing of the latest quota information.

[0073] It should also be noted that the quota adjustment flag can be set to 1 when the target bucket is undergoing its first adjustment and 0 when the target bucket is undergoing its first adjustment. There are no special restrictions here. The flag can be set flexibly according to actual needs and the above-mentioned flag setting principle.

[0074] In some embodiments, it also includes:

[0075] Determine whether all objects in the bucket index shard corresponding to the target object have completed quota information correction;

[0076] If quota information correction has been completed, change the correction status of the bucket index shard to the corrected status.

[0077] Determine whether all bucket index shards corresponding to the target bucket are in a corrected state;

[0078] If all are in the calibrated state, the quota calibration of the target bucket is complete.

[0079] In this embodiment, considering that the order in which the correction baseline information is fed back by each object storage gateway server may differ, and that the correction of multiple bucket index shards under the responsibility of the same object storage gateway server may also differ, and that each bucket index shard corresponds to multiple objects, it is determined whether all objects in the bucket index shard corresponding to the target object have completed quota information correction. Completion of quota information correction means that the historical quota status corresponding to the object has been updated. If the quota information correction has not been completed for all objects, quota correction can continue for the remaining objects in the bucket index shard. If the quota information correction has been completed for all objects, the correction status corresponding to the bucket index shard is changed to the corrected status.

[0080] The system checks whether all index shards corresponding to the target bucket are in a corrected state. If not, it indicates that some object storage gateway servers have not yet returned correction baseline information, and the system continues to wait for them to send it. If all shards are in a corrected state, the quota correction for the target bucket is considered complete. It's understandable that in practical applications, users may correspond to multiple buckets in a distributed storage cluster to store various types of data. Therefore, after the historical quota information of the target bucket is updated to the latest quota information, the user's quota information can be updated synchronously accordingly. No specific limitations are imposed here; it depends on the actual needs.

[0081] In some embodiments, corresponding correction identification information is sent to each object storage gateway server, including:

[0082] When the quota correction conditions are met, corresponding correction identification information is sent to each object storage gateway server. The quota correction conditions are reaching a preset correction period and / or receiving a trigger signal indicating that quota correction is to be performed.

[0083] In this embodiment, quota correction can be performed when the quota correction conditions are met, so as to send the corresponding correction identification information to each object storage gateway server. The quota correction can be performed periodically, and the corresponding quota correction condition is to reach a preset correction period. Alternatively, quota correction can be triggered on the human-computer interaction interface when the user finds that the quota status of the current target bucket is recorded incorrectly. The corresponding quota correction condition is to receive a trigger signal indicating that quota correction is to be performed. Of course, other quota correction conditions can be set according to actual needs, which are not particularly limited here.

[0084] In some embodiments, the step of the object storage gateway server determining calibration baseline information includes:

[0085] The object storage gateway server determines whether the object still exists in the target bucket based on the first identifier of the target bucket and the second identifier of each object in the sharding information of the bucket index shard.

[0086] If not, determine that the object's current storage state is cleared and the corresponding current storage capacity is 0;

[0087] If so, determine that the object's current storage state is normal storage;

[0088] Determine the current storage capacity of the object;

[0089] The object's second identifier, the object's current storage state, and the current storage capacity are used as the corresponding correction reference information for the object.

[0090] This embodiment provides the steps for any object storage gateway server to determine the calibration baseline information. Specifically, after receiving the calibration identifier information, the object storage gateway server calls the object query interface to search the storage pool based on the first identifier of the target bucket in the calibration identifier information and the second identifier of each object recorded in the shard information of each bucket index shard. Since the object may still be in the storage pool or may have been deleted and is no longer in the storage pool (i.e., the search result may be empty), it can be determined whether the object still exists in the target bucket based on the search result. If not, it means that the object has been deleted, and the current storage status of the object is determined to be cleared and the corresponding current storage capacity is 0. If yes, since the search result is not empty and the current storage capacity of the object is displayed, the current storage status of the object is determined to be normal storage, and the current storage capacity of the object is determined. The second identifier of the object, the current storage status of the object, and the current storage capacity are used as the calibration baseline information corresponding to the object. Each object in each bucket index shard is operated in the above manner to obtain the calibration baseline information corresponding to each bucket index shard and provide feedback.

[0091] Please refer to Figure 3 , Figure 3This is a schematic diagram of a quota correction system provided by the present invention.

[0092] This quota adjustment system is applied to the central control server in a distributed storage cluster. The central control server is connected to multiple object storage gateway servers in the distributed storage cluster. The quota adjustment system includes:

[0093] The information distribution unit 21 is used to distribute corresponding correction identification information to each object storage gateway server. The correction identification information includes the first identifier of the target bucket and the fragment information of the bucket index fragment corresponding to the target bucket. The sum of the number of bucket index fragments corresponding to each fragment information is equal to the total number of bucket index fragments corresponding to the target bucket.

[0094] The update unit 22 is used to update the historical quota information corresponding to the target bucket according to the correction baseline information fed back by the object storage gateway server. The correction baseline information represents the current actual storage status of each object in the corresponding bucket index shard.

[0095] For a description of the quota correction system provided in this invention, please refer to the embodiments of the quota correction method described above; further details will not be repeated here.

[0096] In some embodiments, the quota correction system includes a correction identification information determination unit for determining each correction identification information;

[0097] The calibration identification information determination unit includes:

[0098] The grouping unit is used to group all bucket index shards corresponding to the target bucket to obtain X basic correction groups, where X is the first total number of object storage gateway servers, X-1 basic correction groups include S bucket index shards and the remaining basic correction groups include A bucket index shards, and satisfy (X-1)*S+A=M, where M is the total number of bucket index shards, S is obtained by dividing M by X and rounding down, where X is an integer greater than 1 and S is an integer greater than 1;

[0099] The first determining unit is used to determine the segment information corresponding to the basic correction group and the first identifier of the target bucket as a set of correction identification information, so as to obtain multiple sets of correction identification information.

[0100] In some embodiments, the calibration baseline information includes the second identifier and actual storage information of each object in the corresponding bucket index shard;

[0101] Update unit 22 includes:

[0102] The quota information to be corrected determination unit is used to determine the quota information to be corrected corresponding to the target object in the historical quota information of the target bucket based on the second identifier of the target object fed back by the current object storage gateway server as a search benchmark.

[0103] The update sub-unit is used to update the quota information to be corrected using the actual storage information of the target object.

[0104] In some embodiments, the quota correction system further includes:

[0105] The first judgment unit is used to determine whether the target bucket is undergoing its first correction based on the quota correction identifier corresponding to the target bucket after receiving the correction baseline information fed back by the object storage gateway server; if so, it triggers the clearing unit.

[0106] The clear unit is used to clear the historical storage information corresponding to each object in the target bucket.

[0107] In some embodiments, the quota correction system further includes:

[0108] The second judgment unit is used to determine whether all objects in the bucket index shard corresponding to the target object have completed quota information correction; if all have completed quota information correction, the status change unit is triggered.

[0109] The status change unit is used to change the corrected status of the bucket index shard to the corrected status.

[0110] The third judgment unit is used to determine whether the index shards of each bucket corresponding to the target bucket are all in the corrected state; if they are all in the corrected state, the second determination unit is triggered.

[0111] The second determining unit is used to determine that the quota correction of the target bucket is complete.

[0112] In some embodiments, the information dissemination unit 21 specifically includes:

[0113] The information distribution subunit is used to distribute corresponding correction identification information to each object storage gateway server when the quota correction conditions are met. The quota correction conditions are reaching a preset correction period and / or receiving a trigger signal indicating that quota correction is to be performed.

[0114] For a description of the quota correction system provided in this invention, please refer to the embodiments of the quota correction method described above; it will not be repeated here. It is understood that this quota correction system is the central control server described in the above embodiments.

[0115] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of a server provided by the present invention.

[0116] The server includes:

[0117] Memory 31 is used to store computer programs;

[0118] The processor 32 is configured to implement the steps of the quota correction method as described above when executing the computer program.

[0119] For a description of the server provided in this invention, please refer to the embodiments of the quota correction method described above; it will not be repeated here. It is understood that this server is the central control server described in the above embodiments.

[0120] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention.

[0121] The computer-readable storage medium 4 includes:

[0122] The computer-readable storage medium 4 stores a computer program 41, which, when executed by a processor, implements the steps of the quota correction method as described above.

[0123] For a description of the computer-readable storage medium 4 provided in this invention, please refer to the embodiments of the quota correction method described above, which will not be repeated here.

[0124] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. Relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0125] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementation should not be considered beyond the scope of the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A quota adjustment method, characterized in that, A central control server is applied to the storage process of a distributed storage cluster. The central control server is connected to multiple object storage gateway servers in the distributed storage cluster. The quota adjustment method includes: The corresponding correction identification information is sent to each of the object storage gateway servers. The correction identification information includes the first identifier of the target bucket and the shard information of the bucket index shard corresponding to the target bucket. The sum of the number of bucket index shards corresponding to each of the shard information is equal to the total number of bucket index shards corresponding to the target bucket, so that the pressure on each object storage gateway server is more even during quota correction. Based on the calibration baseline information received from the object storage gateway server, the historical quota information corresponding to the target bucket is updated. The calibration baseline information represents the current actual storage status of each object in the corresponding bucket index shard. After receiving the calibration identifier information, the second identifier of multiple objects is recorded based on the shard information corresponding to the bucket index shard, so that the object storage network management server can determine the calibration baseline information representing the current actual storage status of each object from the storage pool based on the second identifier. Correspondingly, the correction reference information includes the second identifier and actual storage information of each object in the corresponding bucket index shard; The step of updating the historical quota information corresponding to the target bucket based on the calibration baseline information received from the object storage gateway server includes: Based on the second identifier of the target object fed back by the current object storage gateway server as the search benchmark, determine the quota information to be corrected corresponding to the target object in the historical quota information of the target bucket; The quota information to be corrected is updated using the actual storage information of the target object; Correspondingly, the step of determining the correction identifier information includes: Group all bucket index shards corresponding to the target bucket to obtain X basic correction groups, where X is the first total number of the object storage gateway servers, X-1 basic correction groups include S bucket index shards and the remaining basic correction groups include A bucket index shards, and satisfy (X-1)*S+A=M, where M is the total number of bucket index shards, S is obtained by dividing M by X and rounding down, where X is an integer greater than 1 and S is an integer greater than 1; The segmentation information corresponding to the basic correction group and the first identifier of the target bucket are determined as a set of correction identifier information to obtain multiple sets of correction identifier information; Correspondingly, based on the business load of each object storage gateway server, a portion of the object storage gateway servers are selected as target object storage gateway servers to perform quota correction tasks.

2. The quota correction method as described in claim 1, characterized in that, After receiving the calibration baseline information from the object storage gateway server, the process also includes: Based on the quota correction identifier corresponding to the target bucket, determine whether the target bucket is undergoing its first correction; If so, clear the historical storage information corresponding to each object in the target bucket.

3. The quota correction method as described in claim 1, characterized in that, Also includes: Determine whether all objects in the bucket index shard corresponding to the target object have completed quota information correction; If quota information correction has been completed, change the correction status of the bucket index shard to the corrected status. Determine whether each bucket index fragment corresponding to the target bucket is in a corrected state; If all are in a calibrated state, the quota calibration of the target bucket is considered complete.

4. The quota correction method as described in claim 1, characterized in that, The step of sending corresponding correction identification information to each of the object storage gateway servers includes: When the quota correction conditions are met, corresponding correction identification information is sent to each of the object storage gateway servers, wherein the quota correction conditions are reaching a preset correction period and / or receiving a trigger signal indicating that quota correction is to be performed.

5. The quota adjustment method according to any one of claims 1 to 4, characterized in that, The step of the object storage gateway server determining the calibration benchmark information includes: The object storage gateway server determines whether the object still exists in the target bucket based on the first identifier of the target bucket and the second identifier of each object in the sharding information of the bucket index shard. If not, determine that the current storage state of the object is cleared and the corresponding current storage capacity is 0; If so, determine that the current storage state of the object is normal storage; Determine the current storage capacity of the object; The second identifier of the object, the current storage state of the object, and the current storage capacity are used as the correction reference information corresponding to the object.

6. A quota adjustment system, characterized in that, A central control server applied in the storage process of a distributed storage cluster, the central control server being connected to multiple object storage gateway servers in the distributed storage cluster, the quota correction system comprising: The information distribution unit is used to distribute corresponding correction identification information to each of the object storage gateway servers. The correction identification information includes the first identifier of the target bucket and the shard information of the bucket index shard corresponding to the target bucket. The sum of the number of bucket index shards corresponding to each of the shard information is equal to the total number of bucket index shards corresponding to the target bucket, so that the pressure on each object storage gateway server is more even during quota correction. The update unit is used to update the historical quota information corresponding to the target bucket according to the correction benchmark information fed back by the object storage gateway server. The correction benchmark information represents the current actual storage status of each object in the corresponding bucket index shard. After receiving the correction identifier information, the second identifier of multiple objects is recorded based on the shard information corresponding to the bucket index shard, so that the object storage network management server can determine the correction benchmark information representing the current actual storage status of each object from the storage pool according to the second identifier. Correspondingly, the correction reference information includes the second identifier and actual storage information of each object in the corresponding bucket index shard; The step of updating the historical quota information corresponding to the target bucket based on the calibration baseline information received from the object storage gateway server includes: Based on the second identifier of the target object fed back by the current object storage gateway server as the search benchmark, determine the quota information to be corrected corresponding to the target object in the historical quota information of the target bucket; The quota information to be corrected is updated using the actual storage information of the target object; Correspondingly, the step of determining the correction identifier information includes: Group all bucket index shards corresponding to the target bucket to obtain X basic correction groups, where X is the first total number of the object storage gateway servers, X-1 basic correction groups include S bucket index shards and the remaining basic correction groups include A bucket index shards, and satisfy (X-1)*S+A=M, where M is the total number of bucket index shards, S is obtained by dividing M by X and rounding down, where X is an integer greater than 1 and S is an integer greater than 1; The segmentation information corresponding to the basic correction group and the first identifier of the target bucket are determined as a set of correction identifier information to obtain multiple sets of correction identifier information; Correspondingly, based on the business load of each object storage gateway server, a portion of the object storage gateway servers are selected as target object storage gateway servers to perform quota correction tasks.

7. A server, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the quota correction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, include: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the quota correction method as described in any one of claims 1 to 5.

Images