A method and apparatus for fragmentation

Abstract

The disclosure provides a sharding method and device, relates to the technical field of computers, and particularly relates to the technical field of cloud storage. A specific implementation scheme is as follows: in response to a sharding instruction for a target bucket, a target shard corresponding to the target bucket is created in an index pool; object information of objects in the target bucket stored in a source shard corresponding to the target bucket in the index pool is synchronized to the target shard, and in a case where a first object is written to the target bucket in the synchronization process, first object information of the first object is acquired, and the first object information is stored in the source shard and the target shard; in a case where object information synchronization is completed, the source shard is deleted; and in a case where object information synchronization fails, the target shard is deleted. The sharding scheme provided in the embodiment of the disclosure can allow objects to be written to a bucket in a sharding process, thereby improving the use experience of a user.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of cloud storage technology, and more particularly to a sharding method and device. Background Art

[0002] In the object storage service, a bucket is a container for storing objects. The index pool stores object information for the objects stored in the bucket and is used to traverse the objects in the bucket. In addition, the object information of different objects in the same bucket can be stored in different shards contained in the index pool. As the number of objects stored in the bucket increases, the object information stored in each shard corresponding to the bucket also increases. When the object storage service is started, the object information stored in the shard needs to be loaded or verified. In this case, if the shard stores too much object information, it will take a long time to load or verify the object information in the shard, which may cause the service to time out.

[0003] To address this situation, the object storage service can increase the number of shards corresponding to the bucket through sharding operations to reduce the amount of object information stored in each shard.

[0004] In addition, when there are fewer objects stored in a bucket and the number of shards corresponding to the bucket in the index pool is large, the object storage service can also reduce the number of shards corresponding to the bucket through sharding operations. Summary of the Invention

[0005] The present disclosure provides a sharding method and device.

[0006] In a first aspect, an embodiment of the present disclosure provides a sharding method, including:

[0007] In response to a sharding instruction for a target bucket, create a target shard corresponding to the target bucket in the index pool;

[0008] Synchronize object information of an object in the target bucket stored in a source shard corresponding to the target bucket in the index pool to the target shard, and when a first object is written to the target bucket during the synchronization process, obtain first object information of the first object, and store the first object information in the source shard and the target shard;

[0009] When the object information synchronization is completed, the source shard is deleted;

[0010] In the event that object information synchronization fails, the target shard is deleted.

[0011] In one embodiment of the present disclosure, synchronizing object information of objects in the target bucket stored in a source shard corresponding to the target bucket in the index pool to the target shard includes:

[0012] Traversing the object information of the object in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written;

[0013] In the process of synchronizing each object information to be written to the target shard, it is detected whether the object information of the object to which the object information to be written belongs already exists in the target shard. If so, the object information to be written is deleted; if not, the object information to be written is written to the target shard.

[0014] In one embodiment of the present disclosure, when the object information includes an object name, detecting whether the object information of the object to which the object information to be written belongs already exists in the target shard includes:

[0015] It is detected whether the object name included in the object information to be written exists in the object name included in the object information stored in the target slice.

[0016] In one embodiment of the present disclosure, the method further includes:

[0017] In a case where the second object in the target bucket is deleted during the synchronization process, deleting the second object information of the second object stored in the source shard;

[0018] Detecting whether the second object information exists in the target slice;

[0019] If it exists, delete the second object information stored in the target slice.

[0020] In one embodiment of the present disclosure, synchronizing object information of objects in the target bucket stored in a source shard corresponding to the target bucket to the target shard includes:

[0021] Traversing the object information of the object in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written;

[0022] Synchronize the object information to be written to the target shard;

[0023] During the synchronization process, if it is detected that the second object information is deleted in the source shard, detecting whether the second object information exists in the object information to be written that has not been synchronized to the target shard;

[0024] If so, the second object information is deleted from the object information to be written that has not been synchronized to the target shard.

[0025] In one embodiment of the present disclosure, the method further includes:

[0026] In response to a sharding instruction for a target bucket, setting the sharding state of the target bucket to a normal state and starting a timer;

[0027] When the timer reaches the first timer, the object information synchronization process is carried out normally, and the timer is restarted;

[0028] When the timing duration reaches a second duration, adjusting the shard status of the target bucket to an abnormal state, wherein the first duration is less than the second duration;

[0029] In the case where the first object is written to the target bucket during the synchronization process, obtaining first object information of the first object and storing the first object information in the source shard and the target shard includes:

[0030] When writing a first object to the target bucket during synchronization, if the shard status of the target bucket is normal, first object information of the first object is obtained and stored in the source shard and the target shard.

[0031] In one embodiment of the present disclosure, the method further includes:

[0032] When writing the first object to the target bucket during the synchronization process, if the shard status of the target bucket is abnormal, it is determined that the synchronization has failed, and the first object information is stored in the source shard.

[0033] In one embodiment of the present disclosure, the first duration is half of the second duration.

[0034] In a second aspect, an embodiment of the present disclosure provides a sharding device, including:

[0035] A shard creation module, configured to create a target shard corresponding to the target bucket in the index pool in response to a shard instruction for the target bucket;

[0036] an information synchronization module, configured to synchronize object information of an object in the target bucket stored in a source shard corresponding to the target bucket in the index pool to the target shard, and, when writing a first object to the target bucket during the synchronization process, obtain first object information of the first object, and store the first object information in the source shard and the target shard;

[0037] A first deletion module, configured to delete the source shard when object information synchronization is completed;

[0038] The second deletion module is configured to delete the target shard if object information synchronization fails.

[0039] In one embodiment of the present disclosure, the information synchronization module includes:

[0040] An information traversal submodule, configured to traverse object information of objects in the target bucket stored in the source shard corresponding to the target bucket, and obtain object information to be written;

[0041] an information writing submodule, configured to detect whether the object information of the object to be written already exists in the target shard during the process of synchronizing each object information to be written into the target shard, and if so, delete the object information to be written; if not, write the object information to be written into the target shard;

[0042] The information storage submodule is used to obtain first object information of the first object when writing the first object to the target bucket during the synchronization process, and store the first object information in the source shard and the target shard.

[0043] In one embodiment of the present disclosure, when the object information includes an object name, the information is written into a submodule, specifically for:

[0044] It is detected whether the object name included in the object information to be written exists in the object name included in the object information stored in the target slice.

[0045] In one embodiment of the present disclosure, the apparatus further comprises:

[0046] an information deletion module, configured to delete the second object information of the second object stored in the source shard when the second object in the target bucket is deleted during the synchronization process;

[0047] an information detection module, configured to detect whether the second object information exists in the target shard, and if so, trigger a third deletion module;

[0048] The third deletion module is used to delete the second object information stored in the target slice.

[0049] In one embodiment of the present disclosure, the information synchronization module is specifically configured to:

[0050] Traversing the object information of the object in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written;

[0051] Synchronize the object information to be written to the target shard;

[0052] During the synchronization process, if it is detected that the second object information is deleted in the source shard, detecting whether the second object information exists in the object information to be written that has not been synchronized to the target shard;

[0053] If so, the second object information is deleted from the object information to be written that has not been synchronized to the target shard.

[0054] In one embodiment of the present disclosure, the apparatus further comprises:

[0055] A state setting module, configured to, in response to a sharding instruction for a target bucket, set the sharding state of the target bucket to a normal state and start timing;

[0056] A first timing module, configured to restart the timing when the object information synchronization process is normally performed when the timing duration reaches a first duration;

[0057] A second timing module is configured to adjust the shard status of the target bucket to an abnormal state when the timing duration reaches a second duration, wherein the first duration is less than the second duration;

[0058] The information synchronization module is specifically used to:

[0059] When writing a first object to the target bucket during synchronization, if the shard status of the target bucket is normal, first object information of the first object is obtained and stored in the source shard and the target shard.

[0060] In one embodiment of the present disclosure, the apparatus further comprises:

[0061] The information storage module is used to determine that the synchronization fails when writing the first object to the target bucket during the synchronization process, if the shard status of the target bucket is abnormal, and store the first object information in the source shard.

[0062] In one embodiment of the present disclosure, the first duration is half of the second duration.

[0063] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0064] at least one processor; and

[0065] a memory communicatively connected to the at least one processor; wherein,

[0066] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method described in the first aspect.

[0067] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any of the methods described in the first aspect.

[0068] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which implements any method described in the first aspect when executed by a processor.

[0069] From the above, it can be seen that when the solution provided by the embodiment of the present disclosure is applied to perform sharding operations, if an object is written to the target bucket during the object information synchronization process, the target object information of the written object can be stored in the source shard and the target shard, thereby allowing the write operation of the target bucket during the sharding operation on the target bucket without blocking the write operation of the target bucket during the sharding process. It can be seen that the sharding solution provided by the embodiment of the present disclosure can optimize the sharding operation and improve the user experience.

[0070] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0072] Figure 1 A schematic diagram of a first sharding method according to an embodiment of the present disclosure;

[0073] Figure 2 A flowchart of an information synchronization process provided by an embodiment of the present disclosure;

[0074] Figure 3 A schematic diagram of a second sharding method according to an embodiment of the present disclosure;

[0075] Figure 4 A schematic structural diagram of a first sharding device provided in an embodiment of the present disclosure;

[0076] Figure 5 It is a block diagram of an electronic device used to implement the slicing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0077] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0078] First, some concepts mentioned in the embodiments of the present disclosure are explained.

[0079] 1. Bucket

[0080] In the object storage service, a bucket is a container for storing objects.

[0081] 2. Index pool

[0082] The index pool can store object information of objects stored in the bucket. When the object storage service needs to traverse which objects are stored in the bucket, it can traverse the object information stored in the index pool and determine the objects stored in the bucket based on the traversed object information.

[0083] 3. Sharding and Sharding Operations (Shard)

[0084] The index pool can be built based on the RocksDB storage engine. RocksDB can contain multiple object mapping (omap) spaces. When the index pool stores object information, the object information can be written to the omap space. An omap space is called a shard.

[0085] After the object storage service creates a bucket, it can divide the index pool into multiple shards, that is, divide it into multiple omap spaces. The operation of dividing multiple shards is called a sharding operation. The shards or omap spaces obtained by the division are the shards corresponding to the created bucket and are used to subsequently store object information of objects stored in the bucket.

[0086] When the object storage service needs to traverse the objects stored in a bucket, it can traverse the object information stored in each shard corresponding to the bucket.

[0087] 4. Resharding

[0088] After the Object Storage service creates a bucket, it can create multiple shards within the index pool. As the number of objects stored in the bucket increases, the amount of object information stored in each shard also increases. If each shard stores too much object information, the Object Storage service can increase the number of shards, at which point you can perform another sharding operation. Alternatively, if each shard stores too little object information, the Object Storage service can reduce the number of shards, at which point you can perform another sharding operation.

[0089] The above-mentioned re-sharding operation can be called a re-sharding operation.

[0090] During a resharding operation, the Object Storage service re-divides the bucket's index pool into a shard. This shard is called the target shard for the bucket, and the shard that originally stored the object information for the bucket is called the source shard. After the target shard is generated, the object information stored in the source shard is synchronized to the target shard. After the synchronization is complete, the source shard is deleted, and the resharding operation is completed.

[0091] For example, when creating a bucket, the object storage service can divide the index pool into 64 shards as the shards corresponding to the bucket. As the number of objects stored in the bucket increases, the object information stored in each of the 64 shards also increases. When the object information stored in each shard reaches a certain threshold, the object storage service can perform a reshard operation to re-divide 1024 shards as the target shards corresponding to the bucket, and then synchronize the object information stored in the original 64 shards to these 1024 shards. After the synchronization is complete, the original 64 shards are deleted, and the sharding operation is completed.

[0092] As can be seen from the above example, the Object Storage service can increase the number of shards associated with a bucket by performing a resharding operation. In addition, resharding can also reduce the number of shards associated with a bucket.

[0093] For example, when there are fewer objects stored in a bucket, the object information stored in the source shard is also less. However, each time the object storage service traverses the objects stored in the bucket, it needs to traverse the object information stored in all shards corresponding to the bucket. This may result in a larger workload for each traversal. In view of this, the object storage service can divide the bucket into fewer target shards, thereby synchronizing the object information of the objects stored in the bucket to a smaller number of shards.

[0094] The following describes the problems existing in the existing re-sharding operation.

[0095] During the existing resharding operation, the object storage service blocks users from writing objects to the bucket. Only after the resharding operation is completed can users write objects to the bucket again. However, when the source shards corresponding to the bucket store a large number of objects, the resharding operation takes a long time. For example, if the source shards corresponding to the bucket store 100 million objects, the resharding operation will take more than an hour, during which time users cannot write objects to the bucket, which seriously affects the user experience.

[0096] To solve the above technical problems, the embodiments of the present disclosure provide a sharding method and apparatus, which are described in detail below through specific embodiments.

[0097] See also Figure 1, a flow chart of a sharding method is provided. In this embodiment, the method includes the following steps S101-S104.

[0098] Step S101: In response to a sharding instruction for a target bucket, a target shard corresponding to the target bucket is created in an index pool.

[0099] The sharding instruction is used to instruct the object storage service to perform a sharding operation. The sharding instruction can be manually issued by an administrator of the object storage service, or triggered by the object storage service when it detects that a preset trigger condition for performing a sharding operation is met.

[0100] For example, the above condition may be that the quantity of object information stored in each shard corresponding to the bucket exceeds a first quantity threshold, or the quantity is lower than a second quantity threshold.

[0101] When creating target shards, the object storage service can determine the number of target shards and create the determined number of shards in the index pool as the target shards corresponding to the target bucket.

[0102] The number of target shards can be determined by either of the following two implementation methods.

[0103] In a first implementation manner, the number of shards of the target shard may be calculated according to the total number of object information stored in the source shards and the expected number of object information stored in each preset target shard.

[0104] In the second implementation method, the number of shards newly configured for the target bucket can be obtained based on manual experience and the storage status of the object information of the source shards.

[0105] Step S102: Synchronize the object information of the object in the target bucket stored in the source shard corresponding to the target bucket in the index pool to the target shard, and when writing the first object to the target bucket during the synchronization process, obtain the first object information of the first object, and store the first object information in the source shard and the target shard.

[0106] The object information may include object meta-information or attribute information, such as object name, object type, object identifier, etc.

[0107] The object storage service can run multiple threads in parallel, each performing a different task. For example, thread a manages objects in a bucket, including writing objects to and deleting objects from the bucket. Thread b manages object information stored in the bucket's corresponding shard, including storing object information in the bucket's corresponding shard and deleting object information about deleted objects from the bucket's corresponding shard. Thread c synchronizes object information stored in the bucket's corresponding source shard to the target shard.

[0108] After the Object Storage service creates the target shard corresponding to the target bucket, thread c can start synchronizing the object information stored in the source shard corresponding to the target bucket to the target shard. The specific synchronization process is as follows:

[0109] See also Figure 2 , shows the synchronization process of object information, from Figure 2 As can be seen from the figure, thread c first traverses the object information stored in the source shard corresponding to the target bucket, obtains a traversal list, which contains the traversed object information, and then thread c writes the object information in the traversal list into the target shard.

[0110] During the synchronization process, thread c can synchronize all object information stored in all source shards at one time, that is, thread c traverses all object information stored in all source shards at one time, and after the traversal is completed, writes the traversed object information to the target shard; or thread c can synchronize in batches, synchronizing only part of the object information each time.

[0111] For example, if the source shard stores a total of 10 million object information, thread c can synchronize 1 million object information at a time, that is, traverse 1 million object information each time and write the traversed object information into the target shard. After writing is completed, traverse the 1 million object information in the source shard again and write it into the target shard. This cycle is repeated ten times, thereby synchronizing 10 million object information to the target shard.

[0112] During thread c's object information synchronization, if the object storage service obtains an object to be stored in the target bucket, thread a can write the object to the target bucket. This object is referred to as the first object. Furthermore, thread b can store the object information of the first object (i.e., the first object information) in the shard corresponding to the target bucket. Since the target bucket corresponds to a source shard and a target shard during the sharding process, thread b can write the first object information of the first object to both the source shard and the target shard.

[0113] Step S103: When the object information synchronization is completed, the source fragment is deleted.

[0114] Specifically, after all object information stored in the source shard is synchronized to the target shard, it can be determined that the object information synchronization is completed. In this case, both the source shard and the target shard store a complete copy of the object information stored in the target bucket. At this time, the source shard is deleted and only the target shard is retained. If thread a subsequently writes an object to the target bucket again, thread b can store the object information of the newly written object in the target shard.

[0115] Step S104: If the object information synchronization fails, the target shard is deleted.

[0116] Since the source shard can store the first object information of the first object newly written to the target bucket during the object information synchronization process, the source shard always stores the object information of all objects in the target bucket. When the object information synchronization fails, some object information in the source shard may not be synchronized to the target shard. Therefore, the object information stored in the target shard is incomplete. In this case, the target shard can be deleted and only the source shard is retained. Subsequently, if thread a writes an object to the target bucket again, thread b can store the object information of the newly written object in the source shard.

[0117] From the above, it can be seen that when the solution provided by the embodiment of the present disclosure is applied to perform sharding operations, if an object is written to the target bucket during the object information synchronization process, the target object information of the written object can be stored in the source shard and the target shard, thereby allowing the write operation of the target bucket during the sharding operation on the target bucket without blocking the write operation of the target bucket during the sharding process, and thus will not affect the user's business. It can be seen that the sharding solution provided by the embodiment of the present disclosure can optimize the sharding operation and improve the user experience.

[0118] Before writing the first object to the target bucket, an older version of the first object may be stored in the target bucket. Therefore, writing the first object to the target bucket can be considered an update of the first object already stored in the target bucket. Furthermore, the source shard already stores the object information for the older version of the first object. When thread a writes the first object to the target bucket, thread b updates the object information for the older version of the first object stored in the source shard with the first object information and writes the first object information to the target shard.

[0119] However, since the event of writing the first object to the target bucket can occur at any time during the object information synchronization process, if thread c traverses and obtains the traversal list but does not write the object information in the traversal list into the target shard, thread a writes the first object to the target bucket, and thread b stores the first object information in the source shard and the target shard, then both the source shard and the target shard store the first object information, and the traversal list contains the object information of the old version of the first object. In this case, when thread c writes the object information in the traversal list into the target shard, the first object information already stored in the target shard will be overwritten by the object information of the old version of the first object contained in the traversal list, so that the object information of the old version of the first object is still stored in the target shard, resulting in an object information synchronization error.

[0120] The following takes object k as an example to explain the steps in the synchronization process when the above phenomenon occurs. In this synchronization process, the target bucket originally stores the old version of object k, which is called k.0 , the source shard originally stored object k 0 object information.

[0121] Step 1: Thread c traverses the object information stored in the source shard and obtains the traversal list. At this time, the traversal list contains object k 0 The source shard stores object k 0 object information;

[0122] Step 2: Thread a writes object k to the target bucket, and thread b stores the object information of object k in the source shard and the target shard. At this point, the object information of object k is stored in both the source shard and the target shard.

[0123] Step 3: Thread c writes the object information contained in the traversal list into the target shard. At this point, the object information of object k is still stored in the source shard, but the object information of object k stored in the target shard is overwritten with object k. 0 object information.

[0124] To solve this problem, in one embodiment of the present disclosure, during the synchronization process, the object storage service (or thread c) can traverse the object information of the objects in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written; and in the process of synchronizing each object information to be written to the target shard, it is detected whether the object information of the object to which the object information to be written belongs already exists in the target shard. If so, the object information to be written is deleted; if not, the object information to be written is written to the target shard.

[0125] Specifically, the above-mentioned object information to be written is the object information contained in the above-mentioned traversal list. After thread c traverses the traversal list, the object information to be written in the list can be written into the target shard one by one, or multiple sub-threads can be run, and each sub-thread is used to write a single object information to be written into the target shard.

[0126] During the writing process of each object information to be written, thread c or child thread can first detect whether the object information of the object to be written already exists in the target shard. If so, it means that the object to which the object information to be written belongs is written to the target bucket by thread a after thread c traversed the traversal list. At this time, if the object information to be written is written into the target shard, the above-mentioned object information overwriting problem will occur. Therefore, the object information to be written can be deleted; if it does not exist, it means that the object to which the object information to be written belongs is an object that has been originally stored in the target bucket. At this time, the object to be written needs to be written into the target shard.

[0127] As can be seen from the above, when applying the solution provided by the embodiment of the present disclosure to perform sharding operations, before writing the object information to be written into the target shard, it is first detected whether the object information of the object to which the object information to be written belongs exists in the target shard. If so, the object information to be written is deleted. This can avoid the problem of the above-mentioned object information being overwritten, thereby improving the reliability and accuracy of the sharding operation.

[0128] The following describes an implementation method for detecting whether the object information of the object to which the object information to be written already exists in the target shard.

[0129] In one embodiment of the present disclosure, the object information stored in the shards of the index pool may include object names. In this case, it may be detected whether the object name included in the object information to be written is included in the object name already stored in the target shard.

[0130] Specifically, during the writing process of each object information to be written, the object name contained in the object information to be written can be compared one by one with the object name contained in the object information already stored in the target shard. If the object name contained in the object information already stored in the target shard is the same as the object name contained in the object information to be written, it means that the object name contained in the object information already stored in the target shard contains the object name contained in the object information to be written, and further means that the object information of the object to which the object information to be written belongs already exists in the target shard.

[0131] If the object names contained in all object information stored in the target shard are different from the object names contained in the object information to be written, it means that the object names contained in the object information stored in the target shard do not contain the object names contained in the object information to be written, which further means that the object information of the object to which the object information to be written belongs does not exist in the target shard.

[0132] In addition, if the object information in the shard is stored in the form of a key-value pair, the key in the key-value pair can be the object name, and the value can be other metadata of the object. In this case, before checking whether the object name contained in the object information stored in the target shard contains the object name contained in the object information to be written, the key in all key-value pairs stored in the target shard can be checked to see if it is the same as the key in the key-value pair of the object information to be written.

[0133] From the above, it can be seen that when applying the solution provided by the embodiment of the present disclosure to perform sharding operations, it is possible to accurately detect whether the object information of the object to be written already exists in the target shard by detecting whether the object name contained in the object information stored in the target shard contains the object name contained in the object information to be written, thereby performing subsequent corresponding operations according to the more accurate detection results to complete the sharding operation. Therefore, applying the sharding solution provided by the embodiment of the present disclosure can improve the accuracy of the sharding operation.

[0134] In another embodiment of the present disclosure, the object information stored in the shards of the index pool may include a unique identifier of the object. In this way, when detecting whether the object information of the object to which the object information to be written belongs already exists in the target shard, it is possible to detect whether the unique identifier contained in the object information already stored in the target shard contains the unique identifier contained in the object information to be written.

[0135] During object information synchronization, users can not only write objects to the target bucket, but also delete objects in the target bucket.

[0136] In view of this situation, in one embodiment of the present disclosure, see Figure 3 , provides a flow chart of a second sharding method. In this embodiment, the above method includes the following steps S301-S307.

[0137] Step S301: In response to a sharding instruction for a target bucket, a target shard corresponding to the target bucket is created in an index pool.

[0138] Step S302: Synchronize the object information of the object in the target bucket stored in the source shard corresponding to the target bucket in the index pool to the target shard, and when writing the first object to the target bucket during the synchronization process, obtain the first object information of the first object, and store the first object information in the source shard and the target shard.

[0139] Step S303: when the second object in the target bucket is deleted during the synchronization process, the second object information of the second object stored in the source shard is deleted;

[0140] Step S304: Detect whether the second object information exists in the target slice. If so, execute step S305.

[0141] Step S305: deleting the second object information stored in the target slice;

[0142] Step S306: When the object information synchronization is completed, the source shard is deleted;

[0143] Step S307: If the object information synchronization fails, the target shard is deleted.

[0144] The above steps S301-S302 are respectively the same as the above steps S101-S102, and the above steps S306-S307 are respectively the same as the above steps S103-S104, and are not described in detail here.

[0145] Specifically, during the synchronization process, the user or other threads of the object storage service may issue an object deletion instruction to delete the second object stored in the target bucket. Thread a may delete the second object in the target bucket in response to the object deletion instruction, and thread b may also delete the second object information of the second object stored in the source shard in response to the object deletion instruction, or thread a may notify thread b that the second object has been deleted, so that thread b synchronously deletes the second object information of the second object.

[0146] Since the event of deleting the second object may occur at any time during the synchronization process, when thread b deletes the second object information of the second object, the second object information may have been synchronized to the target shard or may not have been synchronized to the target shard. Therefore, in addition to deleting the second object information stored in the source shard, thread b also needs to detect whether the second object information exists in the target shard. If so, delete the second object information stored in the target shard.

[0147] From the above, it can be seen that when the solution provided by the embodiment of the present disclosure is applied to perform sharding operations, during the object information synchronization process, not only can objects be written to the target bucket, but objects in the target bucket can also be deleted, so that users can normally perform write and delete object operations during the sharding operation. It can be seen that the sharding solution provided by the embodiment of the present disclosure further optimizes the sharding operation, thereby further improving the user experience.

[0148] Since the event of deleting the second object in the target bucket can occur at any time during the synchronization process, if the event of deleting the second object in the target bucket occurs when thread c traverses the traversal list but has not yet written the object information in the traversal list into the target shard, then after deleting the second object information in the source shard and the target shard, the second object information may still exist in the traversal list, so when thread c writes the object information in the traversal list into the target shard, the second object information will be written into the target shard again.

[0149] In response to the above situation, in one embodiment of the present disclosure, during the synchronization process, the object information of the objects in the target bucket stored in the source shard corresponding to the target bucket can be traversed to obtain the object information to be written; the object information to be written is synchronized to the target shard; during the synchronization process, if it is perceived that the second object information has been deleted in the source shard, it is detected whether the second object information exists in the object information to be written that has not been synchronized to the target shard; if so, the second object information is deleted from the object information to be written that has not been synchronized to the target shard.

[0150] Specifically, during the synchronization process when no object deletion event occurs, thread c can traverse the object information of the object in the target bucket stored in the source shard corresponding to the target bucket, obtain the object information to be written, and synchronize the object information to be written to the target shard.

[0151] During the synchronization process, when thread a deletes the second object in the target bucket and thread b deletes the second object information in the source shard, thread b can notify thread c that it has deleted the second object in the source shard, so that thread c can perceive that the source shard has deleted the second object information during the synchronization process.

[0152] When thread c perceives that the source shard has deleted the second object, since the object information to be written that has not been synchronized to the target shard may contain the second object information, if the object information to be written is directly written to the target shard, the second object information may be re-stored in the target shard. Therefore, when thread c perceives that the source shard has deleted the second object information, it is necessary to detect whether the second object information exists in the object information to be written that has not been synchronized to the target shard.

[0153] If thread c detects that there is second object information in the object information to be written that has not been synchronized to the target shard, the second object information is deleted from the object information to be written to ensure that in subsequent steps, the second object information can be avoided from being stored in the target shard again, thereby improving the reliability of the sharding operation.

[0154] Thread b can be used to write the object information of the object in the bucket to the shard corresponding to the bucket. The shard corresponding to the bucket will have different situations at different times. For example, before the sharding operation, the shard corresponding to the bucket is the source shard; during the sharding process, the shard corresponding to the bucket includes the source shard and the target shard; after the sharding operation is successfully executed, the shard corresponding to the bucket is the target shard; after the sharding operation fails, the shard corresponding to the bucket is the source shard.

[0155] In view of this, the slice into which thread b needs to write object information can be determined by the method mentioned in the following embodiment, and then when thread b needs to write object information, the object information is written to the determined slice.

[0156] In one embodiment of the present disclosure, thread d in the object storage service can respond to a sharding instruction for a target bucket, set the sharding status of the target bucket to a normal state, and start timing. When the timing duration reaches a first duration, the object information synchronization process proceeds normally, and the timing is restarted. When the timing duration reaches a second duration, the sharding status of the target bucket is adjusted to an abnormal state.

[0157] Among them, the first duration is smaller than the second duration.

[0158] Specifically, after the object storage service receives the sharding instruction, thread d may respond to the sharding instruction, set the sharding state of the target bucket to normal, and start timing.

[0159] There are two timing modes: forward timing and countdown. The following describes the subsequent operations of thread d under these two timing modes.

[0160] 1. Positive timing mode

[0161] Thread d can set the target bucket's shard status to normal and start counting from zero. When the count reaches the first duration, it can check whether thread c is running normally. If it is, it means that the object information synchronization process is proceeding normally. At this time, thread d can restart the count from zero. As can be seen, if thread c is running normally, thread d will restart the count from zero every time it reaches the first duration. During this process, the target bucket's shard status remains normal until object information synchronization is completed, or thread c runs abnormally.

[0162] If thread c runs abnormally, it indicates that an abnormality has occurred in the object information synchronization process. At this time, when thread d's timing reaches the first duration, thread d detects that the object synchronization process is not proceeding normally and does not restart the timing. Instead, it continues counting. When the timing reaches the second duration, thread d adjusts the shard status of the target bucket to an abnormal state.

[0163] 2. Countdown method

[0164] Thread d can set the shard status of the target bucket to normal and start counting down from the second duration. When the countdown reaches the second duration minus the first duration, it can detect whether the object information synchronization process is proceeding normally. If so, thread d starts counting down from the second duration again.

[0165] If an exception is detected in the object information synchronization process, the countdown continues until the countdown reaches zero, and thread d adjusts the shard status of the target bucket to an abnormal state.

[0166] In one embodiment of the present disclosure, the first duration is half of the second duration.

[0167] For example, the second duration is 120 seconds and the first duration is 60 seconds.

[0168] By setting the first duration to be half of the second duration, thread d can be easily switched between countdown and countdown, thereby improving the convenience of the slicing operation.

[0169] As can be seen from the above, when the object information synchronization process is proceeding normally, the timing of thread d is always in a continuous cycle, and the sharding status of the target bucket is always normal. It can be seen that the sharding status of the target bucket corresponds to the actual running status of the object information synchronization process.

[0170] When an exception occurs in the object information synchronization process, process d will continue when the timing duration reaches the first duration until the timing duration reaches the second duration. The shard status of the target bucket is adjusted to an abnormal state. It can be seen that when the timing duration reaches the second duration, the shard status of the target bucket also corresponds to the actual running status of the object information synchronization process.

[0171] During the period from the moment the object information synchronization process experienced an exception to the moment the timing reached the second duration, the target bucket's sharding status did not correspond to the actual running state of the object information synchronization process. However, since the maximum duration of this period is only the second duration, i.e., the object information synchronization process experienced an exception just as thread d was retiming, the target bucket's sharding status can only be adjusted once the timing reaches the second duration. Therefore, if the second duration is short, the discrepancy between the target bucket's sharding status and the actual running state of the object information synchronization process can be ignored.

[0172] In summary, the sharding status of the target bucket can be considered to correspond to the running status of the actual object information synchronization process.

[0173] In view of this, thread b can determine the shards into which the object information needs to be written according to different shard states of the target bucket, and then write the object information into the determined shards when the object information needs to be written.

[0174] Specifically, when writing the first object to the target bucket during the synchronization process, if the shard status of the target bucket is normal, it means that the object information synchronization process is proceeding normally. At this time, thread b can obtain the first object information of the first object and store the first object information in the source shard and the target shard, thereby ensuring that the object information of the object in the target bucket will not be missed in the source shard and the target shard, thereby improving the reliability of the sharding operation.

[0175] If the shard status is abnormal, it means that an abnormality has occurred in the object information synchronization process. At this time, the target shard may not have synchronized all object information, and the target shard needs to be discarded. The source shard is still used as the shard corresponding to the target bucket. Therefore, when the shard status is abnormal, thread d can determine that the synchronization has failed and store the first object information in the source shard without storing the first object information in the target shard. This can reduce the workload of storing object information and save processing resources of the object storage service.

[0176] In one embodiment of the present disclosure, the sharding state of the target bucket may be set by setting a state lock.

[0177] When the status lock is in the locked state, it means that the shard status of the target bucket is normal. At this time, thread b writes object information to both the source shard and the target shard. When the status lock is in the unlocked state, it means that the shard status of the target bucket is abnormal. At this time, thread b only writes object information to the source shard.

[0178] Specifically, thread d can respond to the sharding instruction, set the status lock, set the status lock to a locked state, and start timing. When the timing time reaches the first time, if the object information synchronization process proceeds normally, the timing is restarted. When the timing time reaches the second time, the state of the status lock is adjusted to an unlocked state. In this way, when thread b writes the first object information to the shard corresponding to the target bucket, if it is detected that the status lock is in a locked state, the first object information is written to the source shard and target shard corresponding to the target bucket; if it is detected that the status lock is in an unlocked state, the first object information is only written to the source shard corresponding to the target bucket.

[0179] Corresponding to the aforementioned sharding method, the embodiment of the present disclosure also provides a sharding device.

[0180] In one embodiment of the present disclosure, see Figure 4 , provides a structural diagram of a sharding device. In this embodiment, the device includes:

[0181] A shard creation module 401 is configured to create a target shard corresponding to a target bucket in an index pool in response to a shard instruction for the target bucket;

[0182] An information synchronization module 402 is configured to synchronize object information of an object in the target bucket stored in a source shard corresponding to the target bucket in the index pool to the target shard, and when a first object is written to the target bucket during the synchronization process, obtain first object information of the first object, and store the first object information in the source shard and the target shard;

[0183] A first deletion module 403 is configured to delete the source shard when object information synchronization is completed;

[0184] The second deleting module 404 is configured to delete the target shard if object information synchronization fails.

[0185] From the above, it can be seen that when the solution provided by the embodiment of the present disclosure is applied to perform sharding operations, if an object is written to the target bucket during the object information synchronization process, the target object information of the written object can be stored in the source shard and the target shard, thereby allowing the write operation of the target bucket during the sharding operation on the target bucket without blocking the write operation of the target bucket during the sharding process. It can be seen that the sharding solution provided by the embodiment of the present disclosure can optimize the sharding operation and improve the user experience.

[0186] In one embodiment of the present disclosure, the information synchronization module 402 includes:

[0187] An information traversal submodule, configured to traverse object information of objects in the target bucket stored in the source shard corresponding to the target bucket, and obtain object information to be written;

[0188] an information writing submodule, configured to detect whether the object information of the object to be written already exists in the target shard during the process of synchronizing each object information to be written into the target shard, and if so, delete the object information to be written; if not, write the object information to be written into the target shard;

[0189] The information storage submodule is used to obtain first object information of the first object when writing the first object to the target bucket during the synchronization process, and store the first object information in the source shard and the target shard.

[0190] As can be seen from the above, when applying the solution provided by the embodiment of the present disclosure to perform sharding operations, before writing the object information to be written into the target shard, it is first detected whether the object information of the object to which the object information to be written belongs exists in the target shard. If so, the object information to be written is deleted. This can avoid the problem of the above-mentioned object information being overwritten, thereby improving the reliability and accuracy of the sharding operation.

[0191] In one embodiment of the present disclosure, when the object information includes an object name, the information is written into a submodule, specifically for:

[0192] It is detected whether the object name included in the object information to be written exists in the object name included in the object information stored in the target slice.

[0193] From the above, it can be seen that when applying the solution provided by the embodiment of the present disclosure to perform sharding operations, it is possible to accurately detect whether the object information of the object to be written already exists in the target shard by detecting whether the object name contained in the object information stored in the target shard contains the object name contained in the object information to be written, thereby performing subsequent corresponding operations according to the more accurate detection results to complete the sharding operation. Therefore, applying the sharding solution provided by the embodiment of the present disclosure can improve the accuracy of the sharding operation.

[0194] In one embodiment of the present disclosure, the apparatus further comprises:

[0195] an information deletion module, configured to delete the second object information of the second object stored in the source shard when the second object in the target bucket is deleted during the synchronization process;

[0196] an information detection module, configured to detect whether the second object information exists in the target shard, and if so, trigger a third deletion module;

[0197] The third deletion module is used to delete the second object information stored in the target slice.

[0198] From the above, it can be seen that when the solution provided by the embodiment of the present disclosure is applied to perform sharding operations, during the object information synchronization process, not only can objects be written to the target bucket, but objects in the target bucket can also be deleted, so that users can normally perform write and delete object operations during the sharding operation. It can be seen that the sharding solution provided by the embodiment of the present disclosure further optimizes the sharding operation, thereby further improving the user experience.

[0199] In one embodiment of the present disclosure, the information synchronization module 402 is specifically configured to:

[0200] Traversing the object information of the object in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written;

[0201] Synchronize the object information to be written to the target shard;

[0202] During the synchronization process, if it is detected that the second object information is deleted in the source shard, detecting whether the second object information exists in the object information to be written that has not been synchronized to the target shard;

[0203] If so, the second object information is deleted from the object information to be written that has not been synchronized to the target shard.

[0204] In this solution, if it is detected that there is second object information in the object information to be written that has not been synchronized to the target shard, the second object information is deleted from the object information to be written to ensure that in subsequent steps, the second object information can be avoided from being stored in the target shard again, thereby improving the reliability of the sharding operation.

[0205] In one embodiment of the present disclosure, the apparatus further comprises:

[0206] A state setting module, configured to, in response to a sharding instruction for a target bucket, set the sharding state of the target bucket to a normal state and start timing;

[0207] A first timing module, configured to restart the timing when the object information synchronization process is normally performed when the timing duration reaches a first duration;

[0208] A second timing module is configured to adjust the shard status of the target bucket to an abnormal state when the timing duration reaches a second duration, wherein the first duration is less than the second duration;

[0209] The information synchronization module 402 is specifically configured to:

[0210] When writing a first object to the target bucket during synchronization, if the shard status of the target bucket is normal, first object information of the first object is obtained and stored in the source shard and the target shard.

[0211] In this solution, when writing the first object to the target bucket during the synchronization process, if the shard status of the target bucket is normal, it means that the object information synchronization process is proceeding normally. At this time, thread b can obtain the first object information of the first object and store the first object information in the source shard and the target shard, thereby ensuring that the object information of the object in the target bucket will not be missed in the source shard and the target shard, thereby improving the reliability of the sharding operation.

[0212] In one embodiment of the present disclosure, the apparatus further comprises:

[0213] The information storage module is used to determine that the synchronization fails when writing the first object to the target bucket during the synchronization process, if the shard status of the target bucket is abnormal, and store the first object information in the source shard.

[0214] In this solution, if the shard status is abnormal, it means that an abnormality occurs in the object information synchronization process. At this time, the target shard may not have synchronized all object information, and the target shard needs to be discarded. The source shard is still used as the shard corresponding to the target bucket. Therefore, when the shard status is abnormal, thread d can determine that the synchronization has failed and store the first object information in the source shard without storing the first object information in the target shard. This can reduce the workload of storing object information and save processing resources of the object storage service.

[0215] In one embodiment of the present disclosure, the first duration is half of the second duration.

[0216] In this solution, by setting the first duration to half of the second duration, thread d can be easily switched between forward counting and countdown, thereby improving the convenience of slicing operations.

[0217] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0218] It should be noted that the head model in this embodiment is not a head model for a specific user and cannot reflect the personal information of a specific user.

[0219] It should be noted that the two-dimensional face images in this embodiment come from a public dataset.

[0220] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0221] In one embodiment of the present disclosure, an electronic device includes:

[0222] at least one processor; and

[0223] a memory communicatively connected to the at least one processor; wherein,

[0224] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the sharding method mentioned in any of the aforementioned method embodiments.

[0225] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0226] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0227] Various components in device 500 are connected to I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0228] The computing unit 501 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the sharding method. For example, in some embodiments, the sharding method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the sharding method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the sharding method by any other appropriate means (e.g., by means of firmware).

[0229] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0230] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0231] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0232] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0233] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0234] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0235] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0236] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A sharding method, wherein: The method comprises: In response to a sharding instruction for a target bucket, create a target shard corresponding to the target bucket in the index pool; Traversing the object information of the objects in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written; in the process of synchronizing each object information to be written to the target shard, detecting whether the object information of the object to which the object information to be written already exists in the target shard, if so, deleting the object information to be written; if not, writing the object information to be written to the target shard, and when writing a first object to the target bucket during the synchronization process, obtaining the first object information of the first object, and storing the first object information in the source shard and the target shard; When the object information synchronization is completed, the source shard is deleted; In the event that object information synchronization fails, the target shard is deleted.

2. The method according to claim 1, wherein: In a case where the object information includes an object name, detecting whether the object information of the object to which the object information to be written belongs already exists in the target slice includes: It is detected whether the object name included in the object information to be written exists in the object name included in the object information stored in the target slice.

3. The method according to claim 1, wherein: The method further comprises: In a case where the second object in the target bucket is deleted during the synchronization process, deleting the second object information of the second object stored in the source shard; Detecting whether the second object information exists in the target slice; If it exists, delete the second object information stored in the target slice.

4. The method according to claim 3, wherein: The step of synchronizing object information of objects in the target bucket stored in the source shard corresponding to the target bucket to the target shard includes: Traversing the object information of the object in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written; Synchronize the object information to be written to the target shard; During the synchronization process, if it is detected that the second object information is deleted in the source shard, detecting whether the second object information exists in the object information to be written that has not been synchronized to the target shard; If so, the second object information is deleted from the object information to be written that has not been synchronized to the target shard.

5. The method according to claim 1, wherein: The method further comprises: In response to a sharding instruction for a target bucket, setting the sharding state of the target bucket to a normal state and starting a timer; When the timer reaches the first timer, the object information synchronization process is carried out normally, and the timer is restarted; When the timing duration reaches a second duration, adjusting the shard status of the target bucket to an abnormal state, wherein the first duration is less than the second duration; In the case where the first object is written to the target bucket during the synchronization process, obtaining first object information of the first object and storing the first object information in the source shard and the target shard includes: When writing a first object to the target bucket during synchronization, if the shard status of the target bucket is normal, first object information of the first object is obtained and stored in the source shard and the target shard.

6. The method according to claim 5, wherein: The method further comprises: When writing the first object to the target bucket during the synchronization process, if the shard status of the target bucket is abnormal, it is determined that the synchronization has failed, and the first object information is stored in the source shard.

7. The method according to claim 5 or 6, wherein: The first duration is half of the second duration.

8. A slicing device, wherein: The device comprises: A shard creation module, configured to create a target shard corresponding to the target bucket in the index pool in response to a shard instruction for the target bucket; an information synchronization module, configured to synchronize object information of an object in the target bucket stored in a source shard corresponding to the target bucket in the index pool to the target shard, and, when writing a first object to the target bucket during the synchronization process, obtain first object information of the first object, and store the first object information in the source shard and the target shard; A first deletion module, configured to delete the source shard when object information synchronization is completed; A second deletion module is configured to delete the target shard if object information synchronization fails; The information synchronization module includes: An information traversal submodule, configured to traverse object information of objects in the target bucket stored in the source shard corresponding to the target bucket, and obtain object information to be written; an information writing submodule, configured to detect whether the object information of the object to be written already exists in the target shard during the process of synchronizing each object information to be written into the target shard, and if so, delete the object information to be written; if not, write the object information to be written into the target shard; The information storage submodule is used to obtain first object information of the first object when writing the first object to the target bucket during the synchronization process, and store the first object information in the source shard and the target shard.

9. The device according to claim 8, wherein: In the case where the object information includes an object name, the information is written into a submodule, specifically for: It is detected whether the object name included in the object information to be written exists in the object name included in the object information stored in the target slice.

10. The device according to claim 8, wherein: The device further comprises: an information deletion module, configured to delete the second object information of the second object stored in the source shard when the second object in the target bucket is deleted during the synchronization process; an information detection module, configured to detect whether the second object information exists in the target shard, and if so, trigger a third deletion module; The third deletion module is used to delete the second object information stored in the target slice.

11. The device according to claim 10, wherein: The information synchronization module is specifically used to: Traversing the object information of the object in the target bucket stored in the source shard corresponding to the target bucket to obtain the object information to be written; Synchronize the object information to be written to the target shard; During the synchronization process, if it is detected that the second object information is deleted in the source shard, detecting whether the second object information exists in the object information to be written that has not been synchronized to the target shard; If so, the second object information is deleted from the object information to be written that has not been synchronized to the target shard.

12. The device according to claim 8, wherein: The device further comprises: A state setting module, configured to, in response to a sharding instruction for a target bucket, set the sharding state of the target bucket to a normal state and start timing; A first timing module, configured to restart the timing when the object information synchronization process is normally performed when the timing duration reaches a first duration; A second timing module is configured to adjust the shard status of the target bucket to an abnormal state when the timing duration reaches a second duration, wherein the first duration is less than the second duration; The information synchronization module is specifically used to: When writing a first object to the target bucket during synchronization, if the shard status of the target bucket is normal, first object information of the first object is obtained and stored in the source shard and the target shard.

13. The device according to claim 12, wherein: The device further comprises: The information storage module is used to determine that the synchronization fails when writing the first object to the target bucket during the synchronization process, if the shard status of the target bucket is abnormal, and store the first object information in the source shard.

14. The device according to claim 12 or 13, wherein: The first duration is half of the second duration.

15. An electronic device, wherein: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product, wherein The computer program product comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images