Method and apparatus for implementing write operation mutex

Abstract

This specification provides a method and apparatus for implementing mutual exclusion in write operations. The method includes: receiving an append write operation for snapshot data, where the type of the append write operation includes a write data operation or a complete snapshot operation, and the snapshot data includes an original data block and an append record block; updating the append record block to write an append write record corresponding to the append write operation into the append record block; in the case of a failed update, querying whether there is an append write record corresponding to the complete snapshot operation in the append record block; if it exists, terminating the append write operation, otherwise retrying to update the append record block.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular, to a method and apparatus for implementing write operation mutual exclusion. Background Art

[0002] With the continuous development of computer technology, snapshot has gradually become an important technical means for data backup and recovery. In the snapshot service implemented based on object storage, the snapshot service data can usually be stored in blocks. However, since the write operations for the above snapshot data are independent and stateless in scenarios such as distributed cloud environments, it is difficult to solve the mutual exclusion problem in multi-device concurrent scenarios through traditional methods such as kernel mutex variables. Summary of the Invention

[0003] In view of this, this specification provides a method and apparatus for implementing write operation mutual exclusion to solve the deficiencies in the related art.

[0004] Specifically, this specification is implemented through the following technical solutions:

[0005] According to the first aspect of the embodiments of this specification, a method for implementing write operation mutual exclusion is provided. The method includes:

[0006] Receiving an append write operation for snapshot data, where the type of the append write operation includes a write data operation or a complete snapshot operation, and the snapshot data includes an original data block and an append record block;

[0007] Updating the append record block to write an append write record corresponding to the append write operation into the append record block;

[0008] In the case of update failure, querying whether there is an append write record corresponding to the complete snapshot operation in the append record block; if so, terminating the append write operation, otherwise retrying to update the append record block.

[0009] According to the second aspect of the embodiments of this specification, an apparatus for implementing write operation mutual exclusion is provided. The apparatus includes:

[0010] An operation receiving unit, configured to receive an append write operation for snapshot data, where the type of the append write operation includes a write data operation or a complete snapshot operation, and the snapshot data includes an original data block and an append record block;

[0011] An append record block update unit, configured to update the append record block to write an append write record corresponding to the append write operation into the append record block;

[0012] An update failure handling unit, configured to query whether there is an append write record corresponding to the completion of the snapshot operation in the append record block in case of an update failure; if so, terminate the append write operation, otherwise retry updating the append record block.

[0013] According to a third aspect of the embodiments of the present specification, a data storage system is provided, the system includes:

[0014] An object storage device, configured to maintain snapshot data;

[0015] A client, configured to initiate an append write operation for the snapshot data;

[0016] A server, configured to execute the steps of the method described in the first aspect in response to the append write operation.

[0017] According to a fourth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] According to a fifth aspect of the embodiments of the present specification, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the steps of the method described in the first aspect are implemented.

[0019] In the technical solution provided by the present specification, by utilizing the append write mechanism provided by the object storage service, the mutual exclusion of write operations in a multi-device concurrent scenario is ensured according to the append write records in the append record block.

[0020] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present specification. Description of the Drawings

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments recorded in the present specification, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0022] Figure 1 It is a schematic diagram of the architecture of a data storage system shown in an exemplary embodiment of the present specification;

[0023] Figure 2 It is a schematic flowchart of a method for implementing mutual exclusion of write operations shown in an exemplary embodiment of the present specification;

[0024] Figure 3 It is a schematic flowchart of another method for implementing write operation mutual exclusion shown in an exemplary embodiment of this specification;

[0025] Figure 4 It is a schematic flowchart of yet another method for implementing write operation mutual exclusion shown in an exemplary embodiment of this specification;

[0026] Figure 5 It is a schematic flowchart of still another method for implementing write operation mutual exclusion shown in an exemplary embodiment of this specification;

[0027] Figure 6 It is a schematic structural diagram of an electronic device shown in an exemplary embodiment of this specification;

[0028] Figure 7 It is a schematic structural diagram of a device for implementing write operation mutual exclusion shown in an exemplary embodiment of this specification. Detailed implementation manners

[0029] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0030] The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit this specification. The singular forms "a", "the", and "said" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0032] In the related art, when a snapshot operation is completed, it is usually necessary for the user to actively record and specify the number of data blocks to be written and the checksum (such as the SHA256 hash value) of all data blocks to verify the written data. The logic is cumbersome and the execution efficiency is low. In addition, if the verification fails, then even if the previous write is successful, the above-mentioned completion of the snapshot operation will be regarded as a total failure, resulting in the data blocks that the user has successfully written may not be included in the snapshot. In other words, in the related art, there is no breakpoint continuation function for the written data, and the consistency problem of the written data in the multi-device concurrent scenario cannot be solved. Therefore, the present specification proposes the following technical solutions to solve the above problems.

[0033] Figure 1 It is a schematic diagram of the architecture of a data storage system shown in an exemplary embodiment of the present specification. As Figure 1 shown, it may include an object storage device 11, a server 12, and a client group 13.

[0034] The object storage device 11 is an electronic device deployed with an Object Storage Service (OSS). In one embodiment, the object storage device 11 can be used to maintain the above-mentioned snapshot data. Those skilled in the art can understand that the object storage device 11 may adopt a similar distributed architecture, so it is easy to cause concurrent operations. The above object storage service provides a file append mechanism. On the one hand, the above file append mechanism can ensure the atomicity of the write operation by using a mechanism similar to the CAS (compare and swap) mechanism, and implement a save function similar to writing a checkpoint for snapshot data, so that the user does not need to pay attention to or record the historical data information that has been written into the snapshot data during the process of writing snapshot data, but only needs to pay attention to whether the current append write operation is successfully executed, thereby reflecting that the solution of the present application has a breakpoint continuation function. On the other hand, the above file append mechanism provides a technical basis for the mutual exclusion processing of the above concurrent operations. During the operation of the system, the logical space of the object storage device 11 can be divided into blocks of a certain size and the above-mentioned snapshot data can be stored in multiple object files respectively. In addition, Figure 1 the object storage device 11 in only shows the storage structure of the corresponding snapshot in any storage space (Bucket, also known as a bucket) in the object storage service, and the present application does not limit the number of snapshots of the object storage device 11.

[0035] The server 12 is an electronic device with a backend service deployed with a corresponding snapshot data interface API (Application Programming Interface). In one embodiment, the snapshot data interface API provides the ability to directly read and write snapshots and images. The server 12 can execute the steps of a method for implementing write operation mutex described below. The backend service can be used to respond to the call requests of the snapshot data interface API and perform corresponding processing. During the operation of the system, when the server 12 receives a snapshot data write request ① from the client group 13, the backend service can write the corresponding data in the request into the object file of the corresponding original data block in the object storage device 11 (i.e., a normal object file, NormalObject, which only supports overwrite writing but not append writing) through request ②, and update the relevant information of this write operation to the object file of the corresponding append record block (i.e., an append object file, AppendObject, which can perform data append operations) through request ③; when the server 12 receives a complete snapshot request ④ from the client group 13, the backend service can generate snapshot metadata information (i.e., generate or update snapshot metadata information based on the appended data block) and set the snapshot status to complete according to the valid data records in the append record block through request ⑤, and at the same time set the snapshot data to a read-only state.

[0036] The client group 13 contains one or more clients that support the snapshot data interface API. In one embodiment, the client can initiate an append write operation for the snapshot data. During the operation of the system, the user can call the snapshot data interface APIs of multiple clients to achieve concurrent snapshot data write requests to the server 12. Or, after the snapshot data is written, the user can initiate a complete snapshot request to the server 12 through the client to cause the object storage device 11 to generate a snapshot. In addition, the user can use electronic devices of the following types as the above clients: mobile phones, tablet devices, laptop computers, personal digital assistants (PDAs), wearable devices (such as smart glasses, smart watches, etc.). One or more embodiments of this specification do not limit this.

[0037] Among them, the above object storage device 11 and server 12 can correspond to different electronic devices or the same electronic device. The electronic device can be a physical server containing an independent host, or it can be a virtual server hosted by a host cluster. One or more embodiments of this specification do not limit this.

[0038] In addition, the connection methods among the object storage device 11, the server 12, and the client group 13 may include various types of wired or wireless connections, which are not limited in this specification.

[0039] The following elaborates on the technical solutions of this specification in conjunction with Figure 2 the illustrated embodiments. Figure 2 FIG. is a schematic flowchart of a method for implementing write operation mutex shown in an exemplary embodiment of this specification. As Figure 2 shown, the method may include the following steps:

[0040] S201, Receive an append write operation for snapshot data. The types of the append write operation include a write data operation or a complete snapshot operation. The snapshot data includes an original data block and an append record block.

[0041] The above append write operation may be executed relying on the above file append mechanism. For example, as described above, the above server may receive the above append write operation through the snapshot data interface API. Among them, the above append write operation includes two types: a write data operation and a complete snapshot operation, and each append write operation is stateless and independent of each other. The above append write operation may be executed on any machine in a cluster.

[0042] In addition, the above snapshot data may include an original data block and an append record block. Among them, the object file stored in the above original data block only supports overwrite writing and cannot be appended, and can be used to store the actual content corresponding to the above append write operation. The object file stored in the above append record block supports append writing and can be used to record the following append write records. At the same time, the above append record block may be set to any storage structure with orderliness, which is not limited in this specification.

[0043] S202, Update the append record block to write an append write record corresponding to the append write operation into the append record block.

[0044] According to the atomicity of the above file append mechanism, the above append record block can be updated to write an append write record corresponding to the above append write operation into it.

[0045] Before updating the append record block, different processing may be performed according to the type of the append write operation.

[0046] In one embodiment, when the above append write operation is a write data operation, the append data corresponding to the above append write operation is written into the above original data block; if the write fails, it is determined that the above append write operation fails to execute; if the write is successful, the above append record block is updated. Among them, the above write failure may be caused by abnormal situations such as OSS service response timeout, OSS-related device downtime, and network fluctuations during sending, which are not limited in this specification. Since the write operation for the above original data block is a pre-operation for the update operation of the above append record block, in the case of the failure of the above write operation, it can be directly determined that the above append write operation fails to execute, improving the efficiency of processing the above append write operation.

[0047] Before performing the write operation on the above original data block, it is also possible to determine whether it is necessary to perform subsequent write operations and update operations according to the status of the current snapshot.

[0048] In one embodiment, it is possible to determine whether the above snapshot data is in a completed state or whether there is an append write record corresponding to the completed snapshot operation in the above append record block; if so, it is determined that the above append write operation fails to execute; if not, the append data corresponding to the above append write operation is written into the above original data block. Among them, both the above completed state and the above append write record corresponding to the completed snapshot operation can represent that the above snapshot data has completed the snapshot. The above completed state can be recorded in the snapshot metadata information in the Bucket corresponding to the above snapshot data, or any other preset storage space, which is not limited in this specification. As described above, when the above snapshot data completes the snapshot, the snapshot data will be set to a read-only state and the above append write operation cannot be continued. In other words, at this time, it can be determined that the above append write operation fails to execute.

[0049] Those skilled in the art can understand that in the previous embodiment, the main basis for determining that the execution result of the above append write operation is a failure is that the above append write operation is a write data operation. When the above append write operation is a completed snapshot operation, it means that there is a concurrent append write operation that exists and is executed before the above append write operation is executed, so that the above snapshot data completes the snapshot. Therefore, even if the above append write operation is not actually executed, the purpose of the above append write operation has been achieved, that is, it can be determined that the above append write operation is successfully executed. Otherwise, the above append record block still needs to be updated.

[0050] After the append data corresponding to the append write operation is written into the above original data block and before the above append record block is updated, it is also possible to further determine whether it is necessary to perform the above update operation according to the append record block.

[0051] In one embodiment, when the data corresponding to the above append write operation is successfully written into the target original data block, it is possible to query whether there is an append write record corresponding to the target original data block in the above append record block; if it exists, it is determined that the append write operation is successfully executed; if it does not exist, the append record block is updated. Among them, for example, as described above, the above append record block is used to write the append write record corresponding to the above append write operation, that is, the append record block can record any original data block in the above snapshot data that has been written with data. In other words, in this embodiment, when there is an append write record corresponding to the target original data block in the above append record block, it means that the target original data block is not written with data for the first time, and the above append record block does not need to repeatedly write the append write record corresponding to the above append write operation, thereby further improving the efficiency of processing the above append write operation.

[0052] S203, in the case of update failure, query whether there is an append write record corresponding to the completed snapshot operation in the above append record block; if it exists, terminate the above append write operation, otherwise retry updating the above append record block.

[0053] When the update fails, that is, when it is impossible to write the append write record corresponding to the above append write operation into the above append record block, except for abnormal situations such as equipment and network, it is usually caused by concurrent append write operations. When concurrent append write operations cause the update to fail, it is possible to determine whether to terminate the above append write operation or retry updating the above append record block by checking whether there is an append write record corresponding to the completed snapshot operation in the above append record block. Among them, if there is an append write record corresponding to the completed snapshot operation, no matter what type the above append write operation is, it cannot be continued; if there is no append write record corresponding to the completed snapshot operation, it can be determined that the above update failure is related to the concurrent write data operation, and no matter what type the above append write operation is, it is possible to try to retry updating the above append record block. Among them, the above retry can be predefined with limits such as the number of times and the upper limit of the duration, and this specification does not limit this.

[0054] Those skilled in the art can understand that the purpose of retrying to update the appended record block is to avoid the situation of "after the above-mentioned original data block is written, but there is no appended write record corresponding to the above-mentioned append write operation in the above-mentioned appended record block". Therefore, it is possible that different clients initiate the same write data operation for the same appended record block, resulting in multiple appended write records corresponding to the same original data block in the above-mentioned appended record block. The solution of this application can, when the snapshot operation is completed, select the record with the latest update time point from the multiple appended write records corresponding to the same original data block as the valid appended write record. Among them, the update time point is only one of the methods for selecting the above-mentioned valid appended write record, and different selection conditions can be set according to different actual requirements, which are not limited in this specification.

[0055] When the above-mentioned append write operation is terminated, the written data can be cleared to characterize the atomicity of the above-mentioned append write operation.

[0056] In an embodiment, when the above-mentioned append write operation is a write data operation and the above-mentioned append write operation is terminated because there is an appended write record corresponding to the completed snapshot operation in the above-mentioned appended record block, the written data corresponding to the above-mentioned append write operation in the above-mentioned original data block is cleared. In this implementation, the above-mentioned append write operation is executed before the concurrent completion of the snapshot operation. That is, when the snapshot is completed, the above-mentioned append write operation only writes the corresponding data in the above-mentioned original data block, but does not update the above-mentioned appended record block, resulting in the possibility that the above-mentioned original data block and the above-mentioned appended record block may not correspond to each other.

[0057] Those skilled in the art can understand that the update operation of the above-mentioned appended record block may be repeatedly executed due to continuous retry update operations. In other words, there may be multiple iteration processes in the two steps of S202 and S203, which are not limited in this specification.

[0058] Compared with the related art, in the solution of this application, it is possible to query whether there is an appended write record corresponding to the completed snapshot operation in the above-mentioned appended record block without relying on technologies such as mutex variables and distributed lock services.

[0059] In one embodiment, the parity of the append write records corresponding to the above write data operation and the above complete snapshot operation is fixed and different from each other. It is possible to determine whether there is an append write record corresponding to the complete snapshot operation in the above append record block according to the parity of all the data already written in the above append record block. For example: Define the append write record corresponding to the above complete snapshot operation as a fixed-length odd byte, the length of the append write record corresponding to the above write data operation is fixed as a fixed-length even byte, and the above complete snapshot operation is executed successfully at most once. Therefore, when the length of all the data already written in the above append record block is odd, the snapshot data must have completed the snapshot. When the length of all the data already written in the above append record block is even, only the write data operation is performed on the above snapshot data, and no complete snapshot operation is performed. In this embodiment, by using the simple principle that the result of adding an odd number and an even number is odd, it is possible to judge the result after the concurrent write and the complete snapshot operation are superimposed on each other, thereby ensuring the mutual exclusion between each append write operation, and thus solving the consistency problem of the snapshot data under the multi-client concurrent append write operation.

[0060] Taking the mutual exclusion between multiple write data operations as an example below, the technical solution related to the above embodiment in the present application will be elaborated in detail in combination with Figure 3 the following. Figure 3 FIG. is a schematic flowchart of another method for realizing write operation mutual exclusion shown in an exemplary embodiment of the present specification. As Figure 3 shown, this solution may include the following steps:

[0061] S301, initiate an append write operation of the type of write data operation through the snapshot data interface API.

[0062] In one embodiment, the client may perform an append write operation on the snapshot data through the snapshot data interface API to write the target data. Among them, the type of this append write operation is a write data operation.

[0063] S302, write the append data corresponding to the append write operation into the target original data block.

[0064] In one embodiment, the backend service corresponding to the above snapshot data interface API may write the above target data into the object file of the target original data block in the corresponding object storage service Bucket.

[0065] S303, judge whether the write is successful.

[0066] In one embodiment, the above snapshot data interface API may be used to judge whether the above target data is successfully written into the object file of the above target original data block. If it is successful, execute S304. Otherwise, it may be determined that the above append write operation fails.

[0067] S304, query whether there is an append write record corresponding to the target original data block in the append record block.

[0068] In one embodiment, after the above target data is successfully written to the object file of the above target original data block, it is possible to determine whether the target original data block is written with data for the first time by querying whether there is an append write record corresponding to the address information of the target original data block in the above append record block. If it is not the first write, it can be determined that the above append write operation is successfully executed; otherwise, continue to execute S305.

[0069] S305, update the append record block to write an append write record with an even length to the append record block.

[0070] In one embodiment, when it is determined that the target original data block is written with data for the first time, an attempt can be made to update the above append record block to write an append write record corresponding to the above append write operation to the above append record block. Among them, the append write record contains the address information of the above target original data block and has a length of a preset even number of bytes.

[0071] S306, determine whether the update is successful.

[0072] In one embodiment, if the above append record block is successfully updated, it can be determined that the above append write operation is successfully executed; if it fails, it can be determined that there may be a concurrent append write operation that is successfully executed.

[0073] S307, determine whether to retry the update.

[0074] In one embodiment, if there is a concurrent append write operation, it may cause the "S307 - S305 - S306 - S307" loop to execute. In this embodiment, there may be a preset limit for the update operation of the above append record block. Assume that the above preset limit requires that the retry count of the update operation is less than 10. Then when the update operation for the above append record block has been repeated 9 times and is still not successful, it is determined that the above append write operation fails and S308 is executed; otherwise, it is determined that the above append write operation is successfully executed.

[0075] S308, clean up the written data corresponding to the append write operation in the target original data block.

[0076] In one embodiment, since the above append write operation fails and the above target data has been successfully written into the object file of the above target original data block, in order to avoid consistency problems related to the above snapshot data, the target data corresponding to the above append write operation in the target original data block can be cleaned up.

[0077] Below, taking the mutual exclusion between the write data operation and the completion of the snapshot operation as an example, in combination withFigure 4 Elaborate in detail on the technical solutions related to the above embodiments in this application. Figure 4 It is a schematic flowchart of another method for implementing write operation mutual exclusion shown in an exemplary embodiment of this specification. As Figure 4 shown, this solution may include the following steps:

[0078] S401, initiate an append write operation of the type of write data operation through the snapshot data interface API.

[0079] This step is the same as S301 above, and this specification will not elaborate here.

[0080] S402, determine whether the snapshot data is in a completed state.

[0081] In one embodiment, the backend service of the above snapshot data interface API can determine whether the snapshot data is in a completed state according to the snapshot metadata information in the Bucket corresponding to the snapshot data. If so, it is determined that the above append write operation fails; otherwise, S403 is executed.

[0082] S403, determine whether the length of the append record block is odd.

[0083] In one embodiment, even if the above snapshot data is not in a completed state, it does not mean that the above snapshot data has not performed the complete snapshot operation. Therefore, it is possible to determine whether the above snapshot data has performed the complete snapshot operation by determining whether the length of the append record block is odd. If it is odd, it is determined that the above snapshot data has performed the complete snapshot operation, and at the same time, it is determined that the above append write operation fails; if it is even, S404 is executed.

[0084] S404, write the append data corresponding to the append write operation to the target original data block.

[0085] S405, determine whether the write is successful.

[0086] S406, query whether there is an append write record corresponding to the target original data block in the append record block.

[0087] The above steps are the same as S302 - S304 above, and this specification will not elaborate here.

[0088] S407, whether the length of the append record block is odd.

[0089] In one embodiment, when there is no append write record corresponding to the target original data block in the append record block, that is, when it is determined that the target original data block is written with data for the first time, it is possible to further determine whether the length of the append record block is odd, so as to avoid the completion snapshot operation for the above snapshot data being executed after S403 and before S407. If it is odd, S408 is executed; if it is even, S412 is executed.

[0090] S408, update the append record block to write an append write record with an even length to the append record block.

[0091] S409, determine whether the update is successful.

[0092] The above steps are the same as S305 - S306 above, and will not be elaborated herein.

[0093] S410, determine whether the length of the append record block is odd.

[0094] In one embodiment, when the update of the append record block fails, it is possible to determine whether the length of the append record block is odd, so as to determine whether the above snapshot data has been executed with the completion snapshot operation. If it is even, it is determined that the failure to update the above append record block is not caused by a concurrent completion snapshot operation, and S411 is executed; if it is odd, it is determined that the above snapshot data has been executed with the completion snapshot operation, and S412 is executed.

[0095] S411, determine whether to retry the update.

[0096] In one embodiment, if there is a concurrent append write operation, it may cause the loop of "S411 - S408 - S409 - S410 - S411" to execute. In this embodiment, the retry update of the append record operation can be restricted, for example, it is required that the retry duration of the update operation is less than 3 minutes. Then when the update operation for the above append record block is greater than or equal to 3 minutes and is still not successful, it is determined that the above append write operation fails and S412 is executed, otherwise S407 is executed.

[0097] S412, clean up the written data corresponding to the append write operation in the target original data block.

[0098] This step is the same as S308 above, and will not be elaborated herein.

[0099] Next, taking the mutual exclusion between the completion snapshot operation and the write data operation or between the completion snapshot operations as an example, the technical solutions related to the above embodiments in the present application will be elaborated in detail. Figure 5 Elaborate on the technical solutions related to the above embodiments in the present application. Figure 5 It is a flowchart showing another method for implementing write operation mutual exclusion shown in an exemplary embodiment of this specification. AsFigure 3 As shown, the solution may include the following steps:

[0100] S501, initiate an append write operation of the type of completing the snapshot operation through the snapshot data interface API.

[0101] In one embodiment, the client can perform an append write operation on the snapshot data through the snapshot data interface API to complete the snapshot, where the type of the append write operation is the operation of completing the snapshot.

[0102] S502, determine whether the snapshot data is in a completed state.

[0103] In one embodiment, the backend service of the above snapshot data interface API can determine whether the snapshot data is in a completed state according to the snapshot metadata information in the Bucket corresponding to the snapshot data. If so, it is determined that the concurrent completion snapshot operation is executed successfully, and the above append write operation does not need to be repeated; otherwise, execute S503.

[0104] S503, determine whether the length of the appended record block is odd.

[0105] In one embodiment, even if the above snapshot data is not in a completed state, it does not mean that the above snapshot data has not executed the operation of completing the snapshot. Therefore, it is possible to determine whether the above snapshot data has executed the operation of completing the snapshot by determining whether the length of the appended record block is odd. If it is odd, it is determined that a concurrent completion snapshot operation has been executed successfully, and at the same time, it is determined that the above append write operation is executed successfully; if it is even, execute S504.

[0106] S504, update the appended record block to write an appended write record with an odd length into the appended record block.

[0107] In one embodiment, when it is determined that the above appended record block has not executed the operation of completing the snapshot, an appended write record corresponding to the above append write operation can be written into the above appended record block. The appended write record is used to indicate that the state of the above snapshot data is set to completed, and the length of the appended write record is a preset odd number of bytes.

[0108] S505, determine whether the write is successful.

[0109] In one embodiment, if the above appended record block is successfully updated, it can be determined that the above append write operation is executed successfully; if it fails, it can be determined that a concurrent append write operation may be executed successfully.

[0110] S506, determine whether the length of the appended record block is odd.

[0111] In one embodiment, when the update of the append record block fails, it is possible to determine whether the length of the append record block is odd, so as to determine whether the above snapshot data has completed the snapshot operation. If it is even, it is determined that the failure of the above update of the append record block is not caused by the concurrent completion of the snapshot operation, and S507 is executed; if it is odd, it is determined that a concurrent completion of the snapshot operation has been successfully executed, and at the same time it is determined that the above append write operation has been successfully executed.

[0112] S507, determine whether to retry the update.

[0113] In one embodiment, if there is a concurrent append write operation, it may cause the loop of "S507 - S504 - S505 - S506 - S507" to execute. In this embodiment, the retry of the update of the append record operation can be restricted, for example, it is required that the retry duration of the update operation is less than 3 minutes. Then when the update operation for the above append record block is greater than or equal to 3 minutes and still not successful, it is determined that the above append write operation has failed, otherwise S504 is executed.

[0114] As can be seen from the above embodiments, in this specification, the object file append function provided by the object storage service is used to replace the mutex variables or distributed lock services at the system level in the related art, thereby realizing the breakpoint resume function in the stateless object data scenario. At the same time, this application uses the parity of the file length to distinguish between the write data operation and the completion of the snapshot operation, thereby realizing the mutual exclusion between the write data operation and the write data operation, the write data operation and the completion of the snapshot operation, and the completion of the snapshot operation and the completion of the snapshot operation, and solving the consistency problem of snapshot data in the multi-client concurrent operation scenario.

[0115] Figure 6 It is a schematic structural diagram of an electronic device in an exemplary embodiment. Please refer to Figure 6 , at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, other required hardware may also be included. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs, forming a device for realizing the mutual exclusion of write operations at the logical level. Of course, in addition to the software implementation method, this specification does not exclude other implementation methods, such as logical devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and can also be hardware or logical devices.

[0116] Corresponding to the foregoing embodiments of the method for realizing the mutual exclusion of write operations, this specification also provides an embodiment of a device for realizing the mutual exclusion of write operations.

[0117] Please refer to Figure 7 , Figure 7It is a schematic structural diagram of a device for implementing write operation mutual exclusion shown in an exemplary embodiment.

[0118] As Figure 7 shown, in the software implementation, the device may include:

[0119] An operation receiving unit 701, configured to receive an append write operation for snapshot data, where the type of the append write operation includes a write data operation or a complete snapshot operation, and the snapshot data includes an original data block and an append record block;

[0120] An append record block updating unit 702, configured to update the append record block to write an append write record corresponding to the append write operation into the append record block;

[0121] An update failure handling unit 703, configured to query whether there is an append write record corresponding to the complete snapshot operation in the append record block in case of update failure; if so, terminate the append write operation, otherwise retry updating the append record block.

[0122] Optionally, the operation receiving unit 701 is specifically configured to:

[0123] Receive the append write operation through a snapshot data interface API.

[0124] Optionally, the device further includes:

[0125] An original data block writing unit 704, configured to write the append data corresponding to the append write operation into the original data block when the append write operation is a write data operation; if the writing fails, determine that the append write operation fails to execute;

[0126] The append record block updating unit 702 is specifically configured to:

[0127] If the writing is successful, update the append record block.

[0128] Optionally, the device further includes:

[0129] A write condition judging unit 705, configured to determine whether the snapshot data is in a completed state or whether there is an append write record corresponding to the complete snapshot operation in the append record block; if so, determine that the append write operation fails to execute;

[0130] The original data block writing unit 704 is specifically configured to: if not, write the append data corresponding to the append write operation into the original data block.

[0131] Optionally, the device further includes:

[0132] An update condition judgment unit 706 is configured to query whether there is an append write record corresponding to the target original data block in the append record block when the data corresponding to the append write operation is successfully written into the target original data block; if so, it is determined that the append write operation is successfully executed;

[0133] The append record block update unit 702 is specifically configured to:

[0134] If not, update the append record block.

[0135] Optionally, the device further includes:

[0136] A data cleaning unit 707 is configured to determine whether the snapshot data is in a completed state or whether there is an append write record corresponding to the completed snapshot operation in the append record block; if so and the append write operation is a completed snapshot operation, it is determined that the append write operation is successfully executed;

[0137] The append record block update unit 702 is specifically configured to:

[0138] If not and the append write operation is a completed snapshot operation, update the append record block.

[0139] Optionally, the update failure handling unit 703 is specifically configured to:

[0140] In the case where the parities of the append write records corresponding to the write data operation and the completed snapshot operation are fixed and different from each other, determine whether there is an append write record corresponding to the completed snapshot operation in the append record block according to the parity of all the data already written in the append record block.

[0141] The implementation processes of the functions and roles of each unit in the above device are specifically described in detail in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0142] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0143] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or one or more of them in combination. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier to be executed by, or to control the operation of, a data processing apparatus. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode and transmit information to the appropriate receiver apparatus for execution by the data processing apparatus. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0144] The processes and logical flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform the functions by operating on input data and generating output. The processes and logical flows can also be performed by, or the apparatus can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0145] Suitable computers for executing computer programs include, by way of example, general and / or special purpose microprocessors, or any other type of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory and / or a random access memory. Basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, etc., or the computer will be operatively coupled to such mass storage devices to receive data from them or to transfer data to them, or both. However, a computer need not have such devices. In addition, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name just a few.

[0146] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0147] Although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of what is claimed, but rather as primarily describing the features of specific embodiments of particular inventions. Certain features that are described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Additionally, although features may operate in certain combinations as described above and even be claimed as such initially, one or more features from a claimed combination may in some cases be removed from the combination, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination.

[0148] Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or sequentially, or that all illustrated operations be performed, to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system modules and components in the above embodiments should not be understood as required in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0149] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the acts recited in the claims may be performed in a different order and still achieve the desired result. Additionally, the processes depicted in the drawings are not necessarily in the particular order or sequential order shown to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0150] The above is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included within the scope of protection of this specification.

Claims

1. A method for implementing write operation mutual exclusion, characterized in that The method includes: Receiving an append write operation for snapshot data, where the type of the append write operation includes a write data operation or a complete snapshot operation, and the snapshot data includes an original data block and an append record block; Updating the append record block to write an append write record corresponding to the append write operation into the append record block; In the case of update failure, querying whether there is an append write record corresponding to the complete snapshot operation in the append record block; if so, terminating the append write operation, otherwise retrying to update the append record block.

2. The method according to claim 1, characterized in that, The receiving an append write operation for snapshot data includes: Receiving the append write operation through a snapshot data interface API.

3. The method according to claim 1, wherein The method further includes: in the case where the append write operation is a write data operation, writing the append data corresponding to the append write operation into the original data block; if the writing fails, determining that the append write operation fails to execute; The updating the append record block includes: if the writing is successful, updating the append record block.

4. The method according to claim 3, wherein The method further includes: determining whether the snapshot data is in a completed state or whether there is an append write record corresponding to the complete snapshot operation in the append record block; if so, determining that the append write operation fails to execute; The writing the append data corresponding to the append write operation into the original data block includes: if not, writing the append data corresponding to the append write operation into the original data block.

5. The method according to claim 3, wherein The method further includes: in the case where the data corresponding to the append write operation is successfully written into the target original data block, querying whether there is an append write record corresponding to the target original data block in the append record block; if so, determining that the append write operation is successfully executed; The updating the append record block includes: if not, updating the append record block.

6. The method according to claim 1, wherein It further includes: In the case where the append write operation is a write data operation and the append write operation is terminated because there is an append write record corresponding to the complete snapshot operation in the append record block, cleaning the written data corresponding to the append write operation in the original data block.

7. The method according to claim 1, wherein The method further includes: determining whether the snapshot data is in a completed state or whether there is an append write record corresponding to the complete snapshot operation in the append record block; if so and the append write operation is a complete snapshot operation, determining that the append write operation is successfully executed; The updating the append record block includes: if not and the append write operation is a complete snapshot operation, updating the append record block.

8. The method according to claim 1, characterized in that, The querying whether there is an append write record corresponding to the complete snapshot operation in the append record block includes: When the parity of the appended write records corresponding to the write data operation and the complete snapshot operation is fixed and different from each other, determine whether there is an appended write record corresponding to the complete snapshot operation in the appended record block according to the parity of all the data already written in the appended record block.

9. An apparatus for implementing exclusive write operations, characterized in that, The apparatus includes: An operation receiving unit, configured to receive an append write operation for snapshot data, where the type of the append write operation includes a write data operation or a complete snapshot operation, and the snapshot data includes an original data block and an appended record block; An appended record block updating unit, configured to update the appended record block to write an appended write record corresponding to the append write operation into the appended record block; An update failure handling unit, configured to, when an update fails, query whether there is an appended write record corresponding to the complete snapshot operation in the appended record block; if so, terminate the append write operation, otherwise retry updating the appended record block.

10. A data storage system, characterized in that, The system includes: An object storage device, configured to maintain snapshot data; A client, configured to initiate an append write operation for the snapshot data; A server, configured to, in response to the append write operation, execute the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images