Data processing method and apparatus

Abstract

Embodiments of the present specification provide a data processing method and device, wherein the data processing method comprises: in response to a data write request, writing target data to a target object of a storage end, wherein the storage end comprises a primary storage end and a backup storage end; determining initial object metadata of the target object, and adding a data lock to the initial object metadata; sending a pre-commit request for the target data to the storage end; in a case where pre-commit metadata and a pre-commit success instruction returned by the storage end are received, sending a commit request for the target data to the storage end, wherein the pre-commit metadata is metadata of the target data to which a current version number is added, and the current version number is obtained by the storage end; and in a case where a commit success instruction returned by the storage end is received, determining that the target data is written successfully.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and in particular, to a data processing method. BACKGROUND

[0002] With the popularity of cloud computing, more and more users and enterprises choose to store data on the cloud. For some important data, users want the data to be stored in data centers in different regions of the cloud service provider at the same time, in order to achieve disaster recovery, improve data availability and security. For example, when a data center in a certain region of the cloud service provider fails or is damaged due to power, weather, equipment room equipment, etc., the user can switch data access to the data center in other regions of the cloud service provider to restore data services.

[0003] Currently, object storage services with cross-region replication function provide the ability to asynchronously replicate data between data centers in different regions or different machine room clusters. It will asynchronously replicate object creation, update and deletion operations from the source storage space to the target storage space in different regions. When the source end (source storage space) fails, the user can switch read and write to the opposite end cluster (target storage space) to restore services.

[0004] Due to asynchronous replication, the opposite end may "lose" a small part of the data that has not been replicated by the source end cluster after switching. However, many traditional enterprise users (such as financial users) have strong requirements for data consistency and require that there be no data "loss" or inconsistency during disaster recovery switching. SUMMARY

[0005] Therefore, the embodiments of the present specification provide a data processing method. One or more embodiments of the present specification also relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0006] According to a first aspect of the embodiments of the present specification, a data processing method is provided, comprising:

[0007] in response to a data write request, writing target data to a target object of a storage end, wherein the storage end includes a primary storage end and a backup storage end;

[0008] determining initial object metadata of the target object, and adding a data lock to the initial object metadata;

[0009] sending a pre-commit request for the target data to the storage end;

[0010] In a case where the pre-commit metadata and the pre-commit success instruction returned by the storage end are received, a commit request for the target data is sent to the storage end, wherein the pre-commit metadata is metadata of the target data with a current version number added, and the current version number is obtained by the storage end.

[0011] In a case where the commit success instruction returned by the storage end is received, it is determined that the target data is written successfully.

[0012] According to a second aspect of an embodiment of the present specification, a data processing apparatus is provided, comprising:

[0013] A write module is configured to write target data to a target object of a storage end in response to a data write request, wherein the storage end comprises a primary storage end and a backup storage end;

[0014] A lock module is configured to determine initial object metadata of the target object, and add a data lock to the initial object metadata;

[0015] A first request sending module is configured to send a pre-commit request for the target data to the storage end;

[0016] A second request sending module is configured to send a commit request for the target data to the storage end in a case where the pre-commit metadata and the pre-commit success instruction returned by the storage end are received, wherein the pre-commit metadata is metadata of the target data with a current version number added, and the current version number is obtained by the storage end;

[0017] A data write module is configured to determine that the target data is written successfully in a case where the commit success instruction returned by the storage end is received.

[0018] According to a third aspect of an embodiment of the present specification, a computing device is provided, comprising:

[0019] a memory and a processor;

[0020] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions, which realize the steps of the above data processing method when executed by the processor.

[0021] According to a fourth aspect of an embodiment of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, which realize the steps of the above data processing method when executed by the processor.

[0022] According to a fifth aspect of the embodiments of the present specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the data processing method.

[0023] One embodiment of the present specification implements a data processing method and device, wherein the data processing method comprises: in response to a data write request, writing target data into a target object of a storage end, wherein the storage end comprises a main storage end and a backup storage end; determining initial object metadata of the target object, and adding a data lock to the initial object metadata; sending a pre-commit request for the target data to the storage end; in the case of receiving a pre-commit metadata and a pre-commit success instruction returned by the storage end, sending a commit request for the target data to the storage end, wherein the pre-commit metadata is metadata of the target data with a current version number added, and the current version number is obtained by the storage end; in the case of receiving a commit success instruction returned by the storage end, determining that the target data is written successfully.

[0024] Specifically, the data processing method adopts concurrent writing of to-be-written data into the main storage end and the backup storage end, ensuring that the to-be-written data can be written into the main storage end and the backup storage end at any moment, solving the problem of short-term inconsistency of data caused by asynchronous replication, and by adopting the two-phase commit mode of pre-commit and commit, ensuring that in the case of successful commit at one end and failed commit at the other end, the pre-commit metadata of pre-commit can be used as a redo log for data recovery, to ensure the consistency of data of the main storage end and the backup storage end when reading. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a specific application scenario diagram of a data processing method provided by one embodiment of the present specification;

[0026] Figure 2 is a flowchart of a data processing method provided by one embodiment of the present specification;

[0027] Figure 3 is a process flowchart of a data processing method provided by one embodiment of the present specification;

[0028] Figure 4 is a structural schematic diagram of a data processing device provided by one embodiment of the present specification;

[0029] Figure 5 is a structural block diagram of a computing device provided by one embodiment of the present specification. DETAILED DESCRIPTION

[0030] In the following description, numerous specific details are set forth to provide a thorough understanding of the specification. However, the specification can be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure the specification.

[0031] The terminology used in this description of one or more embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the specification. As used in this description and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0032] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, a first entity discussed below could later be discussed as a second entity, and similarly, a second entity discussed below could later be discussed as a first entity without departing from the scope of one or more embodiments. As used herein, the term "if' can be construed to mean "when" or "in response to determining" or "in response to a determination" or "in response to the fact that... "

[0033] First, the noun terms related to one or more embodiments of the specification are explained.

[0034] Asynchronous replication: data is copied from the source storage space to the target storage space, the data of the target storage space is an exact copy of the source storage space, and they have the same Object name, version information, metadata, and content, etc. The data is not real-time copied to the opposite end (target storage space), and there is a certain time interval from the source storage space.

[0035] Synchronous replication: similar to asynchronous replication, data can be copied from the source storage space to the target storage space, but the data replication is real-time, and after the source data is written successfully, the data is visible at the opposite end.

[0036] Pre-commit: pre-commit, which means that after data is written, a pre-commit record is made, but it is not directly visible to users.

[0037] Commit: commit, which means that after data is written, it is persisted to the storage system and is visible to users.

[0038] Atomic write: compare and swap, CAS, is used to implement uninterrupted data exchange operations in multi-threaded programming, thereby avoiding data inconsistencies caused by uncertainty in execution order and unpredictable interruptions when multiple threads simultaneously rewrite a certain data. This operation compares the value in memory with the specified data and replaces the data in memory with the new value when the values ​​are the same.

[0039] object meta: object metadata; request body: request body; bucket: bucket, storage space; versionId: version number; redo log: redo log; failovere: failover; current: current; redo: redo.

[0040] In this specification, a data processing method is provided. This specification also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

[0041] See also Figure 1 , Figure 1 A schematic diagram of a specific application scenario of a data processing method provided according to an embodiment of this specification is shown.

[0042] Figure 1 The system includes an object storage service platform 102, a primary storage end 104, and a backup storage end 106. The primary storage end 104 includes a storage cluster composed of multiple storage spaces, and the backup storage end 106 includes a storage cluster composed of multiple storage spaces.

[0043] Specifically, after receiving the user's data write request, the object storage service platform 102 checks whether the initial object metadata of the target object to which the target data is to be written in the primary storage end 104 and the backup storage end 106 has a data lock. If there is no data lock or the data lock has expired, it means that no other data operations have been performed on the target object. In this case, the target data written by the user can be streamed and the read target data can be concurrently written to the primary storage end 104 and the backup storage end 106 until the target data reading is completed.

[0044] After all target data has been concurrently written to the primary storage end 104 and the backup storage end 106, the object storage service platform 102 adds data locks to the initial object metadata of the target objects on the primary storage end 104 and the backup storage end 106, respectively. The specific operation of adding data locks can be understood as adding a lock mark to the initial object metadata of the target object.

[0045] After the initial object metadata of the target object of the primary storage end 104 and the backup storage end 106 are successfully locked, the object storage service platform 102 concurrently performs pre-committing of the target data on the primary storage end 104 and the backup storage end 106, that is, the object storage service platform 102 sends a pre-committing request for the target data to the primary storage end 104 and the backup storage end 106.

[0046] When the object storage service platform 102 receives the pre-committing success instruction returned by the primary storage end 104 and the backup storage end 106, the object storage service platform 102 again concurrently sends a committing request for the target data to the primary storage end 104 and the backup storage end 106. When the object storage service platform 102 receives the committing success instruction returned by the primary storage end 104 for the target data, the object storage service platform 102 determines that the target data is written to the primary storage end 104 successfully, at which time the object storage service platform 102 asynchronously deletes the pre-committing metadata returned by the primary storage end 104 during pre-committing, wherein the pre-committing metadata can be understood as the metadata of the target data to which the current version number obtained by the primary storage end 104 is added; and the object storage service platform 102 releases the data lock of the initial object metadata of the target object of the primary storage end 104.

[0047] When the object storage service platform 102 receives the committing success instruction returned by the backup storage end 106 for the target data, the object storage service platform 102 determines that the target data is written to the backup storage end 106 successfully, at which time the object storage service platform 102 asynchronously deletes the pre-committing metadata returned by the backup storage end 106 during pre-committing, wherein the pre-committing metadata can be understood as the metadata of the target data to which the current version number obtained by the primary storage end 104 is added; and the object storage service platform 102 releases the data lock of the initial object metadata of the target object of the backup storage end 106.

[0048] The data processing method provided by the embodiment of the present specification guarantees the synchronous writing of the target data on the primary storage end 104 and the backup storage end 106 by synchronously writing the target data to the primary storage end 104 and the backup storage end 106, and guarantees that in the case of a failover of the data writing of one end, the target data can be recovered through the pre-committing metadata according to the consistent relationship between the version number of the object metadata of the target object in which the target data that has been successfully committed by the other end is located and the version number in the pre-committing metadata, so as to implement the successful committing of the target data and guarantee the consistency of the target data of the two ends when the target data is read.

[0049] Referring to Figure 2 , Figure 2 A flowchart of a data processing method according to an embodiment of the present specification is shown, and the method specifically includes the following steps.

[0050] Step 202: In response to a data writing request, write target data to a target object of a storage end, wherein the storage end includes a primary storage end and a backup storage end.

[0051] The primary storage end can be understood as the source end of the above embodiments, and the backup storage end can be understood as the opposite end of the above embodiments. The primary storage end includes a storage cluster formed by a plurality of storage spaces, and the backup storage end includes a storage cluster formed by a plurality of storage spaces.

[0052] The object storage service platform of the above embodiments is the execution subject, and the target data is written into the target object of the storage end in response to the data write request. It can be understood that the object storage service platform writes the target data into the primary storage end and the backup storage end in response to the data write request.

[0053] Specifically, the target data is written into the target object of the storage end in response to the data write request, including:

[0054] receiving a data write request sent by a first user;

[0055] acquiring the target data written by the user according to the data write request through a preset reading mode;

[0056] writing the target data into the target object of the primary storage end and the backup storage end, respectively.

[0057] The first user can be any type of user. The preset reading mode can be set according to actual application, and the present specification does not make any limitation thereon. For example, the preset reading mode can be streaming reading.

[0058] In specific implementation, taking the preset reading mode as streaming reading as an example, the object storage service platform receives a data write request sent by a first user, reads the target data written by the first user in a streaming manner, and writes the read target data into the primary storage end and the backup storage end concurrently.

[0059] In the embodiments of the present specification, after receiving a data write request sent by a first user, the object storage service platform reads the target data written by the first user in a streaming manner, and then writes the streaming read target data into the primary storage end and the backup storage end. The streaming reading mode avoids data accumulation and reduces network burden through the streaming reading mode of writing while reading. At the same time, the read target data is written into the primary storage end and the backup storage end concurrently, thereby ensuring the consistency of data written into the primary storage end and the backup storage end.

[0060] Further, in order to avoid the case that data confusion occurs due to other data operations on the target objects of the primary storage end and the backup storage end in the case of target data storage, the object storage service platform will check whether the initial object metadata of the target objects of the primary storage end and the backup storage end exist data locks before obtaining the target data written by the first user, and if not, the next data acquisition and writing process is performed to ensure that only one data processing operation can be performed on the same target object at the same time. The specific implementation manner is as follows:

[0061] The target data written by the user is obtained according to the data write request through a preset reading mode, and the specific implementation manner is as follows:

[0062] The target object of the primary storage end and the target object of the backup storage end are determined.

[0063] In the case that the initial object metadata of the target objects of the two ends do not exist data locks, the target data written by the user is obtained according to the data write request.

[0064] The initial object metadata of the target objects of the two ends can be understood as the initial object metadata of the target object of the primary storage end and the initial object metadata of the target object of the backup storage end.

[0065] Specifically, after receiving the data write request sent by the first user, the object storage service platform will check whether the initial object metadata of the target objects of the primary storage end and the backup storage end exist data locks, and if the initial object metadata of the target objects of the primary storage end and the backup storage end do not exist data locks, it can be explained that the target objects of the primary storage end and the backup storage end do not exist other data processing operations at present. At this time, the object storage service platform will enter the streaming reading process of the target data, and the target data written by the first user is read in a streaming manner.

[0066] Further, in the case that the initial object metadata of the target objects of the primary storage end and the backup storage end exist data locks, but the data locks have been invalidated, the next data acquisition and writing process can be directly performed to ensure that only one data processing operation can be performed on the same target object at the same time. The specific implementation manner is as follows:

[0067] The target data written by the user is obtained according to the data write request through a preset reading mode, and the specific implementation manner is as follows:

[0068] The target object of the primary storage end and the target object of the backup storage end are determined.

[0069] In a case where the data lock of the initial object metadata of the target object at both ends is determined to be invalid, the target data written by the user is streamed according to the data write request.

[0070] The data lock can be understood as data lock expiration. For example, when the data lock is set, a life cycle of 5 minutes is configured for the data lock. After the life cycle of the data lock ends, the data lock is considered to be invalid.

[0071] Specifically, after the object storage service platform receives the data write request sent by the first user, the object storage service platform checks whether the initial object metadata of the target object at the main storage end and the backup storage end exists a data lock. If the initial object metadata of the target object at the main storage end and the backup storage end exists a data lock, but the data lock is expired, it can also be considered that the target object at the main storage end and the backup storage end currently does not exist other data processing operations. At this time, the object storage service platform enters the streaming reading process of the target data, and the target data written by the first user is streamed.

[0072] Step 204: determining the initial object metadata of the target object, and adding a data lock to the initial object metadata.

[0073] The data lock can be understood as a lock mark. The operation of adding the data lock can be understood as updating the old object meta (initial object metadata) of the target object at the storage end, and adding the data lock mark in the old object meta of the target object at the storage end.

[0074] Specifically, the determining the initial object metadata of the target object and adding the data lock to the initial object metadata comprises:

[0075] respectively determining the initial object metadata of the target object at the main storage end and the backup storage end;

[0076] In a case where the data write of the target data to the target object at the main storage end and the backup storage end is determined to be ended, a data lock is added to the initial object metadata of the target object at the main storage end and the backup storage end.

[0077] Specifically, the object storage service platform determines initial object metadata of the target object of the primary storage end, adds a data lock to the initial object metadata of the target object of the primary storage end in a case where it is determined that the target data is all written to the target object of the primary storage end, and simultaneously determines initial object metadata of the target object of the backup storage end, adds a data lock to the initial object metadata of the target object of the backup storage end in a case where it is determined that the target data is all written to the target object of the backup storage end, so as to add the data lock to the initial metadata of the target object of the two ends in this way, so as to ensure that the target object will not be occupied by other data processing operations in subsequent target data submission to the two ends, and ensure the stability of target data processing.

[0078] Step 206: sending a pre-commit request for the target data to the storage end.

[0079] Specifically, in a case where the storage end includes a primary storage end and a backup storage end, sending a pre-commit request for the target data to the storage end can be understood as that the object storage service platform concurrently pre-commits transactions to the primary storage end and the backup storage end. The specific implementation manner is as follows:

[0080] The sending of the pre-commit request for the target data to the storage end includes:

[0081] Respectively sending a pre-commit request for the to-be-written data to the primary storage end and the backup storage end.

[0082] In order to ensure the consistency of the target data stored by the primary storage end and the backup storage end, the object storage service platform concurrently pre-commits the primary storage end and the backup storage end, so that the primary storage end and the backup storage end can simultaneously pre-commit the target data.

[0083] Step 208: in a case where pre-commit metadata and a pre-commit success instruction returned by the storage end are received, sending a commit request for the target data to the storage end.

[0084] The pre-commit metadata is metadata of the target data to which a current version number is added, and the current version number is obtained by the storage end.

[0085] Specifically, the sending of the commit request for the target data to the storage end in a case where the pre-commit metadata and the pre-commit success instruction returned by the storage end are received includes:

[0086] receiving pre-commit metadata and a pre-commit success instruction for the target data returned by the primary storage end in response to the pre-commit request, and

[0087] In the case that the backup storage end returns the pre-commit metadata and the pre-commit success instruction for the target data in response to the pre-commit request, a commit request for the target data is sent to the storage end.

[0088] In the case that the storage end includes a main storage end and a backup storage end, the current version number is obtained from the object storage service platform by the main storage end and sent to the backup storage end.

[0089] That is, the current versionId is obtained from the version generator module of the object storage service platform by the source end; in actual application, the current versionId can be obtained from the object storage service platform by both the main storage end and the backup storage end; in order to ensure the consistency of the versionId increment, the current versionId is sent to the backup storage end while being obtained by the main storage end.

[0090] In addition, the pre-commit success instruction is generated after the target data is pre-committed successfully by the main storage end or the backup storage end.

[0091] In specific implementation, the object storage service platform sends a pre-commit request to the main storage end and the backup storage end respectively; after receiving the pre-commit request, the main storage end obtains the current version number from the object storage service platform and sends the current version number to the backup storage end.

[0092] After obtaining the current version number, the main storage end and the backup storage end both add the current version number to the metadata of the target data as pre-commit metadata; then the pre-commit metadata and the pre-commit success instruction for the target data are returned to the object storage service platform; in the case that the object storage service platform receives the pre-commit metadata and the pre-commit success instruction for the target data returned by the main storage end and the backup storage end, a commit request is sent to the main storage end and the backup storage end respectively.

[0093] In which, the pre-commit metadata submitted to the object storage service platform by the main storage end and the backup storage end can be used as a redo log to restore data when a failover occurs in the main storage end or the backup storage end, and maintain the data consistency of the main storage end and the backup storage end.

[0094] Specifically, the commit request for the target data sent to the storage end includes:

[0095] A commit request for the target data is sent to the main storage end and the backup storage end respectively.

[0096] In the embodiments of the present specification, when the object storage service platform synchronously stores target data to the main storage end and the backup storage end, a pre-commit stage is added, and in the case that the pre-commit is successful at both the main storage end and the backup storage end, a formal commit process of the target data is performed again, so as to ensure that in the case that the main storage end and the backup storage end appear to be down or other failures during the formal commit process of the target data, the pre-commit metadata can be used as a redo log to recover data, and the consistency of the target data stored in the main storage end and the backup storage end is maintained.

[0097] Step 210: In the case that the commit success instruction returned by the storage end is received, it is determined that the target data is written successfully.

[0098] Specifically, in the case that the commit success instruction for the target data returned by the storage end in response to the commit request is received, it is determined that the target data is written successfully, including:

[0099] In the case that the commit success instruction for the target data returned by the main storage end in response to the commit request is received, it is determined that the target data is written successfully; and / or

[0100] In the case that the commit success instruction for the target data returned by the backup storage end in response to the commit request is received, it is determined that the target data is written successfully.

[0101] The commit success instruction is that the main storage end or the backup storage end, in the case that the version number corresponding to the initial object metadata is determined to be less than or equal to the obtained current version number, the updated object metadata generated according to the initial object metadata of the target object and the metadata of the target object, and the target data are successfully committed.

[0102] In specific implementation, after the object storage service platform concurrently sends the commit request for the target data to the main storage end and the backup storage end, the main storage end and the backup storage end will strictly determine whether the version number corresponding to the initial object metadata of the target object is less than or equal to the obtained current version number, if yes, the updated object metadata of the target object is generated according to the initial object metadata of the target object and the metadata of the target object; and the obtained current version number is taken as the version number corresponding to the updated object metadata; and the updated object metadata is atomically written.

[0103] Among them, when the primary storage end and the backup storage end submit the updated object metadata and target data, there will be no data lock in the updated object metadata. After one end successfully submits the updated object metadata and target data, it will send a submission success instruction to the object storage service platform; and after receiving the submission success instructions from the primary storage end and the backup storage end, the object storage service platform will determine that the target data is successfully written to the primary storage end and the backup storage end synchronously, that is, the synchronous storage is successful.

[0104] After a client successfully submits the target data, the object storage service platform will release the data lock of the initial object metadata of the target object on that client and delete the pre-submitted metadata to the object storage service platform, so that the target object can continue to process other data. The specific implementation method is as follows:

[0105] After determining that the target data is written successfully, the method further includes:

[0106] Deleting the pre-committed metadata of the primary storage end, and releasing the data lock of the initial object metadata of the target object of the primary storage end; and / or

[0107] The pre-committed metadata of the backup storage end is deleted, and the data lock of the initial object metadata of the target object of the backup storage end is released.

[0108] Specifically, when the object storage service platform receives a successful submission instruction for the target data from a certain end, it will release the data lock of the initial object metadata of the target object on that end and delete the pre-submitted metadata submitted by that end to the object storage service platform, so that the target object can continue to perform other data processing operations.

[0109] The data processing method provided in the embodiments of this specification adopts the method of concurrently writing the target data to the primary storage end and the backup storage end, ensuring that the target data can be written to the primary storage end and the backup storage end at the same time at any time, solving the problem of short-term data inconsistency caused by asynchronous replication, and by adopting a two-stage submission method of pre-submission and submission, ensuring that when the submission on one end is successful and the submission on the other end fails, the pre-submitted data can be used as a redo log for data recovery to ensure the consistency of data on the primary storage end and the backup storage end when reading.

[0110] In addition, after the target data is synchronously written to the primary storage end and the backup storage end in the data processing method, when a data read request for the target object is received from another user, the target data of the target object can be obtained from the primary storage end and / or the backup storage end, and the target data can be returned to the client. The specific implementation method is as follows:

[0111] The method further comprises:

[0112] receiving a data read request for a target object of one of the primary storage end or the backup storage end sent by a second user;

[0113] determining object metadata of the target object of the one end according to the data read request;

[0114] reading target data of the target object from the one end in a case where it is determined that the object metadata of the target object of the one end does not exist data lock;

[0115] returning the target data to the second user.

[0116] Wherein, the first user and the second user can be the same user, or can be different users; in a case where the first user and the second user are the same user, first, the data is synchronously stored according to the above method, and then the synchronously stored data is read; in a case where the first user and the second user are different users, the data read is divided into multiple cases, for example, when the data is read, the object metadata of the target object of the two ends does not exist data lock, because the data is stored to the primary storage end or the backup storage end according to the data synchronous storage method in the above data processing method, the data will not exist inconsistency, then the data can be quickly read from any one of the primary storage end or the backup storage end.

[0117] Specifically, one of the primary storage end or the backup storage end can be understood as the primary storage end, or can be understood as the backup storage end, that is, in a case where the primary storage end and the backup storage end do not exist data lock, the target data of the target object can be read from the metadata of the target object of the primary storage end, or the target data of the target object can be read from the backup storage end.

[0118] When the data is read, the data is in the process of being synchronously stored to the primary storage end and the backup storage end, at this time, the object metadata of the target object of the primary storage end and the backup storage end has been locked, and the data lock is expired, then the data read process can be performed after the data of the primary storage end and the backup storage end is synchronously stored. The specific implementation mode is as follows:

[0119] After the object metadata of the target object of the one end is determined according to the data read request, the method further comprises:

[0120] in a case where it is determined that the object metadata of the target object of the one end exists data lock, and the data lock is not invalid, the data read request is suspended.

[0121] Or in the data reading, the data is in the process of synchronously storing to the primary storage end and the backup storage end, at this time, any one of the primary storage end and the backup storage end has successfully submitted the target data, the data lock of the object metadata of the target object has been released, and the data lock of the object metadata of the target object of the other end still exists, but the version number of the updated object metadata of the target object of one end is consistent with the version number in the pre-submission metadata of the other end, which can indicate that when the data is synchronously stored to both ends, the target data is lost due to the failure of the other end due to downtime or network failure. At this time, the lost target data needs to be found according to the pre-submission metadata of the other end, and the target data is submitted again according to the above submission process, and after the target data is successfully submitted, the target data is read from the target object of the primary storage end and / or the backup storage end. The specific implementation is as follows:

[0122] After determining the object metadata of the target object of the one end according to the data reading request, the method further comprises:

[0123] In a case where it is determined that the object metadata of the target object of the one end does not exist a data lock, and the object metadata of the target object of the other end exists an invalid data lock, determining the version number of the object metadata of the one end which does not exist a data lock, and the version number of the pre-submission metadata of the other end which exists an invalid data lock;

[0124] In a case where the version numbers of both ends are the same, determining the pre-submission metadata of the other end;

[0125] According to the pre-submission metadata of the other end, the target data of the other end is recovered;

[0126] In a case where a submission success instruction returned by the other end for the target data is received, reading the target data of the target object from the primary storage end and / or the backup storage end.

[0127] Wherein, reading the target data of the target object from the primary storage end and the backup storage end at the same time can be understood as that the target data of the target object read from the primary storage end and the backup storage end can be compared subsequently to verify the consistency of the data synchronously stored by the primary storage end and the backup storage end.

[0128] The data processing method provided by the embodiments of the present specification adopts the concurrent writing process of the master storage cluster and the backup storage cluster, ensures that data is written to both clusters at any time, and solves the problem of short-term inconsistency of data caused by asynchronous replication; by adopting the pre-commit and commit two-phase commit mode, in the case that one end is successfully committed and the other end fails, the pre-commit data can be used as a redo log for recovery, so that when data is read, according to the data lock of the object metadata of the target object of both ends, the target data is read in an appropriate way, and the consistency of the data of both ends when reading is ensured.

[0129] The following describes the embodiments of the present specification in conjunction with the accompanying Figure 3 The data processing method provided by the embodiments of the present specification is further described by taking an application of synchronously storing data to a source end and a peer end by using the data processing method as an example. Among them, Figure 3 FIG. 1 shows a processing process flow diagram of a data processing method provided by an embodiment of the present specification, which specifically includes the following steps.

[0130] Specifically, the data writing process of synchronous replication is as follows:

[0131] Step 302: The object storage service platform receives a data writing request sent by a user.

[0132] Specifically, the data writing request carries to-be-written data and a target object to which the user wants to write the to-be-written data, such as a storage space or a file name of the to-be-written data.

[0133] Step 304: The object storage service platform reads the to-be-written data of the user in a streaming manner.

[0134] Specifically, after receiving the data writing request sent by the user, the object storage service platform checks whether the old object meta (object metadata) of the target object of the source end (such as the master storage end) and the peer end (such as the backup storage end) has a data lock, and if there is a data lock on both ends (i.e., the source end and the peer end), but the data lock has expired, or there is no data lock on both ends, the object storage service platform reads the to-be-written data of the user in a streaming manner.

[0135] Among them, the data lock has expired, which can be understood as that the data lock will be considered to have expired after a certain time (such as 5 minutes or 10 minutes, etc.) has expired, and other data processing requests (such as other data writing requests) for the target object can continue to operate the target object. The purpose of setting the expiration of the data lock is to prevent the target object from being locked for a long time due to the server being down or network anomalies after the object metadata of the target object is locked, and to ensure the normal processing of subsequent data operations.

[0136] Step 306: The object storage service platform writes the stream read to-be-written data into the target object of the source end until the reading of the to-be-written data from the user ends.

[0137] Step 308: The object storage service platform writes the stream read to-be-written data into the target object of the opposite end until the reading of the to-be-written data from the user ends.

[0138] Step 310: The object storage service platform determines that all to-be-written data writing of the source end and the opposite end ends, and updates the old object meta of the target object of the source end, that is, adds a data lock to the old object meta of the target object.

[0139] Step 312: The object storage service platform updates the old object meta of the target object of the opposite end, that is, adds a data lock to the old object meta of the target object, while updating the old object meta of the target object of the source end, that is, adding a data lock to the old object meta of the target object.

[0140] Wherein, the data lock can be understood as a lock mark; specifically, after the to-be-written data of both ends is written, the object storage service platform can add a data lock to the old object meta of the target object of both ends, and the operation of adding a data lock can be understood as updating the old object meta of the target object of both ends, that is, adding a data lock mark in the old object meta of the target object of both ends.

[0141] Step 314: The object storage service platform receives the version number acquisition request sent by the source end, generates a globally self-incrementing current versionId (version number) according to the version number acquisition request, and sends the current versionId to the source end and the opposite end respectively.

[0142] Wherein, the current versionId is obtained by the source end from the version generator module of the object storage service platform; in actual application, the source end and the opposite end can obtain the current versionId from the object storage service platform, in order to ensure the consistency of the self-incrementing versionId, so the current versionId is sent to the opposite end while the source end obtains the current versionId.

[0143] In addition, after the object storage service platform successfully adds a data lock to the old object meta of the target object of the source end and the opposite end, it concurrently sends a pre-commit request to the source end and the opposite end.

[0144] Step 316: The source end receives the pre-commit request sent by the object storage service platform and executes the pre-commit request.

[0145] Specifically, after receiving the pre-commit request from the object storage service platform, the source end uses the metadata of the data to be written as pre-commit metadata (i.e., pre-commit meta), writes the obtained current versionId into the pre-commit metadata, and returns it to the object storage service platform. The metadata of the data to be written records information such as the address, offset, and length of the data to be written. The metadata of the data to be written can be used to find and read the corresponding data to be written later.

[0146] Step 318: The peer receives the pre-submission request sent by the object storage service platform and executes the pre-submission request.

[0147] Specifically, after receiving the pre-commit request sent by the object storage service platform, the peer uses the metadata of the data to be written as pre-commit meta, and writes the obtained current versionId into the pre-commit meta and returns it to the object storage service platform.

[0148] After receiving the pre-commit meta with the current versionId added from the source and peer ends, the object storage service platform determines that the concurrent pre-commit of the source and peer ends is successful, and concurrently sends commit requests for the data to be written to the source and peer ends respectively.

[0149] Step 320: The source end receives the submission request sent by the object storage service platform and executes the submission request.

[0150] Specifically, after receiving the commit request sent by the object storage service platform, the source side determines whether the versionId corresponding to the old object meta of the target object on the source side is less than or equal to the obtained current versionId. If so, it generates new object meta of the target object based on the old object meta of the target object and the metadata of the data to be written, and adds the current versionId to the new object meta as the version number corresponding to the new object meta.

[0151] At the same time, the new object meta is written atomically; that is, the new object meta is persistently stored in the system, indicating that the data to be written is written to the source target object and the commit is successful. At this time, the data lock of the old object meta of the source target object is released.

[0152] Step 322: The peer receives the submission request sent by the object storage service platform and executes the submission request.

[0153] Specifically, after receiving the commit request sent by the object storage service platform, the peer judges whether the versionId corresponding to the old object meta of the target object of the peer is less than or equal to the current versionId obtained, if yes, generates a new object meta of the target object according to the old object meta of the target object and the metadata of the data to be written, and adds the current versionId to the new object meta as the version number corresponding to the new object meta.

[0154] Meanwhile, the new object meta is atomically written, that is, the new object meta is persistently stored in the system, indicating that the data to be written is written into the target object of the peer and the commit is successful, at this time the data lock of the old object meta of the target object of the peer is released.

[0155] Step 324: After receiving the commit success instruction sent by the source end, the object storage service platform asynchronously deletes the pre-commit meta returned by the source end in the pre-commit stage.

[0156] Step 326: After receiving the commit success instruction sent by the peer, the object storage service platform asynchronously deletes the pre-commit meta returned by the peer in the pre-commit stage.

[0157] Specifically, when the user writes the data to be written into the source end and the peer, and the pre-commit of both ends is successful, if the peer commit is successful but the source end fails to commit successfully, the data write request of the source end fails; at this time, the target object of the source end exists an old object meta, the old object meta exists a data lock, and the pre-commit meta exists in the object storage service platform; while the target object of the peer exists a new object meta, the new object meta has no data lock, and the current versionId corresponding to the new object meta of the peer is consistent with the current versionId corresponding to the pre-commit meta of the source end, which indicates that the data to be written referenced by the source end and the peer is the same; then when the system recovers, the user tries to read the target object of the source end, the object storage service platform will redo according to the pre-commit meta of the source end according to the operation of step four in the reading process, to recover the data to be written which has not been committed by the source end, so as to ensure the consistency of the data read from both ends.

[0158] The data reading process of synchronous replication is as follows:

[0159] Step one: The object storage service platform receives the data reading request sent by the user.

[0160] Wherein, the target object can be understood as the storage space or file name of the source end and the opposite end.

[0161] Step two: the object storage service platform checks whether the initial object metadata of the target object of the source end or the opposite end exists data lock according to the target object carried in the data reading request, if the initial object metadata of the target object of the source end or the opposite end does not exist data lock, then directly read the target data of the target object of the source end or the opposite end, and return the read target data to the user.

[0162] Step three: the object storage service platform checks that the initial object metadata of the target object of the source end and the opposite end exists data lock, but the data lock has expired, which means that the write operation process for the to-be-written data has failed at the pre-commit stage, and the to-be-written data has not been committed, at this time, the old object meta of both ends needs to be updated to remove the lock information and restore to the original state, then the target data of the target object of the source end and the opposite end is read, and the read target data is returned to the user.

[0163] Step four: the object storage service platform checks that the initial object metadata of the target object of the source end exists expired data lock, the initial object metadata of the target object of the opposite end does not exist data lock, and the current versionId corresponding to the new object meta of the opposite end is consistent with the current versionId corresponding to the pre-commit meta of the source end, which means that the data synchronization write is performed on the opposite end and the source end, but the source end fails during data write, and the to-be-written data has not been committed, then in order to ensure the consistency of the data reading, the data write of the source end needs to be redone, that is, using the pre-commit meta of the source end to redo and restore the to-be-written data that has not been committed by the source end, so as to ensure the consistency of the read data of both ends.

[0164] After the source end commits successfully, the target data of the target object of the source end and the opposite end is read and compared, and in the case that the target data of both ends is consistent, the read target data is returned to the user, so as to ensure the consistency of the data reading.

[0165] Step five: the object storage service platform checks that the target object of the source end and the opposite end exists data lock, and the data lock has not expired, then it is judged that the object may be held by a certain request, then hang and retry.

[0166] If the target object is not held by other requests, it may occur in the above-mentioned data lock expiration case, at this time, the data is sorted according to the specific operation of the above-mentioned lock expiration, and then the data reading is performed, so as to ensure the consistency of the read data.

[0167] The data processing method provided by the embodiment of the present specification adopts the concurrent writing process of the master storage end cluster and the backup storage end cluster, ensures that data is written to both clusters at any time, and solves the problem of short-term inconsistency of data caused by asynchronous replication; by adopting the pre-commit and commit two-phase commit mode, in the case of successful submission on one end and failure on the other end, the pre-commit data can be used as a redo log for recovery, ensuring the consistency of data on both ends when reading.

[0168] Corresponding to the method embodiments described above, the present specification also provides data processing device embodiments, Figure 4 The structure diagram of a data processing device provided by one embodiment of the present specification is shown. As shown in the figure, Figure 4 The device comprises:

[0169] The write module 402 is configured to write target data to a target object of a storage end in response to a data write request, wherein the storage end comprises a master storage end and a backup storage end;

[0170] The locking module 404 is configured to determine initial object metadata of the target object and add a data lock to the initial object metadata;

[0171] The first request sending module 406 is configured to send a pre-commit request for the target data to the storage end;

[0172] The second request sending module 408 is configured to send a commit request for the target data to the storage end in the case of receiving pre-commit metadata and a pre-commit success instruction returned by the storage end, wherein the pre-commit metadata is metadata of the target data with a current version number added, and the current version number is obtained by the storage end;

[0173] The data write module 410 is configured to determine that the target data is written successfully in the case of receiving a commit success instruction returned by the storage end.

[0174] Optionally, the write module 402 is further configured to:

[0175] receive a data write request sent by a first user;

[0176] obtain target data written by the user through a preset reading mode according to the data write request;

[0177] write the target data to target objects of the master storage end and the backup storage end, respectively.

[0178] Optionally, the write module 402 is further configured to:

[0179] determine the target object of the primary storage end and the target object of the backup storage end;

[0180] in a case where it is determined that the initial object metadata of the target objects of both ends does not exist data lock, stream the target data written by the user according to the data write request.

[0181] Optionally, the write module 402 is further configured to:

[0182] determine the target object of the primary storage end and the target object of the backup storage end;

[0183] in a case where it is determined that the data lock of the initial object metadata of the target objects of both ends is invalid, stream the target data written by the user according to the data write request.

[0184] Optionally, the lock module 404 is further configured to:

[0185] determine the initial object metadata of the target objects of the primary storage end and the backup storage end, respectively;

[0186] in a case where it is determined that the target data is written into the target objects of the primary storage end and the backup storage end, add data locks to the initial object metadata of the target objects of the primary storage end and the backup storage end, respectively.

[0187] Optionally, the first request sending module 406 is further configured to:

[0188] send pre-commit requests for the target data to the primary storage end and the backup storage end, respectively;

[0189] Correspondingly, the second request sending module 408 is further configured to:

[0190] in a case where the pre-commit metadata returned by the primary storage end in response to the pre-commit request and the pre-commit success instruction for the target data are received, and

[0191] in a case where the pre-commit metadata returned by the backup storage end in response to the pre-commit request and the pre-commit success instruction for the target data are received, send a commit request for the target data to the storage end.

[0192] Optionally, the second request sending module 408 is further configured to:

[0193] send commit requests for the target data to the primary storage end and the backup storage end, respectively;

[0194] Correspondingly, the data writing module 410 is further configured to:

[0195] determine that the target data is written successfully in a case where a commit success instruction for the target data returned by the primary storage end in response to the commit request is received; and / or

[0196] determine that the target data is written successfully in a case where a commit success instruction for the target data returned by the backup storage end in response to the commit request is received;

[0197] The commit success instruction is that the primary storage end or the backup storage end, in a case where it is determined that the version number corresponding to the initial object metadata is less than or equal to the obtained current version number, will generate and successfully commit the target data according to the update object metadata generated based on the initial object metadata and the metadata of the target object.

[0198] Optionally, the apparatus further comprises:

[0199] a post-processing module configured to:

[0200] delete the pre-commit metadata of the primary storage end, and release the data lock of the initial object metadata of the target object of the primary storage end; and / or

[0201] delete the pre-commit metadata of the backup storage end, and release the data lock of the initial object metadata of the target object of the backup storage end.

[0202] Optionally, the apparatus further comprises:

[0203] a first data reading module configured to:

[0204] receive a data reading request for a target object of one of the primary storage end or the backup storage end sent by a second user;

[0205] determine object metadata of the target object of the one end according to the data reading request;

[0206] read target data of the target object from the one end in a case where it is determined that the object metadata of the target object of the one end does not exist a data lock;

[0207] return the target data to the second user.

[0208] Optionally, the apparatus further comprises:

[0209] a second data reading module configured to:

[0210] In a case where it is determined that the object metadata of the target object at the one end exists a data lock, and the data lock is not invalid, the data read request is suspended.

[0211] Optionally, the apparatus further comprises:

[0212] The third data reading module is configured to:

[0213] In a case where it is determined that the object metadata of the target object at the one end does not exist a data lock, and the object metadata of the target object at the other end exists an invalid data lock, determine the version number of the object metadata at the one end where no data lock exists, and the version number of the pre-commit metadata at the other end where the invalid data lock exists;

[0214] In a case where the version numbers at the two ends are the same, determine the pre-commit metadata at the other end;

[0215] Restore the target data of the other end according to the pre-commit metadata of the other end;

[0216] In a case where a commit success instruction returned by the other end for the target data is received, read the target data of the target object from the main storage end and / or the backup storage end.

[0217] The data processing apparatus provided by the embodiments of the present specification adopts concurrent writing of to-be-written data to the main storage end and the backup storage end, ensures that the to-be-written data can be written to the main storage end and the backup storage end at any moment, solves the problem of short-term inconsistency of data caused by asynchronous replication, and ensures that, in a case where the one end commits successfully while the other end fails to commit, the pre-committed data can be used as a redo log to perform data recovery, so as to ensure the consistency of the data of the main storage end and the backup storage end when reading.

[0218] The above is a schematic scheme of the data processing apparatus of the present embodiment. It should be noted that the technical scheme of the data processing apparatus and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0219] Figure 5 A structural block diagram of a computing device 500 according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 through a bus 530, and a database 550 is used to save data.

[0220] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or combinations of such networks, such as the Internet. The access device 540 can include one or more of any type of network interface (for example, a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a global system for mobile communications (GSM) interface, a code division multiple access (CDMA) interface, a Bluetooth interface, a near field communication (NFC) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a Wi-MAX interface, an Ethernet interface, a token ring interface, and / or the like.

[0221] In an embodiment of the present specification, the above-mentioned components of the computing device 500 and other components not shown in the Figure 5 may be connected to each other by a bus, for example. It should be understood that Figure 5 The computing device structure diagram shown is merely for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced by those skilled in the art as needed.

[0222] The computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (for example, a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, and the like), a mobile phone (for example, a smartphone), a wearable computing device (for example, a smart watch, smart glasses, and the like), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 500 can also be a mobile or stationary server.

[0223] The processor 520 is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data processing method.

[0224] The above is a schematic scheme of a computing device according to an embodiment of the present specification. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned data processing method belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the above-mentioned data processing method.

[0225] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned data processing method.

[0226] The above is a schematic solution of the computer readable storage medium of the embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the data processing method described above belong to the same concept, and the details of the technical solution of the storage medium which are not described in detail can be referred to the description of the technical solution of the data processing method.

[0227] An embodiment of the present specification further provides a computer program, which causes a computer to execute the steps of the data processing method when the computer program is executed in the computer.

[0228] The above is a schematic solution of the computer program of the embodiment. It should be noted that the technical solution of the computer program and the technical solution of the data processing method described above belong to the same concept, and the details of the technical solution of the computer program which are not described in detail can be referred to the description of the technical solution of the data processing method.

[0229] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0230] The computer instructions include computer program code, which can be in the form of source code, object code, executable code, or some intermediate form. The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0231] It should be noted that, for the aforementioned method embodiments, the sequences of the described actions are not necessarily required to implement the present application, and certain actions can be performed in other sequences, or even at the same time, in accordance with the present application. Furthermore, certain actions can not be required to implement the present application. Additionally, the described embodiments are not necessarily the only possible implementation of the present application.

[0232] In the above embodiments, the description of each embodiment is focused on the aspects of the embodiment, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0233] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and use the present application. The present application is limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: In response to a data write request, synchronously write the target data to a target object of a storage end, wherein the storage end includes a primary storage end and a backup storage end; Determining initial object metadata of the target object, and adding a data lock to the initial object metadata; Sending a pre-commit request for the target data to the storage end; Upon receiving the pre-submission metadata and the pre-submission success instruction returned by the storage end, sending a submission request for the target data to the storage end, wherein the pre-submission metadata is the metadata of the target data with the current version number added, and the current version number is obtained by the primary storage end and sent to the backup storage end; Upon receiving a commit success instruction returned by the storage end, determining that the target data has been written successfully, wherein the commit success instruction is for the primary storage end or the backup storage end, and upon determining that the version number corresponding to the initial object metadata is less than or equal to the obtained current version number, successfully committing the updated object metadata generated based on the initial object metadata and the metadata of the target object, and the target data; The pre-committed metadata of the storage end is deleted, and the data lock of the initial object metadata of the target object of the storage end is released.

2. The data processing method according to claim 1, wherein in response to a data write request, the step of synchronously writing the target data to the target object on the storage end comprises: receiving a data writing request sent by a first user; Acquiring the target data written by the user through a preset reading method according to the data writing request; The target data is written into the target objects of the primary storage end and the backup storage end respectively.

3. The data processing method according to claim 2, wherein obtaining the target data written by the user through a preset reading method according to the data writing request comprises: Determine the target object of the primary storage end and the target object of the backup storage end; When it is determined that the initial object metadata of the target objects at both ends do not have data locks, the target data written by the user is obtained in a streaming manner according to the data write request.

4. The data processing method according to claim 2, wherein obtaining the target data written by the user through a preset reading method according to the data writing request comprises: Determine the target object of the primary storage end and the target object of the backup storage end; When it is determined that the data locks of the initial object metadata of the target objects at both ends are invalid, the target data written by the user is obtained in a streaming manner according to the data write request.

5. The data processing method according to claim 1, wherein determining the initial object metadata of the target object and adding a data lock to the initial object metadata comprises: Determining initial object metadata of a target object on the primary storage end and the backup storage end respectively; When it is determined that writing of the target data into the target objects of the primary storage end and the backup storage end is completed, data locks are added to the initial object metadata of the target objects of the primary storage end and the backup storage end respectively.

6. The data processing method according to claim 1, wherein the sending of the pre-submission request for the target data to the storage end comprises: Sending a pre-commit request for the target data to the primary storage end and the backup storage end respectively; Accordingly, upon receiving the pre-submission metadata and the pre-submission success instruction returned by the storage end, sending a submission request for the target data to the storage end includes: Upon receiving the pre-commit metadata returned by the primary storage end in response to the pre-commit request and a pre-commit success instruction for the target data, and Upon receiving the pre-commit metadata returned by the backup storage end in response to the pre-commit request and a pre-commit success instruction for the target data, a commit request for the target data is sent to the storage end.

7. The data processing method according to claim 1, wherein the sending of a submission request for the target data to the storage end comprises: Sending a submission request for the target data to the primary storage end and the backup storage end respectively; Accordingly, upon receiving a submission success instruction for the target data returned by the storage end in response to the submission request, determining that the target data is successfully written includes: Upon receiving a commit success instruction for the target data returned by the primary storage end in response to the commit request, determining that the target data is written successfully; and / or When a submission success instruction for the target data is received, which is returned by the backup storage end in response to the submission request, it is determined that the target data is written successfully.

8. The data processing method according to claim 7, wherein the deleting the pre-commit metadata of the storage end and releasing the data lock of the initial object metadata of the target object of the storage end comprises: Deleting the pre-committed metadata of the primary storage end, and releasing the data lock of the initial object metadata of the target object of the primary storage end; and / or The pre-committed metadata of the backup storage end is deleted, and the data lock of the initial object metadata of the target object of the backup storage end is released.

9. The data processing method according to claim 1, further comprising: receiving a data read request for a target object on one of the primary storage end or the backup storage end, sent by a second user; Determining object metadata of the target object at the one end according to the data read request; When it is determined that the object metadata of the target object at the one end does not have a data lock, reading the target data of the target object from the one end; The target data is returned to the second user.

10. The data processing method according to claim 9, after determining the object metadata of the target object at the one end according to the data read request, further comprising: When it is determined that a data lock exists on the object metadata of the target object at the one end and the data lock has not expired, the data read request is suspended.

11. The data processing method according to claim 9, after determining the object metadata of the target object at the one end according to the data read request, further comprising: When it is determined that the object metadata of the target object at the one end has an expired data lock and the object metadata of the target object at the other end does not have a data lock, determine a version number of the object metadata at the end with the data lock and a version number of the pre-commit metadata at the other end without the data lock; When the version numbers of both ends are the same, determining the pre-commit metadata of the other end; Restoring target data of the other end according to the pre-committed metadata of the other end; When a submission success instruction returned by the other end for the target data is received, the target data of the target object is read from the primary storage end and / or the backup storage end.

12. A data processing device comprising: a writing module configured to synchronously write target data to a target object of a storage end in response to a data writing request, wherein the storage end includes a primary storage end and a backup storage end; a locking module, configured to determine initial object metadata of the target object and add a data lock to the initial object metadata; a first request sending module, configured to send a pre-submission request for the target data to the storage end; a second request sending module configured to send a submission request for the target data to the storage end upon receiving the pre-submission metadata and a pre-submission success instruction returned by the storage end, wherein the pre-submission metadata is the metadata of the target data with a current version number added, and the current version number is obtained by the storage end; a data writing module configured to, upon receiving a commit success instruction returned by the storage end, determine that the target data has been successfully written, wherein the commit success instruction is from the primary storage end or the backup storage end, and upon determining that the version number corresponding to the initial object metadata is less than or equal to the obtained current version number, successfully commit and generate updated object metadata generated based on the initial object metadata and the metadata of the target object, and the target data; The pre-committed metadata of the storage end is deleted, and the data lock of the initial object metadata of the target object of the storage end is released.

13. A computing device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the data processing method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 11.

Images