Data synchronization method, system and device

Abstract

Embodiments of the present specification provide a data synchronization method, system, and device, wherein the data synchronization method comprises: sending a log pull request to a log service node, wherein the log pull request carries a version number of a data storage space, and the version number has a corresponding log number in the log service node; receiving a target log record returned by the log service node based on the log pull request, wherein the log number of the target log record is greater than the log number corresponding to the version number in the log service node; and updating the data stored in the data storage space and the version number by executing the target log record, so that the updated version number corresponds to the latest log number of the target log record.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of data processing technology, and in particular to a data synchronization method, system, and device. Background Art

[0002] With the development of the Internet, the amount of data in various online projects is increasing. When the amount of data is large and the storage service node storing the data fails and needs to be switched, the full amount of data needs to be transferred between the storage service nodes.

[0003] However, the transmission of full data may cause network delays and result in service unavailability. Therefore, a more reliable solution is needed. Summary of the invention

[0004] In view of this, the embodiments of this specification provide two data synchronization methods. One or more embodiments of this specification also involve two data synchronization devices, a data synchronization system, two computing devices, and two computer-readable storage media to solve the technical defects existing in the prior art.

[0005] A first aspect of an embodiment of this specification provides a data synchronization method, including:

[0006] Sending a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node;

[0007] Receiving a target log record returned by the log service node based on the log pull request, wherein a log sequence number of the target log record is greater than a log sequence number corresponding to the version sequence number in the log service node;

[0008] By executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record.

[0009] Optionally, when there are multiple target log records, updating the data stored in the data storage space and the version number by executing the target log record includes:

[0010] Determine the execution order of the target log records according to the order of the log sequence numbers from small to large;

[0011] Based on the execution order, the target log record is executed to update the data stored in the data storage space and the version number.

[0012] Optionally, updating the data stored in the data storage space and the version number by executing the target log record includes:

[0013] For a first log record in the target log record, the first log record is executed to update the data stored in the data storage space and update the version number of the data storage space based on the log sequence number of the first log record.

[0014] Optionally, the data synchronization method further includes:

[0015] Receive data change requests submitted by users;

[0016] The data change request is sent to the log service node, so that the log service node generates a corresponding second log record based on the data change request.

[0017] Optionally, the data synchronization method further includes:

[0018] Sending a first log pull request to the log service node, wherein the log pull request carries a first version number of the data storage space;

[0019] Receiving a first target log record returned by the log service node based on the first log pull request, wherein the log sequence number of the first target log record is greater than the log sequence number corresponding to the first version sequence number, and the first target log record includes the second log record;

[0020] By executing the first target log record, the data stored in the data storage space and the first version serial number are updated so that the updated version serial number corresponds to the latest log serial number of the first target log record;

[0021] The update result obtained by executing the second log record and the target version number of the data storage space are returned to the user as the execution result of the data change request.

[0022] Optionally, the data synchronization method further includes:

[0023] Receive a data query request submitted by a user, wherein the data query request carries a query condition and a target version number;

[0024] Determine whether the target version number is greater than the log number corresponding to the version number of the data storage space;

[0025] If yes, execute the step of sending a log pull request to the log service node;

[0026] Correspondingly, after the data stored in the data storage space and the serial number of the version data storage space are updated, the method further includes:

[0027] Searching the updated data storage space for first target data corresponding to the query condition;

[0028] The first target data is returned to the user.

[0029] Optionally, if the result of determining whether the target version number is greater than the log number corresponding to the version number of the data storage space is no, perform the following operations:

[0030] Determine whether there is a data snapshot corresponding to the target version serial number;

[0031] If so, searching the data snapshot for second target data corresponding to the query condition;

[0032] The second target data is returned to the user.

[0033] Optionally, sending a log pull request to a log service node includes:

[0034] When receiving a data query request submitted by a user, a log pull request is sent to the log service node;

[0035] Correspondingly, after the data stored in the data storage space and the version number are updated, the method further includes:

[0036] According to the query condition carried in the data query request, query the data storage space for third target data corresponding to the query condition;

[0037] The third target data is returned to the user.

[0038] Optionally, before updating the data stored in the data storage space and the version number by executing the target log record, the method further includes:

[0039] Obtaining the current version serial number of the data storage space;

[0040] In the case where the current version number is greater than the version number, a second target log record whose log number is greater than the log number corresponding to the current version number is filtered out from the target log record, and the second target record is used as the target log record, and the current version number is used as the version number.

[0041] Optionally, before updating the data stored in the data storage space and the version number by executing the target log record, the method further includes:

[0042] Among the target log records, log records whose target log records are not empty are screened out as the target log records.

[0043] Optionally, the data synchronization method further includes:

[0044] Generate a target data snapshot of the data storage space according to a preset snapshot generation strategy;

[0045] A log deletion instruction is sent to the log service node, wherein the log deletion instruction carries the second version sequence number corresponding to the target data snapshot, so that the log service node deletes historical log records whose log sequence numbers are less than the log sequence number corresponding to the second version sequence number based on the log deletion instruction.

[0046] Optionally, the data synchronization method further includes:

[0047] In the case of adding a new target storage service node, receiving a data synchronization request for the target storage service node;

[0048] Based on the data synchronization request, the target data snapshot is sent to the target storage service node, so that the target storage service node serves as the storage service node, and the second version number corresponding to the target data snapshot is used as the version number, and the step of sending a log pull request to the log service node is executed. A second aspect of the embodiment of this specification provides another data synchronization method, including:

[0049] Receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space;

[0050] Based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number;

[0051] The target log record is returned to the storage service node, so that the storage service node updates the data and version number stored in the data storage space by executing the target log record.

[0052] Optionally, the data synchronization method further includes:

[0053] Receive a data change request submitted by the storage service node, wherein the data change request carries data update operation information;

[0054] Based on the data change operation information, a first log record corresponding to the data change operation information is generated.

[0055] Optionally, the data synchronization method further includes:

[0056] Receiving a log deletion instruction sent by the storage service node;

[0057] Based on the target time point carried in the log deletion instruction, historical log records whose recording time is less than the target time point are deleted.

[0058] A third aspect of the embodiments of this specification provides a data synchronization system, including:

[0059] Storage service nodes and log service nodes;

[0060] The storage service node is configured to send a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node;

[0061] The log service node is configured to receive the log pull request, and based on the log pull request, pull a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number, and return the target log record to the storage service node;

[0062] The storage service node is configured to receive a target log record returned by the log service node based on the log pull request, and update the data stored in the data storage space and the version number by executing the target log record, so that the updated version number corresponds to the latest log number of the target log record.

[0063] A fourth aspect of the embodiments of this specification provides a data synchronization device, including:

[0064] A sending request module is configured to send a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node;

[0065] A record receiving module is configured to receive a target log record returned by the log service node based on the log pull request, wherein the log sequence number of the target log record is greater than the log sequence number corresponding to the version sequence number in the log service node;

[0066] The update module is configured to update the data stored in the data storage space and the version number by executing the target log record, so that the updated version number corresponds to the latest log number of the target log record.

[0067] According to a fifth aspect of the embodiments of this specification, another data synchronization device is provided, including:

[0068] A request receiving module is configured to receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space;

[0069] A record acquisition module is configured to acquire, based on the log pull request, a target log record with a log sequence number greater than the log sequence number corresponding to the version sequence number;

[0070] The return module is configured to return the target log record to the storage service node, so that the storage service node updates the data and version number stored in the data storage space by executing the target log record.

[0071] According to a sixth aspect of the embodiments of this specification, a computing device is provided, including:

[0072] Memory and processor;

[0073] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions:

[0074] Sending a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node;

[0075] Receiving a target log record returned by the log service node based on the log pull request, wherein a log sequence number of the target log record is greater than a log sequence number corresponding to the version sequence number in the log service node;

[0076] By executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record.

[0077] According to a seventh aspect of the embodiments of this specification, a computing device is provided, including:

[0078] Memory and processor;

[0079] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions:

[0080] Receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space;

[0081] Based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number;

[0082] The target log record is returned to the storage service node, so that the storage service node updates the data and version number stored in the data storage space by executing the target log record.

[0083] In an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the data synchronization method are implemented.

[0084] A ninth aspect of the embodiments of this specification provides another computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of another data synchronization method.

[0085] The present specification provides a data synchronization method, wherein the execution subject is a storage service node, which cooperates with a log service node to send a log pull request to the log service node, wherein the log pull request carries a version number of a data storage space, and the version number has a corresponding log number in the log service node; a target log record returned by the log service node based on the log pull request is received, wherein the log number of the target log record is greater than the log number corresponding to the version number in the log service node; and data stored in the data storage space and the version number are updated by executing the target log record, so that the updated version number corresponds to the latest log number of the target log record, thereby achieving consistency correspondence control through the version number and the log number when the data storage service (such as provided by a cluster of storage service nodes) and the log service (such as a cluster of log service nodes) are separated, and incremental log records are pulled, and the data stored in the data storage space and the version number are updated by executing the incremental log record. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 is a processing flow chart of a data synchronization method provided by an embodiment of this specification;

[0087] Figure 2 It is a schematic diagram of the architecture of a data synchronization method provided by an embodiment of this specification;

[0088] Figure 3 is a processing flow chart of another data synchronization method provided by an embodiment of this specification;

[0089] Figure 4 is an interactive processing flow chart of a data synchronization method provided by an embodiment of this specification;

[0090] Figure 5is an interactive processing flow chart of a data synchronization method based on a data change request provided by an embodiment of the present specification;

[0091] Figure 6 It is a structural diagram of a data synchronization system provided by an embodiment of this specification;

[0092] Figure 7 is a schematic diagram of a data synchronization device provided by an embodiment of this specification;

[0093] Figure 8 is a schematic diagram of another data synchronization device provided by an embodiment of this specification;

[0094] Fig. 9 is a structural block diagram of a computing device provided by one embodiment of this specification;

[0095] Fig.10 It is a structural block diagram of another computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0096] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0097] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0098] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0099] In this specification, two data synchronization methods are provided. One or more embodiments of this specification also involve two data synchronization devices, a data synchronization system, two computing devices, and two computer-readable storage media, which are described in detail one by one in the following embodiments.

[0100] An embodiment of a data synchronization method provided in this specification is as follows:

[0101] Figure 1 A processing flow chart of a data synchronization method provided according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0102] Step 102: Send a log pull request to the log service node.

[0103] The log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node.

[0104] The executor of the method embodiment of this specification is a storage service node, and the executor of another method embodiment of this specification is a log service node. This specification realizes updating of data stored in a data storage device by executing newly added log records when the data storage service (provided by the storage service node) and the log service (provided by the log service node) are separated through the cooperation between the storage service node and the log service node.

[0105] Among them, the storage service node refers to a computing device that includes data storage space. In actual applications, the number of storage service nodes can be one or more. When the data storage volume is large and the data update frequency is high, in order to ensure data access efficiency, multiple storage service clusters composed of multiple storage service nodes can be used to store data, and each storage service cluster can be deployed in at least two geographical areas, and the geographical area can be an area based on cities, provinces, or countries, etc., and each storage service cluster can include a front-end service node and at least one back-end storage service node, wherein the front-end service node is used for external interaction (for example, interaction with users or interaction with log service nodes), and the back-end storage service node is used to store and update data.

[0106] The data storage space can be understood as a non-relational database, or a distributed database or other database for providing data storage, and the data storage space has a corresponding version number, which is used to mark a version of the data storage space. In actual applications, each time a data change operation is performed on the data stored in the data storage space, the version number of the data storage space will be updated. Specifically, the data change operation includes but is not limited to: data creation operation, data deletion operation and data modification operation.

[0107] The log service node refers to a computing device used to store log records. In actual applications, the number of log service nodes can be one or more. The deployment of log service nodes is similar to the deployment of the above storage service nodes, which will not be described in detail here. In the case where the log service node is a log service cluster composed of multiple log service nodes, the log service nodes in the log service cluster include a master log service node and at least one slave log service node. The log consistency is achieved between the multiple log service nodes by running a distributed consistency protocol. Specifically, the distributed consistency protocol includes but is not limited to: Paxos protocol, Zookeeper Atomic Broadcast protocol, Raft protocol and ChainReplication protocol.

[0108] The log pull request refers to a pull request for requesting to pull newly added log records from the log service node. Specifically, the log record is used to record data change operations on data stored in the data storage space, and each daily record has a corresponding log sequence number. The log sequence number is used to uniquely mark a log record. The log pull request carries the version number of the data storage space because each version number has a corresponding log sequence number in the log service node, and the version number and the log sequence number both contain a string of incrementable values. When pulling log records based on the log pull request, it is necessary to compare the log sequence number corresponding to the version number with the log sequence number to pull the newly added log record. For example, if the version number of the data storage space is "0023", and the log sequence number corresponding to the version number in the log service node is "0023", it means that when pulling logs, the log record that needs to be pulled is the log record with a log sequence number greater than "0023" (i.e., the newly added target log record).

[0109] It should be noted that, in the embodiments of this specification, since the data storage service (provided by the storage service node) and the log service (provided by the log service node) are in a separate state, in order to ensure the consistency between the data stored in the data storage space and the log records stored in the log service node, it is necessary to continuously pull new log records from the log service node, and synchronize the data update operation to the data storage space by executing the new log records.

[0110] Furthermore, according to the needs of the actual scenario, the trigger conditions for sending the log pull request can be preset. When the preset trigger conditions are met, the log pull request is sent to the log service node. In actual applications, the preset trigger conditions can be various, for example, receiving a data change request for data in the data storage space is used as a trigger condition, and / or a preset time period (every minute, every hour, every day, etc.) is used as a trigger condition, etc., which is not limited here.

[0111] In a specific implementation, in order to ensure that the data queried by the user is not expired data, in an optional implementation mode provided in the embodiment of this specification, the sending of the log pull request to the log service node is specifically implemented in the following manner:

[0112] When receiving a data query request submitted by a user, a log pull request is sent to the log service node.

[0113] The embodiment of this specification uses a data query request as a trigger condition for sending a log pull request to a log service node. When a data query request submitted by a user is received, a log pull request is sent to the log service node.

[0114] Specifically, the data query request refers to a query request for data stored in the data storage space. Each time a data query request from a user is received, a log pull request is sent to the log service node, so that the data stored in the data storage space can be updated according to the pulled log records, and then a data query is performed on the updated data, thereby ensuring that the data queried by the user is updated data.

[0115] In the case where there are multiple storage service nodes and log service nodes, and the nodes are deployed in different cities, Figure 2 As shown, the storage service node includes 3 back-end storage service nodes, each of which is deployed in Beijing, Shanghai and Hangzhou respectively, and a front-end service node is deployed in each region, and the number of log service nodes is 3, which are deployed in Beijing, Shanghai and Shenzhen respectively. Among them, the log service node deployed in Shanghai is the main log service node, and the log service nodes deployed in Beijing and Shenzhen are slave log service nodes. The synchronization between these log service nodes adopts a distributed consistency protocol to achieve log consistency. The front-end service nodes deployed in various regions in the storage service node can receive user requests, and upon receiving user requests, send log pull requests to the main log service node deployed in Shanghai. The main log service node pulls the target log record based on the log pull request, and returns the pulled target log record to the front-end service node. The front-end service node sends the received target log record to the back-end storage service node. The back-end storage service node updates the data stored in the data storage space by executing the target log record.

[0116] Step 104: Receive the target log record returned by the log service node based on the log pull request.

[0117] The log sequence number of the target log record is greater than the log sequence number corresponding to the version sequence number in the log service node.

[0118] Specifically, after sending a log pull request to the log service node, the log service node pulls the target log record based on the log pull request. In actual applications, it is possible that multiple target log records have been added between two log pulls. Therefore, the log records can be pulled in batches based on the log pull request. The log sequence number of the pulled target log record must be greater than the log sequence number corresponding to the version number in the log pull request. This is because the log sequence number corresponding to the version number of the data storage space indicates which log record has been executed for the data storage space. In this case, the log record (target log record) added after the last execution is completed can be pulled, and the pulled target log record can be returned to the storage service node. The storage service node receives the target log record returned by the log service node.

[0119] Continuing with the above example, assuming that the version number of the data storage space carried in the log pull request is "0023", the log number corresponding to the version number is "0023", and the log service node pulls the target log record with a log number greater than "0023" based on this log pull request, and returns the target log record to the storage service node. The storage service node receives the returned target log record, where the target log record contains 2 log records, and the log numbers corresponding to these 2 log records are: "0024" and "0025" respectively.

[0120] In practical applications, considering that there may be reasons such as network delay, which may cause the log pulled by the log pull request to contain executed log records (that is, the log records pulled by executing other log pull requests, and the data in the corresponding data storage space has been updated), in order to avoid repeated execution of old log records, it is necessary to filter out the executed log records in the pulled log records, filter out the unexecuted log records, and update the data in the data storage space. In an optional implementation manner provided by the embodiment of this specification, before the data stored in the data storage space and the version number are updated by executing the target log record, it also includes:

[0121] Obtaining the current version serial number of the data storage space;

[0122] In the case where the current version number is greater than the version number, a second target log record whose log number is greater than the log number corresponding to the current version number is filtered out from the target log record, and the second target record is used as the target log record, and the current version number is used as the version number.

[0123] In specific implementation, when the current version number is greater than the version number, it indicates that the data storage space has been updated in the process of pulling log records in the above-mentioned data pull request. Therefore, in order to avoid repeated updates of the data stored in the data storage space, it is necessary to filter the pulled target log records, and filter out the second target log records (i.e., the newly added log records compared to the data currently stored in the data storage space) whose log numbers are greater than the log numbers corresponding to the current version number, and use the second target log records as the target log records, and use the current version number as the version number, so as to execute the following step 106.

[0124] Using the above example, the version number is "0023" and the received target log record contains 2 log records, the log numbers corresponding to these 2 log records are "0024" and "0025" respectively. The current version number of the data storage space is obtained as "0024". The current version number is greater than the above version number, and the log number corresponding to the current version number is "0024". Then, the log records with log numbers greater than "0024" are selected from the target log records as the log records with log number "0025", and the log record with log number "0025" is used as the target log record, and the current version number "0024" is used as the version number.

[0125] In addition, it is also possible that invalid log records are generated due to the downtime of the log service node. In this case, it is necessary to filter out the invalid log records contained in the pulled target log records and screen out the valid log records to improve the efficiency of executing log records. In an optional implementation manner provided by the embodiment of this specification, before the data stored in the data storage space and the version number are updated by executing the target log record, it also includes:

[0126] Among the target log records, log records whose target log records are not empty are screened out as the target log records.

[0127] In actual applications, the target log record is an empty log record, that is, an invalid log record. By screening out the target log records that are not empty, that is, valid log records, as the target log records, the following step 106 is performed.

[0128] Step 106: Update the data stored in the data storage space and the version number by executing the target log record, so that the updated version number corresponds to the latest log number of the target log record.

[0129] Specifically, the latest log sequence number refers to the log record with the largest log sequence number in the target log records.

[0130] In actual applications, executing target log records means executing the data update operation recorded in the target log records. By executing the data update operation, the data stored in the data storage space and the version number are updated. When all target log records are executed, the updated version number corresponds to the latest log number of the target log record.

[0131] In specific implementation, when the log sequence number is an increasing sequence, the size of the log sequence number indicates the order in which the log records are generated, that is, the log record with a smaller log sequence number is an earlier recorded data update operation, and thus needs to be executed first to ensure the correctness of the data stored in the updated data deposit space. In an optional implementation provided by the embodiment of this specification, when there are multiple target log records, the data stored in the data storage space and the version number are updated by executing the target log record, which is specifically implemented in the following manner:

[0132] Determine the execution order of the target log records according to the order of the log sequence numbers from small to large;

[0133] Based on the execution order, the target log record is executed to update the data stored in the data storage space and the version number.

[0134] In the embodiments of this specification, the execution order of the target log records is determined according to the order of the log sequence numbers from small to large, which means that the target log records with smaller log sequence numbers are executed first, that is, the target log records are executed in sequence starting from the next log record of the log sequence number corresponding to the version number, so as to update the data stored in the data storage space and the version number, thereby ensuring that the data change operations are executed in order according to the order of submission time.

[0135] Using the above example, if the target log record contains two log records, and the log numbers corresponding to these two log records are "0024" and "0025" respectively, the execution order of these two log records is determined according to the order of log numbers from small to large. The log record with log number "0024" is executed first, and then the log record with log number "0025" is executed. In accordance with the determined execution order, the data stored in the data storage space and the version number are updated.

[0136] In the case of receiving a data query request from a user in the above step 102, on the basis of sending a log pull request to the log service node, after updating the data in the data storage space by executing the pulled log record, querying the updated data based on the user's query request, and returning the query result to the user, so that the data queried by the user is the latest data, increasing the validity of the query and improving the user's query experience. In an optional implementation manner provided by the embodiment of this specification, after the data stored in the data storage space and the version number are updated, it also includes:

[0137] According to the query condition carried in the data query request, query the data storage space for third target data corresponding to the query condition;

[0138] The third target data is returned to the user.

[0139] Specifically, the query condition refers to the query condition for querying the data stored in the data storage space, for example, the query condition is to query the sales volume of mobile phone model t1, or to query the user information of user U1, etc., which is not limited here.

[0140] In the embodiment of the present specification, after receiving a data query request, the latest log record is first pulled, the data stored in the data storage space corresponding to the log record is updated by executing it, and the updated data is queried based on the query conditions to obtain the query result and return the query result, that is, the third target data is returned to the user to avoid the user querying outdated data.

[0141] Further, in an optional implementation manner provided by the embodiments of this specification, the updating of the data stored in the data storage space and the version number by executing the target log record is specifically implemented in the following manner:

[0142] For a first log record in the target log record, the first log record is executed to update the data stored in the data storage space and update the version number of the data storage space based on the log sequence number of the first log record.

[0143] The first log record refers to any log record in the target log record. In specific implementation, only when the data update and the version number update are both executed successfully, can it be indicated that the log record is successfully executed. Therefore, the data update and the version number update can be implemented through transactions or through batch write operations. That is, if only the data update is successful or only the version update is successful, the update operation needs to be rolled back, indicating that the log record has not been successfully executed, and the log record needs to be re-executed.

[0144] In actual applications, since the storage service is separated from the log service, the data stored in the data storage space is updated by executing log records. Therefore, when a data change request submitted by a user is received, it is necessary to first generate a new log record according to the log change request, and then execute the new log record to update the data stored in the data storage space, so as to achieve synchronization between the log and the data when receiving the data change request. In an optional implementation manner provided in the embodiment of this specification, the data synchronization method further includes:

[0145] Receive data change requests submitted by users;

[0146] The data change request is sent to the log service node, so that the log service node generates a corresponding second log record based on the data change request.

[0147] Specifically, the data change request refers to a change request that carries a data update operation. In actual applications, after the storage service node receives the data change request, the data change request will be sent to the log service node, so that the log service node generates a second log record, i.e., a log record corresponding to the data change operation, by recording the data change operation information carried in the data change request.

[0148] Further, based on the above generation of the second log record, in an optional implementation manner provided by the embodiment of this specification, the data synchronization method further includes:

[0149] Sending a first log pull request to the log service node, wherein the log pull request carries a first version number of the data storage space;

[0150] Receiving a first target log record returned by the log service node based on the first log pull request, wherein the log sequence number of the first target log record is greater than the log sequence number corresponding to the first version sequence number, and the first target log record includes the second log record;

[0151] By executing the first target log record, the data stored in the data storage space and the first version serial number are updated so that the updated version serial number corresponds to the latest log serial number of the first target log record;

[0152] The update result obtained by executing the second log record and the target version number of the data storage space are returned to the user as the execution result of the data change request.

[0153] Specifically, the first log pull request is similar to the log pull request, and is used to obtain the newly added log record in the log service node; the first target log record is a log record containing the above-mentioned second log record. In actual applications, after receiving the data change request, the newly added log record is pulled, and the newly added log record contains the second log record; the first version number is similar to the above version number, and the above version number can be referred to, and no further details are given here.

[0154] In addition, the specific implementation of the above-mentioned step of sending the first log pull request to the log service node to executing the first target log record to update the data stored in the data storage space and the version number is similar to the above-mentioned steps 102 to 106. The specific implementation of the above-mentioned steps 102 to 106 can be referred to, and will not be repeated here.

[0155] After updating the data stored in the data storage space and the first version number, the execution result of the data change request needs to be returned to the user. Since the first target log record contains the second log record corresponding to the data change operation information carried in the data change request, the second log record will be executed in the process of executing the first target log record, that is, the data change operation information carried in the data change request will be executed, the data stored in the data storage space will be changed, and the corresponding update result will be obtained. The version number of the data storage space (that is, the first version number) is updated to obtain the target version number (this target version number corresponds to the log number of the second log record), and the update result and the target version number are returned to the user as the execution result of the data change request.

[0156] It should be noted that the target version number is returned to the user so that the user can view the data corresponding to the target version number according to the target version number, or to prevent the user from viewing data that is outdated relative to the data corresponding to the target version number.

[0157] When the user obtains the target version number, the user can carry the target version number for querying, so as to avoid the user from querying expired data, thereby improving the accuracy of the query. In an optional implementation manner provided by the embodiment of this specification, the data synchronization method also includes:

[0158] Receive a data query request submitted by a user, wherein the data query request carries a query condition and a target version number;

[0159] Determine whether the target version number is greater than the log number corresponding to the version number of the data storage space;

[0160] If yes, it indicates that the data currently stored in the data storage space is expired data, and the step of sending a log pull request to the log service node is executed;

[0161] Correspondingly, after the data stored in the data storage space and the serial number of the version data storage space are updated, the method further includes:

[0162] Searching the updated data storage space for first target data corresponding to the query condition;

[0163] The first target data is returned to the user.

[0164] In specific implementation, whether the target version number is greater than the version number of the data storage space is determined in order to determine whether the data currently stored in the data storage space is expired data. If so, it indicates that the data currently stored in the data storage space is expired data, and then the above steps 102 to 106 are executed to pull the latest log record from the log service node to update the data stored in the data storage space, and after the update, query the updated data storage space, and return the first target data corresponding to the query condition, that is, the query result, to the user.

[0165] Further, when the user obtains the target version number and there is a data snapshot corresponding to the target version number, the user can query the data snapshot corresponding to the target version number, thereby improving the accuracy of the query. In an optional implementation manner provided by the embodiment of this specification, if the result of the judgment of whether the target version number is greater than the log number corresponding to the version number of the data storage space is no, the following operations are performed:

[0166] Determine whether there is a data snapshot corresponding to the target version serial number;

[0167] If it exists, it indicates that a data snapshot is generated for the data corresponding to the target version serial number, and the second target data corresponding to the query condition is searched in the data snapshot;

[0168] returning the second target data to the user;

[0169] If it does not exist, indicating that no data snapshot has been generated for the target version number, the fourth target data corresponding to the query condition is directly queried in the data storage space, and the fourth target data is returned to the user.

[0170] Specifically, the data snapshot refers to a snapshot generated for the data stored in the data storage space of the target version number. If there is a data snapshot corresponding to the target version number, the data snapshot is queried and the query result, i.e., the second target data, is returned to the user. If there is no data snapshot corresponding to the target version number, the data stored in the data storage space is queried and the query result, i.e., the fourth target data, is returned to the user.

[0171] In practical applications, there are various strategies for generating data snapshots. In an optional implementation provided by the embodiment of this specification, the data synchronization method further includes:

[0172] Generate a target data snapshot of the data storage space according to a preset snapshot generation strategy;

[0173] A log deletion instruction is sent to the log service node, wherein the log deletion instruction carries the second version sequence number corresponding to the target data snapshot, so that the log service node deletes historical log records whose log sequence numbers are less than the log sequence number corresponding to the second version sequence number based on the log deletion instruction.

[0174] Specifically, the snapshot generation strategy can be set according to the needs of the actual scenario, such as periodically generating data snapshots (generating a data snapshot on the 1st of each month, or generating a data snapshot every Wednesday, etc.), generating data snapshots based on trigger instructions submitted by users, or generating data snapshots when the version number of the data storage space is incremented to a preset version number, etc., without limitation here; the recording time refers to the generation time of the log record.

[0175] In a specific implementation, generating the target data snapshot of the data storage space refers to backing up the data stored in the data storage space at a certain point in time, so as to restore the data through the target data snapshot, or query the target data snapshot.

[0176] In order to prevent the log service node from storing too many expired log records and thus occupying too many device resources, in the embodiment of the present specification, after generating the target data snapshot, a log deletion instruction is sent to the log service node, so that the log service node deletes the historical log records whose log sequence numbers are smaller than the log sequence numbers corresponding to the second version sequence number based on the log deletion instruction. This is because the data corresponding to the second version sequence number (i.e., the target data snapshot) has been backed up in the storage service node, and the historical log records whose log sequence numbers are smaller than the log sequence numbers corresponding to the second version sequence number are used to update the data corresponding to the version number before the second version sequence number. Therefore, if the data corresponding to the second version sequence number has been backed up, there is no need to retain the above historical log records.

[0177] In addition, the target time point corresponding to the target data snapshot may also be carried in the log deletion instruction, so that the log service node deletes historical log records whose recording time is less than the target time point based on the log deletion instruction.

[0178] Further, after the target data snapshot is generated, and when at least one target storage service node needs to be added in the storage service node, since there is no data in the data storage space of the target storage service node, it is necessary to synchronize data from the storage service node to the target storage service node. In an optional implementation manner provided in the embodiment of this specification, the data synchronization method further includes:

[0179] In the case of adding a new target storage service node, receiving a data synchronization request for the target storage service node;

[0180] Based on the data synchronization request, the target data snapshot is sent to the target storage service node, so that the target storage service node acts as the storage service node, and the second version number corresponding to the target data snapshot is used as the version number, and the step of sending a log pull request to the log service node is executed.

[0181] In actual applications, when a new target storage service node is added and a target data snapshot exists in the storage service node, the target data snapshot in the storage service node is first synchronized to the newly added target storage service node, and then the target storage service node pulls the log records newly added after the target data snapshot from the log service node, so that the target storage service node generates full data in the data storage space through the target data snapshot and the newly added log records to complete data synchronization. That is, there is no need to transmit the full data between nodes, nor is there any need to transmit the data for providing storage services online, which increases the synchronization efficiency between nodes and reduces the adverse effects on the storage service nodes providing storage services during the synchronization process.

[0182] In specific implementation, when there are multiple data snapshots in the storage service node, if no specific data snapshot is specified in the data synchronization request, the most recently generated data snapshot (i.e. the data snapshot with the largest version number) is usually used as the target data snapshot and synchronized to the target storage service node.

[0183] In summary, the present specification provides a data synchronization method, wherein the execution subject is a storage service node, and by cooperating with a log service node, a log pull request is sent to the log service node, wherein the log pull request carries a version number of a data storage space, and the version number has a corresponding log number in the log service node; a target log record returned by the log service node based on the log pull request is received, wherein the log number of the target log record is greater than the log number corresponding to the version number in the log service node; by executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record, thereby realizing that in a state where a data storage service (such as provided by a cluster of storage service nodes) and a log service (such as a cluster of log service nodes) are separated, consistency correspondence control is performed through the version number and the log number, and incremental log records are pulled, and the data stored in the data storage space and the version number are updated by executing the incremental log record.

[0184] Another data synchronization method embodiment provided in this specification is as follows:

[0185] Another method embodiment provided in this specification corresponds to the above method embodiment. The executor of this embodiment is a log service node, while the executor of the above method embodiment is a storage service node. The two embodiments cooperate with each other to implement updating of data stored in a data storage device by executing newly added log records when the data storage service (provided by the storage service node) and the log service (provided by the log service node) are separated. When reading this embodiment, it can correspond to the above method embodiment. Figure 3 A processing flow chart of another data synchronization method provided according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0186] Step 302: Receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space.

[0187] Step 304: Based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number.

[0188] Step 306: Return the target log record to the storage service node, so that the storage service node updates the data and version number stored in the data storage space by executing the target log record.

[0189] In an optional implementation manner provided in the embodiments of this specification, the data synchronization method further includes:

[0190] Receive a data change request submitted by the storage service node, wherein the data change request carries data change operation information;

[0191] Based on the data change operation information, a first log record corresponding to the data change operation information is generated.

[0192] Specifically, the data change operation information refers to data change operation information for performing a change operation on data. In the embodiment of the present specification, the log service node generates a first log record corresponding to the data change request by receiving a data change request submitted by a user, and records the data change operation information carried in the data change request, so as to update the data stored in the data storage space in the storage service node by executing the first log record.

[0193] In practical applications, in order to prevent the log service node from storing too many expired log records, thereby occupying too many device resources, it is necessary to clean up the historical log records in the log service node. In an optional implementation manner provided in the embodiment of this specification, the data synchronization method further includes:

[0194] Receiving a log deletion instruction sent by the storage service node;

[0195] Based on the second version sequence number carried in the log deletion instruction, historical log records whose log sequence numbers are smaller than the log sequence number corresponding to the second version sequence number are deleted.

[0196] Specifically, after the storage service node generates a target data snapshot corresponding to the second version number (i.e., the data corresponding to the second version number is backed up at the storage service node), due to the existence of the backup data, there is no need to update the data by executing log records with log numbers smaller than the log numbers corresponding to the second version number. Therefore, log records with log numbers smaller than the log numbers corresponding to the second version number are deleted as historical log records.

[0197] In addition, historical log records with a recording time less than a preset time can be deleted according to a preset time interval, and the preset time interval can be set to every month, every week, etc. according to the needs of the actual scenario, and there is no restriction here; the recording time refers to the time when the log record is generated; the preset time can be based on the system time (current time) as a reference, such as three months before the system time, or one month before the system time, etc., and there is no restriction here.

[0198] For example, when the preset time interval is one month, the preset time is three months forward of the system time, and the system time is 2020 / 6 / 1, historical log records with a generation time less than 2020 / 3 / 1 are deleted every month.

[0199] In summary, this specification provides another data synchronization method, in which the execution subject is a log service node, which cooperates with the storage service node to receive a log pull request submitted by the storage service node, wherein the log pull request carries the version number of the data storage space; based on the log pull request, a target log record whose log number is greater than the log number corresponding to the version number is obtained; the target log record is returned to the storage service node, so that the storage service node executes the target log record, and updates the data stored in the data storage space and the version number, thereby realizing that in a state where the data storage service (such as provided by a cluster of storage service nodes) and the log service (such as a cluster of log service nodes) are separated, consistency control is performed through the version number and the log number, incremental log records are pulled, and the data stored in the data storage space and the version number are updated by executing the incremental log record.

[0200] The following combination Figure 4 , taking the application of the data synchronization method provided in this specification in an interactive scenario as an example, the data synchronization method is further described. Figure 4 The following is a flowchart of an interactive process of a data synchronization method provided by an embodiment of the present specification, which specifically includes the following steps:

[0201] Step 402: When a preset trigger condition is met, the front-end service node in the storage service node sends a request for obtaining the version number of the data storage space to the back-end storage service node.

[0202] Specifically, when there are multiple storage service nodes, the storage service nodes include a front-end service node and at least one back-end storage service node, and the back-end storage service node receives the acquisition request sent by the front-end service node.

[0203] Step 404: The backend storage service node obtains the version number of the data storage space based on the acquisition request.

[0204] Step 406: The backend storage service node returns the obtained version number to the frontend service node.

[0205] Specifically, the front-end service node receives the version number sent by the back-end storage service node, and the version number has a corresponding log number in the log service node.

[0206] Step 408: The front-end service node sends a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space.

[0207] Specifically, the log service node receives the log pull request submitted by the front-end service node.

[0208] Step 410: The log service node obtains a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number based on the log pull request.

[0209] Step 412: The log service node returns the target log record to the front-end service node.

[0210] Specifically, the front-end service node receives the target log record returned by the log service node based on the log pull request, wherein the log sequence number of the target log record is greater than the log sequence number corresponding to the version sequence number in the log service node.

[0211] Step 414: the front-end service node sends the target log record to the back-end storage service node.

[0212] Specifically, the backend storage service node receives the target log record.

[0213] Step 416: The backend storage service node updates the data stored in the data storage space and the version number by executing the target log record to obtain an update result.

[0214] Specifically, the updated version serial number corresponds to the latest log serial number of the target log record.

[0215] Step 418: The backend storage service node returns the update result to the frontend service node.

[0216] Specifically, the front-end service node receives the update result.

[0217] In summary, the present specification provides a data synchronization method. In the case where there are multiple storage service nodes, the storage service nodes are divided into front-end service nodes and back-end storage service nodes. The front-end service nodes, back-end storage service nodes cooperate with the log service nodes to send a log pull request to the log service node, wherein the log pull request carries the version number of the data storage space, and the version number has a corresponding log number in the log service node; receiving a target log record returned by the log service node based on the log pull request, wherein the log number of the target log record is greater than the log number corresponding to the version number in the log service node; by executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record, so that in a state where the data storage service (such as provided by a cluster composed of storage service nodes) and the log service (such as a cluster composed of log service nodes) are separated, consistency correspondence control is performed through the version number and the log number, and incremental log records are pulled, and the data stored in the data storage space and the version number are updated by executing the incremental log record.

[0218] The following combination Figure 5 , taking the application of the data synchronization method provided in this specification in an interactive scenario as an example, the data synchronization method is further described. Figure 5 The following is a flowchart of an interactive processing of a data synchronization method based on a data change request provided by an embodiment of the present specification, which specifically includes the following steps:

[0219] Step 502: A front-end service node in the storage service node receives a data change request submitted by a user.

[0220] Step 504: The front-end service node encapsulates the data change request, generates a consistency proposal, and sends the consistency proposal to the log service node.

[0221] Specifically, the log service node receives the consistency proposal submitted by the front-end service node.

[0222] Step 506: The log service node returns a message accepting the consistency proposal to the front-end service node.

[0223] Specifically, the log service node accepting the consistency proposal means that most of the log service nodes included in the log service node accept the consistency proposal.

[0224] The log service node accepts the consistency protocol and generates a first log record corresponding to the data change operation information based on the data change operation information carried in the consistency proposal.

[0225] Step 508: The front-end service node sends a first log pull request to the log service node, wherein the log pull request carries a first version number of the data storage space;

[0226] Specifically, the log service node receives the first log pull request sent by the front-end service node.

[0227] Step 510: The log service node obtains a first target log record whose log sequence number is greater than the version sequence number based on the first log pull request.

[0228] Step 512: The log service node returns the first target log record to the front-end service node.

[0229] Specifically, the front-end service node receives the first target log record sent by the log service node.

[0230] Step 514: The front-end service node sends the first target log record to a back-end storage service node in the storage service nodes.

[0231] Specifically, the backend storage service node receives the first target log record sent by the frontend service node.

[0232] Step 516: The backend storage service node updates the data stored in the data storage space and the first version number stored in the data storage space by executing the first target log record to obtain an update result.

[0233] Specifically, the updated version serial number corresponds to the latest log serial number of the first target log record.

[0234] Step 518: The backend storage service node returns the update result to the frontend service node.

[0235] Specifically, the front-end service node receives the update result sent by the back-end storage service node.

[0236] Step 520: The front-end service node returns the update result to the user.

[0237] In summary, the present specification provides a data synchronization method. In the case where there are multiple storage service nodes, the storage service nodes are divided into front-end service nodes and back-end storage service nodes. The front-end service nodes, back-end storage service nodes cooperate with the log service nodes to send a log pull request to the log service node, wherein the log pull request carries the version number of the data storage space, and the version number has a corresponding log number in the log service node; receiving a target log record returned by the log service node based on the log pull request, wherein the log number of the target log record is greater than the log number corresponding to the version number in the log service node; by executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record, so that in a state where the data storage service (such as provided by a cluster composed of storage service nodes) and the log service (such as a cluster composed of log service nodes) are separated, consistency correspondence control is performed through the version number and the log number, and incremental log records are pulled, and the data stored in the data storage space and the version number are updated by executing the incremental log record.

[0238] An embodiment of a data synchronization system provided in this specification is as follows:

[0239] A system embodiment provided in this specification corresponds to the above method embodiment, and when reading this embodiment, it can correspond to the above method embodiment. Figure 6 A schematic diagram of the structure of a data synchronization system provided according to an embodiment of the present specification is shown.

[0240] The data synchronization system comprises:

[0241] Storage service node 602 and log service node 604;

[0242] The storage service node 602 is configured to send a log pull request to the log service node 604, wherein the log pull request carries a version number of the data storage space;

[0243] The log service node 604 is configured to receive the log pull request, pull a target log record whose log sequence number is greater than the version sequence number based on the log pull request, and return the target log record to the storage service node 602;

[0244] The storage service node 602 is configured to receive a target log record returned by the log service node 604 based on the log pull request, wherein the log sequence number of the target log record is greater than the version sequence number; by executing the target log record, the data stored in the data storage space and the version sequence number are updated so that the updated version sequence number corresponds to the latest log sequence number of the target log record.

[0245] Optionally, when there are multiple storage service nodes 602, the storage service node 602 includes a front-end service node and at least one back-end storage service node.

[0246] Accordingly, the front-end service node is configured to receive a data change request submitted by a user; encapsulate the data change request, generate a consistency proposal, and send the consistency proposal to the log service node 604;

[0247] The log service node 604 is further configured to receive the consistency proposal, and send a message to the front-end service node accepting the consistency proposal when more than a node quantity threshold in the log service node 604 accept the consistency proposal.

[0248] Specifically, the log service node 604 includes at least one log service node. When the log service node 604 includes multiple log service nodes, the log service node is a distributed log service node. Accordingly, the consistency proposal is a proposal of the distributed consistency protocol. When most nodes in the distributed log service node agree with the proposal, it indicates that the proposal is accepted.

[0249] Optionally, the front-end service node is further configured to send a request for obtaining a version number of the data storage space to the back-end storage service node;

[0250] The backend storage service node is configured to receive the acquisition request; and based on the acquisition request, acquire the version number of the data storage space, and return the version number to the frontend service node;

[0251] The front-end service node is further configured to accept the version number and send the log pull request to the log service node 604 .

[0252] Optionally, the front-end service node is further configured to receive the target log record returned by the log service node 604 based on the log pull request; and send the target log record to the back-end storage service node;

[0253] The backend storage service node is also configured to accept the target log record, and by executing the target log record, update the data stored in the data storage space and the version number so that the updated version number corresponds to the latest log number of the target log record.

[0254] Optionally, the log service node 604 is further configured to implement log consistency between nodes of the log service node based on a distributed consistency protocol, wherein the distributed consistency protocol includes: Paxos protocol, Zookeeper Atomic Broadcast protocol, Raft protocol and Chain Replication protocol.

[0255] In actual applications, when the log service node 604 includes multiple log service nodes, it generally includes an odd number of log service nodes. In addition, the log service node 604 includes: a master log service node and at least one slave log service node. The log service node 604 is deployed in at least one geographical area.

[0256] In summary, the present specification provides a data synchronization system, which cooperates with the storage service node and the log service node to send a log pull request to the log service node, wherein the log pull request carries the version number of the data storage space, and the version number has a corresponding log sequence number in the log service node; receives the target log record returned by the log service node based on the log pull request, wherein the log sequence number of the target log record is greater than the log sequence number corresponding to the version number in the log service node; by executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log sequence number of the target log record, and realizes that in a state where the data storage service (such as provided by a cluster composed of storage service nodes) and the log service (such as a cluster composed of log service nodes) are separated, consistency correspondence control is performed through the version number and the log sequence number, and incremental log records are pulled, and the data stored in the data storage space and the version number are updated by executing the incremental log record.

[0257] The above is a schematic scheme of a data synchronization system of this embodiment. It should be noted that the technical scheme of the data synchronization system and the technical scheme of the above data synchronization method belong to the same concept, and the details not described in detail in the technical scheme of the data synchronization device can be referred to the description of the technical scheme of the above data synchronization method.

[0258] An embodiment of a data synchronization device provided in the specification is as follows:

[0259] Corresponding to the above method embodiment, this specification also provides a data synchronization device embodiment, Figure 7 FIG. 1 is a schematic diagram of a data synchronization device provided by an embodiment of the present specification. Figure 7 As shown, the device comprises:

[0260] The request sending module 702 is configured to send a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node;

[0261] The record receiving module 704 is configured to receive a target log record returned by the log service node based on the log pull request, wherein the log sequence number of the target log record is greater than the log sequence number corresponding to the version sequence number in the log service node;

[0262] The update module 706 is configured to update the data stored in the data storage space and the version number by executing the target log record, so that the updated version number corresponds to the latest log number of the target log record.

[0263] Optionally, when there are multiple target log records, the updating module 706 includes:

[0264] A determination sequence submodule is configured to determine the execution sequence of the target log records according to the sequence of the log sequence numbers from small to large;

[0265] The data update submodule is configured to execute the target log record based on the execution order, and update the data stored in the data storage space and the version number.

[0266] Optionally, the updating module 706 is further configured to:

[0267] For a first log record in the target log record, the first log record is executed to update the data stored in the data storage space and update the version number of the data storage space based on the log sequence number of the first log record.

[0268] Optionally, the data synchronization device further includes:

[0269] A first request receiving module is configured to receive a data change request submitted by a user;

[0270] The second request sending module is configured to send the data change request to the log service node, so that the log service node generates a corresponding second log record based on the data change request.

[0271] Optionally, the data synchronization device further includes:

[0272] A third request sending module is configured to send a first log pull request to the log service node, wherein the log pull request carries a first version number of the data storage space;

[0273] A second record receiving module is configured to receive a first target log record returned by the log service node based on the first log pull request, wherein the log sequence number of the first target log record is greater than the log sequence number corresponding to the first version sequence number, and the first target log record includes the second log record;

[0274] A second updating module is configured to update the data stored in the data storage space and the first version number by executing the first target log record, so that the updated version number corresponds to the latest log number of the first target log record;

[0275] The first returning module is configured to return the update result obtained by executing the second log record and the target version number of the data storage space to the user as the execution result of the data change request.

[0276] Optionally, the data synchronization device further includes:

[0277] A second request receiving module is configured to receive a data query request submitted by a user, wherein the data query request carries a query condition and a target version number;

[0278] A first judgment module is configured to judge whether the target version number is greater than the log number corresponding to the version number of the data storage space;

[0279] If yes, execute the sending request module 702;

[0280] The data synchronization device further includes:

[0281] A first query module is configured to query the updated data storage space for first target data corresponding to the query condition;

[0282] The second returning module is configured to return the first target data to the user.

[0283] Optionally, if the judgment result of the first judgment module is no, the following modules are executed:

[0284] A second determination module is configured to determine whether there is a data snapshot corresponding to the target version serial number;

[0285] If so, running a second query module, wherein the second query module is configured to query the data snapshot for second target data corresponding to the query condition;

[0286] The third returning module is configured to return the second target data to the user.

[0287] Optionally, the sending request module 702 is further configured to:

[0288] When receiving a data query request submitted by a user, a log pull request is sent to the log service node;

[0289] Correspondingly, the data synchronization device further includes:

[0290] A third query module is configured to query the data storage space for third target data corresponding to the query condition according to the query condition carried in the data query request;

[0291] The fourth returning module is configured to return the third target data to the user.

[0292] Optionally, the data synchronization device further includes:

[0293] A serial number acquisition module is configured to acquire a current version serial number of the data storage space;

[0294] The first filtering module is configured to, when the current version number is greater than the version number, filter out a second target log record whose log number is greater than the log number corresponding to the current version number in the target log record, use the second target record as the target log record, and use the current version number as the version number.

[0295] Optionally, the data synchronization device further includes:

[0296] The second filtering module is configured to filter out log records whose target log records are not empty from the target log records as the target log records.

[0297] Optionally, the data synchronization device further includes:

[0298] A snapshot generation module is configured to generate a target data snapshot of the data storage space according to a preset snapshot generation strategy;

[0299] The sending instruction module is configured to send a log deletion instruction to the log service node, wherein the log deletion instruction carries the second version sequence number corresponding to the target data snapshot, so that the log service node deletes historical log records whose log sequence numbers are less than the log sequence number corresponding to the second version sequence number based on the log deletion instruction.

[0300] Optionally, the data synchronization device further includes:

[0301] A third request receiving module is configured to receive a data synchronization request for a target storage service node when a new target storage service node is added;

[0302] The snapshot sending module is configured to send the target data snapshot to the target storage service node based on the data synchronization request, so that the target storage service node acts as the storage service node, and the second version number corresponding to the target data snapshot acts as the version number, and executes the step of sending a log pull request to the log service node.

[0303] In summary, the present specification provides a data synchronization device, which cooperates with a log service node to send a log pull request to the log service node, wherein the log pull request carries a version number of a data storage space, and the version number has a corresponding log number in the log service node; receives a target log record returned by the log service node based on the log pull request, wherein the log number of the target log record is greater than the log number corresponding to the version number in the log service node; updates the data stored in the data storage space and the version number by executing the target log record, so that the updated version number corresponds to the latest log number of the target log record, and realizes consistency correspondence control through the version number and the log number in a state where the data storage service (such as provided by a cluster composed of storage service nodes) and the log service (such as a cluster composed of log service nodes) are separated, pulls incremental log records, and updates the data stored in the data storage space and the version number by executing the incremental log record.

[0304] The above is a schematic scheme of a data synchronization device of this embodiment. It should be noted that the technical scheme of the data synchronization device and the technical scheme of the above data synchronization method belong to the same concept, and the details not described in detail in the technical scheme of the data synchronization device can be referred to the description of the technical scheme of the above data synchronization method.

[0305] Another data synchronization device embodiment provided in this specification is as follows:

[0306] Corresponding to the above method embodiment, this specification also provides a data synchronization device embodiment, Figure 8 FIG. 2 is a schematic diagram of another data synchronization device provided by an embodiment of the present specification. Figure 8 As shown, the device comprises:

[0307] The request receiving module 802 is configured to receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space;

[0308] The record acquisition module 804 is configured to acquire, based on the log pull request, a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number;

[0309] The return module 806 is configured to return the target log record to the storage service node, so that the storage service node updates the data and version number stored in the data storage space by executing the target log record.

[0310] Optionally, the data synchronization device further includes:

[0311] A request receiving module is configured to receive a data change request submitted by the storage service node, wherein the data change request carries data update operation information;

[0312] The record generation module is configured to generate a first log record corresponding to the data change operation information based on the data change operation information.

[0313] Optionally, the data synchronization device further includes:

[0314] An instruction receiving module is configured to receive a log deletion instruction sent by the storage service node;

[0315] The record deletion module is configured to delete historical log records whose log sequence numbers are less than the log sequence numbers corresponding to the second version number based on the second version number carried in the log deletion instruction. In summary, this specification provides another data synchronization device, which cooperates with a storage service node to receive a log pull request submitted by a storage service node, wherein the log pull request carries the version number of the data storage space; based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version number; return the target log record to the storage service node so that the storage service node executes the target log record, updates the data stored in the data storage space and the version number, and realizes that when the data storage service (such as provided by a cluster composed of storage service nodes) and the log service (such as a cluster composed of log service nodes) are separated, the consistency correspondence control is performed through the version number and the log sequence number, and the incremental log record is pulled, and the data stored in the data storage space and the version number are updated by executing the incremental log record.

[0316] The above is a schematic scheme of another data synchronization device of this embodiment. It should be noted that the technical scheme of the data synchronization device and the technical scheme of the above another data synchronization method belong to the same concept, and the details of the technical scheme of the other data synchronization device that are not described in detail can all be referred to the description of the technical scheme of the above another data synchronization method.

[0317] An embodiment of a computing device provided in this specification is as follows:

[0318] Fig. 9 The structure block diagram of a computing device 900 provided according to an embodiment of the present specification is shown. The components of the computing device 900 include but are not limited to a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0319] The computing device 900 also includes an access device 940 that enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0320] In one embodiment of the present specification, the above components of the computing device 900 and Fig. 9 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig. 9 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0321] The computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 900 may also be a mobile or stationary server.

[0322] This specification provides a computing device, including a memory 910, a processor 920, and computer instructions stored in the memory and executable on the processor, wherein the processor 920 is configured to execute the following computer executable instructions:

[0323] Sending a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node;

[0324] Receiving a target log record returned by the log service node based on the log pull request, wherein a log sequence number of the target log record is greater than a log sequence number corresponding to the version sequence number in the log service node;

[0325] By executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record.

[0326] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data synchronization method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data synchronization method.

[0327] Another computing device embodiment provided in this specification is as follows:

[0328] Fig.10The structure block diagram of another computing device 1000 provided according to an embodiment of the present specification is shown. The components of the computing device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0329] The computing device 1000 also includes an access device 1040 that enables the computing device 1100 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0330] In one embodiment of the present specification, the above components of the computing device 1000 and Fig.10 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.10 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0331] The computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 1000 may also be a mobile or stationary server.

[0332] This specification provides another computing device, including a memory 1010, a processor 1020, and computer instructions stored in the memory and executable on the processor, wherein the processor 1020 is configured to execute the following computer executable instructions:

[0333] Receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space;

[0334] Based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number;

[0335] The target log record is returned to the storage service node, so that the storage service node updates the data and version number stored in the data storage space by executing the target log record.

[0336] The above is a schematic scheme of another computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above another data synchronization method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above another data synchronization method.

[0337] An embodiment of a computer-readable storage medium provided in this specification is as follows:

[0338] This specification provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to:

[0339] Sending a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node;

[0340] Receiving a target log record returned by the log service node based on the log pull request, wherein a log sequence number of the target log record is greater than a log sequence number corresponding to the version sequence number in the log service node;

[0341] By executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record.

[0342] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data synchronization method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data synchronization method.

[0343] Another computer-readable storage medium embodiment provided in this specification is as follows:

[0344] This specification provides another computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to:

[0345] Receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space;

[0346] Based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number;

[0347] The target log record is returned to the storage service node, so that the storage service node updates the data and version number stored in the data storage space by executing the target log record.

[0348] The above is a schematic scheme of another computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above another data synchronization method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above another data synchronization method.

[0349] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0350] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0351] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0352] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0353] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data synchronization method, applied to a storage service node, comprising: Sending a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node; Receiving a target log record returned by the log service node based on the log pull request, wherein a log sequence number of the target log record is greater than a log sequence number corresponding to the version sequence number in the log service node; By executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record, wherein the executing the target log record includes: executing the data update operation recorded in the target log record.

2. The data synchronization method according to claim 1, wherein when there are multiple target log records, updating the data stored in the data storage space and the version number by executing the target log record comprises: Determine the execution order of the target log records according to the order of the log sequence numbers from small to large; Based on the execution order, the target log record is executed to update the data stored in the data storage space and the version number.

3. The data synchronization method according to claim 1, wherein the updating of the data stored in the data storage space and the version number by executing the target log record comprises: For a first log record in the target log record, the first log record is executed to update the data stored in the data storage space and update the version number of the data storage space based on the log sequence number of the first log record.

4. The data synchronization method according to claim 1, further comprising: Receive data change requests submitted by users; The data change request is sent to the log service node, so that the log service node generates a corresponding second log record based on the data change request.

5. The data synchronization method according to claim 4, further comprising: Sending a first log pull request to the log service node, wherein the log pull request carries a first version number of the data storage space; Receiving a first target log record returned by the log service node based on the first log pull request, wherein the log sequence number of the first target log record is greater than the log sequence number corresponding to the first version sequence number, and the first target log record includes the second log record; By executing the first target log record, the data stored in the data storage space and the first version serial number are updated so that the updated version serial number corresponds to the latest log serial number of the first target log record; The update result obtained by executing the second log record and the target version number of the data storage space are returned to the user as the execution result of the data change request.

6. The data synchronization method according to claim 1, further comprising: Receive a data query request submitted by a user, wherein the data query request carries a query condition and a target version number; Determine whether the target version number is greater than the log number corresponding to the version number of the data storage space; If yes, execute the step of sending a log pull request to the log service node; Correspondingly, after the data stored in the data storage space and the version number of the data storage space are updated, the method further includes: Searching the updated data storage space for first target data corresponding to the query condition; The first target data is returned to the user.

7. According to the data synchronization method of claim 6, if the result of judging whether the target version number is greater than the log number corresponding to the version number of the data storage space is no, the following operations are performed: Determine whether there is a data snapshot corresponding to the target version serial number; If so, searching the data snapshot for second target data corresponding to the query condition; The second target data is returned to the user.

8. According to the data synchronization method of claim 1, the sending of a log pull request to a log service node comprises: When receiving a data query request submitted by a user, send a log pull request to the log service node; Correspondingly, after updating the data stored in the data storage space and the version number, the method further includes: According to the query condition carried in the data query request, query the data storage space for third target data corresponding to the query condition; The third target data is returned to the user.

9. The data synchronization method according to claim 1, before updating the data stored in the data storage space and the version number by executing the target log record, further comprising: Obtaining the current version serial number of the data storage space; In the case where the current version number is greater than the version number, a second target log record whose log number is greater than the log number corresponding to the current version number is filtered out from the target log record, and the second target record is used as the target log record, and the current version number is used as the version number.

10. The data synchronization method according to claim 1, before updating the data stored in the data storage space and the version number by executing the target log record, further comprising: Among the target log records, log records whose target log records are not empty are screened out as the target log records.

11. The data synchronization method according to claim 1, further comprising: Generate a target data snapshot of the data storage space according to a preset snapshot generation strategy; A log deletion instruction is sent to the log service node, wherein the log deletion instruction carries the second version sequence number corresponding to the target data snapshot, so that the log service node deletes historical log records whose log sequence numbers are less than the log sequence number corresponding to the second version sequence number based on the log deletion instruction.

12. The data synchronization method according to claim 11, further comprising: In the case of adding a new target storage service node, receiving a data synchronization request for the target storage service node; Based on the data synchronization request, the target data snapshot is sent to the target storage service node, so that the target storage service node acts as the storage service node, and the second version number corresponding to the target data snapshot is used as the version number, and the step of sending a log pull request to the log service node is executed.

13. A data synchronization method, applied to a log service node, comprising: Receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space; Based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number; The target log record is returned to the storage service node so that the storage service node updates the data and version number stored in the data storage space by executing the target log record, wherein the storage service node executes the target log record including: the storage service node executes the data update operation recorded in the target log record.

14. The data synchronization method according to claim 13, further comprising: Receive a data change request submitted by the storage service node, wherein the data change request carries data update operation information; Based on the data change operation information, a first log record corresponding to the data change operation information is generated.

15. The data synchronization method according to claim 13, further comprising: Receiving a log deletion instruction sent by the storage service node; Based on the second version sequence number carried in the log deletion instruction, historical log records whose log sequence numbers are smaller than the log sequence number corresponding to the second version sequence number are deleted.

16. A data synchronization system, comprising: Storage service nodes and log service nodes; The storage service node is configured to send a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node; The log service node is configured to receive the log pull request, and based on the log pull request, pull a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number, and return the target log record to the storage service node; The storage service node is configured to receive a target log record returned by the log service node based on the log pull request, and to update the data stored in the data storage space and the version number by executing the target log record so that the updated version number corresponds to the latest log number of the target log record, wherein executing the target log record includes: executing the data update operation recorded in the target log record.

17. A data synchronization device, applied to a storage service node, comprising: A sending request module is configured to send a log pull request to a log service node, wherein the log pull request carries a version number of a data storage space, and the version number has a corresponding log number in the log service node; A record receiving module is configured to receive a target log record returned by the log service node based on the log pull request, wherein the log sequence number of the target log record is greater than the log sequence number corresponding to the version sequence number in the log service node; The update module is configured to update the data stored in the data storage space and the version number by executing the target log record so that the updated version number corresponds to the latest log number of the target log record, wherein the execution of the target log record includes: executing the data update operation recorded in the target log record.

18. A data synchronization device, applied to a log service node, comprising: A request receiving module is configured to receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space; A record acquisition module is configured to acquire, based on the log pull request, a target log record with a log sequence number greater than the log sequence number corresponding to the version sequence number; The return module is configured to return the target log record to the storage service node so that the storage service node updates the data stored in the data storage space and the version number by executing the target log record, wherein the storage service node executes the target log record including: the storage service node executes the data update operation recorded in the target log record.

19. A computing device, applied to a storage service node, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the following method: Sending a log pull request to the log service node, wherein the log pull request carries a version number of the data storage space, and the version number has a corresponding log number in the log service node; Receiving a target log record returned by the log service node based on the log pull request, wherein a log sequence number of the target log record is greater than a log sequence number corresponding to the version sequence number in the log service node; By executing the target log record, the data stored in the data storage space and the version number are updated so that the updated version number corresponds to the latest log number of the target log record, wherein the executing the target log record includes: executing the data update operation recorded in the target log record.

20. A computing device, applied to a log service node, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the following method: Receive a log pull request submitted by a storage service node, wherein the log pull request carries a version number of the data storage space; Based on the log pull request, obtain a target log record whose log sequence number is greater than the log sequence number corresponding to the version sequence number; The target log record is returned to the storage service node so that the storage service node updates the data and version number stored in the data storage space by executing the target log record, wherein the storage service node executes the target log record including: the storage service node executes the data update operation recorded in the target log record.

21. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the data synchronization method according to any one of claims 1 to 12.

22. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the data synchronization method described in any one of claims 13 to 15.

Images