Data synchronization method and apparatus, electronic device, and storage medium

Abstract

The application provides a data synchronization method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a first target data block in a first copy; performing data verification on to-be-verified data in the first target data block to obtain a data verification result; if it is determined that the first target data block has not completed data updating based on the data verification result, updating first metadata corresponding to the first target data block based on a second target data block corresponding to the first target data block in a second copy, and then updating the to-be-verified data in the first target data block to complete data synchronization processing of the first copy. The method provided by the application reduces the number of ios of data synchronization, simplifies the data synchronization process, and improves the efficiency of data synchronization.

Description

Technical Field

[0001] This application relates to the field of distributed technology, and in particular to a data synchronization method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of distributed technology, distributed block storage architecture is commonly used to store data to meet the data access requirements of distributed systems. In distributed block storage architecture, multiple replicas are typically used for data redundancy. This requires ensuring that the data content within multiple replicas is completely consistent to avoid reading different data from different replicas, which could lead to system errors.

[0003] Therefore, if a system or replica fails, it is necessary to check whether the data content stored between the replicas is consistent after the system is restored. If they are inconsistent, data synchronization needs to be performed in a timely manner, and data read and write services can only be provided to the outside world after the data synchronization is completed.

[0004] In related technologies, in order to ensure consistency among multiple replicas, it is necessary to update the data of each replica in a consistent manner. However, the existing data update method has a relatively complex process, resulting in high system latency and affecting system performance. Summary of the Invention

[0005] This application provides a data synchronization method, apparatus, electronic device, and storage medium to solve the technical problem in the prior art where, after system failure recovery, data synchronization of each copy involves a complex data update process and high latency, affecting the overall performance of the system.

[0006] Firstly, this application provides a data synchronization method, including:

[0007] Identify a first target data block in the first replica; wherein the first replica represents the replica that has failed; the first target data block represents the data block that is updated when the first replica fails;

[0008] The data to be verified within the first target data block is verified to obtain a data verification result; wherein, the data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has completed the data update.

[0009] If, based on the data verification result, it is determined that the first target data block has not completed data updates, then, based on the second target data block in the second replica corresponding to the first target data block, the first metadata corresponding to the first target data block is first updated, and then the data to be verified in the first target data block is updated to complete the data synchronization processing of the first replica; wherein, the second replica represents a replica that has not experienced a failure; and the first metadata represents the attribute information of the data to be verified.

[0010] In one example, determining the first target data block in the first replica includes:

[0011] Obtain the target sequence number information corresponding to the first replica; wherein, the target sequence number information is used to determine the data block in the first replica that has completed data update before the failure occurs;

[0012] Based on the target sequence number information, the first target data block is determined from the data blocks included in the first copy.

[0013] In one example, the method further includes:

[0014] After completing the data synchronization process for the first copy, the target serial number information is updated.

[0015] In one example, determining the first target data block from the data blocks included in the first copy based on the target sequence number information includes:

[0016] Determine the second metadata corresponding to each data block contained in the first replica; wherein the second metadata includes global sequence number information; the global sequence number information represents the data update order of each data block contained in the first replica before data synchronization;

[0017] Based on the target sequence number information, target identifier data is determined from the global sequence number information corresponding to each data block contained in the first copy, and the data block corresponding to the target identifier data is determined as the first target data block; wherein, the value of the target identifier data is greater than the value of the target sequence number information.

[0018] In one example, before performing data verification on the data to be verified within the first target data block, the method further includes:

[0019] Determine the data update information included in the first metadata corresponding to the first target data block; wherein, the data update information includes update start position information and update length information;

[0020] Based on the update start position information and the update length information, the data to be verified is determined from the first target data block.

[0021] In one example, the step of performing data verification on the data to be verified within the first target data block to obtain the data verification result includes:

[0022] Calculate the verification calculation data corresponding to the data to be verified according to the pre-set data verification method;

[0023] The verification calculation data is compared with the verification value in the first metadata corresponding to the first target data block to obtain the first verification result;

[0024] Obtain the first version number information included in the first metadata corresponding to the first target data block, and the second version number information included in the third metadata corresponding to the second target data block, and compare the first version number information and the second version number information to obtain the second verification result;

[0025] Based on the first verification result and the second verification result, the data verification result is determined.

[0026] In one example, determining the data verification result based on the first verification result and the second verification result includes:

[0027] If the first verification result indicates that the verification calculation data and the second verification value are the same, and the second verification result indicates that the first version number information and the second version number information are the same, then it is determined that the data verification result indicates that the first target data block has completed data synchronization.

[0028] If the first verification result indicates that the verification calculation data and the second verification value are different, and / or the second verification result indicates that the first version number information and the second version number information are different, then it is determined that the data verification result indicates that the first target data block has not completed data synchronization.

[0029] Secondly, this application provides a data synchronization device, comprising:

[0030] The first determining unit is configured to determine a first target data block in the first replica; wherein the first replica represents a replica that has failed; and the first target data block represents a data block that is updated when the first replica fails.

[0031] A data verification unit is used to perform data verification on the data to be verified in the first target data block and obtain a data verification result; wherein, the data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has completed the data update.

[0032] The data synchronization unit is configured to, if based on the data verification result it is determined that the first target data block has not completed data updates, then based on the second target data block in the second replica corresponding to the first target data block, first update the first metadata corresponding to the first target data block, and then update the data to be verified in the first target data block, thereby completing the data synchronization processing of the first replica; wherein, the second replica represents a replica that has not experienced a failure; and the first metadata represents the attribute information of the data to be verified.

[0033] In one example, the first determining unit is used for:

[0034] Obtain the target sequence number information corresponding to the first replica; wherein, the target sequence number information is used to determine the data block in the first replica that has completed data update before the failure occurs;

[0035] Based on the target sequence number information, the first target data block is determined from the data blocks included in the first copy.

[0036] In one example, the device also includes:

[0037] The update unit is used to update the target sequence number information after completing the data synchronization process of the first copy.

[0038] In one example, the first determining unit is used for:

[0039] Determine the second metadata corresponding to each data block contained in the first replica; wherein the second metadata includes global sequence number information; the global sequence number information represents the data update order of each data block contained in the first replica before data synchronization;

[0040] Based on the target sequence number information, target identifier data is determined from the global sequence number information corresponding to each data block contained in the first copy, and the data block corresponding to the target identifier data is determined as the first target data block; wherein, the value of the target identifier data is greater than the value of the target sequence number information.

[0041] In one example, the device also includes:

[0042] The second determining unit is used to determine the data update information included in the first metadata corresponding to the first target data block before performing data verification on the data to be verified in the first target data block; wherein, the data update information includes update start position information and update length information;

[0043] Based on the update start position information and the update length information, the data to be verified is determined from the first target data block.

[0044] In one example, the data validation unit is used for:

[0045] Calculate the verification calculation data corresponding to the data to be verified according to the pre-set data verification method;

[0046] The verification calculation data is compared with the verification value in the first metadata corresponding to the first target data block to obtain the first verification result;

[0047] Obtain the first version number information included in the first metadata corresponding to the first target data block, and the second version number information included in the third metadata corresponding to the second target data block, and compare the first version number information and the second version number information to obtain the second verification result;

[0048] Based on the first verification result and the second verification result, the data verification result is determined.

[0049] In one example, the data validation unit is used for:

[0050] If the first verification result indicates that the verification calculation data and the second verification value are the same, and the second verification result indicates that the first version number information and the second version number information are the same, then it is determined that the data verification result indicates that the first target data block has completed data synchronization.

[0051] If the first verification result indicates that the verification calculation data and the second verification value are different, and / or the second verification result indicates that the first version number information and the second version number information are different, then it is determined that the data verification result indicates that the first target data block has not completed data synchronization.

[0052] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0053] The memory stores computer-executed instructions;

[0054] The processor executes computer execution instructions stored in the memory to implement the method described in the first aspect.

[0055] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0056] Fifthly, this application provides a computer program product comprising: computer execution instructions stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer execution instructions from the readable storage medium, and the at least one processor executes the computer execution instructions to cause the electronic device to perform the method described in the first aspect.

[0057] The data synchronization method, apparatus, electronic device, and storage medium provided in this application can acquire a first target data block in the first replica after the first replica recovers from a failure. The first target data block represents the data block for which data updates were performed when the first replica failed. Then, data verification can be performed on the data to be verified within the first target data block to obtain a data verification result. If, based on the data verification result, it is determined that the first target data block has not completed data synchronization, then based on the second target data block in the second replica corresponding to the first target data block, the first metadata corresponding to the first target data block is updated first, and then the data to be verified in the first target data block is updated, completing the data synchronization process for the first replica. This implementation, when it is determined that the data to be verified in the failed first replica has not completed data updates and data synchronization is required, can update the first metadata first, and then update the data to be verified, based on the unfailed second replica. Compared to existing data update methods, this reduces the number of I / O operations, thereby alleviating disk pressure and reducing I / O path latency during data updates, improving data synchronization efficiency, and ultimately enhancing the overall performance of the storage system. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0059] Figure 1 A schematic diagram of a distributed block storage provided in an embodiment of this application;

[0060] Figure 2 A schematic diagram illustrating a data update process in the prior art, provided as an embodiment of this application;

[0061] Figure 3 A flowchart illustrating a data synchronization method provided in an embodiment of this application;

[0062] Figure 4A schematic diagram of a data synchronization process provided in an embodiment of this application;

[0063] Figure 5 A flowchart illustrating another data synchronization method provided in an embodiment of this application;

[0064] Figure 6 This is a schematic diagram illustrating global sequence number information corresponding to a data block, provided in an embodiment of this application.

[0065] Figure 7 A schematic diagram of verification calculation data provided in an embodiment of this application;

[0066] Figure 8 A schematic diagram illustrating a data verification result provided in an embodiment of this application;

[0067] Figure 9 This is a schematic diagram of the structure of a data synchronization device provided in an embodiment of this application;

[0068] Figure 10 This is a schematic diagram of another data synchronization device provided in an embodiment of this application;

[0069] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0070] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0072] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0073] In the field of distributed technology, distributed block storage architecture is typically used to store data in order to meet the data access needs of distributed systems.

[0074] For example, Figure 1 This is a schematic diagram of a distributed block storage provided in an embodiment of this application, as shown below. Figure 1 As shown, a replica can include multiple data blocks, each with its own metadata area and data area. When storing data in each data block within the replica, version number information and size information of the data to be stored can be stored in the metadata area, and the corresponding data content can be stored in the data area. For example, if the replica contains... Figure 1 In the case of the three data blocks shown, "version number information = 1, size = 1M" can be stored in the metadata area of ​​data block 1, and "data content" can be stored in the metadata area of ​​data block 1; "version number information = 3, size = 1M" can be stored in the metadata area of ​​data block 2, and "data content" can be stored in the metadata area of ​​data block 2; "version number information = 1, size = 1M" can be stored in the metadata area of ​​data block 3, and "data content" can be stored in the metadata area of ​​data block 3.

[0075] In a distributed block storage architecture, multiple replicas are typically used for data redundancy. In this case, it is necessary to ensure that the data content in multiple replicas is completely consistent to avoid reading different data from different replicas, which could lead to system errors.

[0076] For example, during the data update process, if a copy / system fails, the data update operation may not be completed. In this case, after the system recovers, it is necessary to check whether the data content stored between the copies is consistent. If they are inconsistent, data synchronization needs to be performed in a timely manner. That is, the data of the failed copy is updated, and data read and write services are provided to the outside world only after the data update is completed.

[0077] In one example, Figure 2 This application provides a schematic diagram of a prior art process for data updating, as illustrated in the embodiments of this application. Figure 2 As shown, in the prior art, a flag can be set in the metadata area of ​​the copy, for example, such as Figure 2 The modify flag shown is used to identify data blocks that need to be updated.

[0078] For example, such as Figure 2 As shown, when the data block requiring data update is "data block 2", when writing data to data block 2 in the replica, the modify flag in the metadata of data block 2 can be updated first. For example, as shown... Figure 2As shown, the modify flag can be changed to "modify=true". Then, the data is written to data block 2. After the data is successfully written, the metadata of data block 2 can be updated. For example, if the current version number of data block 2 is 1, the version number of data block 2 can be updated to 2, and the modify flag can be modified, that is, as shown. Figure 2 As shown, change the modify flag to "modify=false", and the data update will be completed.

[0079] according to Figure 2 As is known, in existing technologies, updating data within a replica requires three underlying I / O (i.e., input / output) updates. Furthermore, I / O operations typically consume the most time in the storage software stack; for example, a single I / O operation on a mechanical hard drive takes approximately 3ms. Therefore, the latency of three I / O operations reaches 9ms. Consequently, existing data update methods, due to the large number of I / O operations, not only increase latency along the I / O path but also increase the storage pressure on the system, thus reducing system performance.

[0080] The data synchronization method provided in this application aims to solve the above-mentioned technical problems of the prior art.

[0081] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0082] Figure 3 This is a flowchart illustrating a data synchronization method provided in an embodiment of this application, as shown below. Figure 3 As shown, the data synchronization method includes:

[0083] S301. Determine the first target data block in the first copy.

[0084] Here, the first replica represents the replica that has failed; the first target data block represents the data block that is updated when the first replica fails.

[0085] In one example, when updating data blocks included in the first replica in the event of a failure, the following situations may occur: no data block is updated, only a partial data block is updated, or the data block is fully updated.

[0086] Since the update progress is unknown when the first replica fails, a consistency check needs to be performed on the data blocks that were updated during the failure of the first replica—that is, the first target data block—after the first replica recovers. If the consistency check fails, the first target data block can be synchronized according to the data synchronization method provided in this application. If the consistency check passes, data synchronization can be omitted, and data read and write services can be provided directly.

[0087] In one example, the number of first target data blocks can be one or more.

[0088] S302. Perform data verification on the data to be verified in the first target data block to obtain the data verification result.

[0089] The data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has been updated.

[0090] In one example, the data to be verified can be determined based on the first metadata corresponding to the first target data block. The first metadata corresponding to the first target data block may include at least the following: data block ID, version number information corresponding to the data update, checksum corresponding to the data update, update start position information corresponding to the data update, update length information corresponding to the data update, and global sequence number information corresponding to the data update. In this case, the data to be verified can be determined based on the data block ID, the update start position information, and the update length information.

[0091] In one example, based on the failed second copy, data verification can be performed on the data to be verified within the first target data block to obtain a data verification result. This result can include: verification passed or verification failed. If the verification result is "verification passed," it can be determined that the first target data block has completed the data update; if the verification result is "verification failed," it can be determined that the first target data block has not completed the data update.

[0092] In one example, if multiple replicas in a distributed block storage architecture fail, the second replica can be understood as a reference replica determined from the multiple replicas. This reference replica is used to verify the data to be verified in the first target data block and to synchronize the data of the first replica based on the reference replica.

[0093] S303. If, based on the data verification result, it is determined that the first target data block has not completed data update, then based on the second target data block in the second replica that corresponds to the first target data block, first update the first metadata corresponding to the first target data block, and then update the data to be verified in the first target data block to complete the data synchronization processing of the first replica.

[0094] The second copy represents the copy that has not experienced a failure; the first metadata represents the attribute information of the data to be verified.

[0095] For example, Figure 4 This is a schematic diagram illustrating a data synchronization process provided in an embodiment of this application. Figure 4 As shown, in this embodiment of the application, when performing data synchronization, the metadata can be updated first. For example, the version number information, checksum, update start position information, update length information, and global sequence number information in the metadata (e.g., the first metadata corresponding to the first target data block) can be updated according to the data that needs to be updated. Then, the data in the first target data block (i.e., the data to be checked) is updated. Figure 4 The data to be written is shown.

[0096] As described above, in this embodiment, after the first replica recovers from a failure, a first target data block can be obtained from the first replica. This first target data block represents the data block for which data updates were performed when the first replica failed. Then, the data to be verified within the first target data block can be verified to obtain a verification result. If, based on the verification result, it is determined that the first target data block has not completed data synchronization, then based on the second target data block in the second replica corresponding to the first target data block, the first metadata corresponding to the first target data block is updated first, and then the data to be verified in the first target data block is updated, completing the data synchronization process for the first replica. This implementation method, when it is determined that the data to be verified in the failed first replica has not completed data updates and data synchronization is required, updates the first metadata first, and then updates the data to be verified, based on the unfailed second replica. Compared to existing data update methods, this reduces the number of I / O operations, thereby alleviating disk pressure, reducing I / O path latency during data updates, improving data synchronization efficiency, and ultimately enhancing the overall performance of the storage system.

[0097] Figure 5 A flowchart illustrating another data synchronization method provided in this application embodiment is shown below. Figure 5 As shown, the data synchronization method includes:

[0098] S501. Obtain the target serial number information corresponding to the first copy.

[0099] The target sequence number information is used to identify the data blocks in the first replica that have completed data synchronization before a failure occurs.

[0100] In one example, the target sequence number information can be understood as the maximum sequence number identifier corresponding to the data block that has been updated in the first replica. For example, we can first determine the latest updated data block in the first replica before the failure, and then determine the global sequence number information in the metadata corresponding to the latest updated data block as the maximum sequence number identifier, that is, the target sequence number information.

[0101] In one example, corresponding field information can be set for the target serial number, and this target serial number information can be saved to the disk at the corresponding storage location. Then, the target serial number information can be determined by querying the specific content corresponding to the field information on the disk.

[0102] S502. Based on the target sequence number information, determine the first target data block from the data blocks included in the first copy.

[0103] In one example, when determining the first target data block from the data blocks included in the first replica based on the target sequence number information, the second metadata corresponding to each data block included in the first replica can be determined first. Then, based on the target sequence number information, the target identifier data is determined from the global sequence number information corresponding to each data block included in the first replica, and the data block corresponding to the target identifier data is determined as the first target data block.

[0104] The second metadata includes global sequence number information; the global sequence number information represents the data update order of each data block contained in the first copy before data synchronization.

[0105] The value of the target identifier data is greater than the value of the target sequence number information.

[0106] In one example, the data block whose global sequence number information in the second metadata corresponding to each data block included in the first replica is greater than the target sequence information can be identified as the first target data block.

[0107] For example, Figure 6 This is a schematic diagram illustrating global sequence number information corresponding to a data block, as provided in an embodiment of this application. For example... Figure 6 As shown, if the updated data block is stored in the first replica, the global sequence number information will be updated. At this point, the global sequence number information can be obtained as follows: Figure 6As shown: Global serial number information = n, Global serial number information = n+1, Global serial number information = n+2, Global serial number information = n+3, Global serial number information = n+4. In this case, if the target serial number information is determined to be: Global serial number information = n, then Global serial number information = n+1, Global serial number information = n+2, Global serial number information = n+3, and Global serial number information = n+4 can be identified as target identifier data, and the data block corresponding to this target identifier data is identified as the first target data block.

[0108] At this point, the metadata of each data block within the first replica, in addition to including global sequence number information, may also include, for example: Figure 6 The data block ID, version number, checksum, update start position, update length, etc. are shown, and will not be described in detail here.

[0109] The above implementation method can determine the first target data block to be updated when the first copy fails based on the target sequence number information. This allows the first target data block to be determined by checking the modify flag of each data block contained in the first copy one by one, which simplifies the process of determining the first target data block and improves efficiency.

[0110] S503. Determine the data update information included in the first metadata corresponding to the first target data block.

[0111] The data update information includes the start position information and the update length information.

[0112] S504. Based on the update start position information and update length information, determine the data to be verified from the first target data block.

[0113] For example, in the first target data block is Figure 4 In the case of “data block 2” shown, the update start position is “0” and the update length information is “4096”. At this time, the data in the first target data block that starts from the update start position information and has a length equal to the update length information can be identified as the data to be verified.

[0114] This implementation method can determine the relevant information of the updated data in each data block in the first copy as the attribute information of the data block, that is, the first metadata, so that the data to be verified can be determined simply and quickly based on the first metadata.

[0115] S505. Perform data verification on the data to be verified in the first target data block to obtain the data verification result.

[0116] The data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has been updated.

[0117] In one example, the process of performing data verification on the data to be verified within the first target data block and obtaining the data verification result can be described as follows.

[0118] First, according to the pre-set data verification method, calculate the verification calculation data corresponding to the data to be verified.

[0119] The pre-set data verification method can be a pre-set cyclic redundancy check method, such as CRC64. There is no specific limitation on the pre-set data verification method, as long as it can be implemented.

[0120] In one example, Figure 7 This is a schematic diagram of verification calculation data provided in an embodiment of this application. Figure 7 As shown, assuming that the verification value of the data to be verified is "verification value = 0x12345678" before the data to be verified is updated, then, if the data to be verified has not been written (i.e., Figure 7 In case one (as shown), the calculated verification data for the data to be verified is "verification value = 0x12345678"; in the case where the data to be verified is partially written (that is, Figure 7 In scenario two (as shown), the calculated verification data for the data to be verified is "verification value = 0x87659076"; this is assuming the data to be verified has been written (i.e., Figure 7 In case 3 shown, the calculated verification data of the data to be verified is "verification value = 0x23345409".

[0121] Then, the verification calculation data is compared with the verification value in the first metadata corresponding to the first target data block to obtain the first verification result.

[0122] In one example, the first verification result may include: verification passed information and verification failed information. Verification passed information indicates that the calculated verification data and the second verification value are the same, while verification failed information indicates that the calculated verification data and the second verification value are different.

[0123] Next, the first version number information included in the first metadata corresponding to the first target data block and the second version number information included in the third metadata corresponding to the second target data block are obtained, and the first version number information and the second version number information are compared to obtain the second verification result.

[0124] The second verification result can include: verification passed information and verification failed information. Verification passed information indicates that the first version number and the second version number are the same, while verification failed information indicates that the first version number and the second version number are different.

[0125] Finally, based on the first and second verification results, the data verification result is determined.

[0126] In one example, if the first verification result indicates that the verification calculation data and the second verification value are the same, and the second verification result indicates that the first version number information and the second version number information are the same, then it is determined that the data verification result indicates that the first target data block has completed data synchronization.

[0127] If the first verification result indicates that the verification calculation data and the second verification value are different, and / or the second verification result indicates that the first version number information and the second version number information are different, then it is determined that the data verification result indicates that the first target data block has not completed data synchronization.

[0128] In one example, Figure 8 This is a schematic diagram illustrating a data verification result provided in an embodiment of this application. For example... Figure 8 As shown, for data block ID=2, the version number information of the first replica and the second replica are the same (that is, the second verification result is verification passed), but the verification value on the first replica fails (that is, the first verification result is verification failed). In this case, it is necessary to synchronize the data block ID=2 in the first replica with the data block ID=2 in the second replica to restore consistency. For data block ID=10, since the version number information and verification value of both replicas pass verification, no data synchronization is required. For data block ID=3, although the verification value of the first replica and the second replica pass verification, the version number information is different, so data synchronization is also required.

[0129] In the above implementation, the data to be verified can be verified based on the metadata of the data block, and the data verification result can be obtained. In this way, the inconsistent data between replicas can be detected based on the local data write situation, which can improve the efficiency of consistency detection between replicas and thus improve the efficiency of data synchronization.

[0130] S506. If, based on the data verification result, it is determined that the first target data block has not completed data update, then based on the second target data block in the second replica that corresponds to the first target data block, first update the first metadata corresponding to the first target data block, and then update the data to be verified in the first target data block to complete the data synchronization processing of the first replica.

[0131] The second copy represents the copy that has not experienced a failure; the first metadata represents the attribute information of the data to be verified.

[0132] In one example, this step can be referred to the steps described in S303 above, and will not be repeated in detail here.

[0133] S507. After completing the data synchronization process for the first copy, update the target serial number information.

[0134] In one example, after completing the data synchronization process for the first copy, the target sequence number information can be updated based on the global sequence number information corresponding to the last data block that has completed data synchronization. At this time, the updated target sequence number information can be used to overwrite the original target sequence number information, thereby enabling timely updates of the target sequence number information and facilitating the next data synchronization.

[0135] Figure 9 This is a schematic diagram of the structure of a data synchronization device provided in an embodiment of this application, as shown below. Figure 9 As shown, the data synchronization device 900 includes:

[0136] The first determining unit 901 is used to determine the first target data block in the first replica; wherein the first replica represents the replica that has failed; and the first target data block represents the data block that is updated when the first replica fails.

[0137] The data verification unit 902 is used to perform data verification on the data to be verified in the first target data block and obtain the data verification result; wherein, the data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has completed the data update.

[0138] The data synchronization unit 903 is used to, if based on the data verification result it is determined that the first target data block has not completed data update, then based on the second target data block in the second replica corresponding to the first target data block, first update the first metadata corresponding to the first target data block, and then update the data to be verified in the first target data block, thereby completing the data synchronization process of the first replica; wherein, the second replica represents a replica that has not experienced a failure; and the first metadata represents the attribute information of the data to be verified.

[0139] Figure 10 This is a schematic diagram of another data synchronization device provided in an embodiment of this application, as shown below. Figure 10 As shown, the data synchronization device 1000 includes:

[0140] The first determining unit 1001 is used to determine the first target data block in the first replica; wherein the first replica represents the replica that has failed; and the first target data block represents the data block that is updated when the first replica fails.

[0141] The data verification unit 1002 is used to perform data verification on the data to be verified in the first target data block and obtain the data verification result; wherein, the data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has completed the data update.

[0142] The data synchronization unit 1003 is used to, if based on the data verification result it is determined that the first target data block has not completed data update, then based on the second target data block in the second replica corresponding to the first target data block, first update the first metadata corresponding to the first target data block, and then update the data to be verified in the first target data block, thereby completing the data synchronization process of the first replica; wherein, the second replica represents a replica that has not experienced a failure; and the first metadata represents the attribute information of the data to be verified.

[0143] In one example, the first determining unit 1001 is used for:

[0144] Obtain the target sequence number information corresponding to the first replica; wherein, the target sequence number information is used to identify the data block in the first replica that has completed data update before the failure occurs;

[0145] Based on the target sequence number information, the first target data block is determined from the data blocks included in the first copy.

[0146] In one example, the device also includes:

[0147] The update unit 1004 is used to update the target sequence number information after the data synchronization process of the first copy is completed.

[0148] In one example, the first determining unit 1001 is used for:

[0149] Determine the second metadata corresponding to each data block contained in the first replica; wherein, the second metadata includes global sequence number information; the global sequence number information represents the data update order of each data block contained in the first replica before data synchronization;

[0150] Based on the target sequence number information, target identifier data is determined from the global sequence number information corresponding to each data block contained in the first copy, and the data block corresponding to the target identifier data is determined as the first target data block; wherein, the value of the target identifier data is greater than the value of the target sequence number information.

[0151] In one example, the device also includes:

[0152] The second determining unit 1005 is used to determine the data update information included in the first metadata corresponding to the first target data block before performing data verification on the data to be verified in the first target data block; wherein, the data update information includes update start position information and update length information;

[0153] Based on the update start position information and update length information, the data to be verified is determined from the first target data block.

[0154] In one example, data verification unit 1002 is used for:

[0155] Calculate the verification calculation data corresponding to the data to be verified according to the pre-set data verification method;

[0156] Identify the second target data block within the second copy that corresponds to the first target data block, and obtain the verification reference data in the second target data block that corresponds to the data to be verified;

[0157] Determine the second check value corresponding to the check reference data, and compare the check calculation data and the second check value to obtain the first check result;

[0158] Obtain the first version number information included in the first metadata corresponding to the first target data block, and the second version number information included in the third metadata corresponding to the second target data block, and compare the first version number information and the second version number information to obtain the second verification result;

[0159] Based on the first and second verification results, the data verification result is determined.

[0160] In one example, data verification unit 1002 is used for:

[0161] If the first verification result indicates that the verification calculation data and the second verification value are the same, and the second verification result indicates that the first version number information and the second version number information are the same, then it is determined that the data verification result indicates that the first target data block has completed data synchronization.

[0162] If the first verification result indicates that the verification calculation data and the second verification value are different, and / or the second verification result indicates that the first version number information and the second version number information are different, then it is determined that the data verification result indicates that the first target data block has not completed data synchronization.

[0163] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 11 As shown, the electronic device 1100 includes: a memory 1101 and a processor 1102.

[0164] Memory 1101; a memory used to store executable instructions of processor 1102.

[0165] The processor 1102 is configured to perform the method provided in the above embodiments.

[0166] The electronic device also includes a receiver 1103 and a transmitter 1104. The receiver 1103 is used to receive instructions and data sent by an external device, and the transmitter 1104 is used to send instructions and data to an external device.

[0167] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, perform the steps of the data synchronization method described in the above method embodiments. The storage medium can be either volatile or non-volatile computer-readable storage.

[0168] This application also provides a computer program product that carries computer execution instructions. The computer execution instructions include instructions that can be used to execute the steps of the data synchronization method in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0169] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0170] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0171] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0172] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0173] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0174] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0175] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0176] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0177] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data synchronization method, characterized in that, The method includes: Identify a first target data block in the first replica; wherein the first replica represents the replica that has failed; the first target data block represents the data block that is updated when the first replica fails; The data to be verified within the first target data block is verified to obtain a data verification result; wherein, the data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has completed the data update. If, based on the data verification result, it is determined that the first target data block has not completed data updates, then, based on the second target data block in the second replica corresponding to the first target data block, the first metadata corresponding to the first target data block is first updated, and then the data to be verified in the first target data block is updated to complete the data synchronization processing of the first replica; wherein, the second replica represents a replica that has not experienced a failure; and the first metadata represents the attribute information of the data to be verified.

2. The method according to claim 1, characterized in that, The determination of the first target data block in the first copy includes: Obtain the target sequence number information corresponding to the first replica; wherein, the target sequence number information is used to determine the data block in the first replica that has completed data update before the failure occurs; Based on the target sequence number information, the first target data block is determined from the data blocks included in the first copy.

3. The method according to claim 2, characterized in that, The method further includes: After completing the data synchronization process for the first copy, the target serial number information is updated.

4. The method according to claim 2, characterized in that, Based on the target sequence number information, the first target data block is determined from the data blocks included in the first copy, including: Determine the second metadata corresponding to each data block contained in the first replica; wherein the second metadata includes global sequence number information; the global sequence number information represents the data update order of each data block contained in the first replica before data synchronization; Based on the target sequence number information, target identifier data is determined from the global sequence number information corresponding to each data block contained in the first copy, and the data block corresponding to the target identifier data is determined as the first target data block; wherein, the value of the target identifier data is greater than the value of the target sequence number information.

5. The method according to claim 1, characterized in that, Before performing data verification on the data to be verified within the first target data block, the method further includes: Determine the data update information included in the first metadata corresponding to the first target data block; wherein, the data update information includes update start position information and update length information; Based on the update start position information and the update length information, the data to be verified is determined from the first target data block.

6. The method according to claim 1, characterized in that, Data verification is performed on the data to be verified within the first target data block to obtain the data verification result, including: Calculate the verification calculation data corresponding to the data to be verified according to the pre-set data verification method; The verification calculation data is compared with the verification value in the first metadata corresponding to the first target data block to obtain the first verification result; Obtain the first version number information included in the first metadata corresponding to the first target data block, and the second version number information included in the third metadata corresponding to the second target data block, and compare the first version number information and the second version number information to obtain the second verification result; Based on the first verification result and the second verification result, the data verification result is determined.

7. The method according to claim 6, characterized in that, Determining the data verification result based on the first verification result and the second verification result includes: If the first verification result indicates that the verification calculation data and the second verification value are the same, and the second verification result indicates that the first version number information and the second version number information are the same, then it is determined that the data verification result indicates that the first target data block has completed data synchronization. If the first verification result indicates that the verification calculation data and the second verification value are different, and / or the second verification result indicates that the first version number information and the second version number information are different, then it is determined that the data verification result indicates that the first target data block has not completed data synchronization.

8. A data synchronization device, characterized in that, include: The first determining unit is configured to determine a first target data block in the first replica; wherein the first replica represents a replica that has failed; and the first target data block represents a data block that is updated when the first replica fails. A data verification unit is used to perform data verification on the data to be verified in the first target data block and obtain a data verification result; wherein, the data to be verified represents the data that was updated when the first copy failed; the data verification result is used to determine whether the first target data block has completed the data update. The data synchronization unit is configured to, if based on the data verification result it is determined that the first target data block has not completed data updates, then based on the second target data block in the second replica corresponding to the first target data block, first update the first metadata corresponding to the first target data block, and then update the data to be verified in the first target data block, thereby completing the data synchronization processing of the first replica; wherein, the second replica represents a replica that has not experienced a failure; and the first metadata represents the attribute information of the data to be verified.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the data synchronization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data synchronization method as described in any one of claims 1 to 7.

Images