Data consistency verification methods, apparatus, computer equipment, and storage media
By using the identifier sequence of data blocks in a distributed storage system for data consistency verification, the new identifier sequence is written only for the data blocks that need to be updated, which solves the problem of high resource overhead in the prior art and achieves low-overhead and high-efficiency data consistency verification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2026-04-03
AI Technical Summary
In existing technologies, data consistency verification methods in distributed storage systems require writing data version information to all data blocks, resulting in significant resource overhead, especially in random lower-write application scenarios where system write performance is reduced.
The identifier sequence of data blocks is used as data version information. The new identifier sequence is only written to the data blocks that need to be updated. Data consistency is checked by generating and matching the identifier sequence, thus avoiding writing the new identifier sequence to the data blocks that are not being updated.
It saves resource consumption, improves the write performance of distributed storage systems, and can accurately reflect data consistency within data block groups, balancing low overhead and verification accuracy.
Smart Images

Figure CN116680281B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data consistency verification method, apparatus, computer equipment, storage medium, and computer program product. Background Technology
[0002] Distributed storage systems typically contain data blocks with interconnected update relationships, such as data blocks within the same verification calculation group in Erasure Coding (EC) mode. These interconnected data blocks form a group of related data blocks. Because distributed storage systems may experience data storage anomalies due to software or hardware failures, it is usually necessary to perform periodic or ad-hoc data consistency checks or inspections on each data block in the group of related data blocks to monitor the correctness of the stored data.
[0003] In related technologies, data consistency can be verified by using data version information such as timestamps or version numbers recorded in each data block of a related data block group. Theoretically, when data is written, each data block should record the same data version information. If the data version information currently recorded in each data block is different, the data consistency verification fails.
[0004] However, the method of using data version information such as timestamps or version numbers for data consistency verification requires writing new data version information to all data blocks in the associated data block group at the same time when writing data to any data block in the associated data block group in order to meet the requirements of data consistency monitoring within the associated data block group, resulting in a large resource overhead. Summary of the Invention
[0005] Therefore, it is necessary to provide a data consistency verification method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can save resource consumption in response to the above-mentioned technical problems.
[0006] Firstly, this application provides a data consistency verification method. The method includes:
[0007] In response to a data consistency verification request, the identifier sequence of the current record of each data block in the target data block group is obtained; the identifier sequence is generated when the data block is updated and is used to characterize the update sequence of each data block in the target data block group when the current data block is updated;
[0008] Based on the baseline identifier sequence corresponding to the target data block group and the preset sequence matching rules, the baseline identifier sequence is matched with the identifier sequence currently recorded in each data block to obtain the matching result of each data block; the baseline identifier sequence is used to characterize the global update sequence of the target data block group.
[0009] The data consistency verification result of the target data block group is determined based on the matching results of each data block.
[0010] In the above method, an identifier sequence representing the update sequence of each data block is used as data version information for data consistency verification. When writing data, only the data block that needs to be updated needs to be written with a new identifier sequence. It is not necessary to write a new identifier sequence for the data block that does not need to be updated. Therefore, data consistency verification based on the above method can save resource overhead.
[0011] In one embodiment, the step of matching the reference identifier sequence with the identifier sequence currently recorded in each data block according to the reference identifier sequence corresponding to the target data block group and a preset sequence matching rule to obtain the matching result of each data block includes:
[0012] Obtain the baseline identifier sequence corresponding to the target data block group from the check block corresponding to the target data block group;
[0013] For each data block, the identifier information preceding the identifier information of the data block in the baseline identifier sequence is determined as the identifier information to be removed;
[0014] Remove the identifier information to be removed from the baseline identifier sequence to obtain the matching sequence, and remove the identifier information to be removed from the identifier sequence currently recorded in the data block to obtain the identifier sequence to be verified;
[0015] The sequence to be matched is compared with the sequence of identifiers to be verified to obtain the matching result of the data block.
[0016] In the above embodiments, the matching results of each data block are obtained by comparing the identification sequence to be verified and the matching sequence to be matched corresponding to each data block. This can accurately reflect whether there are data anomalies in the target data group, that is, it can obtain accurate data consistency verification results. Therefore, this method can balance low overhead and verification accuracy.
[0017] In one embodiment, the method further includes:
[0018] In response to a data block update request, the target data block that needs to be updated in the target data block group is determined, and the reference identifier sequence corresponding to the target data block group is obtained from the check block corresponding to the target data block group;
[0019] In the baseline identifier sequence corresponding to the target data block group, the identifier information of the target data block is moved before the identifier information of other data blocks in the baseline identifier sequence to obtain the target identifier sequence. Based on the target identifier sequence, the identifier sequence currently recorded for the target data block is updated.
[0020] In the above embodiments, when a data block is updated, a baseline identifier sequence representing the global update time sequence of the target data block group can be read from the verification block, and the baseline identifier sequence can be updated based on the identifier information of the data block to be updated. During data consistency inspection, the consistency verification result can be obtained based on the baseline identifier sequence in the verification block and the identifier sequence currently recorded in each data block, balancing low overhead and verification accuracy.
[0021] In one embodiment, the method further includes:
[0022] In response to a data block update request, determine the target data block in the target data block group that needs to be updated;
[0023] Determine a reference identifier sequence based on the identifier sequences currently recorded in other data blocks in the target data block group, excluding the target data block;
[0024] The identifier information of the target data block in the reference identifier sequence is moved before the identifier information of other data blocks in the reference identifier sequence to obtain the target identifier sequence, and the identifier sequence currently recorded for the target data block is updated based on the target identifier sequence.
[0025] In the above embodiments, a new identifier sequence can be generated based on the identifier sequence of other data blocks and the identifier information of the data block to be updated. In scenarios where the computer device adopts a writing method that does not read the old check code from the check block, it can avoid reading data from the check block again and save resource overhead.
[0026] In one embodiment, determining the reference identifier sequence based on the identifier sequences currently recorded for other data blocks in the target data block group besides the target data block includes:
[0027] The identifier sequences of the currently recorded data blocks other than the target data block in the target data block group are used as candidate identifier sequences, and the local longest common subsequence of any two candidate identifier sequences is calculated.
[0028] From each of the local longest common subsequences, the global longest common subsequence is determined, and from the candidate identifier sequence containing the global longest common subsequence, a reference identifier sequence is determined.
[0029] In the above embodiments, by determining a reference identifier sequence from other identifier sequences containing the longest common subsequence, the reference identifier sequence can characterize or approximately characterize the global update timing of the target data block group. Thus, it is possible to avoid reading additional data from the check block, thus achieving both low overhead and accuracy.
[0030] In one embodiment, updating the identifier sequence of the currently recorded target data block based on the target identifier sequence includes:
[0031] According to the preset mapping rules, the target index identifier corresponding to the target identifier sequence is determined;
[0032] Update the index identifier corresponding to the identifier sequence currently recorded in the target data block to the target index identifier.
[0033] In the above embodiments, by mapping the identifier sequence to the index identifier, the index identifier corresponding to the identifier sequence can be recorded in the data block and the check block. The index identifier occupies less memory, thus saving memory overhead. Furthermore, the sequence mapping table can be shared among the redundant groups (when the subgroup width is the same) within the distributed storage system, further saving memory overhead.
[0034] Secondly, this application also provides a data consistency verification device. The device includes:
[0035] The acquisition module is used to acquire the identifier sequence of the current record of each data block in the target data block group in response to the data consistency verification request; the identifier sequence is generated when the data block is updated and is used to characterize the update time sequence of each data block in the target data block group when the current data block is updated;
[0036] The matching module is used to match the benchmark identifier sequence with the identifier sequence currently recorded in each data block according to the benchmark identifier sequence corresponding to the target data block group and the preset sequence matching rules, so as to obtain the matching result of each data block; the benchmark identifier sequence is used to characterize the global update time sequence of the target data block group;
[0037] The first determining module is used to determine the data consistency verification result of the target data block group based on the matching result of each data block.
[0038] In one embodiment, the matching module is specifically used for:
[0039] Obtain the baseline identifier sequence corresponding to the target data block group from the verification block corresponding to the target data block group; for each data block, determine the identifier information in the baseline identifier sequence that precedes the identifier information of the data block as the identifier information to be removed; remove the identifier information to be removed from the baseline identifier sequence to obtain the matching sequence, and remove the identifier information to be removed from the identifier sequence currently recorded in the data block to obtain the verification identifier sequence; compare the matching sequence with the verification identifier sequence to obtain the matching result of the data block.
[0040] In one embodiment, the matching module is specifically used for:
[0041] If the sequence to be matched is different from the sequence to be verified, the matching result of the data block is determined to be a failed match; if the sequence to be matched is the same as the sequence to be verified, the matching result of the data block is determined to be a successful match.
[0042] In one embodiment, the device further includes:
[0043] The second determining module is used to respond to a data block update request, determine the target data block in the target data block group that needs to be updated, and obtain the reference identifier sequence corresponding to the target data block group from the check block corresponding to the target data block group;
[0044] The first update module is used to move the identifier information of the target data block in the reference identifier sequence corresponding to the target data block group to before the identifier information of other data blocks in the reference identifier sequence to obtain the target identifier sequence, and update the identifier sequence currently recorded by the target data block based on the target identifier sequence.
[0045] In one embodiment, the device further includes:
[0046] The third determining module is used to determine the target data block that needs to be updated in the target data block group in response to the data block update request;
[0047] The fourth determining module is used to determine a reference identifier sequence based on the identifier sequences currently recorded in other data blocks in the target data block group, excluding the target data block.
[0048] The second update module is used to move the identification information of the target data block in the reference identification sequence before the identification information of other data blocks in the reference identification sequence to obtain the target identification sequence, and update the identification sequence currently recorded by the target data block based on the target identification sequence.
[0049] In one embodiment, the fourth determining module is specifically used for:
[0050] The identifier sequences of the currently recorded data blocks other than the target data block in the target data block group are used as candidate identifier sequences. The longest local common subsequence of any two candidate identifier sequences is calculated. From each of the longest local common subsequences, the longest global common subsequence is determined, and a reference identifier sequence is determined from the candidate identifier sequences that contain the longest global common subsequence.
[0051] In one embodiment, the first update module or the second update module is specifically used for:
[0052] According to the preset mapping rules, the target index identifier corresponding to the target identifier sequence is determined; the index identifier corresponding to the identifier sequence currently recorded in the target data block is updated to the target index identifier.
[0053] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in the first aspect.
[0054] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.
[0055] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.
[0056] The aforementioned data consistency verification methods, apparatus, computer equipment, storage media, and computer program products generate an identifier sequence representing the update sequence of each data block at any time when any data block within a data block group is updated, based on the identifier information of each data block in the data block group. This sequence is recorded in the data block to be updated and may also be recorded in a check block or other database. During data consistency verification, the computer equipment can obtain a baseline identifier sequence representing the global update sequence of the target data block group from the check block or other database and match it with the identifier sequence currently recorded in each data block. Since the identifier sequence can represent the update sequence of each data block, if the update sequence represented by the identifier sequence recorded in a data block conflicts with the update sequence represented by the baseline identifier sequence, the data consistency verification fails, indicating that the data block group may have data anomalies due to system failure. In this method, the identifier sequence representing the update sequence of each data block is used as data version information for data consistency verification. When writing data, only the data block that needs to be updated needs to have a new identifier sequence written; it is not necessary to write a new identifier sequence for data blocks that do not need to be updated. Therefore, performing data consistency verification based on the above method can save resource overhead. Attached Figure Description
[0057] Figure 1 This is a flowchart illustrating a data consistency verification method in one embodiment;
[0058] Figure 2 This is a flowchart illustrating the matching process in one embodiment;
[0059] Figure 3This is a flowchart illustrating the data consistency verification method in another embodiment;
[0060] Figure 4 This is a flowchart illustrating the data consistency verification method in another embodiment;
[0061] Figure 5 This is a structural block diagram of a data consistency verification device in one embodiment;
[0062] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0064] First, before introducing the technical solutions of the embodiments of this application, the technical background or evolution of the embodiments of this application will be introduced. Since distributed storage systems may experience data anomalies due to software or hardware failures, in order to monitor data correctness, it is usually necessary to periodically or irregularly perform data consistency checks or data consistency inspections on each data block in the associated data block group (or redundancy group). In related technologies, data consistency checks can be performed using data version information such as timestamps or version numbers recorded by each data block in the associated data block group. However, this method suffers from high resource overhead and severely reduces the write performance of the distributed storage system. For example, for an erasure coding system with a redundancy ratio of N+M (i.e., N data blocks and M check blocks), when writing data to any data block among the N data blocks, the current timestamp (or version number) needs to be recorded in the N data blocks and M check blocks, which can generally be stored as part of the metadata. During data consistency checks, the timestamps currently recorded in each data block can be compared. If the timestamps are different, the data consistency check fails, indicating data loss or metadata recording errors, or other data anomalies. Business personnel can promptly troubleshoot faults based on consistency verification results, ensuring the normal operation of the distributed storage system. However, for scenarios involving random lower-write operations, data typically only needs to be written to a subset of data blocks. Timestamp-based methods, on the other hand, require writing timestamps to all data blocks for each write operation, consuming significant resources and degrading system write performance.
[0065] Against this backdrop, the applicant, through long-term research and development and experimental verification, proposes a data consistency verification method. When any data block within a data block group is updated, an identifier sequence representing the update sequence of each data block is generated based on the identifier information of each data block in the data block group, and recorded in the updated data block and the verification block. During data consistency verification, a baseline identifier sequence representing the global update sequence of the target data block group can be obtained from the verification block and matched with the currently recorded identifier sequences of each data block. Since the identifier sequence can represent the update sequence of each data block, if the update sequence represented by the identifier sequence recorded in a data block conflicts with the update sequence represented by the baseline identifier sequence, the data consistency verification fails, indicating that the data block group may have data anomalies due to system failure. In this method, the identifier sequence representing the update sequence of each data block is used as data version information for data consistency verification. When writing data, only the data blocks that need to be updated need to have a new identifier sequence written; it is not necessary to write a new identifier sequence for data blocks that do not need to be updated. Therefore, data consistency verification based on the above method can save resource overhead. In addition, it should be noted that the applicant has devoted a great deal of creative effort to discovering the technical problems of this application and to developing the technical solutions described in the following embodiments.
[0066] In one embodiment, such as Figure 1 As shown, a data consistency verification method is provided, which can be applied to computer devices in a distributed storage system. The computer devices can be, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, servers, server clusters, etc. In this embodiment, the method includes the following steps:
[0067] Step 101: In response to the data consistency verification request, obtain the identifier sequence of the current record of each data block in the target data block group.
[0068] In implementation, the target data block group can be a data stripe containing data blocks and parity blocks; alternatively, it can be a group of data blocks obtained by grouping the data blocks contained in a data stripe, meaning the target data block group contains only data blocks and no parity blocks. For example, in an erasure coding system with a redundancy ratio of N+M (e.g., 4+2), the target data block group can contain N data blocks and M parity blocks. In some examples, if there are many data blocks within the redundancy group, such as in a 20+2 mode (i.e., N=20), each data block can be divided into multiple subgroups, such as 5 subgroups, each containing 4 data blocks (subgroup width is 4). Each subgroup then constitutes a target data block group, containing only data blocks and no parity blocks. The data block grouping method can be to divide the data blocks within a redundancy group into multiple subgroups of target subgroup width (e.g., 4) according to their logical address from low to high.
[0069] Each data block in the target data block group has identification information, and the sequence of these identification information is called the identification sequence. For example, for a target data block group containing 4 data blocks, the identification information of each data block can be set to 1, 2, 3, 4 (or other forms of identification can be used), and the identification sequence can be (1, 2, 3, 4) or (2, 1, 3, 4), etc. The identification sequence currently recorded for each data block is generated when the data block is updated, and can be used to characterize the update time sequence of each data block in the target data block group when the data block is updated. The identification sequence can be stored as metadata of the data blocks. The identification information of each data block in the identification sequence can be ordered sequentially from the most recent to the oldest update time of the data blocks. That is, the update time of the data block with the identification information ranked earlier is not earlier than (but can be later, i.e., the update time is more recent or equal to) the update time of the data block ranked later. If only one data block is updated each time data is written, then in the identifier sequence of that data block, such as (1, 2, 3, 4), at the moment of the update, the data block with identifier 1 (let's call it D1) that is first in the sequence has the most recent update time (because the identifier sequence is generated when the data block is updated, the current data block is the first in the sequence), and the data block with identifier 4 that is last in the sequence has the oldest update time. It's understandable that if more than one data block is updated each time, the data blocks being updated simultaneously can record the same identifier sequence. For example, if data blocks D3 and D4 are updated simultaneously, their recorded identifier sequences can both be (3, 4, 1, 2) or both can be (4, 3, 1, 2). It should be noted that the recorded identifier sequences are the same for data blocks that are updated simultaneously.
[0070] Optionally, a data consistency verification period can be set to automatically trigger data consistency verification requests at regular intervals, or users can actively trigger data consistency verification requests. In response to the data consistency verification request, the computer device can send read requests to each data block to obtain the identifier sequence currently recorded in each data block. It is understood that a data block can record the identifier sequence itself, or it can record the index identifier corresponding to the identifier sequence. If the data block records the index identifier corresponding to the identifier sequence, the computer device can obtain the index identifier currently recorded in each data block and determine the identifier sequence corresponding to that index identifier based on a preset mapping rule.
[0071] Step 102: Based on the baseline identifier sequence corresponding to the target data block group and the preset sequence matching rules, the baseline identifier sequence is matched with the identifier sequence of the current record of each data block to obtain the matching result of each data block.
[0072] The baseline identifier sequence is used to characterize the global update sequence of the target data block group. The global update sequence reflects the latest update order of each data block in the target data block group. Since the identifier sequence of a data block record is generated when that data block is updated, it can characterize the update sequence of each data block at that time. Therefore, before the verification time, the identifier sequence of the data block record that was most recently updated before the verification time can characterize the latest update order of each data block. To obtain the baseline identifier sequence characterizing the global update sequence of the target data block group, the identifier sequence generated for the updated data block can be permanently stored during data updates, such as storing it locally on the computer device or storing it in the corresponding verification block of the target data block group (i.e., the M verification blocks in the N+M mode, which can be stored as metadata of the verification blocks). Thus, the computer device can obtain the baseline identifier sequence characterizing the global update sequence of the target data block group from the local database or the verification blocks.
[0073] In implementation, the computer device can match the baseline identifier sequence corresponding to the target data block group with the identifier sequence currently recorded for each data block. If a data block does not match the baseline identifier sequence, it indicates a conflict between the update time sequence of the data blocks represented by the identifier sequence currently recorded for that data block and the update time sequence represented by the baseline identifier sequence; if they match, there is no conflict. Thus, the matching result for each data block can be obtained.
[0074] Step 103: Determine the data consistency verification result of the target data block group based on the matching results of each data block.
[0075] During implementation, after the computer equipment determines the matching results of each data block, it can determine the data consistency verification result based on the matching results. For example, if all data blocks match successfully, the data consistency verification result can be determined as successful; if any data block fails to match, the data consistency verification result can be determined as failed, indicating that there is a data anomaly in the target data block group. The computer equipment can output a success or failure message, as well as the identifier of the data block that failed to match, to remind the business to promptly investigate system faults.
[0076] In the aforementioned data consistency verification method, when any data block within a data block group is updated, an identifier sequence representing the update sequence of each data block at that time is generated based on the identifier information of each data block in the data block group, and recorded in the data block where the data is updated, and may also be recorded in the check block or other databases. During data consistency verification, the computer device can obtain the baseline identifier sequence from the check block or other databases and match it with the identifier sequence currently recorded in each data block. Since the identifier sequence can represent the update sequence of each data block, if the update sequence represented by the identifier sequence recorded in a data block conflicts with the global update sequence of the data block group represented by the baseline identifier sequence, the data consistency verification fails, indicating that the data block group may have data anomalies due to system failure. In this method, the identifier sequence representing the update sequence of each data block is used as data version information for data consistency verification. When writing data, only the data block that needs to be updated needs to have a new identifier sequence written; it is not necessary to write a new identifier sequence for data blocks that do not need to be updated. Therefore, performing data consistency verification based on the above method can save resource overhead.
[0077] In one embodiment, such as Figure 2 As shown, the matching process in step 102 specifically includes the following steps:
[0078] Step 201: Obtain the reference identifier sequence corresponding to the target data block group from the check block corresponding to the target data block group.
[0079] In implementation, when updating any data block in the target data block group, an identifier sequence representing the update sequence of each data block at that time can be generated for the data block to be updated. This identifier sequence is recorded as metadata of the updated data block, and it can also be recorded as metadata of a check block. That is, regardless of which data block is updated, the check block will record the latest updated identifier sequence, which represents the global update sequence. Therefore, the computer device can obtain the reference identifier sequence representing the global update sequence of the target data group from the check block.
[0080] Step 202: For each data block, the identifier information preceding the identifier information of the data block in the baseline identifier sequence is determined as the identifier information to be removed.
[0081] In implementation, for each data block, the computer device can determine the identifier information preceding the identifier information of that data block in the baseline identifier sequence as the identifier information to be removed. For example, if the baseline identifier sequence is (3, 2, 1, 4), when matching data block D2, the identifier information "3" preceding the identifier information "2" of D2 in the baseline identifier sequence can be determined as the identifier information to be removed. In this example, for data block D3, since its identifier information "3" is the first character in the sequence, its corresponding identifier information to be removed is empty.
[0082] Step 203: Remove the identifier information to be removed from the baseline identifier sequence to obtain the sequence to be matched, and remove the identifier information to be removed from the identifier sequence currently recorded in the data block to obtain the identifier sequence to be verified.
[0083] In implementation, the computer device can remove the identifier information to be removed from the baseline identifier sequence. As in the previous example, for data block D2, removing "3" from the baseline identifier sequence yields the matching sequence (2, 1, 4) for data block D2. Furthermore, the computer device can remove the identifier information "3" from the currently recorded identifier sequence of data block D2 (e.g., (2, 4, 3, 1)) to obtain the verification identifier sequence (2, 4, 1) for data block D2.
[0084] Step 204: Compare the sequence to be matched with the sequence to be verified to obtain the matching result of the data block.
[0085] In implementation, the computer device can compare the sequence to be matched (2, 1, 4) and the sequence to be verified (2, 4, 1) corresponding to data block D2. If they are the same, the match is successful or consistent. If they are different, the match fails or is inconsistent. Thus, the computer device can determine the corresponding sequence to be matched and the sequence to be verified for each data block, and compare them separately to obtain the matching result for each data block.
[0086] In one example, the computer device can perform matching processing on each data block in order of its update time, from most recent to oldest, according to the update sequence represented by the baseline identifier sequence. As shown in Table 1, the identifier sequences currently recorded for data blocks D1 to D4 are (1, 2, 3, 4), (2, 4, 3, 1), (3, 2, 4, 1), and (4, 2, 3, 1), respectively, while the baseline identifier sequence is (3, 2, 4, 1). Based on the update sequence represented by the baseline identifier sequence, the update time of data block D3 is relatively recent (no earlier than other data blocks), and the update time of data block D1 is relatively earliest (no later than other data blocks). Therefore, the matching order can be D3, D2, D4, D1. First, matching processing is performed on data block D3. At this point, the identifier information to be removed is empty, so the identifier sequence (3, 2, 4, 1) recorded in D3 is directly compared with the baseline identifier sequence (3, 2, 4, 1). Since they are the same, the match is successful. Then, the computer device can perform matching processing on data block D2. At this time, the identifier information to be removed is "3", the sequence to be matched is (2, 4, 1), and the identifier sequence to be verified is (2, 4, 1). Since they are the same, the match is successful. This process is repeated for all data blocks to complete the matching processing and obtain the matching results for each data block.
[0087] Table 1
[0088]
[0089] Understandably, if multiple data blocks update data simultaneously before data consistency verification, the current record identifier sequences of multiple data blocks may be the same. Therefore, when matching data blocks one by one, the sequence to be matched can be compared with the remaining unmatched data blocks simultaneously. As shown in Table 1, when matching data block D2, the identifier information "3" corresponding to data block D2 can be removed from the identifier sequences of each unmatched data block (D1, D2, D4) to obtain their respective identifier sequences to be verified. Then, the identifier sequence to be matched corresponding to data block D2 is compared with the identifier sequences to be verified corresponding to each unmatched data block. If the identifier sequence to be verified of a data block is the same as the sequence to be matched, the corresponding data block is matched successfully; if the identifier sequence to be verified of data block D2 is different from the sequence to be matched, the data block D2 is matched but the result is a failure; for other data blocks besides data block D2, if the sequence is different from the sequence to be matched, the matching is incomplete and will proceed to the next round of matching. Therefore, this method can improve matching efficiency when multiple data blocks have the same identifier sequence.
[0090] In this embodiment, the identification sequence to be verified and the matching sequence corresponding to each data block are compared. Furthermore, the identification sequences recorded for each data block are compared one by one based on the update time sequence of each data block as represented by the baseline identification sequence. For other data blocks whose update time is earlier than the current data block, the relative order of their identification information in the identification sequence recorded in the current data block should be consistent with the relative order in the baseline identification sequence. If they are inconsistent, it means that the update time sequence of each data block reflected in the identification sequence of this data block conflicts with the update time sequence reflected in the baseline identification sequence, indicating that there is a data anomaly in the target data group. Therefore, the matching results of each data block can accurately reflect whether there is a data anomaly within the target data group, that is, an accurate data consistency verification result can be obtained. Thus, this method can balance low overhead and verification accuracy.
[0091] In one embodiment, such as Figure 3 As shown, the provided data consistency verification method also includes a data update process, specifically including the following steps:
[0092] Step 301: In response to the data block update request, determine the target data block in the target data block group that needs to be updated, and obtain the reference identifier sequence corresponding to the target data block group from the check block corresponding to the target data block group.
[0093] In implementation, the computer device can respond to a data block update request and determine the target data block in the target data block group whose data needs to be updated. For example, the computer device can determine the target data block based on the key value contained in the data block update request, or based on the attribute information of the data to be written contained in the data block update request. Then, the computer device can obtain metadata from the checksum block corresponding to the target data block group and extract the base identifier sequence corresponding to the target data block group from the metadata of the checksum block. The process of the computer device generating a new checksum based on the data to be written and sending the data to be written to the target data block for storage, as well as sending the new checksum to the checksum block for storage, will not be described in detail here.
[0094] Step 302: Move the identifier information of the target data block in the baseline identifier sequence corresponding to the target data block group before the identifier information of other data blocks in the baseline identifier sequence to obtain the target identifier sequence, and update the identifier sequence of the current record of the target data block based on the target identifier sequence.
[0095] In implementation, after the computer device obtains the reference identifier sequence corresponding to the target data block group, it can move the identifier information of the target data block in the reference identifier sequence before other identifier information. For example, if the reference identifier sequence obtained from the check block is (1, 2, 3, 4), and the target data block is D2 (i.e., the data block to be updated this time), then the identifier information "2" of data block D2 can be moved before other identifier information (to the first position in the sequence), resulting in the target identifier sequence (2, 1, 3, 4). It is understandable that if there are multiple target data blocks, the identifier information of each target data block can be randomly ordered. For example, if there are two target data blocks, D2 and D4, then the identifier information "2" and "4" can be moved before other identifier information, resulting in the target identifier sequence (2, 4, 1, 3) or (4, 2, 1, 3).
[0096] Then, the computer device can send write requests to the target data block and the check block to update the identifier sequence currently recorded in the target data block to the target identifier sequence, and to update the reference identifier sequence corresponding to the target data block group currently recorded in the check block to the target identifier sequence.
[0097] In one example, in their initial state—when no data has been written or updated—each data block and check block can record the same invalid identifier sequence in the metadata. Each invalid identifier in the sequence corresponds one-to-one with a data block. As shown in Table 2, each data block (e.g., D1 to D4) and check block (P, Q) in the target data block group can initially record an invalid identifier sequence of (I1, I2, I3, I4), where I1 corresponds to data block D1 and I4 corresponds to D4. When data is written, the invalid identifier information corresponding to the target data block to be updated is replaced with the identifier information of the data block and moved before the invalid identifier information to generate a new identifier sequence.
[0098] Table 2
[0099]
[0100] As shown in Table 2, when data is written to data blocks D1, D2, and D3, an identifier sequence (1, 2, 3, I4) can be generated (where the identifier information "1", "2", and "3" can be randomly ordered), and this new identifier sequence is recorded in data blocks D1, D2, and D3 and check blocks P and Q. Subsequently, if the computer device receives a data update request for data block D4, a new identifier sequence (4, 1, 2, 3) can be generated and updated in data block D4 and check blocks P and Q.
[0101] In this embodiment, when a data block is updated, a baseline identifier sequence can be read from the check block, and the baseline identifier sequence can be updated based on the identifier information of the data block to be updated, resulting in a new identifier sequence, which is recorded in both the data block to be updated and the check block. Therefore, the identifier sequence can be used as data version information. When updating any data block in the data block group, it is not necessary to write new data version information to the data blocks that are not being updated. Furthermore, during data consistency checks, the consistency check result can be obtained based on the baseline identifier sequence in the check block and the currently recorded identifier sequence of each data block. Thus, this method can balance low overhead and high check accuracy.
[0102] In one embodiment, such as Figure 4 As shown, the provided data consistency verification method also includes another data update process, which specifically includes the following steps:
[0103] Step 401: In response to the data block update request, determine the target data block in the target data block group that needs to be updated.
[0104] In practice, computer equipment can respond to a data block update request and determine the target data block in the target data block group that needs to be updated.
[0105] Step 402: Determine the reference identifier sequence based on the identifier sequences of the currently recorded data blocks other than the target data block in the target data block group.
[0106] In implementation, the other data blocks in the target data block group, excluding the target data block, are the data blocks that do not currently need to be updated. The computer device can obtain the identifier sequence of its current record from the other data blocks and determine the reference identifier sequence from each identifier sequence. The reference identifier sequence may or may not be the same as the baseline identifier sequence recorded in the check block.
[0107] Step 403: Move the identifier information of the target data block in the reference identifier sequence before the identifier information of other data blocks in the reference identifier sequence to obtain the target identifier sequence, and update the identifier sequence of the currently recorded target data block based on the target identifier sequence.
[0108] In implementation, the computer device can move the identifier information of the target data block in the reference identifier sequence before other identifier information to obtain the target identifier sequence. Then, the computer device can update the identifier sequence of the currently recorded target data block based on the target identifier sequence. The specific process can be referred to the description in step 302, and will not be repeated here.
[0109] In this embodiment, when the target data block is updated, a reference identifier sequence can be determined from the identifier sequences recorded in other data blocks within the target data block group. The reference identifier sequence can characterize or approximately characterize the global update sequence of the target data block group, thus allowing the new identifier sequence of the target data block to be updated based on the reference identifier sequence. That is, the computer device can read the identifier sequence from other data blocks instead of from the check block. It is understandable that when updating data in a data block group, two writing methods are typically used. One mode is for scenarios where the old checksum needs to be read from the check block, such as RMW (read modify write) mode. This mode is generally used when updating data in a small number of data blocks, allowing a new checksum to be obtained based on the new data and the old checksum, and then written into the check block. In this mode, data such as the checksum needs to be read from the check block, so the reference identifier sequence can be read from the check block simultaneously without consuming additional read resources. The other mode is for scenarios where the old checksum does not need to be read from the check block, such as RCW (read construct write) mode. In this mode, old data is read from other data blocks that do not need updating, and a new checksum is directly generated based on the old data in the other data blocks and the newly written data in the data block to be updated. Therefore, when reading old data from other data blocks, the identifier sequence can be read simultaneously, and the reference identifier sequence can be determined, thus eliminating the need to consume additional resources to read the data in the checksum block. Therefore, in scenarios where the computer device uses a write method that does not read old checksums from the checksum block, a new identifier sequence can be generated based on the identifier sequence of other data blocks and the identifier information of the data block to be updated, avoiding the need to read data from the checksum block and saving resource overhead.
[0110] In one embodiment, the process of determining the reference identifier sequence in step 402 specifically includes the following steps: taking the identifier sequences of the currently recorded data blocks other than the target data block in the target data block group as candidate identifier sequences, calculating the local longest common subsequence of any two candidate identifier sequences; determining the global longest common subsequence from each local longest common subsequence, and determining the reference identifier sequence from the candidate identifier sequences containing the global longest common subsequence.
[0111] In implementation, the computer device can obtain the identifier sequences of records from other data blocks within the target data block group that do not currently require updating, as candidate identifier sequences. Then, the computer device can calculate the longest local common subsequence (LCS) of any two candidate identifier sequences. The calculation method can employ a Generalized Suffix Tree (GST) algorithm or other algorithms. Afterward, the computer device can determine the longest LCS among the LCS sequences as the global longest common subsequence (GLS) of the target data block group. There can be one or more GLS sequences. The computer device can randomly select one of the candidate identifier sequences containing the GLS sequence as a reference identifier sequence. Optionally, if among the candidate identifier sequences containing the GLS sequence, there exists a candidate identifier sequence containing invalid identifier information (i.e., when generating the identifier sequence, at least one data block had not been updated and was still in its initial state), then the candidate identifier sequence containing invalid identifier information can be removed, and one can be selected as the reference identifier sequence from other candidate identifier sequences that contain the GLS sequence and do not contain invalid identifier information.
[0112] In one example, as shown in Table 3, if the current record identifier sequences of each data block D1 to D4 in the data block group are (1, 2, 3, I4), (2, 3, 1, I4), (2, 3, 1, I4), and (4, 2, 3, 1), respectively, when a write request for data block D2 is received, the current record identifier sequences of data blocks D1, D3, and D4 can be used as candidate identifier sequences, namely (1, 2, 3, I4), (2, 3, 1, I4), and (4, 2, 3, 1).
[0113] Table 3
[0114]
[0115] Then, the computer device can calculate the longest common subsequence (2, 3, 1) contained in at least two sequences within the group based on each candidate identifier sequence. Since the identifier sequences D3 and D4 contain this longest common subsequence, a reference identifier sequence can be determined from the identifier sequences D3 and D4. Specifically, since the identifier sequence (2, 3, 1, I4) of D3 contains the invalid identifier information "I4", while the identifier sequence of D4 does not contain invalid identifier information, and the specification states that the most recent update time of D4 is later than that of D3, the identifier sequence of D3 can be removed, and the identifier sequence of D4 can be determined as the reference identifier sequence. In this example, the determined reference identifier sequence (4, 2, 3, 1) is consistent with the baseline identifier sequence (4, 2, 3, 1) of the check block record; therefore, this reference identifier sequence can characterize the global update timeline of the data block group.
[0116] Upon receiving a write request for data block D3, two globally longest common subsequences can be determined based on the identifier sequences of data blocks D1, D2, and D4. These are the common subsequences (2, 3) contained in the sequences of D1 and D4, and (3, 1) contained in the sequences of D2 and D4. Since the identifier sequence of D1 contains invalid identifier information, this identifier sequence is removed, and a reference identifier sequence can be randomly selected from the identifier sequences of D2 and D4. As shown in Table 3, for case (a), if the identifier sequence (2, 4, 3, 1) of D2 is determined as the reference identifier sequence, the target identifier sequence (3, 2, 4, 1) can be generated and written to data block D3. In this case, the determined reference identifier sequence is consistent with the baseline identifier sequence of the check block record. For case (b), if the identifier sequence (4, 2, 3, 1) of D4 is determined as the reference identifier sequence, the target identifier sequence (3, 4, 2, 1) can be generated and written to data block D3. In this case, the determined reference identifier sequence can approximately characterize the global update sequence of the data block group.
[0117] Understandably, when all data blocks in the target data block group need to be updated, the target identifier sequence can be randomly generated based on the identifier information of each data block, thereby improving efficiency.
[0118] In this embodiment, a reference identifier sequence is determined from the other identifier sequences containing the longest common subsequence of the identifier sequences of other data blocks that do not currently need to be updated. This reference identifier sequence can characterize or approximately characterize the global update sequence of the data block group, thereby avoiding the need to read additional data from the check block, thus balancing low overhead and accuracy.
[0119] In one embodiment, the process of updating the identifier sequence in step 302 or step 403 specifically includes the following steps: determining the target index identifier corresponding to the target identifier sequence according to a preset mapping rule; updating the index identifier corresponding to the identifier sequence currently recorded in the target data block to the target index identifier.
[0120] In implementation, mapping rules can be pre-set, such as establishing a sequence mapping table to store the index identifiers corresponding to the possible sequence forms of the identification information of each data block. For example, for a scenario where the data block group width is 4 (i.e., the group contains four data blocks), the sequence mapping table used can be as shown in Table 4. It contains a total of 64 sequence forms with index identifiers from 0 to 63, among which I1, I2, I3, and I4 are invalid identification information.
[0121] Table 4 Sequence Mapping Table
[0122]
[0123]
[0124] After determining the target identifier sequence, the computer device can retrieve the corresponding index identifier from the sequence mapping table. Then, the computer device can distribute the index identifier to the data blocks and check blocks that need to be updated, causing them to update the records. That is, the data blocks and check blocks can record the index identifier corresponding to the identifier sequence, rather than directly recording the identifier sequence itself. The computer device can obtain the index identifier corresponding to the identifier sequence of the current record from the data block, and the index identifier corresponding to the base identifier sequence from the check block. Then, according to the mapping rule, it queries the corresponding sequence, generates the target identifier sequence based on the queried sequence, and determines the target index identifier corresponding to the target identifier sequence. Therefore, in the target data blocks and check blocks that need to be updated, the recorded index identifier can be updated to the target index identifier.
[0125] Understandably, if there are multiple target data block groups, such as in a 20+2 mode, the data blocks in the redundant group can be divided into 5 subgroups. The update of the identifier sequences of each subgroup is not coupled; that is, the computer device can generate a target identifier sequence for each data block in the target data block group that needs to be updated. The target identifier sequence of each subgroup can be mapped to a target index identifier. Then, the computer device can record the set of target index identifiers for each subgroup (or historical index identifiers for subgroups that do not currently need updating) as metadata in each data block and check block that needs to be updated. During data consistency verification, the index identifier corresponding to the target data block group to which the data block belongs can be extracted from the metadata recorded in the data block record, and then mapped to an identifier sequence for verification.
[0126] In this embodiment, by mapping the identifier sequence to the index identifier, the index identifier corresponding to the identifier sequence can be recorded in the data block and the check block. The memory occupied by the index identifier is generally smaller than that of the identifier sequence, thus saving memory overhead. Furthermore, the sequence mapping table can be shared among the redundant groups (when the subgroup width is the same) within the distributed storage system, further saving memory overhead.
[0127] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0128] Based on the same inventive concept, this application also provides a data consistency verification device for implementing the data consistency verification method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data consistency verification device embodiments provided below can be found in the limitations of the data consistency verification method described above, and will not be repeated here.
[0129] In one embodiment, such as Figure 5 As shown, a data consistency verification device 500 is provided, including: an acquisition module 501, a matching module 502, and a first determination module 503, wherein:
[0130] The acquisition module 501 is used to acquire the identifier sequence of the current record of each data block in the target data block group in response to the data consistency verification request; the identifier sequence is generated when the data block is updated and is used to characterize the update time sequence of each data block in the target data block group when the current data block is updated.
[0131] The matching module 502 is used to match the benchmark identifier sequence with the identifier sequence of the current record of each data block according to the benchmark identifier sequence corresponding to the target data block group and the preset sequence matching rules, so as to obtain the matching result of each data block; the benchmark identifier sequence is used to characterize the global update time sequence in the target data block group.
[0132] The first determining module 503 is used to determine the data consistency verification result of the target data block group based on the matching results of each data block.
[0133] In one embodiment, the matching module 502 is specifically configured to: obtain the baseline identifier sequence corresponding to the target data block group from the verification block corresponding to the target data block group; for each data block, determine the identifier information in the baseline identifier sequence that precedes the identifier information of the data block as the identifier information to be removed; remove the identifier information to be removed from the baseline identifier sequence to obtain the matching sequence, and remove the identifier information to be removed from the identifier sequence currently recorded in the data block to obtain the verification identifier sequence; compare the matching sequence with the verification identifier sequence to obtain the matching result of the data block.
[0134] In one embodiment, the matching module 502 is specifically used to: determine that the matching result of the data block is a matching failure when the sequence to be matched is different from the identifier sequence to be verified; and determine that the matching result of the data block is a matching success when the sequence to be matched is the same as the identifier sequence to be verified.
[0135] In one embodiment, the device further includes a second determining module and a first updating module, wherein:
[0136] The second determining module is used to respond to the data block update request, determine the target data block in the target data block group that needs to be updated, and obtain the reference identifier sequence corresponding to the target data block group from the check block corresponding to the target data block group.
[0137] The first update module is used to move the identifier information of the target data block in the baseline identifier sequence corresponding to the target data block group before the identifier information of other data blocks in the baseline identifier sequence to obtain the target identifier sequence, and update the identifier sequence of the currently recorded target data block based on the target identifier sequence.
[0138] In one embodiment, the device further includes a third determining module, a fourth determining module, and a second updating module, wherein:
[0139] The third determination module is used to determine the target data block that needs to be updated in the target data block group in response to the data block update request.
[0140] The fourth determining module is used to determine the reference identifier sequence based on the identifier sequence of the currently recorded data blocks other than the target data block in the target data block group.
[0141] The second update module is used to move the identification information of the target data block in the reference identification sequence before the identification information of other data blocks in the reference identification sequence to obtain the target identification sequence, and update the identification sequence of the currently recorded target data block based on the target identification sequence.
[0142] In one embodiment, the fourth determining module is specifically used to: take the currently recorded identifier sequences of other data blocks in the target data block group, excluding the target data block, as candidate identifier sequences, calculate the local longest common subsequence of any two candidate identifier sequences; determine the global longest common subsequence from each local longest common subsequence, and determine the reference identifier sequence from the candidate identifier sequences containing the global longest common subsequence.
[0143] In one embodiment, the first update module or the second update module is specifically used to: determine the target index identifier corresponding to the target identifier sequence according to a preset mapping rule; and update the index identifier corresponding to the identifier sequence currently recorded in the target data block to the target index identifier.
[0144] Each module in the aforementioned data consistency verification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0145] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data required for or generated by the aforementioned data consistency verification method. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a data consistency verification method.
[0146] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0149] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A data consistency verification method, characterized in that, The method includes: In response to a data consistency verification request, the identifier sequence of the current record of each data block in the target data block group is obtained; the identifier sequence is generated when the data block is updated and is used to characterize the update sequence of each data block in the target data block group when the current data block is updated; Based on the baseline identifier sequence corresponding to the target data block group and the preset sequence matching rules, the baseline identifier sequence is matched with the identifier sequence currently recorded in each data block to obtain the matching result of each data block; the baseline identifier sequence is used to characterize the global update sequence of the target data block group. The data consistency verification result of the target data block group is determined based on the matching results of each data block; The step of matching the reference identifier sequence with the identifier sequence currently recorded in each data block according to the reference identifier sequence corresponding to the target data block group and a preset sequence matching rule to obtain the matching result of each data block includes: Obtain the baseline identifier sequence corresponding to the target data block group from the verification block corresponding to the target data block group; for each data block, determine the identifier information in the baseline identifier sequence that precedes the identifier information of the data block as the identifier information to be removed; remove the identifier information to be removed from the baseline identifier sequence to obtain the matching sequence, and remove the identifier information to be removed from the identifier sequence currently recorded in the data block to obtain the verification identifier sequence; compare the matching sequence with the verification identifier sequence to obtain the matching result of the data block.
2. The method according to claim 1, characterized in that, The method further includes: In response to a data block update request, the target data block that needs to be updated in the target data block group is determined, and the reference identifier sequence corresponding to the target data block group is obtained from the check block corresponding to the target data block group; In the baseline identifier sequence corresponding to the target data block group, the identifier information of the target data block is moved before the identifier information of other data blocks in the baseline identifier sequence to obtain the target identifier sequence. Based on the target identifier sequence, the identifier sequence currently recorded for the target data block is updated.
3. The method according to claim 1, characterized in that, The method further includes: In response to a data block update request, determine the target data block in the target data block group that needs to be updated; Determine a reference identifier sequence based on the identifier sequences currently recorded in other data blocks in the target data block group, excluding the target data block; The identifier information of the target data block in the reference identifier sequence is moved before the identifier information of other data blocks in the reference identifier sequence to obtain the target identifier sequence, and the identifier sequence currently recorded for the target data block is updated based on the target identifier sequence.
4. The method according to claim 3, characterized in that, Determining the reference identifier sequence based on the identifier sequences currently recorded in other data blocks in the target data block group besides the target data block includes: The identifier sequences of the currently recorded data blocks other than the target data block in the target data block group are used as candidate identifier sequences, and the local longest common subsequence of any two candidate identifier sequences is calculated. From each of the local longest common subsequences, the global longest common subsequence is determined, and from the candidate identifier sequence containing the global longest common subsequence, a reference identifier sequence is determined.
5. The method according to claim 2 or 3, characterized in that, The step of updating the identifier sequence of the currently recorded target data block based on the target identifier sequence includes: According to the preset mapping rules, the target index identifier corresponding to the target identifier sequence is determined; Update the index identifier corresponding to the identifier sequence currently recorded in the target data block to the target index identifier.
6. A data consistency verification device, characterized in that, The device includes: The acquisition module is used to acquire the identifier sequence of the current record of each data block in the target data block group in response to the data consistency verification request; the identifier sequence is generated when the data block is updated and is used to characterize the update time sequence of each data block in the target data block group when the current data block is updated; The matching module is used to match the benchmark identifier sequence with the identifier sequence currently recorded in each data block according to the benchmark identifier sequence corresponding to the target data block group and the preset sequence matching rules, so as to obtain the matching result of each data block; the benchmark identifier sequence is used to characterize the global update time sequence of the target data block group; The first determining module is used to determine the data consistency verification result of the target data block group based on the matching result of each data block; The matching module is specifically used to obtain the baseline identifier sequence corresponding to the target data block group from the verification block corresponding to the target data block group; for each data block, the identifier information in the baseline identifier sequence that precedes the identifier information of the data block is determined as the identifier information to be removed; the identifier information to be removed is removed from the baseline identifier sequence to obtain the matching sequence, and the identifier information to be removed is removed from the identifier sequence currently recorded in the data block to obtain the verification identifier sequence; the matching sequence to be matched is compared with the verification identifier sequence to obtain the matching result of the data block.
7. The apparatus according to claim 6, characterized in that, The matching module is specifically used to: determine that the matching result of the data block is a matching failure when the sequence to be matched is different from the identifier sequence to be verified; and determine that the matching result of the data block is a matching success when the sequence to be matched is the same as the identifier sequence to be verified.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Distributed storage CEPH based erasure correction code overwriting method
CN105930103A
Data verification method, device and equipment
CN110019278A