A data consistency verification method and device, electronic equipment, storage medium and product

Abstract

This invention discloses a data consistency verification method, apparatus, electronic device, storage medium, and product, relating to the field of distributed storage technology. The data consistency verification method includes: dividing disk data into multiple data sub-intervals based on the numerical value domain space of data identifiers, and determining an interval check code for each data sub-interval; obtaining the interval check codes of each data sub-interval within the same disk redundancy group, and updating the timestamp of the corresponding data sub-interval according to the interval check codes; determining a consistency timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and determining global data consistency according to the current time and the consistency timestamp. In embodiments of this invention, full consistency verification does not require traversing the entire dataset, improving the efficiency of global data consistency verification; simultaneously, it enables real-time data consistency verification during data read / write tasks, without pausing business operations or interrupting service, thus improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and in particular to a data consistency verification method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] Distributed storage typically stores multiple copies or verifications of data, often referred to as replica redundancy mode and erasure coding redundancy mode. These copies or verifications are stored separately on different devices to ensure data redundancy, and these copies or verifications need to be consistent.

[0003] Typically, each copy or checksum of a data record has its own checksum. The data is considered consistent when all copies or checksums of a data record have the same checksum. The checksum is calculated only when the copy or checksum is written to disk and is written along with the data. Unless the disk is damaged, the checksum and data are considered consistent. A naive method to verify the consistency of all data in distributed storage is to compare the checksums of each data record. If all data records are identical, then the entire data is consistent. Alternatively, the checksum of all data on a single disk can be calculated. If the checksums of all data on all redundant disks are identical, then the entire data is consistent; in this model, disks belonging to the same group have completely identical data.

[0004] Regardless of the mode used, write operations must be paused during consistency verification. When write operations are in progress, a single piece of data is concurrently written to different disks, completing the write operation on some disks first and others later. When comparing the checksums of each piece of data, if some replicas of the data being written have already been written while others have not, then that data is inconsistent, and the full data consistency check fails. If the disk data is redundantly grouped, and some replicas of the data have already been written while others have not, then the checksums calculated for the entire dataset on different disks will be inconsistent, and the full data consistency check will also fail. Therefore, how to perform full data consistency verification without pausing data read and write operations has become a pressing problem to solve. Summary of the Invention

[0005] This invention provides a data consistency verification method, apparatus, electronic device, storage medium, and product to solve the problem in the prior art that it is impossible to perform consistency verification on all data without pausing data read and write.

[0006] According to one aspect of the present invention, a data consistency verification method is provided, wherein the method includes: The numerical value range space based on data identifiers divides disk data into multiple data sub-intervals and determines the interval check code for each data sub-interval. Obtain the interval check code of each data sub-interval within the same disk redundancy group, and update the timestamp of the corresponding data sub-interval according to the interval check code; A consistent timestamp is determined based on the timestamps of all data sub-intervals in the same disk redundancy group, and global data consistency is determined based on the current time and the consistent timestamp.

[0007] According to another aspect of the present invention, a data consistency verification apparatus is provided, wherein the apparatus comprises: The check code determination module is used to divide disk data into multiple data sub-intervals based on the numerical value range space of the data identifier, and determine the interval check code of each data sub-interval. The timestamp update module is used to obtain the interval check code of each data sub-interval within the same disk redundancy group, and update the timestamp of the corresponding data sub-interval according to the interval check code. The consistency verification module is used to determine the consistency timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and to determine global data consistency based on the current time and the consistency timestamp.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data consistency verification method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data consistency verification method according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the data consistency verification method of any embodiment of the present invention.

[0011] The technical solution of this invention divides disk data into multiple data sub-intervals based on the numerical value space of data identifiers, determines the interval check code for each data sub-interval, obtains the interval check codes of each data sub-interval within the same disk redundancy group, updates the timestamp of the corresponding data sub-interval according to the interval check code, determines the consistency timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and determines global data consistency according to the current time and the consistency timestamp. This achieves full consistency verification without traversing the entire data, improving the efficiency of global data consistency verification. Simultaneously, it enables real-time data consistency verification during data read / write tasks without pausing business operations or interrupting service, thus improving the user experience.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a data consistency verification method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a data consistency verification method provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a data consistency verification device according to Embodiment 4 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the data consistency verification method of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] In one embodiment, two data consistency verification methods in the prior art are illustrated using three replicas as an example.

[0018] Mode 1: There are six disks, A, B, C, D, E, and F. Data x has three copies, located on any three blocks of these six disks, let's say A, C, and D; data y also has three copies, also located on any three blocks of these six disks, let's say D, E, and F. In this mode, it's impossible to calculate a checksum for all the data on the disks because each disk contains different data; only each data entry can be compared.

[0019] Mode 2: There are six disks, A, B, C, D, E, and F, divided into two groups: Group 1 (A, B, C) and Group 2 (D, E, F). There are two data items, x and y, each with three copies. These three copies of x and y reside either on the disks in Group 1 or Group 2. In this way, each group of disks has the same data. Therefore, a checksum for all data can be calculated for each disk, and comparing the checksums within the same group confirms the consistency of the entire data.

[0020] To address the inability of the two aforementioned modes to accurately verify data consistency during write operations, this invention proposes a scheme that divides the data into multiple sub-ranges and verifies data consistency using timestamps.

[0021] Example 1 Figure 1 This is a flowchart of a data consistency verification method according to Embodiment 1 of the present invention. This embodiment is applicable to the situation of performing consistency verification on disks with data redundancy. The method can be executed by a data consistency verification device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes: S110. The numerical value domain space based on the data identifier divides the disk data into multiple data sub-intervals, and determines the interval check code for each data sub-interval.

[0022] In this context, a data identifier can be understood as a unique number for each piece of data on the disk, used to locate and distinguish different data. Generally, a data identifier can be an identifier (ID) for each piece of data. Data identifiers can be pre-assigned during data generation. The numeric value range refers to the entire range of possible values ​​for the data identifier. For example, the numeric value range can be the complete range of 64-bit unsigned integers. A data sub-interval refers to a continuous segment divided according to the numeric value range. This allows the full amount of data on the disk to be split into multiple smaller blocks, facilitating fragment verification and incremental maintenance. Disk data refers to the full amount of data on the disk. The interval checksum refers to the data content checksum determined by all data within each data sub-interval. In practice, the interval checksum can be calculated for each data sub-interval through incremental updates, or by extracting all data within each data sub-interval and calculating the checksum using the ID and checksum of the data content within the full interval.

[0023] In this embodiment, a data identifier for the disk data can be determined, and the value range of the data identifier can be defined as the numerical value range space. In actual operation, the numerical value range space may include, but is not limited to, [other uses]. The numerical value space is then divided into multiple non-overlapping, fully covered data sub-intervals using a fixed step size or a fixed number of data segments. Generally, the number of data sub-intervals can be an integer power of 2. For each data sub-interval, an interval check code is calculated either through incremental verification or by extracting all data within each sub-interval and calculating the interval check code for that sub-interval.

[0024] S120. Obtain the interval check code of each data sub-interval within the same disk redundancy group, and update the timestamp of the corresponding data sub-interval according to the interval check code.

[0025] The disk redundancy group contains at least two disks that store exactly the same data. In other words, the disk redundancy group can be a group of disks that store multiple copies or erasure codes of the same batch of data. The same timestamp of the disk data in the same disk redundancy group is used to mark the time of the last verification of the data sub-interval, so as to quickly determine whether it is consistent.

[0026] In this embodiment, the interval checksums of data sub-intervals with the same number can be read from each disk in the same disk redundancy group. The interval checksums of data sub-intervals with the same number are compared for consistency. If all interval checksums corresponding to the same number are consistent, it can be determined that the data update for the current data sub-interval is complete, and the timestamp of the data sub-interval corresponding to that number can be updated to the current time. If the interval checksums corresponding to the same number are inconsistent, the timestamp is not updated, and the previous timestamp is retained. This process of judgment and timestamp update is performed on all data sub-intervals one by one until the timestamps of all data sub-intervals are updated.

[0027] S130. Determine a consistent timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and determine global data consistency based on the current time and the consistent timestamp.

[0028] Among them, the consistent timestamp refers to the latest timestamp that is valid for all data sub-intervals of all disks in the disk redundancy group, which is the time point when all data is consistent.

[0029] In this embodiment, the timestamps of all data sub-intervals within the same disk redundancy group can be traversed, and the minimum value among all data sub-interval timestamps can be determined as the consistency timestamp. Then, the current time is determined. If the consistency timestamp is close to the current time, global data consistency can be determined; if the consistency timestamp is much earlier than the current time, inconsistency between sub-intervals can be determined, resulting in global data inconsistency. In actual operation, the difference between the current time and the consistency timestamp can be determined. If the difference is less than a preset threshold, global data consistency can be determined to have passed; if the difference is greater than or equal to the preset threshold, global data consistency can be determined to have failed. In one embodiment, the preset threshold can be one hour, etc.

[0030] This invention divides disk data into multiple sub-intervals based on a numerical value space of data identifiers, determines the interval check code for each sub-interval, obtains the interval check codes for each sub-interval within the same disk redundancy group, updates the timestamp of the corresponding sub-interval based on the interval check codes, determines a consistent timestamp based on the timestamps of all sub-intervals in the same disk redundancy group, and determines global data consistency based on the current time and the consistent timestamp. This achieves full consistency verification without traversing the entire dataset, improving the efficiency of global data consistency verification. Simultaneously, it enables real-time data consistency verification during data read / write tasks without interrupting business operations or services, thus improving the user experience.

[0031] In one embodiment, before dividing the disk data into multiple data sub-intervals based on the numerical value range space of the data identifier, and determining the interval check code for each data sub-interval, the method further includes: The disks are divided into at least one disk redundancy group according to the data redundancy strategy.

[0032] Data redundancy strategy refers to a data protection method that, to ensure data is not lost and can be recovered, stores data redundantly in the form of multiple copies or check blocks and distributes them across different devices. In practical applications, data redundancy strategies may include, but are not limited to, multi-copy redundancy strategies, which involve copying the same data into multiple complete copies and storing them on different disks and different nodes.

[0033] In this embodiment, the disk can be divided into at least one disk redundancy group by means of a multi-replica redundancy strategy, which facilitates subsequent data consistency verification.

[0034] Example 2 Figure 2 This is a flowchart of a data consistency verification method according to Embodiment 2 of the present invention. This embodiment is a further optimization and extension based on the above embodiments, and can be combined with various optional technical solutions in the above embodiments. Figure 2 As shown, the method includes: S210. Extract the data identifiers of all data in the disk, determine the value range space of the data identifiers, and divide the value range space into a preset number of data sub-intervals.

[0035] The preset quantity refers to the number of data sub-intervals that need to be divided in advance. Generally, the preset quantity is an integer power of 2, such as 32768.

[0036] In this embodiment, the target disk can be scanned, data identifiers of all data can be extracted, the maximum and minimum values ​​in the data identifiers can be determined, the numerical value range space of the data identifiers can be determined by the maximum and minimum values, and the numerical value range space can be divided into a preset number of data sub-intervals.

[0037] S220. Map the disk data to the corresponding data sub-intervals according to the corresponding data identifiers, and determine the interval check code according to the data in each data sub-interval.

[0038] In this embodiment, each piece of data on the disk can be traversed, and its corresponding data sub-range can be matched according to the data identifier of the disk data to complete the mapping and assignment of data to sub-ranges. Then, an incremental verification algorithm is used to traverse the IDs and check values ​​of all data within the data sub-range and calculate the interval check code of that data sub-range. Alternatively, the IDs and check values ​​of all data within the data sub-range can be read directly through a full read method to calculate the interval check code of the data sub-range.

[0039] S230. Extract the interval check code of the data sub-interval with the same index within the same disk redundancy group according to the preset period and use it as the first interval check code.

[0040] The preset period refers to the pre-set data verification period, which can be set according to user needs. For example, the preset period may include, but is not limited to, 30 seconds, 1 minute, and 2 minutes. The first interval check code refers to the set of interval check codes for the same index data sub-intervals extracted from each disk in the redundancy group within the preset period.

[0041] In this embodiment, the interval check code of the data sub-interval of the same index within the same disk redundancy group can be determined based on a preset period, and the interval check code corresponding to the same index within the same disk redundancy group can be used as the first interval check code.

[0042] S240. When the first interval check codes are the same, update the timestamp of the data sub-interval corresponding to the first interval check code to the current time.

[0043] In one embodiment, if the first interval check codes are determined to be the same, it can be determined that the data in the data sub-interval to which the first interval check codes belong is the same, and the timestamp of the data sub-interval corresponding to the first interval check codes can be updated to the current time. In another embodiment, when the first interval check codes are different, the timestamp of the data sub-interval corresponding to the first interval check codes is maintained.

[0044] S250. Determine the minimum value of the timestamps of all data sub-intervals in the same disk redundancy group as the consistent timestamp.

[0045] In this embodiment, the minimum timestamp of all data sub-intervals in the same disk redundancy group can be determined and used as the consistent timestamp.

[0046] S260. Determine the difference between the consistent timestamp and the current time. If the difference is less than a preset threshold, determine that the global data consistency is passed.

[0047] The preset threshold refers to the upper limit of the allowed consistency lag, used to determine whether the current global data is effectively consistent. Generally, the preset threshold can be customized according to business needs, such as 30 minutes, 1 hour, etc.

[0048] In this embodiment, the difference between the consistent timestamp and the current time can be calculated, and the relationship between the difference and a preset threshold can be determined. If the difference is less than the preset threshold, the global data consistency can be determined to be passed.

[0049] S270. When the difference is greater than or equal to the preset difference threshold, global data consistency is determined to fail.

[0050] In this embodiment, if the difference is determined to be greater than or equal to a preset difference threshold, global data consistency can be considered to be unsuccessful.

[0051] In this embodiment of the invention, by extracting the data identifiers of all data in the disk, the numerical value space of the data identifiers is determined. The value space is divided into a preset number of data sub-intervals. The disk data is mapped to the corresponding data sub-intervals according to the corresponding data identifiers. An interval check code is determined according to the data in each data sub-interval. The interval check codes of the data sub-intervals with the same index in the same disk redundancy group are extracted according to a preset period as the first interval check code. When the first interval check codes are the same, the timestamp of the data sub-interval corresponding to the first interval check code is updated to the current time, realizing the group update of timestamps. By determining the minimum value of the timestamps of all data sub-intervals in the same disk redundancy group as the consistent timestamp, the difference between the consistent timestamp and the current time is determined. When the difference is less than a preset threshold, the global data consistency is determined to be passed, realizing an intuitive and fast determination of whether the global data is consistent. At the same time, data reading and writing and verification are performed concurrently, without interrupting business or blocking services, meeting the high availability requirements of the production environment.

[0052] Example 3 In one embodiment, six disks (A, B, C, D, E, and F) are divided into two disk redundancy groups. The disks are grouped according to data redundancy, and disks within a group contain identical data. Each data entry consists of an ID and a CHECKSUM (data content checksum). Generally, both the ID and CHECKSUM are 64-bit unsigned integers, and their values ​​range from [value range missing]. For example, further explanation of data consistency verification methods. Dynamic consistency verification methods include: 1. Initialize a time stamp array of size N. The default value is 0.

[0053] 2. Execute the following steps at regular intervals (1 minute interval in this implementation): (1) Obtain the range checksum of all disks within a given disk group. (Three-replica redundancy mode means three 256KiB data sets.) ).

[0054] (2) Loop through all intervals from 0 to N-1 using I, where I is the interval index. If the interval check codes at index I are consistent, compare whether the check codes of intervals with the same index are consistent. Then update the consistent time of that interval to the current time, that is... ( (The current timestamp).

[0055] (3) Find the smallest timestamp among all interval timestamps as the last consistent timestamp (consistent timestamp). .

[0056] The interval checksum is calculated by dividing the ID value space into N intervals, with a checksum for each interval. N is a power of 2; when the ID generation method is sufficiently hashed, the amount of data within each interval is sufficiently evenly distributed. In this embodiment, N is 32768, and the interval checksum is a 64-bit unsigned integer, i.e., 8 bytes, occupying a total memory of 256 KiB. The interval checksum is calculated using an incrementally updateable algorithm. When a piece of data is written within an interval, the interval checksum is updated as follows: , where x is the old checksum of the interval, i is the ID of the data, o is the old checksum of the data, n is the new checksum of the data, and y is the new checksum of the interval.

[0057] 3. If This indicates that the global data across the disks in the disk redundancy group is dynamically consistent. The current DELAY (preset threshold) is 3600, which is one hour.

[0058] Since the data space is divided into N equal-sized intervals, when N is large enough, even with high-concurrency writes, the data being written will fall into M intervals, where M is much smaller than N. At this point, there are N - M intervals that have not been written to within a short period of time. The periodically executed dynamic consistency check will update the consistency time of some, or even most, intervals. After several rounds of dynamic consistency check execution, the consistency time of each interval has been updated, so all data is consistent up to the oldest interval consistency time (the smallest interval consistency timestamp) (other intervals reached consistency during the update).

[0059] Conversely, if a certain piece of data cannot reach a consensus, the consensus time of the interval to which that data belongs will not be updated. If consensus has never been reached, the timestamp will be 0. If consensus was reached at some point, the consensus time of the interval will be the time when consensus was last reached.

[0060] The last consensus time is updated over time. In actual testing, the last consensus time is 5 to 15 minutes later than the current time. Therefore, a last consensus time that is more than 1 hour later than the current time is considered inconsistent.

[0061] In this method, the parameters N and DELAY are adjusted based on the actual running results. They ensure that the metadata of the interval check code occupies less space, the time consumed by periodically comparing the interval check codes is less, and the calculation cost of updating the check code is less.

[0062] Example 4 Figure 3 This is a schematic diagram of a data consistency verification device according to Embodiment 4 of the present invention. Figure 3As shown, the device includes: a check code determination module 31, a timestamp update module 32, and a consistency verification module 33.

[0063] The check code determination module 31 is used to divide the disk data into multiple data sub-intervals based on the numerical value range space of the data identifier, and determine the interval check code of each data sub-interval.

[0064] The timestamp update module 32 is used to obtain the interval check code of each data sub-interval within the same disk redundancy group, and update the timestamp of the corresponding data sub-interval according to the interval check code.

[0065] The consistency verification module 33 is used to determine the consistency timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and to determine global data consistency based on the current time and the consistency timestamp.

[0066] The technical solution of this invention divides disk data into multiple data sub-intervals based on the numerical value space of the data identifier using a check code determination module, and determines the interval check code for each data sub-interval. A timestamp update module obtains the interval check codes of each data sub-interval within the same disk redundancy group and updates the timestamp of the corresponding data sub-interval based on the interval check codes. A consistency verification module determines a consistent timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and determines global data consistency based on the current time and the consistent timestamp. This achieves full consistency verification without traversing the entire dataset, improving the efficiency of global data consistency verification. Simultaneously, it enables real-time data consistency verification during data read / write tasks without pausing business operations or interrupting service, thus improving the user experience.

[0067] In one embodiment, the verification code determination module 31 includes: The space partitioning unit is used to extract the data identifiers of all data in the disk, determine the value range space of the data identifiers, and divide the value range space into a preset number of data sub-intervals. The check code determination unit is used to map disk data to corresponding data sub-intervals according to the corresponding data identifiers, and to determine the interval check code according to the data in each data sub-interval.

[0068] In one embodiment, the timestamp update module 32 includes: The check code lookup unit is used to extract the interval check code of the data sub-interval with the same index within the same disk redundancy group as the first interval check code according to a preset period. The timestamp update unit is used to update the timestamp of the data sub-interval corresponding to the first interval check code to the current time when the first interval check codes are the same.

[0069] In one embodiment, the consistency verification module 33 includes: The timestamp identification unit is used to determine the minimum value of the timestamps of all data sub-intervals in the same disk redundancy group as the consistent timestamp; The first verification unit is used to determine the difference between the consistent timestamp and the current time. When the difference is less than a preset threshold, the global data consistency is determined to be passed. The second verification unit is used to determine that global data consistency fails when the difference is greater than or equal to a preset difference threshold.

[0070] In one embodiment, the disk redundancy group comprises at least two disks storing identical data.

[0071] In one embodiment, the data consistency verification device further includes: The redundancy group partitioning module is used to partition disks into at least one disk redundancy group according to the data redundancy strategy.

[0072] The data consistency verification device provided in the embodiments of the present invention can execute the data consistency verification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0073] Example 5 Figure 4 This is a schematic diagram of the structure of an electronic device implementing the data consistency verification method of the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0074] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0075] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0076] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data consistency verification methods.

[0077] In some embodiments, the data consistency verification method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data consistency verification method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the data consistency verification method by any other suitable means (e.g., by means of firmware).

[0078] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0079] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0080] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0081] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0082] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0083] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0084] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the data consistency verification method of any embodiment of the present invention.

[0085] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0086] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0087] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of data consistency checking, the method comprising: include: The numerical value range space based on data identifiers divides disk data into multiple data sub-intervals, and determines the interval check code for each data sub-interval. Obtain the interval check code of each data sub-interval within the same disk redundancy group, and update the timestamp of the corresponding data sub-interval according to the interval check code; A consistent timestamp is determined based on the timestamps of all data sub-intervals in the same disk redundancy group, and global data consistency is determined based on the current time and the consistent timestamp.

2. The method of claim 1, wherein, The numerical value range space based on data identifiers divides disk data into multiple data sub-intervals, and determines the interval check code for each data sub-interval, including: Extract the data identifiers of all data in the disk, determine the value range space of the data identifiers, and divide the value range space into a preset number of data sub-intervals; Disk data is mapped to corresponding data sub-intervals according to the corresponding data identifiers, and the interval check code is determined according to the data in each of the data sub-intervals.

3. The method of claim 1, wherein, The step of obtaining the interval check code of each data sub-interval within the same disk redundancy group and updating the timestamp of the corresponding data sub-interval according to the interval check code includes: According to a preset period, the interval check code of the data sub-interval with the same index in the same disk redundancy group is extracted as the first interval check code. When the first interval check codes are the same, update the timestamp of the data sub-interval corresponding to the first interval check code to the current time.

4. The method of claim 1, wherein, The step of determining a consistent timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and determining global data consistency based on the current time and the consistent timestamp, includes: The minimum timestamp of all data sub-intervals in the same disk redundancy group is determined as the consistent timestamp; The difference between the consistent timestamp and the current time is determined. If the difference is less than a preset threshold, the global data consistency is determined to be passed. If the difference is greater than or equal to a preset difference threshold, the global data consistency is determined to be unsuccessful.

5. The method according to claim 1, characterized in that, The disk redundancy group comprises at least two disks storing identical data.

6. The method according to claim 1, characterized in that, Before dividing the disk data into multiple data sub-intervals based on the numerical value range space of the data identifier, and determining the interval check code for each data sub-interval, the method further includes: The disks are divided into at least one disk redundancy group according to the data redundancy strategy.

7. A data consistency verification device, characterized in that, include: The check code determination module is used to divide disk data into multiple data sub-intervals based on the numerical value range space of the data identifier, and determine the interval check code of each data sub-interval. The timestamp update module is used to obtain the interval check code of each data sub-interval within the same disk redundancy group, and update the timestamp of the corresponding data sub-interval according to the interval check code. The consistency verification module is used to determine the consistency timestamp based on the timestamps of all data sub-intervals in the same disk redundancy group, and to determine global data consistency based on the current time and the consistency timestamp.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data consistency verification method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data consistency verification method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the data consistency verification method according to any one of claims 1-6.

Images