Data verification method, device and system

Abstract

The present invention discloses a data verification method, device and system. The method comprises: reading first data stored in a storage end, wherein the first data is generated by a data generator and stored in the storage end; obtaining second data generated by the data generator; comparing the first data and the second data to obtain a data verification result. The present invention solves the technical problem in the related art that when a distributed storage system detects data correctness, cached data occupies more memory.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage systems, and in particular to a data verification method, device and system. Background Art

[0002] In a distributed storage system, there are two main ways to implement end-to-end data verification. The first method is to generate test data on the client, write the test data to the storage end and keep the test data in the memory. After writing, randomly read the written data fragments or the entire content of the written data, and then compare it byte by byte with the data retained in the memory to see if the data is consistent. The second method is that the client generates data and writes the data to the storage end, and then saves the test data to the disk, or directly reads the test data on the disk and writes it to the storage end, then reads the data and compares the data byte by byte or reads all the data, and then calculates the MD5 (Message Digest Algorithm, Message Digest Algorithm Fifth Edition) of the two files to compare whether the data is consistent.

[0003] However, when checking the correctness of data, the first method mentioned above saves the test data on the test side and then compares the data with the data retrieved from the storage system by byte. However, this method requires that all test data be kept in the memory on the test side before it can be compared with the data retrieved from the storage side. If concurrent multi-threaded testing or writing a large amount of data for testing, the data in the memory will take up a lot of memory space, which is unacceptable in many cases. The second method mentioned above is to write the test data to the disk on the client, read the disk data and the data on the storage side respectively, and then compare them by byte to see if the data is consistent. This method does not use much memory, but it needs to read the data from the client, which is a long process and wastes disk bandwidth and CPU resources. Compared with the first method, the data verification time is longer, and the data is fixed, so there is a leakage test case scenario.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a data verification method, device and system to at least solve the technical problem in the related art that cached data occupies too much memory when the distributed storage system detects the correctness of data.

[0006] According to one aspect of an embodiment of the present invention, a data verification method is provided, comprising: reading first data stored in a storage end, wherein the first data is generated by a data generator and stored in the storage end; obtaining second data generated by the data generator; and comparing the first data with the second data to obtain a data verification result.

[0007] Optionally, the storage end includes: multiple data layers, wherein reading the first data stored in the storage end includes: determining a current reading position; determining whether data is stored at the current reading position of the current data layer; if no data is stored at the current reading position of the current data layer, pushing the current data layer into the stack, determining the next data layer as the current data layer, and returning to execute the step of determining whether data is stored at the current reading position of the current data layer, until the current data layer is the last data layer among the multiple data layers; if data is stored at the current reading position of the current data layer, reading the data stored at the current reading position of the current data layer.

[0008] Optionally, before reading the data stored at the current read position of the current data layer, the method also includes: obtaining the first data layer stored in the stack; determining the target position where the data is stored in the first data layer to obtain the first target position; judging whether the first target position is smaller than the current read position; if the first target position is larger than the current read position, reading the data stored at the current read position of the current data layer.

[0009] Optionally, if the first target position is smaller than the current read position, the method further includes: popping the first data layer from the stack; using the first data layer as the current data layer, and reading data stored at the current read position of the current data layer.

[0010] Optionally, the storage end includes: multiple data layers. Before reading the first data stored in the storage end, the method also includes: obtaining third data generated by a data generator corresponding to the current data layer; writing the third data to the storage end; after generating a snapshot corresponding to the current data layer, determining that the previous data layer is the current data layer, and returning to execute the step of obtaining the third data generated by the data generator corresponding to the current data layer, until the current data layer is the first data layer among the multiple data layers.

[0011] Optionally, obtaining the third data generated by the data generator corresponding to the current data layer includes: generating a seed number corresponding to the current data layer; initializing the data generator based on the seed number; obtaining the data range corresponding to the current data layer; and obtaining the third data generated by the data generator based on the data range.

[0012] Optionally, after obtaining the data range corresponding to the current data layer, the data generator and the data range are stored in the current data layer.

[0013] Optionally, obtaining the second data generated by the data generator includes: determining a target data layer corresponding to the first data; obtaining the data generator and data range stored in the target data layer; and obtaining the second data generated by the data generator based on the data range.

[0014] Optionally, before reading the first data stored in the storage end, the method also includes: determining a data segment set, wherein the interval between any two data segments in the data segment set is a preset interval; determining the offset and length of each data segment in the data segment set; and based on the offset and length of each data segment, reading the first data stored in the storage end.

[0015] Optionally, based on the offset and length of each data segment, the first data and the second data are compared to obtain a data verification result.

[0016] According to another aspect of an embodiment of the present invention, a data verification device is also provided, including: a reading module, used to read first data stored in a storage end, wherein the first data is generated by a data generator and stored in the storage end; an acquisition module, used to acquire second data generated by the data generator; and a comparison module, used to compare the first data and the second data to obtain a data verification result.

[0017] According to another aspect of an embodiment of the present invention, a data verification system is also provided, including: a storage end, for storing data; a client, including: a data generator, the client is used to read first data stored in the storage end, obtain second data generated by the data generator, and compare the first data and the second data to obtain a data verification result, wherein the first data is generated by the data generator and stored in the storage end.

[0018] Optionally, the storage end includes: multiple data layers and a stack; wherein the storage end is also used to determine the current reading position, determine whether data is stored at the current reading position of the current data layer, and if no data is stored at the current reading position of the current data layer, push the current data layer into the stack, determine the next data layer as the current data layer, and return to execute the step of determining whether data is stored at the current reading position of the current data layer, until the current data layer is the last data layer among the multiple data layers; if data is stored at the current reading position of the current data layer, read the data stored at the current reading position of the current data layer.

[0019] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned data verification method.

[0020] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the above-mentioned data verification method is executed when the program is running.

[0021] In an embodiment of the present invention, by comparing the first data read from the storage end with the second data generated by the data generator, a data verification result can be obtained, thereby achieving the purpose of end-to-end data verification through the data generator. Compared with the related art, there is no need to store the data written to the storage end in the memory or disk, thereby achieving the technical effect of reducing the storage amount of test data, increasing the concurrency capability of stress testing, and saving the time for verifying data, thereby solving the technical problem in the related art that the cached data occupies more memory when the distributed storage system detects the correctness of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 is a flow chart of a data verification method according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of an optional method of reading data from a storage end according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of an optional data writing storage end according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of a data verification device according to an embodiment of the present invention; and

[0027] Figure 5 is a schematic diagram of a data verification system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] The technical names or technical terms appearing in the embodiments of the present invention are explained as follows:

[0031] Data generator test data verification: After the data generator generates data, it can write the data to osd. The test end does not retain the data, but retains the data generator. When it is necessary to detect whether the stored data is correct, the stored data is read out, and the client's data generator generates the data again, and then the two data are compared to see if they are correct.

[0032] End-to-end data verification: End-to-end can refer to the client (including the test end) and the storage end; the test end first writes data to the client, and then reads the data from the storage end to compare the accuracy of the data.

[0033] OSD: Object Storage Device, object storage device can refer to a process responsible for data storage. Replicas and EC (Erasure type Ceph) are also implemented through OSD services. Each hard disk has an OSD process.

[0034] MON: Monitor, monitoring service, can be used to monitor the status of the entire cluster.

[0035] Data layering technology: Due to the existence of snapshot technology, the data at the bottom layer of Ceph is stored in layers. When reading, one or several layers of data are read according to the needs.

[0036] Stack: stack, also known as stack, can be a data structure, which is a linear list with restricted operations. A linear list that is limited to insertion and deletion operations only at the end of the list. This end is called the top of the stack, and the other end is called the bottom of the stack. Inserting a new element into a stack is also called pushing, pushing or pushing, which is to put the new element on top of the top element of the stack to make it the new top element of the stack; deleting an element from a stack is also called popping or popping, which is to delete the top element of the stack so that its adjacent element becomes the new top element of the stack.

[0037] Example 1

[0038] According to an embodiment of the present invention, a data verification method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 1 is a flow chart of a data verification method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0040] Step S102, reading first data stored in the storage end, wherein the first data is generated by a data generator and stored in the storage end.

[0041] The storage end in the above steps may refer to an OSD, but is not limited thereto. The data generator may be a linear pseudo-random data generator, but is not limited thereto.

[0042] It should be noted that the seed number needs to be passed in when the data generator is initialized. The product of the seed number multiplied by the multiplication factor and divided by the modulus factor is the remainder to get the target number. The multiplication factor and the modulus factor are both fixed. As long as the seed number is determined, the target number generated each time is the same. Therefore, the data generator can be used to test data verification, and the seed used during initialization can be saved in the data generator.

[0043] In an optional embodiment, before the test, the test end may generate a target number through a data generator and store it in the storage end. When the data storage in the storage end is reliable, the first data read from the storage end by the test end is the same as the data stored in the storage end, that is, the first data read from the test end is the same as the target number generated by the data generator. However, when the data storage in the storage end is unreliable, the first data read from the storage end by the test end may be the same as the data stored in the storage end, that is, the first data read from the test end is different from the target number generated by the data generator.

[0044] Step S104, obtaining second data generated by the data generator.

[0045] The data generator in the above steps is the same as the data generator in step S102.

[0046] In an optional embodiment, after the test end reads the first data from the storage end, the target number may be generated again by the data generator, that is, the above-mentioned second data is obtained.

[0047] Step S106, comparing the first data and the second data to obtain a data verification result.

[0048] Since the data generators are the same, the generated target number is the same as the target number stored in the storage end. Therefore, if the data storage in the storage end is reliable, it can be determined that the data stored in the storage end, the first data read from the storage end, and the second data generated by the data generator are all the same; if the data storage in the storage end is unreliable, it can be determined that the second data read from the storage end is different from the second data generated by the data generator. Therefore, by comparing the first data and the second data, an end-to-end data verification result can be obtained.

[0049] Through the solution provided by the above-mentioned embodiment of the present invention, by comparing the first data read from the storage end with the second data generated by the data generator, a data verification result can be obtained, thereby achieving the purpose of end-to-end data verification through the data generator. Compared with the related technology, there is no need to store the data written to the storage end in the memory or disk, thereby achieving the technical effect of reducing the storage amount of test data, increasing the concurrency capability of stress testing, and saving the time for verifying data, thereby solving the technical problem in the related technology that the cached data occupies more memory when the distributed storage system detects the correctness of the data.

[0050] Optionally, the storage end includes: multiple data layers, wherein reading the first data stored in the storage end includes: determining a current reading position; determining whether data is stored at the current reading position of the current data layer; if no data is stored at the current reading position of the current data layer, pushing the current data layer into the stack, determining the next data layer as the current data layer, and returning to execute the step of determining whether data is stored at the current reading position of the current data layer, until the current data layer is the last data layer among the multiple data layers; if data is stored at the current reading position of the current data layer, reading the data stored at the current reading position of the current data layer.

[0051] The storage end in the above steps can be a storage system with snapshot data layering technology, such as ceph, so it can contain multiple data layers. The snapshot is used to retain data at a certain time. After the snapshot is taken, the data written will be written to the new head data layer (indicates the top layer in the data layer), and the snapshot data remains unchanged. The current read position can refer to the position where the data needs to be read currently, represented by pos.

[0052] It should be noted that for multiple data layers, the newer the data, the higher the priority. In the process of reading data, the data layer with higher priority will be read first. For example, Figure 2As shown, taking three data layers as an example, there are two layers of snapshots and one layer of current data, and the reading order of the data layers is layer1, layer2 and layer3.

[0053] For example, Figure 2 The three data layers shown are used as examples for explanation, target represents the data to be read in the end, that is, the first data mentioned above; the filled area (such as the black area, the horizontal line area and the vertical line area) represents that there is data stored in the data layer, and the unfilled blank area represents that there is no data stored in the data layer; cur points to the current data layer layer; layer1 represents snapshot snap1, layer2 represents snapshot snap2, and layer3 represents the head data layer; Stack represents a stack with a first-in-last-out feature.

[0054] Read the head layer data. When there is no data stored in this layer, read the data in the order of layer2 and layer3. For the pos position, there is no data at the pos position of the head layer. You can put layer1 into the stack, and then read the pos position of the layer2 data. It is found that there is no data at the pos position of layer2. You can put layer2 into the stack and read the pos position of layer3. It is found that there is data at the pos position of layer3. Then read the data at the pos position of layer3 and put the read data into the target.

[0055] Optionally, before reading the data stored at the current read position of the current data layer, the method also includes: obtaining the first data layer stored in the stack; determining the target position where the data is stored in the first data layer to obtain the first target position; judging whether the first target position is smaller than the current read position; if the first target position is larger than the current read position, reading the data stored at the current read position of the current data layer.

[0056] The target position in the above steps may refer to the starting position of the data layer in the stack that stores data closest to the pos position, which may be represented by next.

[0057] For example, Figure 2 The three data layers shown are used as an example to illustrate that when reading data, it is necessary to compare the pos position with the next of the first layer in the stack (ie, layer2). If pos is less than next, the data at the pos position of layer3 can be directly read.

[0058] Optionally, if the first target position is smaller than the current read position, the method further includes: popping the first data layer from the stack; using the first data layer as the current data layer, and reading data stored at the current read position of the current data layer.

[0059] For example, Figure 2 Take the three data layers shown as an example to explain, if pos is greater than next, layer2 can be taken out of the stack and the data at the pos position of layer2 can be read. At this time, layer2 can be considered to become the current data layer. Similarly, since layer1 is still stored in the stack, the pos position and the first layer in the stack, that is, the only layer1, can be compared. If pos is less than next, the data at the pos position of layer2 can be directly read; if pos is greater than next, layer1 can be taken out of the stack and the data at the pos position of layer1 can be read.

[0060] The above data reading process can ensure that the read data is read according to the priority and the correctness of the data is finally read.

[0061] Optionally, the storage end includes: multiple data layers. Before reading the first data stored in the storage end, the method also includes: obtaining third data generated by a data generator corresponding to the current data layer; writing the third data to the storage end; after generating a snapshot corresponding to the current data layer, determining that the previous data layer is the current data layer, and returning to execute the step of obtaining the third data generated by the data generator corresponding to the current data layer, until the current data layer is the first data layer among the multiple data layers.

[0062] In the embodiment of the present invention, one snapshot corresponds to one layer, and one data generator is placed in each layer. Figure 3 Take the example of taking two snapshots. When the test end needs to write data to the storage end, the test end can read the third data generated by the data generator placed in layer1 and write it to the storage end. At this time, no snapshot has been taken, so the head pointer points to layer1. Take snapshot snap1. If no input is written after taking the snapshot, the snap1 pointer and the head pointer both point to layer1. Then read the third data generated by the data generator placed in layer2 and write it to the storage end. At this time, the head pointer points to layer2. Take snapshot snap2. If no input is written after taking the snapshot, the snap2 pointer and the head pointer both point to layer2. Then read the third data generated by the data generator placed in layer3 and write it to the storage end. At this time, the head pointer points to layer3.

[0063] Optionally, obtaining the third data generated by the data generator corresponding to the current data layer includes: generating a seed number corresponding to the current data layer; initializing the data generator based on the seed number; obtaining the data range corresponding to the current data layer; and obtaining the third data generated by the data generator based on the data range.

[0064] The number of seeds corresponding to different data layers can be the same or different. For example, Figure 3 Taking two snapshots as an example, the seed number corresponding to layer 1 is seed1, the seed number corresponding to layer 2 is seed2, and the seed number corresponding to layer 3 is seed3.

[0065] In an optional embodiment, a seed number seed1 may be generated first, a data generator generator1 may be initialized according to the seed number, and then a data range interval1 to be generated may be recorded, and then a piece of data may be generated according to interval1 and written to the storage end.

[0066] Optionally, after obtaining the data range corresponding to the current data layer, the data generator and the data range are stored in the current data layer.

[0067] For example, Figure 3 Taking the two snapshots shown as an example, generator1 and interval1 can be placed in layer1, generator2 and interval2 can be placed in layer2, and generator3 and interval3 can be placed in layer3.

[0068] Optionally, obtaining the second data generated by the data generator includes: determining a target data layer corresponding to the first data; obtaining the data generator and data range stored in the target data layer; and obtaining the second data generated by the data generator based on the data range.

[0069] It should be noted that the method of generating the second data from the data here is the same as that described in the above embodiment, and will not be described in detail here.

[0070] Optionally, before reading the first data stored in the storage end, the method also includes: determining a data segment set, wherein the interval between any two data segments in the data segment set is a preset interval; determining the offset and length of each data segment in the data segment set; and based on the offset and length of each data segment, reading the first data stored in the storage end.

[0071] The preset interval in the above steps can be set according to actual needs, so that data can be read from the storage end non-continuously.

[0072] In an optional embodiment, a set of data segments at a preset interval may be calculated according to a certain method, and then the offset offset and data segment length len of each data segment may be saved in a memory, and the first data in the storage end may be read.

[0073] Optionally, based on the offset and length of each data segment, the first data and the second data are compared to obtain a data verification result.

[0074] In an optional embodiment, after reading the first data in the storage end, the first data and the second data generated by the data generator can be compared for consistency based on the data segment information in the memory (ie, the offset and the length len), thereby achieving the purpose of data verification.

[0075] The solution provided by the above-mentioned embodiment of the present invention can solve the problem that the cached data occupies too much memory when checking the data correctness during the concurrent stress test of the client in the storage system. It can also greatly reduce the test data stored in the client, increase the concurrency of the stress test, and do not need to read the disk again, saving the time of verifying the data. For storage systems with snapshot data tiering technology such as ceph, it is also possible to simulate the data tiering technology in the upper layer and flexibly read the required data. It has the following advantages over the prior art: compared with full memory data, it has the advantages of fast speed and can compare the specific location of inconsistent output data; compared with the prior art, it occupies less memory; it can simulate the storage tiering technology on the client, and combine multiple layers of data into valid data, which is flexible and convenient.

[0076] Example 2

[0077] According to an embodiment of the present invention, a data verification device is provided, which can execute the data verification method in the above-mentioned embodiment 1. The specific implementation scheme and application scenario are the same as those in the above-mentioned embodiment 1, and will not be repeated here.

[0078] Figure 4 is a schematic diagram of a data verification device according to an embodiment of the present invention. Figure 4 As shown, the device comprises:

[0079] A reading module 42, configured to read first data stored in the storage end, wherein the first data is generated by a data generator and stored in the storage end;

[0080] An acquisition module 44, used for acquiring second data generated by a data generator;

[0081] The comparison module 46 is used to compare the first data with the second data to obtain a data verification result.

[0082] Optionally, the storage end includes: multiple data layers, wherein the reading module includes: a first determination unit, used to determine the current reading position; a judgment unit, used to judge whether data is stored at the current reading position of the current data layer; a push unit, used to push the current data layer into the stack if no data is stored at the current reading position of the current data layer, determine the next data layer as the current data layer, and return to execute the step of judging whether data is stored at the current reading position of the current data layer, until the current data layer is the last data layer among the multiple data layers; and a reading unit, used to read the data stored at the current reading position of the current data layer if data is stored at the current reading position of the current data layer.

[0083] Optionally, the device also includes: an acquisition module for acquiring the first data layer stored in the stack; a first determination module for determining a target position where data is stored in the first data layer to obtain a first target position; a judgment module for judging whether the first target position is smaller than a current read position; and a reading module for reading data stored at a current read position of the current data layer if the first target position is larger than the current read position.

[0084] Optionally, the device also includes: a popping module, used to pop the first data layer from the stack if the first target position is smaller than the current reading position; the reading module is also used to use the first data layer as the current data layer and read the data stored at the current reading position of the current data layer.

[0085] Optionally, the storage end includes: multiple data layers, and the device also includes: an acquisition module also used to obtain third data generated by a data generator corresponding to the current data layer; a writing module used to write the third data to the storage end; a second determination module used to determine that the previous data layer is the current data layer after generating a snapshot corresponding to the current data layer, and return to execute the function of the acquisition module until the current data layer is the first data layer among the multiple data layers.

[0086] Optionally, the acquisition module includes: a generation unit for generating a seed number corresponding to the current data layer; an initialization unit for initializing a data generator based on the seed number; a first acquisition unit for acquiring a data range corresponding to the current data layer; and a second acquisition unit for acquiring third data generated by the data generator based on the data range.

[0087] Optionally, the device further includes: a storage module, configured to store the data generator and the data range in the current data layer after acquiring the data range corresponding to the current data layer.

[0088] Optionally, the acquisition module includes: a second determination unit, used to determine the target data layer corresponding to the first data; a third acquisition unit, used to acquire the data generator and data range stored in the target data layer; and a second acquisition unit, used to acquire the second data generated by the data generator based on the data range.

[0089] Optionally, the device also includes: a third determination module, used to determine a data segment set, wherein the interval between any two data segments in the data segment set is a preset interval; a fourth determination module, used to determine the offset and length of each data segment in the data segment set; and the reading module is also used to read the first data stored in the storage end based on the offset and length of each data segment.

[0090] Optionally, the comparison module is further used to compare the first data and the second data based on the offset and the length of each data segment to obtain a data verification result.

[0091] Example 3

[0092] According to an embodiment of the present invention, a data verification system is provided, which can execute the data verification method in the above-mentioned embodiment 1. The specific implementation scheme and application scenario are the same as those in the above-mentioned embodiment 1, and will not be repeated here.

[0093] Figure 5 is a schematic diagram of a data verification system according to an embodiment of the present invention. Figure 5 As shown, the system includes:

[0094] A storage terminal 52, used for storing data;

[0095] The client 54 includes: a data generator 542, the client 54 is used to read the first data stored in the storage end, obtain the second data generated by the data generator 542, and compare the first data with the second data to obtain a data verification result, wherein the first data is generated by the data generator and stored in the storage end.

[0096] Alternatively, if Figure 5 As shown, the storage end 52 includes: multiple data layers 522 and a stack 524, wherein the storage end 52 is also used to determine the current reading position, determine whether data is stored at the current reading position of the current data layer, if no data is stored at the current reading position of the current data layer, push the current data layer into the stack, determine the next data layer as the current data layer, and return to execute the step of determining whether data is stored at the current reading position of the current data layer, until the current data layer is the last data layer among the multiple data layers; if data is stored at the current reading position of the current data layer, read the data stored at the current reading position of the current data layer.

[0097] Optionally, the storage end 52 is also used to obtain the first data layer stored in the stack before reading the data stored at the current read position of the current data layer, determine the target position where the data is stored in the first data layer, obtain the first target position, and determine whether the first target position is smaller than the current read position; if the first target position is larger than the current read position, read the data stored at the current read position of the current data layer.

[0098] Optionally, the storage end 52 is further configured to pop the first data layer from the stack if the first target position is smaller than the current read position, use the first data layer as the current data layer, and read the data stored at the current read position of the current data layer.

[0099] Alternatively, if Figure 5 As shown, the storage end includes: multiple data layers, and the client 54 includes: multiple data generators 542; the client 54 is also used to obtain the third data generated by the data generator corresponding to the current data layer, write the third data to the storage end, and after generating a snapshot corresponding to the current data layer, determine that the previous data layer is the current data layer, and return to execute the step of obtaining the third data generated by the data generator corresponding to the current data layer, until the current data layer is the first data layer among the multiple data layers.

[0100] Optionally, the client 54 is further used to generate a seed number corresponding to the current data layer, initialize the data generator based on the seed number, obtain a data range corresponding to the current data layer, and obtain third data generated by the data generator based on the data range.

[0101] Optionally, the client 54 is further configured to store the data generator and the data range in the current data layer after acquiring the data range corresponding to the current data layer.

[0102] Optionally, the client 54 is further configured to determine a target data layer corresponding to the first data; obtain a data generator and a data range stored in the target data layer; and obtain second data generated by the data generator based on the data range.

[0103] Optionally, the client 54 is also used to determine a data segment set, determine the offset and length of each data segment in the data segment set, and read the first data stored in the storage end based on the offset and length of each data segment, wherein the interval between any two data segments in the data segment set is a preset interval.

[0104] Optionally, the client 54 is further configured to compare the first data with the second data based on the offset and length of each data segment to obtain a data verification result.

[0105] Example 4

[0106] According to an embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the data verification method in the above-mentioned embodiment 1.

[0107] Example 5

[0108] According to an embodiment of the present invention, a processor is provided, and the processor is used to run a program, wherein the data verification method in the above-mentioned embodiment 1 is executed when the program is running.

[0109] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0110] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0113] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0115] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data verification method, characterized in that: include: Based on the current read position, determine whether data is stored at the current read position of the storage end, and obtain a determination result. If data is not stored at the current read position of the current data layer, push the current data layer into the stack, determine the next data layer as the current data layer, and return to execute the step of determining whether data is stored at the current read position of the current data layer, until the current data layer is the last data layer among the multiple data layers; Determine whether the current read position of the current data layer satisfies a preset position condition based on the stack, and if data is stored at the current read position of the current data layer and the current read position satisfies the preset position condition, read first data stored at the current read position of the current data layer, wherein the first data is generated by a data generator and stored in the storage end; Determine a target data layer for the first data, and generate second data according to a data generator corresponding to the target data layer; The first data and the second data are compared to obtain a data verification result.

2. The method according to claim 1, characterized in that Determining whether the current reading position meets a preset position condition based on the stack includes: Get the first data layer stored in the stack; Determine a target location where data is stored in the first data layer to obtain a first target location; Determining whether the first target position is smaller than the current reading position; If the first target position is greater than the current reading position, it is determined that the current reading position meets the preset position condition.

3. The method according to claim 2, characterized in that If the first target position is smaller than the current read position, the method further comprises: Popping the first data layer from the stack; The first data layer is used as the current data layer, and the data stored at the current reading position of the current data layer is read.

4. The method according to claim 1, characterized in that: The storage end includes: a plurality of data layers. Before reading the first data stored in the storage end, the method further includes: Obtain third data generated by the data generator corresponding to the current data layer; Writing the third data into the storage end; After generating a snapshot corresponding to the current data layer, determine that the previous data layer is the current data layer, and return to execute the step of obtaining the third data generated by the data generator corresponding to the current data layer until the current data layer is the first data layer among the multiple data layers.

5. The method according to claim 4, characterized in that Obtaining the third data generated by the data generator corresponding to the current data layer includes: Generate the seed number corresponding to the current data layer; Initializing the data generator based on the seed number; Obtaining the data range corresponding to the current data layer; The third data generated by the data generator based on the data range is acquired.

6. The method according to claim 5, characterized in that After acquiring the data range corresponding to the current data layer, the data generator and the data range are stored in the current data layer.

7. The method according to claim 6, characterized in that Generating second data according to a data generator corresponding to the target data layer includes: Obtaining a data generator and a data range stored in the target data layer; The second data generated by the data generator based on the data range is obtained.

8. The method according to claim 1, characterized in that: Before reading the first data stored in the storage end, the method further includes: Determine a data segment set, wherein an interval between any two data segments in the data segment set is a preset interval; Determining the offset and length of each data segment in the set of data segments; Based on the offset and length of each data segment, the first data stored in the storage end is read.

9. The method according to claim 8, characterized in that Based on the offset and length of each data segment, the first data and the second data are compared to obtain the data verification result.

10. A data verification device, characterized in that: include: A reading module, used for judging whether data is stored at the current reading position of the storage end based on the current reading position, obtaining a judgment result, and if data is not stored at the current reading position of the current data layer, pushing the current data layer into the stack, determining the next data layer as the current data layer, and returning to execute the step of judging whether data is stored at the current reading position of the current data layer, until the current data layer is the last data layer among the multiple data layers; Determine whether the current read position of the current data layer satisfies a preset position condition based on the stack, and if data is stored at the current read position of the current data layer and the current read position satisfies the preset position condition, read first data stored at the current read position of the current data layer, wherein the first data is generated by a data generator and stored in the storage end; An acquisition module, used to determine a target data layer of the first data, and generate second data according to a data generator corresponding to the target data layer; A comparison module is used to compare the first data with the second data to obtain a data verification result.

11. A data verification system, characterized in that: include: Storage end, used to store data; A client, including: a data generator, wherein the client is used to determine whether data is stored at the current reading position of the storage end based on the current reading position, and obtain a judgment result. If no data is stored at the current reading position of the current data layer, the current data layer is pushed into a stack, the next data layer is determined to be the current data layer, and the step of determining whether data is stored at the current reading position of the current data layer is returned to execute until the current data layer is the last data layer among multiple data layers; based on the stack, determine whether the current reading position of the current data layer meets a preset position condition; if data is stored at the current reading position of the current data layer and the current reading position meets the preset position condition, read the first data stored at the current reading position of the current data layer, determine the target data layer of the first data, generate second data according to the data generator corresponding to the target data layer, and compare the first data with the second data to obtain a data verification result, wherein the first data is generated by the data generator and stored in the storage end.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the data verification method according to any one of claims 1 to 9.

13. A processor, characterized in that: The processor is used to run a program, wherein the program executes the data verification method according to any one of claims 1 to 9 when running.

Images