Method, system, device and medium for data verification of storage devices

Abstract

The application discloses a kind of to the method for carrying out data check to storage device, comprising the following steps: setting unit data, multiple reference data and respectively calculating the size and check value of each reference data;Into the first preset quantity of unit data is written in storage device and the second preset quantity of unit data is as to be checked data;Determine the maximum number of process that can be used to carry out data check and utilize the size of to-be-checked data divided by maximum number to obtain preliminary split data size;Utilize the size of to-be-checked data divided by first reference data not less than preliminary split data size to obtain first split quantity and based on first split quantity to to-be-checked data is split to obtain multiple split data;Utilize multiple processes to read a split data respectively to carry out check value calculation and compare with the check value of first reference data.The application also discloses a kind of system, computer device and readable storage medium.

Description

Technical Field

[0001] This invention relates to the field of storage, and more specifically to a method, system, device, and storage medium for verifying data in storage devices. Background Technology

[0002] For integrity verification of large datasets, the current testing approach involves reading the data and calculating the checksum, then calculating the checksum of the original written data, and finally comparing the checksum of the read data with that of the original written data to obtain the verification result. However, calculating the checksum of the original written data increases testing time, leading to low testing efficiency. Moreover, as the amount of data to be verified increases, the time required to calculate the checksum also increases. Summary of the Invention

[0003] In view of this, in order to overcome at least one aspect of the above-mentioned problems, embodiments of the present invention propose a method for data verification of a storage device, comprising the following steps:

[0004] Set up unit data and obtain multiple benchmark data based on the unit data, and calculate the size and check value of each benchmark data separately;

[0005] Write a first preset number of unit data into the storage device and use a second preset number of unit data as the data to be verified;

[0006] Determine the maximum number of processes that can be used for data verification and divide the size of the data to be verified by the maximum number to obtain the initial split data size;

[0007] Starting with the smallest benchmark data, compare it sequentially with the size of the initially split data until a first benchmark data not smaller than the size of the initially split data is obtained from multiple benchmark data;

[0008] The first number of splits is obtained by dividing the size of the data to be verified by the first base data, and the data to be verified is split into multiple split data based on the first number of splits;

[0009] Multiple processes are used to read one of the split data to calculate the check value and compare it with the check value of the first benchmark data. The comparison result is used as the data check result of the storage device.

[0010] In some embodiments, starting with the smallest benchmark data, the data is sequentially compared with the size of the initially split data until a first benchmark data not smaller than the size of the initially split data is obtained from a plurality of benchmark data, further comprising:

[0011] Sort the multiple benchmark data in ascending order;

[0012] For each of the aforementioned baseline data, an index is set that increments from 0 according to the order;

[0013] Starting from index 0, the size of the corresponding baseline data is compared with the size of the initially split data in ascending order until a first baseline data not less than the size of the initially split data is obtained from multiple baseline data, and the index of the first baseline data is recorded.

[0014] In some embodiments, it also includes:

[0015] In response to the fact that the size of all the benchmark data is smaller than the size of the initially split data, the largest of the multiple benchmark data is taken as the first benchmark data.

[0016] In some embodiments, dividing the size of the data to be verified by the first base data to obtain a first number of splits, and splitting the data to be verified based on the first number of splits to obtain multiple split data, further includes:

[0017] Determine whether the first number of splits is an integer;

[0018] In response to the first splitting quantity being an integer, the first splitting quantity is obtained by dividing the size of the data to be verified by the first base data, and the data to be verified is split based on the first splitting quantity to obtain multiple splitting data.

[0019] In response to the fact that the first number of splits is not an integer, the first number of splits is rounded down to obtain the second number of splits;

[0020] The size of the first remaining data is obtained by subtracting the product of the second number of splits and the size of the first baseline data from the size of the data to be verified.

[0021] Use the index of the first baseline data as the current index, the first remaining data as the current remaining data, and the second splitting quantity as the current splitting quantity;

[0022] Starting from the current index, compare the corresponding base data with the size of the current remaining data in descending order until a base data that is not greater than the size of the current remaining data is obtained;

[0023] The current remaining data is split using the benchmark data that is no larger than the current remaining data to obtain split data and the remaining data after splitting;

[0024] In response to the fact that the remaining data after the split is not 0, the current index is updated to the index corresponding to the base data that is not greater than the size of the current remaining data, the current remaining data is updated to the remaining data after the split, the current split number is incremented by 1, and the step of comparing the corresponding base data with the size of the current remaining data in descending order from the current index until a base data that is not greater than the size of the current remaining data is obtained is returned.

[0025] In response to the fact that the remaining data after the split is 0, the baseline data corresponding to each split data is recorded.

[0026] In some embodiments, the process of using multiple processes to read a split data segment to calculate a checksum and comparing it with the checksum of the first benchmark data, and using the comparison result as the data verification result of the storage device, further includes:

[0027] Compare the current number of splits with the maximum number of processes;

[0028] In response to the fact that the current number of splits is not greater than the maximum number of processes, a process is created for each split data to read one of the split data to calculate the check value and compare it with the check value of the corresponding base data. The comparison result is used as the data check result of the storage device.

[0029] In response to the current number of splits being greater than the maximum number of processes, each process is polled to read one of the split data to calculate a check value and compare it with the check value of the first benchmark data. The comparison result is then used as the data check result of the storage device.

[0030] In some embodiments, setting unit data and obtaining multiple baseline data based on the unit data further includes:

[0031] Copy the unit data 2 times respectively. n Multiple benchmark data were obtained in this way.

[0032] In some embodiments, determining the maximum number of processes that can be used for data verification further includes:

[0033] Get the number of available cores for each processor whose utilization is less than the threshold;

[0034] Add up all the available cores to get the maximum number of processes that can be used for data verification.

[0035] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a system for data verification of a storage device, comprising:

[0036] The module is configured to set unit data and obtain multiple benchmark data based on the unit data, and calculate the size and check value of each benchmark data separately.

[0037] The write module is configured to write a first preset number of unit data to the storage device and use a second preset number of unit data as data to be verified.

[0038] The determination module is configured to determine the maximum number of processes that can be used for data verification and to obtain the initial data size by dividing the size of the data to be verified by the maximum number.

[0039] The comparison module is configured to compare the smallest benchmark data with the size of the initially split data sequentially, until a first benchmark data not smaller than the size of the initially split data is obtained from multiple benchmark data.

[0040] The splitting module is configured to divide the size of the data to be verified by the first base data to obtain a first splitting number, and split the data to be verified based on the first splitting number to obtain multiple splitting data.

[0041] The verification module is configured to use multiple processes to read one of the split data to calculate the verification value and compare it with the verification value of the first benchmark data, and use the comparison result as the data verification result of the storage device.

[0042] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a computer device, comprising:

[0043] At least one processor; and

[0044] A memory storing a computer program executable on the processor, wherein the processor, when executing the program, performs the steps of any of the methods described above for verifying data on a storage device.

[0045] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of any of the methods for data verification of a storage device as described above.

[0046] The present invention has one of the following beneficial technical effects: The solution proposed in this invention addresses the problem of integrity verification of big data in storage devices by using a multi-process data reading method and comparing it with a preset verification value. This avoids having to recalculate the verification value of the original data for each verification, thereby improving the efficiency of big data verification. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0048] Figure 1 A flowchart illustrating a method for verifying data on a storage device, provided as an embodiment of the present invention;

[0049] Figure 2 A schematic diagram illustrating the representation of checksum values ​​in the pattern pool provided in an embodiment of the present invention;

[0050] Figure 3 A flowchart illustrating a method for data verification of a storage device provided in an embodiment of the present invention;

[0051] Figure 4 Another flowchart illustrating a method for data verification of a storage device provided in an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the splitting rules provided for embodiments of the present invention;

[0053] Figure 6 A schematic diagram of a system for verifying data in a storage device, provided as an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the structure of a computer device provided for an embodiment of the present invention;

[0055] Figure 8 A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0057] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0058] In embodiments of the present invention, idle represents the percentage of idle CPU; the higher the value, the more idle the core.

[0059] According to one aspect of the present invention, embodiments of the present invention provide a method for data verification of a storage device, such as... Figure 1 As shown, it may include the following steps:

[0060] S1, set unit data and obtain multiple benchmark data based on the unit data, and calculate the size and check value of each benchmark data respectively;

[0061] S2, Write a first preset number of unit data to the storage device and use a second preset number of unit data as the data to be verified;

[0062] S3, determine the maximum number of processes that can be used for data verification and obtain the initial split data size by dividing the size of the data to be verified by the maximum number;

[0063] S4, starting from the smallest of the benchmark data, compare it sequentially with the size of the initially split data, until a first benchmark data not smaller than the size of the initially split data is obtained from multiple benchmark data;

[0064] S5, divide the size of the data to be verified by the first base data to obtain the first number of splits, and split the data to be verified based on the first number of splits to obtain multiple split data;

[0065] S6, using multiple processes to read one of the split data to calculate the verification value and compare it with the verification value of the first benchmark data, and using the comparison result as the data verification result of the storage device.

[0066] The solution proposed in this invention addresses the problem of integrity verification of large data in storage devices. It adopts a multi-process data reading method and compares the data with a preset verification value, which can avoid recalculating the verification value of the original data for each verification and improve the efficiency of large data verification.

[0067] In some embodiments, setting unit data and obtaining multiple baseline data based on the unit data further includes:

[0068] Copy the unit data 2 times respectively. n Multiple benchmark data were obtained in this way.

[0069] Specifically, the method used in this invention involves allocating different processes to different CPU cores, reading data and calculating its checksum (e.g., MD5 value), and comparing it with pre-set checksums (e.g., MD5 values) in the pattern pool. Testers first pre-set the pattern pool: the pattern pool is configured according to 2... nThe rules pre-prepare checksums (e.g., MD5 values) for data of 1M, 2M, 4M, 8M, 16M, 32M, 64M, 128M, 256M, 512M, 1024M, 2048M, 4096M, and 8192M, and name the data. The checksums stored in the pattern pool can be expressed as follows: Figure 2 As shown.

[0070] It should be noted that this invention verifies the data integrity of the storage device, that is, the data read and the data written are consistent. Therefore, it can be repeated twice with a unit of data, such as 1MB of data. n Multiple benchmark data are obtained, such as 1M, 2M, 4M, 8M, 16M, 32M, 64M, 128M, 256M, 512M, 1024M, 2048M, 4096M, and 8192M. Then, the check value of each benchmark data is calculated to construct the pattern pool.

[0071] In some embodiments, such as Figure 3 The design diagram shown splits the large dataset to be read into several smaller datasets according to predefined rules in the pattern pool. The principle of splitting is to complete all processes at once within the available CPU cores. Each process is bound to a core and is responsible for verifying one split dataset, comparing the verification value (e.g., MD5 value) with a predefined verification value (e.g., MD5 value) in the pattern pool, and placing the comparison result in a list. The number of cores refers to the physical number of CPU cores in a computer, while the number of processes refers to the number of processes a CPU can process simultaneously. A core can only handle one process at a time. When the number of running processes equals the number of CPU cores, each core can serve a single process, maximizing CPU resource utilization and improving execution speed. When the number of running processes exceeds the number of CPU cores, the operating system needs to switch between different processes, which increases overhead and slows down the execution speed of each process. Therefore, generally, when the number of running processes equals the number of CPU cores, reducing process switching results in higher execution efficiency. Finally, the results of the process verification are continuously retrieved from the result list, and the verification result of each process is judged. When the result of each process passes, the data integrity verification is successful. If the result of a process fails, the data integrity verification fails.

[0072] In some embodiments, starting with the smallest benchmark data, the data is sequentially compared with the size of the initially split data until a first benchmark data not smaller than the size of the initially split data is obtained from a plurality of benchmark data, further comprising:

[0073] Sort the multiple benchmark data in ascending order;

[0074] For each of the aforementioned baseline data, an index is set that increments from 0 according to the order;

[0075] Starting from index 0, the size of the corresponding baseline data is compared with the size of the initially split data in ascending order until a first baseline data not less than the size of the initially split data is obtained from multiple baseline data, and the index of the first baseline data is recorded.

[0076] In some embodiments, it also includes:

[0077] In response to the fact that the size of all the benchmark data is smaller than the size of the initially split data, the largest of the multiple benchmark data is taken as the first benchmark data.

[0078] In some embodiments, dividing the size of the data to be verified by the first base data to obtain a first number of splits, and splitting the data to be verified based on the first number of splits to obtain multiple split data, further includes:

[0079] Determine whether the first number of splits is an integer;

[0080] In response to the first splitting quantity being an integer, the first splitting quantity is obtained by dividing the size of the data to be verified by the first base data, and the data to be verified is split based on the first splitting quantity to obtain multiple splitting data.

[0081] In response to the fact that the first number of splits is not an integer, the first number of splits is rounded down to obtain the second number of splits;

[0082] The size of the first remaining data is obtained by subtracting the product of the second number of splits and the size of the first baseline data from the size of the data to be verified.

[0083] Use the index of the first baseline data as the current index, the first remaining data as the current remaining data, and the second splitting quantity as the current splitting quantity;

[0084] Starting from the current index, compare the corresponding base data with the size of the current remaining data in descending order until a base data that is not greater than the size of the current remaining data is obtained;

[0085] The current remaining data is split using the benchmark data that is no larger than the current remaining data to obtain split data and the remaining data after splitting;

[0086] In response to the fact that the remaining data after the split is not 0, the current index is updated to the index corresponding to the base data that is not greater than the size of the current remaining data, the current remaining data is updated to the remaining data after the split, the current split number is incremented by 1, and the step of comparing the corresponding base data with the size of the current remaining data in descending order from the current index until a base data that is not greater than the size of the current remaining data is obtained is returned.

[0087] In response to the fact that the remaining data after the split is 0, the baseline data corresponding to each split data is recorded.

[0088] In some embodiments, the process of using multiple processes to read a split data segment to calculate a checksum and comparing it with the checksum of the first benchmark data, and using the comparison result as the data verification result of the storage device, further includes:

[0089] Compare the current number of splits with the maximum number of processes;

[0090] In response to the fact that the current number of splits is not greater than the maximum number of processes, a process is created for each split data to read one of the split data to calculate the check value and compare it with the check value of the corresponding base data. The comparison result is used as the data check result of the storage device.

[0091] In response to the current number of splits being greater than the maximum number of processes, each process is polled to read one of the split data to calculate a check value and compare it with the check value of the first benchmark data. The comparison result is then used as the data check result of the storage device.

[0092] Specifically, such as Figure 4 The flowchart shown illustrates the integrity verification process. First, it obtains the size and starting address (totalSize, startAddr) of the data involved in the calculation. Then, it obtains the number of available CPU cores (cpuNum) with idle time ≥ 50. Next, it creates a list (splitList) to store the data, with each element containing the data size (splitSize) and the starting address. Then, it splits the data according to a pre-created pattern pool, ensuring the splitting is within the available CPU core count and ideally, creating all processes at once. Figure 5 As shown, the specific principles are as follows:

[0093] 1. Initial split size: splitSize = totalSize / cpuNum;

[0094] 2. PatternIndexMax: Represents the number of elements in the pattern pool;

[0095] 3. i is the index of the pattern value, and patternSize[i] represents the pattern value at index i, sorted in ascending order, starting from i=0:

[0096] (i) When i < PatternIndexMax, if patternSize[i] < splitSize, continue to compare, making index i = i + 1, until patternSize[i] ≥ splitSize. At this time, the size of the data to be split is splitSize = patternSize[i], and the index is endIndex = i.

[0097] (ii) If i ≥ PatternIndexMax, the data size to be split is splitSize = patternSize[PatternIndexMax-1], and the index is endIndex = PatternIndexMax-1.

[0098] 4. Split according to splitSize. The initial number of splits is splitNum ≤ totalSize / splitSize. Rounding totalSize / splitSize gives the initial number of splits. Store the split data in splitList, which contains splitNum elements. Each element contains the size of the data (splitSize) and the starting address of the data.

[0099] 5. The size of the remaining data is DelSize = totalSize - splitSize * splitNum. Check if the remaining data is 0. If it is 0, end the splitting. If it is not 0, continue splitting. The splitting method is as follows:

[0100] (i) j is the index of the pattern value, and patternSize[j] represents the pattern value at index j. The indices are from largest to smallest, and j starts from the index j = endIndex. When j ≥ 0, if patternSize[j] > DelSize, the comparison continues until j = j - 1, until patternSize[j] ≤ DelSize. At this point, the size of the data to be split is patternSize[j]. Add patternSize[j] to splitList. At this point, the number of splits is updated to splitNum = splitNum + 1, and the remaining data becomes DelSize = DelSize - patternSize[j]. At this point, the index is updated to endIndex = j.

[0101] (ii) Determine if the remaining data DelSize is 0. If it is not 0, continue to split according to step i above until the remaining data DelSize is 0.

[0102] For example, if there are currently 60 available cores and the data size is 5058M, according to the initial split size splitSize = 5058M / 60, the initial split data size is 84.3M. 84.3M is then compared with the pattern rule, and 128M is greater than 84.3M. Therefore, 128M is chosen as the splitting rule. The initial number of splits is 5058M / 128M = 39.5, which is rounded down to 39, resulting in 39 128M data segments. The remaining data is... 5058M - 128M * 39 = 66M. Starting from 128M, we compare data from largest to smallest. We find that 64M is less than 66M, so we choose 64M for splitting. The number of splits is 39 + 1 = 40. The remaining data is 66 - 64 = 2M. Starting from 64M, we index the data from largest to smallest. This is exactly the same as the 2M data in the pattern rule, so we choose 2M for splitting. The number of splits is updated to 40 + 1 = 41. The final number of splits is 39 data of 128M, 1 data of 64M, and 1 data of 2M.

[0103] In some embodiments, determining the maximum number of processes that can be used for data verification further includes:

[0104] Get the number of available cores for each processor whose utilization is less than the threshold;

[0105] Add up all the available cores to get the maximum number of processes that can be used for data verification.

[0106] Specifically, such as Figure 6As shown, within the limits of the number of CPU cores, multiple processes are created to read data, calculate its checksum (e.g., MD5 value), and compare it with the checksum (e.g., MD5 value) of the corresponding pattern size in the pattern pool. The highest utilization is achieved when only one process runs per available core. The process for handling the number of processes created and how tasks are allocated is as follows: Figure 4 As shown. The rules for creating processes and assigning tasks are as follows: First, compare the number of data splits (splitNum) with the number of available CPU cores (cpuNum). If splitNum is less than or equal to cpuNum, create a process for each data split; if splitNum is greater than cpuNum, create a process for each available core, and complete all processes in multiple steps.

[0107] In some embodiments, to facilitate the collection and merging of results from each process, each process, after calculating the checksum (e.g., MD5 value) of a piece of data and comparing it with a preset checksum (e.g., MD5 value) in the pattern pool, places the final comparison result into a list. The process continuously retrieves checksum results from the list. If the number of retrieved results is less than or equal to the number of data splits, and a process checksum fails, the check should be terminated immediately; in this case, the data integrity check fails. The data integrity check succeeds only when the number of retrieved results equals the number of data splits, and the checksum results of each process pass.

[0108] The proposed solution can split large datasets based on preset pattern rules and the number of available CPU cores. It uses a multi-process approach to calculate checksums and compares them with pre-set checksums, improving the verification rate, shortening test case execution time, and increasing testing efficiency. Furthermore, by selecting CPU cores with an idle value ≥ 50 and creating processes less than or equal to the number of CPU cores, the system can complete tasks faster, with shorter response times, improving overall system efficiency. For large dataset verification, it avoids CPU resource overload or idleness, maintaining a balanced and efficient use of CPU resources, further enhancing testing efficiency.

[0109] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a system 400 for data verification of a storage device, such as... Figure 6 As shown, it includes:

[0110] Module 401 is configured to set unit data and obtain multiple reference data based on the unit data, and calculate the size and check value of each reference data respectively.

[0111] The write module 402 is configured to write a first preset number of unit data to the storage device and use a second preset number of unit data as data to be verified.

[0112] The determination module 403 is configured to determine the maximum number of processes that can be used for data verification and to obtain the initial split data size by dividing the size of the data to be verified by the maximum number.

[0113] The comparison module 404 is configured to compare the smallest reference data with the size of the initially split data sequentially, until a first reference data not smaller than the size of the initially split data is obtained from multiple reference data.

[0114] The splitting module 405 is configured to divide the size of the data to be verified by the first base data to obtain a first splitting number, and split the data to be verified based on the first splitting number to obtain multiple splitting data.

[0115] The verification module 406 is configured to use multiple processes to read one of the split data to calculate the verification value and compare it with the verification value of the first benchmark data, and use the comparison result as the data verification result of the storage device.

[0116] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 7 As shown, embodiments of the present invention also provide a computer device 501, comprising:

[0117] At least one processor 520; and

[0118] The memory 510 stores a computer program 511 that can be run on a processor. When the processor 520 executes the program, it performs the steps of any of the above methods for verifying data on the storage device.

[0119] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 8 As shown, embodiments of the present invention also provide a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, it performs the steps of any of the above-described methods for data verification of a storage device.

[0120] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods.

[0121] Furthermore, it should be understood that the computer-readable storage medium (e.g., memory) described herein may be volatile memory or non-volatile memory, or may include both volatile memory and non-volatile memory.

[0122] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0123] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0124] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0125] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0126] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0127] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for verifying data on a storage device, characterized in that, Includes the following steps: Set up unit data and obtain multiple benchmark data based on the unit data, and calculate the size and check value of each benchmark data separately; Write a first preset number of unit data into the storage device and use a second preset number of unit data as the data to be verified; Determine the maximum number of processes that can be used for data verification and divide the size of the data to be verified by the maximum number to obtain the initial split data size; Starting with the smallest benchmark data, compare it sequentially with the size of the initially split data until a first benchmark data not smaller than the size of the initially split data is obtained from multiple benchmark data; The first number of splits is obtained by dividing the size of the data to be verified by the first base data, and the data to be verified is split into multiple split data based on the first number of splits; Multiple processes are used to read one of the split data to calculate the check value and compare it with the check value of the first benchmark data. The comparison result is used as the data check result of the storage device.

2. The method as described in claim 1, characterized in that, Starting with the smallest benchmark data, it is sequentially compared with the size of the initially split data until a first benchmark data not smaller than the size of the initially split data is obtained from multiple benchmark data, further including: Sort the multiple benchmark data in ascending order; For each of the aforementioned baseline data, an index is set that increments from 0 according to the order; Starting from index 0, the size of the corresponding baseline data is compared with the size of the initially split data in ascending order until a first baseline data not less than the size of the initially split data is obtained from multiple baseline data, and the index of the first baseline data is recorded.

3. The method as described in claim 2, characterized in that, Also includes: In response to the fact that the size of all the benchmark data is smaller than the size of the initially split data, the largest of the multiple benchmark data is taken as the first benchmark data.

4. The method as described in claim 2, characterized in that, The method further includes dividing the size of the data to be verified by the first baseline data to obtain a first number of splits, and then splitting the data to be verified based on the first number of splits to obtain multiple split data. Determine whether the first number of splits is an integer; In response to the first splitting quantity being an integer, the first splitting quantity is obtained by dividing the size of the data to be verified by the first base data, and the data to be verified is split based on the first splitting quantity to obtain multiple splitting data. In response to the fact that the first number of splits is not an integer, the first number of splits is rounded down to obtain the second number of splits; The size of the first remaining data is obtained by subtracting the product of the second number of splits and the size of the first baseline data from the size of the data to be verified. Use the index of the first baseline data as the current index, the first remaining data as the current remaining data, and the second splitting quantity as the current splitting quantity; Starting from the current index, compare the corresponding base data with the size of the current remaining data in descending order until a base data that is not greater than the size of the current remaining data is obtained; The current remaining data is split using the benchmark data that is no larger than the current remaining data to obtain split data and the remaining data after splitting; In response to the fact that the remaining data after the split is not 0, the current index is updated to the index corresponding to the base data that is not greater than the size of the current remaining data, the current remaining data is updated to the remaining data after the split, the current split number is incremented by 1, and the step of comparing the corresponding base data with the size of the current remaining data in descending order from the current index until a base data that is not greater than the size of the current remaining data is obtained is returned. In response to the fact that the remaining data after the split is 0, the baseline data corresponding to each split data is recorded.

5. The method as described in claim 4, characterized in that, The method further includes: using multiple processes to read a segment of the split data to calculate a check value and comparing it with the check value of the first benchmark data, and using the comparison result as the data check result of the storage device; Compare the current number of splits with the maximum number of processes; In response to the fact that the current number of splits is not greater than the maximum number of processes, a process is created for each split data to read one of the split data to calculate the check value and compare it with the check value of the corresponding base data. The comparison result is used as the data check result of the storage device. In response to the current number of splits being greater than the maximum number of processes, each process is used in a round-robin fashion to read one of the split data to calculate a check value and compare it with the check value of the first benchmark data. The comparison result is then used as the data check result of the storage device.

6. The method as described in claim 1, characterized in that, Setting unit data and obtaining multiple benchmark data based on the unit data further includes: Copy the unit data 2 times respectively. n Multiple benchmark data were obtained in this way.

7. The method as described in claim 1, characterized in that, Determine the maximum number of processes that can be used for data validation, further including: Get the number of available cores for each processor whose utilization is less than the threshold; Add up all the available cores to get the maximum number of processes that can be used for data verification.

8. A system for verifying data on a storage device, characterized in that, include: The module is configured to set unit data and obtain multiple benchmark data based on the unit data, and calculate the size and check value of each benchmark data separately. The write module is configured to write a first preset number of unit data to the storage device and use a second preset number of unit data as data to be verified. The determination module is configured to determine the maximum number of processes that can be used for data verification and to obtain the initial data size by dividing the size of the data to be verified by the maximum number. The comparison module is configured to compare the smallest benchmark data with the size of the initially split data sequentially, until a first benchmark data not smaller than the size of the initially split data is obtained from multiple benchmark data. The splitting module is configured to divide the size of the data to be verified by the first base data to obtain a first splitting number, and split the data to be verified based on the first splitting number to obtain multiple splitting data. The verification module is configured to use multiple processes to read one of the split data to calculate the verification value and compare it with the verification value of the first benchmark data, and use the comparison result as the data verification result of the storage device.

9. A computer device, comprising: At least one processor; as well as A memory storing a computer program executable on the processor, characterized in that the processor executes the program by performing the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1-7.

Images