Data verification method, device, equipment and computer-readable storage medium

Abstract

A data verification method, apparatus, device, and computer-readable storage medium. The method includes: after starting a data verification task, determining the number of tokens required for the data verification task; if the number of tokens in the token bucket is greater than or equal to the number of tokens required for the data verification task, subtracting the number of tokens required for the data verification task from the number of tokens in the token bucket and running the data verification task. Through this application, the execution of the data verification task is controlled based on a token allocation mechanism, effectively balancing the relationship between the data verification requirements and the system operation efficiency, ensuring both the timely execution of data verification and avoiding the degradation of system performance caused by a large number of data verifications.

Description

Technical Field

[0001] This application relates to the technical field of data storage, and in particular, to a data verification method, apparatus, device, and computer-readable storage medium. Background Art

[0002] The data stored in the storage system is prone to data storage errors due to disk damage. Based on this, it is necessary to run a data verification task to verify the correctness of the data in the storage system. How to reduce the impact of executing the data verification task on system performance is a technical problem that needs to be solved urgently. Summary of the Invention

[0003] To reduce the impact of executing the data verification task on system performance, this application provides a data verification method, apparatus, device, and computer-readable storage medium.

[0004] In a first aspect, an embodiment of this application provides a data verification method, which includes:

[0005] After the data verification task is started, determine the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task;

[0006] Detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, where the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset time period;

[0007] If the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens is greater than or equal to N2, then subtract N1 from the number of bandwidth tokens in the token bucket and subtract N2 from the number of operation tokens in the token bucket, and run the data verification task.

[0008] In combination with the first aspect, in an implementation, the determining the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task includes:

[0009] Determine the quotient of the length of the data to be verified corresponding to the data verification task divided by the amount of data that each bandwidth token allows to verify, and round up the quotient to obtain the required number of bandwidth tokens N1;

[0010] Determine the number of operation tokens N2 required according to the number of operations required for the data verification task.

[0011] In combination with the first aspect, in an implementation, updating the number of tokens in the token bucket includes:

[0012] When the update time point is reached, calculate the sum value S1 of C1 and P1 and calculate the sum value S2 of C2 and P2, where C1 is the number of bandwidth tokens in the current token bucket, P1 is the first preset value, C2 is the number of operation tokens in the current token bucket, and P2 is the second preset value;

[0013] If S1 ≤ M1, update the number of bandwidth tokens in the token bucket to S1; if S1 > M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket;

[0014] If S2 ≤ M2, update the number of operation tokens in the token bucket to S2; if S2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

[0015] Combined with the first aspect, in an implementation manner, updating the number of tokens in the token bucket includes:

[0016] When the update time point is reached, calculate the sum value K1 of C1 and Q1 and calculate the sum value K2 of C2 and Q2, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the IO latency of the storage system within the most recent second preset duration, and Q1 is inversely proportional to the magnitude of the IO latency of the storage system within the most recent second preset duration, C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the IO latency of the storage system within the most recent second preset duration, and Q2 is inversely proportional to the magnitude of the IO latency of the storage system within the most recent second preset duration;

[0017] If K1 ≤ M1, update the number of bandwidth tokens in the token bucket to K1; if K1 > M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket;

[0018] If K2 ≤ M2, update the number of operation tokens in the token bucket to K2; if K2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

[0019] Combined with the first aspect, in an implementation manner, after detecting whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, it further includes:

[0020] If the number of bandwidth tokens in the token bucket is less than N1 or the number of operation tokens is less than N2, control the data verification task to enter the sleep state and release the storage system resources occupied by the data verification task;

[0021] When the duration for the data verification task to enter the sleep state reaches the third preset duration, wake up the data verification task and return to check whether the number of bandwidth tokens in the detection token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2.

[0022] Combined with the first aspect, in an implementation, the third preset duration is determined according to the IO latency of the storage system within the most recent fourth preset duration, and the length of the third preset duration is directly proportional to the magnitude of the IO latency of the storage system within the most recent fourth preset duration.

[0023] In a second aspect, an embodiment of the present application provides a data verification device, which includes:

[0024] A determination module, configured to determine the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task after the data verification task is started;

[0025] A detection module, configured to detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, where the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset duration;

[0026] An execution module, configured to subtract N1 from the number of bandwidth tokens in the token bucket and subtract N2 from the number of operation tokens in the token bucket and run the data verification task if the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens is greater than or equal to N2.

[0027] Combined with the second aspect, in an implementation, the data verification device further includes a first update module, configured to:

[0028] When reaching the update time point, calculate the sum value K1 of C1 and Q1 and calculate the sum value K2 of C2 and Q2, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the IO latency of the storage system within the most recent second preset duration, and Q1 is inversely proportional to the magnitude of the IO latency of the storage system within the most recent second preset duration, C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the IO latency of the storage system within the most recent second preset duration, and Q2 is inversely proportional to the magnitude of the IO latency of the storage system within the most recent second preset duration;

[0029] If K1≤M1, update the number of bandwidth tokens in the token bucket to K1; if K1>M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket;

[0030] If K2 ≤ M2, update the number of operation tokens in the token bucket to K2; if K2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

[0031] In a third aspect, an embodiment of the present application provides a data verification device, which includes a processor, a memory, and a data verification program stored on the memory and executable by the processor. When the data verification program is executed by the processor, the steps of the data verification method described in the first aspect are implemented.

[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a data verification program is stored. When the data verification program is executed by a processor, the steps of the data verification method described in the first aspect are implemented.

[0033] The beneficial effects brought by the technical solutions provided in the embodiments of the present application include:

[0034] The execution of the data verification task is controlled based on the token allocation mechanism, effectively balancing the relationship between the data verification requirements and the system operation efficiency, ensuring the timely execution of data verification and avoiding the degradation of system performance caused by a large number of data verifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a schematic flowchart of an embodiment of the data verification method of the present application;

[0036] Figure 2 is a schematic diagram of the functional modules of an embodiment of the data verification device of the present application;

[0037] Figure 3 is a schematic diagram of the hardware structure of the data verification device involved in the solution of the embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0039] To make the purpose, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] In a first aspect, an embodiment of the present application provides a data verification method.

[0041] In one embodiment, referring to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the data verification method of the present application. As Figure 1 shown, the data verification method includes:

[0042] Step S10, after the data verification task is started, determine the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task;

[0043] In this embodiment, after the data verification task is started, the data verification task is not immediately run, but it is judged whether the conditions for running the data verification task are met. Specifically: detect whether the number of bandwidth tokens and the number of operation tokens in the current token bucket can meet the number of bandwidth tokens and the number of operation tokens required for the data verification task. Then, the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task need to be determined according to the data volume and the number of operations involved in the data verification task.

[0044] Further, in one embodiment, step S10 includes:

[0045] Step S101, determine the quotient of the length of the data to be verified corresponding to the data verification task divided by the amount of data that each bandwidth token allows to verify, round up the quotient to obtain the number of bandwidth tokens N1 required;

[0046] Step S102, determine the number of operation tokens N2 required according to the number of operations required for the data verification task.

[0047] In this embodiment, it should be noted that step S101 and step S102 can be executed sequentially or in parallel, and there is no limitation here. The amount of data that each bandwidth token allows to verify is a preset value, for example, 10M. If the length of the data to be verified corresponding to the data verification task is 25M, then the number of bandwidth tokens N1 = 3 can be calculated.

[0048] The number of operations required for a data verification task can be determined based on the operations involved in its running process. Further, the number of operations required for a data verification task can also be fixed to 1, then the number of operation tokens N2 required for a data verification task = 1.

[0049] Step S20, detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, where the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset time period;

[0050] Step S30: If the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens is greater than or equal to N2, subtract N1 from the number of bandwidth tokens in the token bucket and subtract N2 from the number of operation tokens in the token bucket, and run the data verification task.

[0051] In this embodiment, after determining N1 and N2, the number of bandwidth tokens and the number of operation tokens in the token bucket at this time can be compared with N1 and N2 respectively. Only when the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens is greater than or equal to N2, it is determined that the condition for running the data verification task is met, so as to run the data verification task, and subtract N1 from the number of bandwidth tokens in the token bucket and subtract N2 from the number of operation tokens in the token bucket.

[0052] In the existing data correctness scanning and verification scheme, when verifying the correctness of a certain segment of data, the actual data needs to be read. If no traffic control is performed, frequent reading of disk data will affect the normal IO performance of the storage system. Through the embodiment of the present application, the execution of the data verification task is controlled based on the token allocation mechanism, effectively balancing the relationship between the data verification requirement and the system operation efficiency, ensuring the timely execution of data verification and avoiding the system performance degradation caused by a large amount of data verification.

[0053] Further, in one embodiment, updating the number of tokens in the token bucket includes:

[0054] When reaching the update time point, calculate the sum value S1 of C1 and P1 and calculate the sum value S2 of C2 and P2, where C1 is the number of bandwidth tokens in the current token bucket, P1 is the first preset value, C2 is the number of operation tokens in the current token bucket, and P2 is the second preset value;

[0055] If S1 ≤ M1, update the number of bandwidth tokens in the token bucket to S1; if S1 > M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket;

[0056] If S2 ≤ M2, update the number of operation tokens in the token bucket to S2; if S2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

[0057] In this embodiment, taking the first preset duration as 10s as an example, that is, every 10s when reaching the update time point, updating the number of bandwidth tokens and operation tokens in the token bucket based on the pre-set P1 and P2 and combining with the constraint mechanism effectively avoids the risk of token overflow.

[0058] Further, in one embodiment, updating the number of tokens in the token bucket includes:

[0059] When the update time point is reached, calculate the sum value K1 of C1 and Q1 and calculate the sum value K2 of C2 and Q2, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the I / O latency of the storage system in the most recent second preset duration, and Q1 is inversely proportional to the magnitude of the I / O latency of the storage system in the most recent second preset duration, C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the I / O latency of the storage system in the most recent second preset duration, and Q2 is inversely proportional to the magnitude of the I / O latency of the storage system in the most recent second preset duration;

[0060] If K1 ≤ M1, update the number of bandwidth tokens in the token bucket to K1; if K1 > M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket;

[0061] If K2 ≤ M2, update the number of operation tokens in the token bucket to K2; if K2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

[0062] In this embodiment, taking the first preset duration as 10s as an example, that is, every 10s when the update time point is reached, based on the I / O latency of the storage system in the most recent second preset duration, the values of Q1 and Q2 are dynamically determined, so as to update the number of bandwidth tokens and operation tokens in the token bucket according to Q1 and Q2 and effectively avoid the risk of token overflow in combination with the constraint mechanism. Among them, the second preset duration is a preset value.

[0063] Among them, the I / O latency of the storage system refers to the time interval from the initiation of an I / O request to the completion of I / O processing, usually in milliseconds (ms).

[0064] Q1 / Q2 is inversely proportional to the magnitude of the I / O latency, that is, the larger the I / O latency, the smaller the value of Q1 / Q2. This is because when the I / O latency is large, it means that the storage system is already under greater pressure. At this time, fewer data verification tasks should be performed to avoid further increasing the burden on the storage system, thereby ensuring the stable operation of the entire storage system. On the contrary, when the I / O latency is small, it means that the storage system has more idle resources that can be utilized. At this time, data verification tasks are allowed to be performed more within a certain time range to fully exploit the processing potential of the storage system and improve the efficiency of data verification. It can be seen that by dynamically adjusting the value of Q1 / Q2 according to the I / O latency, the number of bandwidth / operation tokens in the token bucket is matched with the I / O latency, thus balancing the relationship between the data verification requirements and the system operation efficiency.

[0065] Further, in one embodiment, after step S20, it further includes:

[0066] If the number of bandwidth tokens in the token bucket is less than N1 or the number of operation tokens is less than N2, the data verification task is controlled to enter the sleep state, and the storage system resources occupied by the data verification task are released;

[0067] When the duration for which the data verification task has been in the sleep state reaches the third preset duration, the data verification task is woken up and returns to execute step S20.

[0068] In this embodiment, if the number of bandwidth tokens in the token bucket is less than N1 or the number of operation tokens is less than N2, the data verification task is controlled to enter the sleep state, and the storage system resources (such as CPU resources) occupied by the data verification task are released. After waiting for the third preset duration, the data verification task is woken up and returns to step S20.

[0069] Among them, the third preset duration can be preset according to actual needs, that is, the third preset duration is a fixed value.

[0070] Further, in one embodiment, the third preset duration is determined according to the IO latency of the storage system within the most recent fourth preset duration, and the length of the third preset duration is directly proportional to the magnitude of the IO latency of the storage system within the most recent fourth preset duration.

[0071] In this embodiment, considering that in the case of a large IO latency, it indicates that the storage system is already under great pressure, and at this time, the data verification task should do less; in the case of a small IO latency, it means that there are more idle resources in the storage system that can be utilized, and at this time, the data verification task is allowed to do more within a certain time range. Based on this, the length of the third preset duration can be dynamically determined according to the IO latency of the storage system within the most recent fourth preset duration, and the greater the IO latency of the storage system within the most recent fourth preset duration, the greater the length of the third preset duration; the smaller the IO latency of the storage system within the most recent fourth preset duration, the smaller the length of the third preset duration, thus balancing the relationship between the data verification requirements and the system operation efficiency. Among them, the fourth preset duration is a preset value.

[0072] In a second aspect, an embodiment of the present application further provides a data verification device.

[0073] In one embodiment, referring to Figure 2 , Figure 2 is a schematic diagram of the functional modules of an embodiment of the data verification device of the present application. As Figure 2 shown, the data verification device includes:

[0074] A determination module 10, configured to determine the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task after the data verification task is started;

[0075] A detection module 20, configured to detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, wherein the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset time period;

[0076] An execution module 30, configured to, if the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens is greater than or equal to N2, subtract N1 from the number of bandwidth tokens in the token bucket and subtract N2 from the number of operation tokens in the token bucket, and run the data verification task.

[0077] Further, in an embodiment, a determination module 10 is specifically configured to:

[0078] Determine the quotient of the length of the data to be verified corresponding to the data verification task divided by the amount of data that each bandwidth token allows to be verified, and round up the quotient to obtain the required number of bandwidth tokens N1;

[0079] Determine the required number of operation tokens N2 according to the number of operations required for the data verification task.

[0080] Further, in an embodiment, the data verification device further includes a first update module, configured to:

[0081] When reaching the update time point, calculate the sum value K1 of C1 and Q1 and calculate the sum value K2 of C2 and Q2, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the IO latency of the storage system in the most recent second preset time period, and Q1 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset time period, C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the IO latency of the storage system in the most recent second preset time period, and Q2 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset time period;

[0082] If K1 ≤ M1, update the number of bandwidth tokens in the token bucket to K1; if K1 > M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket;

[0083] If K2 ≤ M2, update the number of operation tokens in the token bucket to K2; if K2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

[0084] Further, in an embodiment, the data verification device further includes a second update module, configured to:

[0085] When the update time point is reached, calculate the sum value S1 of C1 and P1 and calculate the sum value S2 of C2 and P2, where C1 is the number of bandwidth tokens in the current token bucket, P1 is the first preset value, C2 is the number of operation tokens in the current token bucket, and P2 is the second preset value;

[0086] If S1 ≤ M1, update the number of bandwidth tokens in the token bucket to S1; if S1 > M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket;

[0087] If S2 ≤ M2, update the number of operation tokens in the token bucket to S2; if S2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

[0088] Further, in one embodiment, the data verification device further includes a sleep wake-up module for:

[0089] If the number of bandwidth tokens in the token bucket is less than N1 or the number of operation tokens is less than N2, control the data verification task to enter the sleep state and release the storage system resources occupied by the data verification task;

[0090] When the duration for which the data verification task enters the sleep state reaches the third preset duration, wake up the data verification task and notify the detection module 20 to detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2.

[0091] Further, in one embodiment, the third preset duration is determined according to the IO latency of the storage system within the most recent fourth preset duration, and the length of the third preset duration is in direct proportion to the magnitude of the IO latency of the storage system within the most recent fourth preset duration.

[0092] Among them, the function implementation of each module in the above data verification device corresponds to each step in the above data verification method embodiment, and its function and implementation process will not be elaborated here one by one.

[0093] In a third aspect, an embodiment of the present application provides a data verification device, and the data verification device may be a device with data processing functions such as a personal computer (PC), a laptop computer, a server, etc.

[0094] Refer to Figure 3 , Figure 3 which is a schematic diagram of the hardware structure of the data verification device involved in the embodiment solution of the present application. In the embodiment of the present application, the data verification device may include a processor, a memory, a communication interface, and a communication bus.

[0095] Among them, the communication bus can be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0096] The communication interface includes interfaces such as input / output (I / O) interfaces, physical interfaces, and logical interfaces for implementing the interconnection of components inside the data verification device, as well as interfaces for implementing the interconnection between the data verification device and other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0097] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical memory, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0098] The processor can be a general-purpose processor, and the general-purpose processor can call the data verification program stored in the memory and execute the data verification method provided by the embodiments of the present application. For example, the general-purpose processor can be a central processing unit (CPU). Among them, the method executed when the data verification program is called can refer to the various embodiments of the data verification method of the present application, which will not be elaborated here.

[0099] Those skilled in the art can understand that Figure 3 the hardware structure shown in does not constitute a limitation to the present application, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0100] In a fourth aspect, the embodiments of the present application further provide a computer-readable storage medium.

[0101] A data verification program is stored on the computer-readable storage medium of the present application. When the data verification program is executed by a processor, the steps of the data verification method as described above are implemented.

[0102] Among them, the method implemented when the data verification program is executed can refer to the various embodiments of the data verification method of the present application, which will not be elaborated here.

[0103] It should be noted that the serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0104] The terms "including" and "having" and any variations thereof in the description of the specification, claims and drawings of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices. The descriptions of terms such as "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit that "first", "second" and "third" are different types.

[0105] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for instance" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary", "for example" or "for instance" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary", "for example" or "for instance" is intended to present relevant concepts in a specific manner.

[0106] In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two.

[0107] In some processes described in the embodiments of the present application, a plurality of operations or steps appear in a specific order. However, it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or may be executed in parallel. The serial numbers of the operations are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0108] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device to execute the methods described in various embodiments of the present application.

[0109] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A data verification method, characterized in that, The data verification method includes: After the data verification task is started, determining the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task; Detecting whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, wherein the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset duration; If the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens is greater than or equal to N2, then subtracting N1 from the number of bandwidth tokens in the token bucket and subtracting N2 from the number of operation tokens in the token bucket, and running the data verification task; Updating the number of tokens in the token bucket includes: When the update time point is reached, calculating the sum value K1 of C1 and Q1 and calculating the sum value K2 of C2 and Q2, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the IO latency of the storage system in the most recent second preset duration, and Q1 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset duration, C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the IO latency of the storage system in the most recent second preset duration, and Q2 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset duration; If K1≤M1, then updating the number of bandwidth tokens in the token bucket to K1; if K1>M1, then updating the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket; If K2≤M2, then updating the number of operation tokens in the token bucket to K2; if K2>M2, then updating the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

2. The data verification method according to claim 1, wherein The determining the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task includes: Determining the quotient of the length of the data to be verified corresponding to the data verification task divided by the amount of data that each bandwidth token allows to verify, and rounding up the quotient to obtain the required number of bandwidth tokens N1; Determining the number of operation tokens N2 required according to the number of operations required for the data verification task.

3. The data verification method according to claim 1, characterized in that, After the detecting whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, it further includes: If the number of bandwidth tokens in the token bucket is less than N1 or the number of operation tokens is less than N2, then controlling the data verification task to enter the sleep state and releasing the storage system resources occupied by the data verification task; When the duration for which the data verification task enters the sleep state reaches the third preset duration, waking up the data verification task and returning to execute the detection of whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2.

4. The data verification method according to claim 3, wherein The third preset duration is determined according to the IO latency of the storage system in the most recent fourth preset duration, and the length of the third preset duration is directly proportional to the magnitude of the IO latency of the storage system in the most recent fourth preset duration.

5. A data verification device, characterized in that, The data verification device includes: A determining module, configured to determine the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task after the data verification task is started; A detection module, configured to detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, wherein the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset time period; An execution module, configured to, if the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens is greater than or equal to N2, subtract N1 from the number of bandwidth tokens in the token bucket and subtract N2 from the number of operation tokens in the token bucket, and run the data verification task; A first update module, configured to: When the update time point is reached, calculate the sum value K1 of C1 and Q1 and calculate the sum value K2 of C2 and Q2, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the IO latency of the storage system in the most recent second preset time period, and Q1 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset time period, C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the IO latency of the storage system in the most recent second preset time period, and Q2 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset time period; If K1 ≤ M1, update the number of bandwidth tokens in the token bucket to K1; if K1 > M1, update the number of bandwidth tokens in the token bucket to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket; If K2 ≤ M2, update the number of operation tokens in the token bucket to K2; if K2 > M2, update the number of operation tokens in the token bucket to M2, where M2 is the maximum number of operation tokens allowed in the token bucket.

6. A data verification device, characterized in that, The data verification device includes a processor, a memory, and a data verification program stored on the memory and executable by the processor, wherein when the data verification program is executed by the processor, the steps of the data verification method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that, A data verification program is stored on the computer-readable storage medium, wherein when the data verification program is executed by a processor, the steps of the data verification method according to any one of claims 1 to 4 are implemented.

Images