Data verification method, device and equipment and computer readable storage medium

Abstract

The invention discloses a data verification method, device and equipment and a computer readable storage medium. The method comprises the steps that after a data verification task is started, the number of tokens needed by the data verification task is determined; and if the number of the tokens in the token bucket is greater than or equal to the number of the tokens required by the data verification task, subtracting the number of the tokens required by the data verification task from the number of the tokens in the token bucket, and running the data verification task. According to the data verification method and device, execution of the data verification task is controlled based on the token distribution mechanism, the relation between the data verification requirement and the system operation efficiency is effectively balanced, it is guaranteed that data verification is conducted in time, and system performance reduction caused by verification of a large amount of data is avoided.

Description

Technical Field

[0001] This application relates to the technical field of data storage, and in particular, to a data verification method, apparatus, device, and computer-readable storage medium. Background Art

[0002] The data stored in the storage system is prone to data storage errors due to disk damage. Based on this, it is necessary to run a data verification task to verify the correctness of the data in the storage system. How to reduce the impact of executing the data verification task on the system performance is a technical problem that needs to be solved urgently. Summary of the Invention

[0003] To reduce the impact of executing the data verification task on the system performance, this application provides a data verification method, apparatus, device, and computer-readable storage medium.

[0004] In a first aspect, an embodiment of this application provides a data verification method, which includes: After the data verification task is started, determine the number of bandwidth tokens N 1 and the number of operation tokens N 2 ; Detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and whether the number of operation tokens is greater than or equal to N 2 , where the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset time period; If the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and the number of operation tokens is greater than or equal to N 2 , then subtract N from the number of bandwidth tokens in the token bucket 1 and subtract N from the number of operation tokens in the token bucket 2 , and run the data verification task.

[0005] In combination with the first aspect, in an implementation, the determining the number of bandwidth tokens N 1 and the number of operation tokens N 2 includes: Determine the quotient of the length of the data to be verified corresponding to the data verification task divided by the amount of data that each bandwidth token allows to verify, and round up the quotient to obtain the number of bandwidth tokens N 1 ; Determine the number of operation tokens N according to the number of operations required for the data verification task 2 .

[0006] In combination with the first aspect, in an implementation, updating the number of tokens in the token bucket includes: When the update time point is reached, calculate C 1 and the sum value S of C 1 and P 1 and calculate C 2 and the sum value S of C 2 and P 2 , where C 1 is the number of bandwidth tokens in the current token bucket, P 1 is the first preset value, C 2 is the number of operation tokens in the current token bucket, P 2 is the second preset value; If S 1 ≤M 1 , then update the number of bandwidth tokens in the token bucket to S 1 ; if S 1 >M 1 , then update the number of bandwidth tokens in the token bucket to M 1 , M 1 is the maximum number of bandwidth tokens allowed in the token bucket; If S 2 ≤M 2 , then update the number of operation tokens in the token bucket to S 2 ; if S 2 >M 2 , then update the number of operation tokens in the token bucket to M 2 , M 2 is the maximum number of operation tokens allowed in the token bucket.

[0007] Combined with the first aspect, in an implementation manner, updating the number of tokens in the token bucket includes: When the update time point is reached, calculate the sum value K of C 1 and Q 1 and calculate the sum value K of C 1 and Q 2 and Q 2 and the sum value K of C 2 , where C 1 is the number of bandwidth tokens in the current token bucket, Q 1 is determined according to the IO latency of the storage system within the most recent second preset duration, and Q 1 is inversely proportional to the magnitude of the IO latency of the storage system within the most recent second preset duration, C 2 is the number of operation tokens in the current token bucket, Q 2 is determined according to the IO latency of the storage system within the most recent second preset duration, and Q 2 is inversely proportional to the magnitude of the IO latency of the storage system within the most recent second preset duration; If K 1 ≤M 1, then update the number of bandwidth tokens in the token bucket to K 1 ; if K 1 > M 1 , then update the number of bandwidth tokens in the token bucket to M 1 , M 1 is the maximum number of bandwidth tokens allowed in the token bucket; If K 2 ≤ M 2 , then update the number of operation tokens in the token bucket to K 2 ; if K 2 > M 2 , then update the number of operation tokens in the token bucket to M 2 , M 2 is the maximum number of operation tokens allowed in the token bucket.

[0008] Combined with the first aspect, in one implementation, after detecting whether the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and whether the number of operation tokens is greater than or equal to N 2 , it further includes: If the number of bandwidth tokens in the token bucket is less than N 1 or the number of operation tokens is less than N 2 , then control the data verification task to enter the sleep state and release the storage system resources occupied by the data verification task; When the duration for which the data verification task enters the sleep state reaches the third preset duration, wake up the data verification task and return to execute the detection of whether the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and whether the number of operation tokens is greater than or equal to N 2 .

[0009] Combined with the first aspect, in one implementation, the third preset duration is determined according to the IO latency of the storage system within the most recent fourth preset duration, and the length of the third preset duration is directly proportional to the magnitude of the IO latency of the storage system within the most recent fourth preset duration.

[0010] Second aspect, an embodiment of the present application provides a data verification device, the data verification device includes: A determination module, configured to determine the number of bandwidth tokens N 1 and the number of operation tokens N 2 required by the data verification task after the data verification task is started; A detection module, configured to detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and whether the number of operation tokens is greater than or equal to N 2 , wherein the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset duration; An execution module is used to execute if the number of bandwidth tokens in the token bucket is greater than or equal to N 1 And the number of operation tokens is greater than or equal to N 2 , then the number of bandwidth tokens in the token bucket is reduced by N 1 And the number of operation tokens in the token bucket is reduced by N 2 , and run the data verification task.

[0011] In conjunction with the second aspect, in one implementation, the data verification device further includes a first updating module, configured to: When the update time point is reached, calculate C 1 With Q 1 The sum value K 1 And calculate C 2 With Q 2 The sum value K 2 , where C 1 is the number of bandwidth tokens in the current token bucket, Q 1 Determined based on the IO latency of the storage system within the most recent second preset time period, and Q 1 It is inversely proportional to the IO latency of the storage system within the second most recent preset time. 2 is the number of operation tokens in the current token bucket, Q 2 Determined based on the IO latency of the storage system within the most recent second preset time period, and Q 2 Inversely proportional to the IO latency of the storage system within the second most recent preset time period; If K 1 ≤M 1 , then update the number of bandwidth tokens in the token bucket to K 1 ; if K 1 >M 1 , then update the number of bandwidth tokens in the token bucket to M 1 , M 1 The maximum number of bandwidth tokens allowed in the token bucket; If K 2 ≤M 2 , then update the number of operation tokens in the token bucket to K 2 ; if K 2 >M 2 , then update the number of operation tokens in the token bucket to M 2 , M 2 The maximum number of operation tokens allowed in the token bucket.

[0012] In a third aspect, an embodiment of the present application provides a data verification device, which includes a processor, a memory, and a data verification program stored on the memory and executable by the processor. When the data verification program is executed by the processor, the steps of the data verification method described in the first aspect are implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a data verification program is stored. When the data verification program is executed by a processor, the steps of the data verification method described in the first aspect are implemented.

[0014] The beneficial effects brought by the technical solutions provided in the embodiments of the present application include: The execution of the data verification task is controlled based on a token allocation mechanism, effectively balancing the relationship between the data verification requirements and the system operation efficiency, ensuring the timely execution of data verification and avoiding the degradation of system performance caused by a large number of data verifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a schematic flowchart of an embodiment of the data verification method of the present application; Figure 2 is a schematic diagram of the functional modules of an embodiment of the data verification device of the present application; Figure 3 is a schematic diagram of the hardware structure of the data verification device involved in the solution of the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0017] In order to make the purpose, technical solution, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0018] In a first aspect, an embodiment of the present application provides a data verification method.

[0019] In one embodiment, referring to Figure 1 , Figure 1 is a schematic flowchart of the first embodiment of the data verification method of the present application. As Figure 1 shown, the data verification method includes: Step S10, after the data verification task is started, determine the number of bandwidth tokens N required for the data verification task 1 and the number of operation tokens N 2 ; In this embodiment, after the data verification task is started, the data verification task is not immediately run, but it is judged whether the conditions for running the data verification task are met. Specifically: detect whether the number of bandwidth tokens and the number of operation tokens in the current token bucket can meet the number of bandwidth tokens and the number of operation tokens required for the data verification task. Then, the number of bandwidth tokens N required for the data verification task needs to be determined according to the data volume and the number of operations involved in the data verification task 1 and the number of operation tokens N 2 .

[0020] Further, in one embodiment, step S10 includes: Step S101, determine the quotient of the length of the data to be verified corresponding to the data verification task divided by the data volume allowed to be verified by each bandwidth token, round up the quotient to obtain the number of bandwidth tokens N required 1 ; Step S102, determine the number of operation tokens N required according to the number of operations required for the data verification task 2 .

[0021] In this embodiment, it should be noted that step S101 and step S102 can be executed sequentially or in parallel, and there is no limitation here. The data volume allowed to be verified by each bandwidth token is a preset value, for example, 10M. If the length of the data to be verified corresponding to the data verification task is 25M, then the number of bandwidth tokens N required can be calculated 1 = 3.

[0022] The number of operations required for a data verification task can be determined based on the operations involved in its running process. Further, the number of operations required for a data verification task can also be fixed at 1, then the number of operation tokens N required for a data verification task 2 = 1.

[0023] Step S20, detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and whether the number of operation tokens is greater than or equal to N 2 , where the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset time interval; Step S30, if the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and the number of operation tokens is greater than or equal to N 2 , then subtract N from the number of bandwidth tokens in the token bucket 1 and subtract N from the number of operation tokens in the token bucket2 , and run the data verification task.

[0024] In this embodiment, after determining N 1 and N 2 , the number of bandwidth tokens and the number of operation tokens in the token bucket at this time can be respectively compared with N 1 and N 2 . Only when the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and the number of operation tokens is greater than or equal to N 2 , it is determined that the condition for running the data verification task is met, so as to run the data verification task, subtract N 1 from the number of bandwidth tokens in the token bucket and subtract N 2 from the number of operation tokens in the token bucket.

[0025] In the existing data correctness scanning and verification scheme, when verifying the correctness of a certain section of data, it is necessary to read the actual data. If no traffic control is performed, frequent reading of disk data will affect the normal IO performance of the storage system. Through the embodiment of the present application, the execution of the data verification task is controlled based on the token allocation mechanism, effectively balancing the relationship between the data verification requirement and the system operation efficiency, ensuring the timely execution of data verification and avoiding the system performance degradation caused by a large amount of data verification.

[0026] Further, in one embodiment, updating the number of tokens in the token bucket includes: When reaching the update time point, calculate the sum value S 1 of C 1 and P 1 , and calculate the sum value S 2 of C 2 and P 2 , where C 1 is the number of bandwidth tokens in the current token bucket, P 1 is the first preset value, C 2 is the number of operation tokens in the current token bucket, and P 2 is the second preset value; If S 1 ≤M 1 , then update the number of bandwidth tokens in the token bucket to S 1 ; if S 1 >M 1 , then update the number of bandwidth tokens in the token bucket to M 1 , and M 1 is the maximum number of bandwidth tokens allowed in the token bucket; If S 2 ≤M 2 , then update the number of operation tokens in the token bucket to S 2 ; if S2 > M 2 Then update the number of operation tokens in the token bucket to M 2 , M 2 is the maximum number of operation tokens allowed in the token bucket.

[0027] In this embodiment, taking the first preset duration as 10s as an example, that is, every 10s when the update time point is reached, based on the pre-set P 1 and P 2 Updating the number of bandwidth tokens and operation tokens in the token bucket and combining the constraint mechanism effectively avoids the risk of token overflow.

[0028] Furthermore, in one embodiment, updating the number of tokens in the token bucket includes: When the update time point is reached, calculate C 1 and the sum value K of Q 1 1 and calculate C 2 and the sum value K of Q 2 2 , where C 1 is the number of bandwidth tokens in the current token bucket, Q 1 is determined according to the IO latency of the storage system in the most recent second preset duration, and Q 1 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset duration, C 2 is the number of operation tokens in the current token bucket, Q 2 is determined according to the IO latency of the storage system in the most recent second preset duration, and Q 2 is inversely proportional to the magnitude of the IO latency of the storage system in the most recent second preset duration; If K 1 ≤ M 1 , then update the number of bandwidth tokens in the token bucket to K 1 ; if K 1 > M 1 , then update the number of bandwidth tokens in the token bucket to M 1 , M 1 is the maximum number of bandwidth tokens allowed in the token bucket; If K 2 ≤ M 2 , then update the number of operation tokens in the token bucket to K 2 ; if K 2 > M 2 , then update the number of operation tokens in the token bucket to M 2 , M 2 is the maximum number of operation tokens allowed in the token bucket.

[0029] ​​In this embodiment, taking the first preset duration as 10s as an example, that is, every 10s when the update time point is reached, based on the IO latency of the storage system within the most recent second preset duration, Q is dynamically determined. 1 and Q 2 values, so as to update the number of bandwidth tokens and operation tokens in the token bucket according to Q 1 and Q 2 and effectively avoid the risk of token overflow by combining with the constraint mechanism. Among them, the second preset duration is a preset value.

[0030] Among them, the IO latency of the storage system refers to the time interval from the initiation of an I / O request to the completion of I / O processing, usually in milliseconds (ms).

[0031] Q 1 / Q 2 is inversely proportional to the magnitude of the IO latency, that is, the greater the IO latency, the smaller the value of Q 1 / Q 2 . This is because when the IO latency is large, it indicates that the storage system is already under great pressure. At this time, fewer data verification tasks should be performed to avoid further increasing the burden on the storage system, thereby ensuring the stable operation of the entire storage system. On the contrary, when the IO latency is small, it means that the storage system has more idle resources that can be utilized. At this time, it is allowed to perform more data verification tasks within a certain time range to fully exploit the processing potential of the storage system and improve the efficiency of data verification. It can be seen that by dynamically adjusting the value of Q 1 / Q 2 , the number of bandwidth / operation tokens in the token bucket is matched with the IO latency, thus balancing the relationship between the data verification requirements and the system operation efficiency.

[0032] Furthermore, in one embodiment, after step S20, it further includes: If the number of bandwidth tokens in the token bucket is less than N 1 or the number of operation tokens is less than N 2 , then control the data verification task to enter the sleep state and release the storage system resources occupied by the data verification task; When the duration for which the data verification task enters the sleep state reaches the third preset duration, wake up the data verification task and return to execute step S20.

[0033] In this embodiment, if the number of bandwidth tokens in the token bucket is less than N 1 or the number of operation tokens is less than N 2 , then control the data verification task to enter the sleep state and release the storage system resources (such as CPU resources) occupied by the data verification task. After waiting for the third preset duration, wake up the data verification task and return to step S20.

[0034] Among them, the third preset duration can be preset according to actual needs, that is, the third preset duration is a fixed value.

[0035] Further, in one embodiment, the third preset duration is determined according to the IO latency of the storage system within the most recent fourth preset duration, and the length of the third preset duration is directly proportional to the magnitude of the IO latency of the storage system within the most recent fourth preset duration.

[0036] In this embodiment, considering that when the IO latency is large, it indicates that the storage system is already facing greater pressure, and at this time, fewer data verification tasks should be performed; when the IO latency is small, it means that the storage system has more idle resources that can be utilized, and at this time, more data verification tasks are allowed to be performed within a certain time range. Based on this, the length of the third preset duration can be dynamically determined according to the IO latency of the storage system within the most recent fourth preset duration, and the greater the IO latency of the storage system within the most recent fourth preset duration, the greater the length of the third preset duration; the smaller the IO latency of the storage system within the most recent fourth preset duration, the smaller the length of the third preset duration, thereby balancing the relationship between data verification requirements and system operation efficiency. Among them, the fourth preset duration is a preset value.

[0037] In a second aspect, an embodiment of the present application further provides a data verification device.

[0038] In one embodiment, referring to Figure 2 , Figure 2 is a schematic diagram of the functional modules of an embodiment of the data verification device of the present application. As Figure 2 shown, the data verification device includes: A determination module 10, configured to determine the number of bandwidth tokens N required for the data verification task after the data verification task is started 1 and the number of operation tokens N 2 ; A detection module 20, configured to detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and whether the number of operation tokens is greater than or equal to N 2 , where the number of bandwidth tokens and the number of operation tokens in the token bucket are updated every first preset duration; An execution module 30, configured to subtract N from the number of bandwidth tokens in the token bucket 1 and subtract N from the number of operation tokens in the token bucket 2 if the number of bandwidth tokens in the token bucket is greater than or equal to N 1 and the number of operation tokens is greater than or equal to N 2 , and run the data verification task.

[0039] Further, in one embodiment, the determination module 10 is specifically configured to: Determine the length of the data to be verified corresponding to the data verification task divided by the amount of data allowed to be verified by each bandwidth token, round up the quotient to get the required number of bandwidth tokens N 1 ; Determine the number of operation tokens N required based on the number of operations required for the data verification task 2 .

[0040] Furthermore, in one embodiment, the data verification device further includes a first updating module, which is used to: When the update time point is reached, calculate C 1 With Q 1 The sum value K 1 And calculate C 2 With Q 2 The sum value K 2 , where C 1 is the number of bandwidth tokens in the current token bucket, Q 1 Determined based on the IO latency of the storage system within the most recent second preset time period, and Q 1 It is inversely proportional to the IO latency of the storage system within the second most recent preset time. 2 is the number of operation tokens in the current token bucket, Q 2 Determined based on the IO latency of the storage system within the most recent second preset time period, and Q 2 Inversely proportional to the IO latency of the storage system within the second most recent preset time period; If K 1 ≤M 1 , then update the number of bandwidth tokens in the token bucket to K 1 ; if K 1 >M 1 , then update the number of bandwidth tokens in the token bucket to M 1 , M 1 The maximum number of bandwidth tokens allowed for the token bucket; If K 2 ≤M 2 , then update the number of operation tokens in the token bucket to K 2 ; if K 2 >M 2 , then update the number of operation tokens in the token bucket to M 2 , M 2 The maximum number of operation tokens allowed in the token bucket.

[0041] Furthermore, in one embodiment, the data verification device further includes a second updating module, which is used to: When the update time point is reached, calculate C 1 With P 1 The sum of S 1 And calculate C 2With P 2 The sum of S 2 , where C 1 is the number of bandwidth tokens in the current token bucket, P 1 is the first preset value, C 2 is the number of operation tokens in the current token bucket, P 2 is a second preset value; If S 1 ≤M 1 , then update the number of bandwidth tokens in the token bucket to S 1 If S 1 >M 1 , then update the number of bandwidth tokens in the token bucket to M 1 , M 1 The maximum number of bandwidth tokens allowed in the token bucket; If S 2 ≤M 2 , then update the number of operation tokens in the token bucket to S 2 If S 2 >M 2 , then update the number of operation tokens in the token bucket to M 2 , M 2 The maximum number of operation tokens allowed in the token bucket.

[0042] Furthermore, in one embodiment, the data verification device further includes a sleep and wake-up module, which is used to: If the number of bandwidth tokens in the token bucket is less than N 1 Or the number of operation tokens is less than N 2 , the data verification task is controlled to enter a dormant state and the storage system resources occupied by the data verification task are released; When the duration of the data verification task entering the dormant state reaches a third preset duration, the data verification task is awakened and the detection module 20 is notified to execute the detection of whether the number of bandwidth tokens in the token bucket is greater than or equal to N. 1 And whether the number of operation tokens is greater than or equal to N 2 .

[0043] Furthermore, in one embodiment, the third preset duration is determined based on the IO latency of the storage system within the most recent fourth preset duration, and the length of the third preset duration is directly proportional to the size of the IO latency of the storage system within the most recent fourth preset duration.

[0044] Among them, the functional implementation of each module in the above-mentioned data verification device corresponds to each step in the above-mentioned data verification method embodiment, and its functions and implementation processes are no longer repeated here.

[0045] In a third aspect, an embodiment of the present application provides a data verification device, which may be a device with data processing functions such as a personal computer (PC), a laptop computer, a server, etc.

[0046] Referring to Figure 3 , Figure 3 FIG. is a schematic diagram of the hardware structure of the data verification device involved in the solution of the embodiment of the present application. In the embodiment of the present application, the data verification device may include a processor, a memory, a communication interface, and a communication bus.

[0047] Among them, the communication bus can be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0048] The communication interface includes interfaces such as input / output (I / O) interfaces, physical interfaces, and logical interfaces for interconnecting components inside the data verification device, as well as interfaces for interconnecting the data verification device with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0049] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical memory, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0050] The processor can be a general-purpose processor, and the general-purpose processor can call the data verification program stored in the memory and execute the data verification method provided by the embodiment of the present application. For example, the general-purpose processor can be a central processing unit (CPU). Among them, the method executed when the data verification program is called can refer to the various embodiments of the data verification method of the present application and will not be elaborated here.

[0051] Those skilled in the art can understand that Figure 3 the hardware structure shown in FIG. does not constitute a limitation to the present application, and may include more or fewer components than shown, or combine some components, or arrange different components.

[0052] Fourthly, an embodiment of the present application further provides a computer-readable storage medium.

[0053] A data verification program is stored on the computer-readable storage medium of the present application. When the data verification program is executed by a processor, the steps of the data verification method as described above are implemented.

[0054] The method implemented when the data verification program is executed may refer to various embodiments of the data verification method of the present application, which will not be elaborated herein.

[0055] It should be noted that the serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0056] The terms "including" and "having" and any variations thereof in the description of the embodiments of the present application, as well as in the specification and claims of the present application and the above-mentioned drawings, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices. The descriptions of terms such as "first", "second", and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit that "first", "second", and "third" are of different types.

[0057] In the description of the embodiments of the present application, "exemplary", "for example", or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary", "for example", or "for instance" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary", "for example", or "for instance" is intended to present related concepts in a specific manner.

[0058] In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two.

[0059] In some processes described in the embodiments of the present application, there are multiple operations or steps that appear in a specific order. However, it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or may be executed in parallel. The serial numbers of the operations are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0060] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes several instructions for causing a terminal device to execute the methods described in the various embodiments of the present application.

[0061] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A data verification method, characterized in that: The data verification method comprises: After the data verification task is started, the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task are determined; Detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, wherein the number of bandwidth tokens and the number of operation tokens in the token bucket are updated once every first preset time period; If the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens in the token bucket is greater than or equal to N2, the number of bandwidth tokens in the token bucket is reduced by N1 and the number of operation tokens in the token bucket is reduced by N2, and the data verification task is executed.

2. The data verification method according to claim 1, characterized in that: The determination of the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task includes: Determine the quotient of the length of the data to be verified corresponding to the data verification task divided by the amount of data allowed to be verified by each bandwidth token, and round up the quotient to obtain the required number of bandwidth tokens N1; The required number of operation tokens N2 is determined according to the number of operations required for the data verification task.

3. The data verification method according to claim 1, characterized in that: Updating the number of tokens in the token bucket includes: When the update time point is reached, the sum S1 of C1 and P1 is calculated, and the sum S2 of C2 and P2 is calculated, where C1 is the number of bandwidth tokens in the current token bucket, P1 is the first preset value, C2 is the number of operation tokens in the current token bucket, and P2 is the second preset value; If S1≤M1, the number of bandwidth tokens in the token bucket is updated to S1; if S1>M1, the number of bandwidth tokens in the token bucket is updated to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket; If S2≤M2, the number of operation tokens in the token bucket is updated to S2; if S2>M2, the number of operation tokens in the token bucket is updated to M2, where M2 is the maximum number of operation tokens allowed by the token bucket.

4. The data verification method according to claim 1, characterized in that: Updating the number of tokens in the token bucket includes: When the update time point is reached, the sum K1 of C1 and Q1 is calculated, and the sum K2 of C2 and Q2 is calculated, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the IO latency of the storage system within the most recent second preset time period, and Q1 is inversely proportional to the size of the IO latency of the storage system within the most recent second preset time period, and C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the IO latency of the storage system within the most recent second preset time period, and Q2 is inversely proportional to the size of the IO latency of the storage system within the most recent second preset time period; If K1≤M1, the number of bandwidth tokens in the token bucket is updated to K1; if K1>M1, the number of bandwidth tokens in the token bucket is updated to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket; If K2≤M2, the number of operation tokens in the token bucket is updated to K2; if K2>M2, the number of operation tokens in the token bucket is updated to M2, where M2 is the maximum number of operation tokens allowed by the token bucket.

5. The data verification method according to claim 1, characterized in that: After detecting whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, the method further includes: If the number of bandwidth tokens in the token bucket is less than N1 or the number of operation tokens is less than N2, the data verification task is controlled to enter a dormant state and the storage system resources occupied by the data verification task are released; When the duration of the data verification task entering the dormant state reaches a third preset duration, the data verification task is awakened and returns to execute detection whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2.

6. The data verification method according to claim 5, characterized in that: The third preset duration is determined according to the IO latency of the storage system within the most recent fourth preset duration, and the length of the third preset duration is directly proportional to the size of the IO latency of the storage system within the most recent fourth preset duration.

7. A data verification device, characterized in that: The data verification device comprises: A determination module, used to determine the number of bandwidth tokens N1 and the number of operation tokens N2 required for the data verification task after the data verification task is started; A detection module, used to detect whether the number of bandwidth tokens in the token bucket is greater than or equal to N1 and whether the number of operation tokens is greater than or equal to N2, wherein the number of bandwidth tokens and the number of operation tokens in the token bucket are updated once every first preset time period; The execution module is used to subtract N1 from the number of bandwidth tokens in the token bucket and N2 from the number of operation tokens in the token bucket if the number of bandwidth tokens in the token bucket is greater than or equal to N1 and the number of operation tokens in the token bucket is greater than or equal to N2, and run the data verification task.

8. The data verification device according to claim 7, characterized in that: The data verification device further includes a first updating module, which is used to: When the update time point is reached, the sum K1 of C1 and Q1 is calculated, and the sum K2 of C2 and Q2 is calculated, where C1 is the number of bandwidth tokens in the current token bucket, Q1 is determined according to the IO latency of the storage system within the most recent second preset time period, and Q1 is inversely proportional to the size of the IO latency of the storage system within the most recent second preset time period, and C2 is the number of operation tokens in the current token bucket, Q2 is determined according to the IO latency of the storage system within the most recent second preset time period, and Q2 is inversely proportional to the size of the IO latency of the storage system within the most recent second preset time period; If K1≤M1, the number of bandwidth tokens in the token bucket is updated to K1; if K1>M1, the number of bandwidth tokens in the token bucket is updated to M1, where M1 is the maximum number of bandwidth tokens allowed in the token bucket; If K2≤M2, the number of operation tokens in the token bucket is updated to K2; if K2>M2, the number of operation tokens in the token bucket is updated to M2, where M2 is the maximum number of operation tokens allowed by the token bucket.

9. A data verification device, characterized in that: The data verification device includes a processor, a memory, and a data verification program stored in the memory and executable by the processor, wherein when the data verification program is executed by the processor, the steps of the data verification method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a data verification program, wherein when the data verification program is executed by a processor, the steps of the data verification method according to any one of claims 1 to 6 are implemented.

Images