A RAID reconstruction management method and system

Abstract

The present invention provides a RAID reconstruction management method and system, the method comprising obtaining the IO load of the current business; adjusting the reconstruction load and back-end bandwidth utilization of the RAID reconstruction based on the IO load and the current business scenario. The present invention dynamically adjusts the reconstruction load and back-end bandwidth during the RAID reconstruction process through the information of the front-end business IO load, shortens the reconstruction time of the whole machine RAID, and thus improves the availability of the whole machine system. The management of the RAID reconstruction process is implemented through a RAID reconstruction management module set on a board programmable device, which dynamically adjusts the back-end bandwidth and reconstruction pressure load without occupying system resources, further improving the availability of the whole machine system.

Description

Technical Field

[0001] The present invention relates to the technical field of disk reconstruction, and in particular to a RAID (Redundant Arrays of Independent Disks, an array with redundancy composed of independent disks, also known as disk array) reconstruction management method and system. Background Art

[0002] In order to improve the performance and reliability of storage systems, disk arrays are widely used in storage systems. In large-scale storage systems, disk failures have become a normal state. Although a disk array can still ensure the availability of the disk array and data integrity when one or even several disks fail simultaneously. However, in order to avoid the unavailability of the disk array and data incompleteness caused by other disk failures, it is necessary to reconstruct the data on the failed disk.

[0003] Currently, unified storage in some industries pays more and more attention to the single-disk reconstruction duration. For example, some financial customers directly put the single-disk reconstruction time less than xx minutes in the centralized procurement rejection items. Generally speaking, the reconstruction time of the mainstream distributed RAID is strongly related to the single-disk capacity, stripe width, number of member disks, backend bandwidth, etc. However, in the actual reconstruction process, factors such as CPU, memory, and front-end service pressure may all affect the reconstruction performance, resulting in a deviation between the actual business and the theoretical calculation, and further affecting the availability of the entire machine system. Summary of the Invention

[0004] The present invention provides a RAID reconstruction management method and system for solving the problem that the existing RAID reconstruction management does not fully consider various influencing factors, thus affecting the availability of the entire machine system.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] In the first aspect of the present invention, a RAID reconstruction management method is provided, and the method includes the following steps:

[0007] Obtain the IO load of the current service;

[0008] Based on the IO load and the current service scenario, adjust the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction.

[0009] Further, the method further includes the steps of:

[0010] Obtain the single-disk bandwidth and the backend bandwidth;

[0011] Calculate the reconstruction time, and display the reconstruction time through a graphical user interface, where the single-disk bandwidth and the backend bandwidth are used as calculation parameters for the reconstruction time.

[0012] Furthermore, the calculation of the reconstruction time is specifically as follows:

[0013]

[0014] In the formula, T is the reconstruction time, K is the reconstruction scenario parameter, the single-disk capacity, stripe width, and number of member disks are all user-set parameters, the reconstruction load and the backend bandwidth utilization rate are adjustment values; min() represents taking the minimum value.

[0015] Furthermore, the service scenarios include a reconstruction-priority scenario and a normal scenario.

[0016] Furthermore, in the reconstruction-priority scenario, based on the IO load, the specific process of adjusting the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction is as follows:

[0017] If the IO load is not greater than a, the reconstruction load is not less than m, and the backend bandwidth utilization rate is A%;

[0018] If the IO load is less than a, the reconstruction load is less than n, and the backend bandwidth utilization rate is less than B%; where m is greater than n, and A is greater than B.

[0019] Furthermore, in the normal scenario, based on the IO load, the specific process of adjusting the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction is as follows:

[0020] In the normal scenario, the reconstruction load is not greater than n, and the backend bandwidth utilization rate is not higher than B%.

[0021] The second aspect of the present invention provides a RAID reconstruction management system, and the system includes:

[0022] An OS module, configured to offload the reconstruction task of the RAID to the RAID reconstruction management module, obtain the IO load of the current service, and send it to the RAID reconstruction management module;

[0023] A RAID reconstruction management module, which adjusts the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction based on the IO load and the current service scenario.

[0024] Furthermore, the system further includes:

[0025] An array module, configured to obtain the single-disk bandwidth;

[0026] A control expansion module, configured to obtain the backend bandwidth;

[0027] A GUI module, configured to display the reconstruction time calculated by the RAID reconstruction management module, and the single-disk bandwidth and the backend bandwidth are used as calculation parameters for the reconstruction time.

[0028] Furthermore, the system further includes:

[0029] A serial port module for information interaction of the RAID reconstruction management module and setting of reconstruction parameters;

[0030] A wireless module for converting the signals of the serial port module into wireless signals;

[0031] A display module for displaying the real-time status of the current RAID reconstruction management module.

[0032] A third aspect of the present invention provides a computer storage medium, in which computer instructions are stored. The feature is that when the computer instructions run on the system according to any one of claims 7-9, the system executes the steps of the method.

[0033] The RAID reconstruction management system according to the second aspect of the present invention can implement the method in the first aspect and each implementation manner of the first aspect, and achieve the same effect.

[0034] The effects provided in the summary of the invention are only the effects of the embodiments, rather than all the effects of the invention. One of the technical solutions in the above technical solutions has the following advantages or beneficial effects:

[0035] The present invention dynamically adjusts the reconstruction load and the backend bandwidth during the RAID reconstruction process through the information of the front-end service IO load, shortens the overall machine RAID reconstruction duration, and thus improves the availability of the overall machine system. And the management of the RAID reconstruction process is implemented through the RAID reconstruction management module set on the board programmable device, dynamically adjusting the backend bandwidth and the reconstruction pressure load without occupying system resources, further improving the availability of the overall machine system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

[0037] Figure 1 is a flowchart of the method embodiment of the present invention;

[0038] Figure 2 is a structural diagram of the system embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] To clearly illustrate the technical features of this solution, the present invention will be elaborated in detail below through specific embodiments and in conjunction with its accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. The present invention omits the description of well-known components, processing techniques, and processes to avoid unnecessarily limiting the present invention.

[0040] As Figure 1 shown, an embodiment of the present invention provides a RAID reconstruction management method, and the method includes the following steps:

[0041] S1. Obtain the IO load of the current service;

[0042] S2. Based on the IO load and the current service scenario, adjust the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction.

[0043] In one implementation of the method of the present invention, the method further includes the steps of:

[0044] Obtain the single-disk bandwidth and the backend bandwidth;

[0045] Calculate the reconstruction time and display the reconstruction time through a graphical user interface, and the single-disk bandwidth and the backend bandwidth are used as calculation parameters for the reconstruction time.

[0046] The calculation of the reconstruction time is specifically as follows:

[0047]

[0048] In the formula, T is the reconstruction time, K is the reconstruction scenario parameter, the single-disk capacity, the stripe width, and the number of member disks are all user-set parameters, the reconstruction load and the backend bandwidth utilization rate are adjustment values; min() represents taking the minimum of the two within the brackets.

[0049] The service scenario includes a reconstruction priority scenario and a normal scenario. The reconstruction scenario parameter K is K1 in the reconstruction priority scenario and K2 in the normal scenario. The value of K1 is 1, and the value of K2 is 1.1.

[0050] In step S2, in the reconstruction priority scenario, based on the IO load, the specific process of adjusting the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction is as follows:

[0051] If the IO load is not greater than a, the reconstruction load is not less than m, and the backend bandwidth utilization rate is A%;

[0052] If the IO load is less than a, the reconstruction load is less than n, and the backend bandwidth utilization rate is less than B%; where m is greater than n, and A is greater than B.

[0053] In a normal scenario, based on the IO load, the specific process of adjusting the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction is as follows:

[0054] In a normal scenario, the reconstruction load is not greater than n, and the backend bandwidth utilization rate is not higher than B%.

[0055] As one achievable way, the value of a is 30%, the value of m is 80%, the value of n is 60%, the value of A is 95, and the value of B is 75.

[0056] As Figure 2 shown, an RAID reconstruction management system provided by an embodiment of the present invention, the system includes an OS module, a reconstruction management module, a disk array module, a control expansion module, and a GUI module.

[0057] The OS module is used to offload the reconstruction task of the RAID to the RAID reconstruction management module without occupying system resources such as the whole machine's CPU and memory; and obtain the IO load of the current service and send it to the RAID reconstruction management module.

[0058] The RAID reconstruction management module is used for the management of the RAID reconstruction process, and adjusts the reconstruction load and the backend bandwidth utilization rate of the RAID reconstruction based on the IO load and the current service scenario. The RAID reconstruction management module is located on a board, such as a programmable logic device such as ARM. This module can communicate out-of-band with the control expansion module through control instructions such as MRPC using TWI, UART, Ethernet, etc., and can also communicate in-band with the control expansion module through SAS / PCIE, etc., and can also communicate with the disk array module using a low-speed bus such as I2C.

[0059] The RAID reconstruction management module calculates the expected single-disk reconstruction time according to the following formula based on parameter information such as single-disk capacity, stripe width, number of member disks, and backend bandwidth, and displays it through the GUI module.

[0060]

[0061] In the formula, T is the reconstruction time, K is the reconstruction scenario parameter, the single-disk capacity, stripe width, and number of member disks are all user-set parameters, and the reconstruction load and the backend bandwidth utilization rate are adjustment values; min() represents taking the minimum of the two inside the parentheses.

[0062] In this way, a quantitative judgment of the expected reconstruction time can be made when establishing a RAID. Among them, the single-disk capacity, stripe width, number of member disks, etc. are all parameters set by the customer; K is a reconstruction scenario parameter, which is K1 in the reconstruction priority scenario and K2 in the normal scenario; the backend bandwidth can be obtained in real time from the control expansion module through in-band or out-of-band methods, and the single-disk bandwidth can be obtained in real time from the disk array module through out-of-band methods.

[0063] The adjustment of the reconstruction load degree and the backend bandwidth utilization rate of the RAID reconstruction by the RAID reconstruction management module is specifically as follows: in the reconstruction priority scenario, if the IO load degree is not greater than a, the reconstruction load degree is not less than m, and the backend bandwidth utilization rate is A%; if the IO load degree is less than a, the reconstruction load degree is less than n, and the backend bandwidth utilization rate is less than B%; where m is greater than n and A is greater than B.

[0064] In the normal scenario, the reconstruction load degree is not greater than n, and the backend bandwidth utilization rate is not higher than B%.

[0065] As one realizable way, the value of a is 30%, the value of m is 80%, the value of n is 60%, the value of A is 95, and the value of B is 75.

[0066] The disk array module is a SAS disk array or an NVMe disk array, which is controlled by the RAID reconstruction management module; the GUI module manages the software interface and can display the expected single-disk reconstruction duration when establishing a RAID.

[0067] The system further includes a serial port module, a wireless module, and a display module. The serial port module is used for information interaction of the RAID reconstruction management module and setting of reconstruction parameters; the wireless module is used to convert the signal of the serial port module into a wireless signal; the display module is used to display the real-time status of the current RAID reconstruction management module.

[0068] An embodiment of the present invention further provides a computer storage medium, in which computer instructions are stored. When the computer instructions run on the system, the system is enabled to execute the steps of the method.

[0069] Although the specific implementation manners of the present invention are described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative labor are still within the protection scope of the present invention.

Claims

1. A RAID reconstruction management method, characterized in that: The method comprises the following steps: Get the IO load of the current business; Based on the IO load and the current business scenario, adjust the reconstruction load and backend bandwidth utilization of the RAID reconstruction; The method further comprises the steps of: Get single disk bandwidth and backend bandwidth; Calculate the reconstruction time, and display the reconstruction time through a graphical user interface, using the single disk bandwidth and the backend bandwidth as calculation parameters for the reconstruction time; The calculation of the reconstruction time is specifically as follows: Where T is the reconstruction time, K is the reconstruction scenario parameter, single disk capacity, stripe width and number of member disks are user-set parameters, reconstruction load and backend bandwidth utilization are adjustment values; min() means taking the minimum value.

2. The RAID reconstruction management method according to claim 1, characterized in that: The business scenarios include reconstruction priority scenarios and common scenarios.

3. The RAID reconstruction management method according to claim 2, characterized in that: In the reconstruction priority scenario, based on the IO load, the specific process of adjusting the reconstruction load and backend bandwidth utilization of RAID reconstruction is as follows: If the IO load is not greater than a, the reconstruction load is not less than m, and the backend bandwidth utilization is A%. If the IO load is less than a, the reconstruction load is less than n, and the backend bandwidth utilization is less than B%; wherein m is greater than n, and A is greater than B.

4. The RAID reconstruction management method according to claim 2, characterized in that: In a common scenario, based on the IO load, the specific process of adjusting the reconstruction load of RAID reconstruction and the backend bandwidth utilization is as follows: In common scenarios, the reconstruction load is no greater than n, and the backend bandwidth utilization is no higher than B%.

5. A RAID reconstruction management system, characterized in that: The system is used to implement the method according to claim 1, and the system comprises: The OS module is used to offload the RAID reconstruction task to the RAID reconstruction management module, and obtain the IO load of the current business and send it to the RAID reconstruction management module; A RAID reconstruction management module, which adjusts the reconstruction load and backend bandwidth utilization of the RAID reconstruction based on the IO load and the current business scenario; The system further comprises: Disk array module, used to obtain the bandwidth of a single disk; Control extension module to obtain backend bandwidth; The GUI module is used to display the reconstruction time calculated by the RAID reconstruction management module, and the single disk bandwidth and the backend bandwidth are used as calculation parameters of the reconstruction time.

6. The RAID reconstruction management system according to claim 5, characterized in that: The system further comprises: Serial port module, used for information exchange with RAID reconstruction management module and setting of reconstruction parameters; A wireless module, used for converting the signal of the serial port module into a wireless signal; The display module is used to display the real-time status of the current RAID reconstruction management module.

7. A computer storage medium having computer instructions stored therein, characterized in that: When the computer instructions are executed on the system according to any one of claims 5 to 6, the system is caused to execute the steps of the method according to any one of claims 1 to 4.

Images