Storage pool fault risk prediction method and device and product

Abstract

The invention provides a storage pool fault risk prediction method and device and a product, and relates to the technical field of cloud computing. The method comprises the following steps: acquiring a health score of each storage unit according to hardware information of at least one storage unit included in a storage pool node; calculating a health score mean value of different types of storage units in the storage pool node according to the health score of each storage unit; and obtaining the health state score of the storage pool node according to the health score mean value of the plurality of different types of storage units and the weight value corresponding to the corresponding type of storage unit. According to the embodiment of the invention, the health score mean value of the storage unit of the corresponding type is calculated according to the health score obtained by the at least one storage unit included in the storage pool node by adopting the hardware information, and the health state score of the whole storage pool node is obtained by combining the weight value of the type; therefore, the overall hardware health state of the storage pool is scored in a unified manner, and the change of the hardware state is seen from the perspective of pooling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, in particular to a storage pool failure risk prediction method, device and product. BACKGROUND

[0002] The storage pool often has the problem of poor performance caused by single hard disk sub-health, which leads to cluster degradation or performance decline. Therefore, major manufacturers have existing sub-health detection mechanisms. The specific detection method is to monitor the error read / write times and delay time of the disk, set a certain threshold, and once the threshold is exceeded, it is judged that the disk has sub-health problems, triggering an alarm. However, there are some problems:

[0003] 1. The detection mechanism of sub-health is simple, and the threshold is set relatively loosely, which is not much different from failure detection.

[0004] 2. The influence of cluster throughput on indicators is not considered. For I / O (Input / Output Operations Per Second) slow or I / O blocking, etc., the strategy cannot be flexibly adjusted, and the amount of data written cannot be dynamically distinguished between busy and idle times.

[0005] 3. The health status of various indicators of the hardware is not dynamically tracked, and the state of the disk is not tracked in the time dimension, so the aging speed and failure tendency of the hardware cannot be observed.

[0006] 4. There is no unified scoring for the overall hardware health status of the storage pool. Macroscopically, the hardware state change cannot be seen from the perspective of pooling. Microscopically, there is no detection mechanism for the state of the SSD (Solid State Drive) disk and array card. SUMMARY

[0007] The purpose of the present application is to provide a storage pool failure risk prediction method, device and product, which solves the problem that the overall hardware health status of the storage pool is not scored uniformly in the prior art, causing the hardware state change to be unable to be seen from the perspective of pooling.

[0008] To achieve the above purpose, an embodiment of the present application provides a storage pool failure risk prediction method, which comprises:

[0009] According to the hardware information of at least one storage unit included in the storage pool node, the health score of each storage unit is obtained;

[0010] According to the health score of each storage unit, the health score average of different types of storage units in the storage pool node is calculated;

[0011] According to the health score average of the plurality of different types of storage units and the weight value corresponding to the type of storage unit, a health status score of the storage pool node is obtained.

[0012] Optionally, the method further comprises:

[0013] In a case where the health status score is lower than a first preset threshold, the storage pool node is set to a read-only state.

[0014] Optionally, the method, wherein the health score of each storage unit is obtained according to hardware information of at least one storage unit included in the storage pool node, comprises:

[0015] At least one failure prediction index is extracted from the hardware information, and each failure prediction index is expanded according to a feature generation strategy to obtain an index set; wherein the failure prediction index comprises one or more of the number of reallocated sectors, the read-write error rate, the seek error rate, and the number of rotation retries; and the feature generation strategy comprises one or more of the current value, the critical value difference, the statistical quantity feature, the time sequence change rate feature, and the sliding mean feature;

[0016] The index set is input into a trained prediction model to obtain a failure risk probability; wherein the prediction model is used to predict the failure risk probability of the storage unit according to the index set using a gradient boosting tree algorithm;

[0017] The health score is obtained according to the failure risk probability.

[0018] Optionally, the method, wherein the health score is obtained according to the failure risk probability, comprises:

[0019] A first health score is obtained according to the failure risk probability;

[0020] In a case where the first health score is lower than a second preset threshold, a first difference value is obtained; wherein the first difference value is a first difference value of the delay of a plurality of throughput collection points in a first curve graph and the delay in a standard curve graph; the standard curve graph is obtained by fitting the throughput and delay of a standard storage unit in historical data; and the first curve graph is obtained by fitting the throughput and delay of the storage unit;

[0021] The first difference values corresponding to the plurality of throughput collection points are averaged to obtain an error amount;

[0022] The health score corresponding to the storage unit is obtained according to the error amount and the first health score.

[0023] Optionally, the method further comprises:

[0024] In a case where the first health score is not lower than the second preset threshold, the first health score is determined as the health score.

[0025] Optionally, the method, wherein the health score of each storage unit is obtained according to hardware information of at least one storage unit included in the storage pool node, comprises:

[0026] The growth rate of at least one first state and at least one first parameter in a first preset period is obtained according to the hardware information of the storage unit, wherein the first state comprises one or more of a storage unit state, a battery state and a battery health state; and the first parameter comprises one or more of a read / write failure number, a predictable failure number and other failure numbers.

[0027] In a case where the first state is normal, a score value is obtained as a first value; or in a case where the first state is abnormal, the score value is obtained as a second value.

[0028] In a case where the growth rate of at least one of the first parameters does not exceed a third preset threshold, the sum of the score values of at least one of the first states is determined as the health score.

[0029] Optionally, the method further comprises:

[0030] In a case where the growth rate of at least one of the first parameters in the first preset period exceeds the third preset threshold, the health score is determined as the second value.

[0031] Optionally, the method, wherein the storage unit is a first type of storage unit, and the health score average of the first type of storage unit in the storage pool node is calculated according to the health score of each storage unit, comprises:

[0032] In a case where the health score is lower than a fourth preset threshold, the storage unit is counted as a failed unit.

[0033] The total number of failed units in the plurality of storage units of the first type is obtained, and the average value of the health score of the plurality of storage units of the first type is obtained.

[0034] The risk coefficient is obtained according to the redundancy strategy of the storage pool node and the total number of failed units.

[0035] The health score average of the first type of storage unit in the storage pool node is obtained according to the risk coefficient and the average value of the health score.

[0036] Optionally, the method, wherein, according to the redundancy strategy of the storage pool node and the total number of the failed units, a risk coefficient is obtained, comprising:

[0037] In the case that the redundancy strategy of the storage pool node is a erasure code redundancy strategy, according to the total number of the failed units and the number of check blocks set in the erasure code redundancy strategy, the risk coefficient is obtained;

[0038] In the case that the redundancy strategy of the storage pool node is a multi-copy redundancy strategy, according to the total number of the failed units, the position of the failed units and the number of copies set in the multi-copy redundancy strategy, the risk coefficient is obtained.

[0039] Optionally, the method, wherein the storage units are second-type storage units, and according to the health scores of each storage unit, a health score average of the second-type storage units in the storage pool node is calculated, comprising:

[0040] The health scores of the plurality of second-type storage units are averaged to obtain the health score average of the second-type storage units in the storage pool node.

[0041] Optionally, the method, wherein the storage units are third-type storage units, and according to the health scores of each storage unit, a health score average of the third-type storage units in the storage pool node is calculated, comprising:

[0042] The health scores of the plurality of third-type storage units are averaged to obtain the health score average of the third-type storage units in the storage pool node.

[0043] Optionally, the method, wherein according to the health score averages of a plurality of different types of storage units and the weight values corresponding to the respective types of storage units, a health status score of the storage pool node is obtained, comprising:

[0044] The products of the health score averages of the plurality of different types of storage units and the corresponding weight values are added to obtain the health status score.

[0045] Optionally, the method, wherein the hardware information comprises one or more of the following:

[0046] Self-monitoring, analysis and reporting technology information;

[0047] Hard disk alarm information;

[0048] Operation and maintenance event logs.

[0049] To achieve the above-mentioned purposes, embodiments of the present application provide a storage pool failure risk prediction device, comprising:

[0050] The first obtaining module is configured to obtain a health score of each storage unit according to hardware information of at least one storage unit included in the storage pool node;

[0051] The first calculating module is configured to calculate a health score mean value of different types of storage units in the storage pool node according to the health score of each storage unit;

[0052] The second obtaining module is configured to obtain a health state score of the storage pool node according to the health score mean value of the plurality of different types of storage units and a weight value corresponding to the type of storage unit.

[0053] To achieve the above object, an embodiment of the present application provides an electronic device, comprising a transceiver, a processor, a memory and a program or instruction stored in the memory and executable on the processor; wherein the processor implements the prediction method of the storage pool failure risk as described above when executing the program or instruction.

[0054] To achieve the above object, an embodiment of the present application provides a readable storage medium, which stores a program or instruction, wherein the program or instruction is executed by a processor to implement the steps in the prediction method of the storage pool failure risk as described above.

[0055] To achieve the above object, an embodiment of the present application provides a computer program product, which comprises computer instructions, and the computer instructions are executed by a processor to implement the steps in the prediction method of the storage pool failure risk as described above.

[0056] The above technical solution of the present application has the following advantages:

[0057] According to the embodiment of the present application, the health score mean value of the corresponding type of storage unit is calculated according to the health score obtained by the hardware information of at least one storage unit included in the storage pool node, and the health state score of the entire storage pool node is obtained in combination with the weight value of the type, so that the hardware state change is seen from the perspective of pool by uniformly scoring the overall hardware health state of the storage pool. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The schematic diagram of the prediction method of the storage pool failure risk is described in the embodiment of the present application;

[0059] Figure 2 The flowchart of the prediction method of the storage pool failure risk is described in the embodiment of the present application;

[0060] Figure 3 The standard curve diagram of the prediction method of the storage pool failure risk is described in the embodiment of the present application;

[0061] Figure 4 This is a schematic diagram of the risk coefficient of the storage pool failure risk prediction method described in the embodiments of the present invention;

[0062] Figure 5 This is a schematic diagram of the storage pool failure risk prediction device according to an embodiment of the present invention. Detailed Implementation

[0063] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0064] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0065] In various embodiments of the present invention, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0066] In addition, the terms "system" and "network" are often used interchangeably in this article.

[0067] In the embodiments provided by this invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0068] For ease of understanding, the following describes some aspects of the embodiments of the present invention:

[0069] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for predicting storage pool failure risk, which includes:

[0070] S10, based on the hardware information of at least one storage unit included in the storage pool node, obtain the health score of each storage unit;

[0071] It should be noted that, as Figure 2 As shown, in step S10, the hardware information of the network cloud resource pool storage cluster is obtained, that is, the hardware information of each storage unit included in the storage pool node is obtained, and then the health score of each storage unit is obtained according to the hardware information.

[0072] S20, calculate the average health score of different types of storage units in the storage pool node based on the health score of each storage unit;

[0073] It should be noted that, as Figure 2 As shown, in step S20, health scores are performed on the HDD (Hard Disk Drive) main disk, SSD (Solid State Drive) system disk, SSD cache disk and RAID card in the storage pool, and risky disks or risky hardware are identified. That is, based on the health score of each storage unit, the average health score of different types of storage units in the storage pool node is calculated.

[0074] S30, obtain the health status score of the storage pool node based on the average health score of multiple different types of storage units and the weight value corresponding to the corresponding type of storage unit;

[0075] It should be noted that, as Figure 2 As shown, in step S30, the redundancy strategy and role of the nodes are obtained, the weight values ​​and risk coefficients are determined, and the health score of the nodes is calculated according to the formula, and risky nodes are identified. That is, the health status score of all nodes in the storage pool is obtained by multiplying the average health score of different types of storage units by their corresponding weight values ​​and then summing the results.

[0076] In this embodiment, the average health score of the corresponding type of storage unit is calculated based on the health score obtained by the hardware information of at least one storage unit included in the storage pool node. Combined with the weight value of the type, the health status score of the entire storage pool node is obtained. Thus, by uniformly scoring the overall hardware health status of the storage pool, the changes in hardware status can be seen from the perspective of pooling.

[0077] Optionally, the method further includes:

[0078] If the health status score is lower than a first preset threshold, the storage pool node is set to read-only mode.

[0079] In this embodiment, such as Figure 2 As shown, after step S30, a self-healing strategy is implemented for risk nodes. That is, when the health status score is lower than the first preset threshold, the storage pool node is set to read-only state. If it is determined that the node has a large risk of failure, the node is isolated in time, and the business data is read-only and not written, so as to ensure that the business does not have data loss problems and does not have stability abnormalities.

[0080] Optionally, the method, wherein step S10 includes:

[0081] At least one fault prediction indicator is extracted from the hardware information, and each fault prediction indicator is expanded according to the feature generation strategy to obtain an indicator set; wherein, the fault prediction indicator includes one or more of the following: number of reassigned sectors, read / write error rate, seek error rate, and number of rotational retries; the feature generation strategy includes one or more of the following: current value, critical value difference, statistical feature, time series change rate feature, and moving average feature;

[0082] The indicator set is input into the trained prediction model to obtain the failure risk probability; wherein, the prediction model is used to predict the failure risk probability of the storage unit based on the indicator set using the gradient spanning tree algorithm.

[0083] The health score is obtained based on the probability of failure risk.

[0084] In this embodiment, software tools are used to collect SMART (Self-Monitoring, Analysis and Reporting Technology) information, hard disk alarm information, and operation and maintenance event logs, and filter out valid data as the fault prediction indicators. Since the number of faulty disks is much smaller than that of normal disks, faulty disks are oversampled and normal disks are undersampled to avoid model overfitting. For hard disk fault prediction scenarios, not all indicators in the original data of the fault prediction indicators may be useful for prediction. Based on the type of storage unit, useful indicators in the original data of the fault prediction indicators are selected according to expert experience and the true physical meaning of the indicators, so that the fault prediction indicators include one or more of the following: number of reallocated sectors, read / write error rate, seek error rate, and number of rotational retries.

[0085] The original data of the fault prediction indicators contains limited information and needs to be expanded. Feature generation strategies include the difference between current and critical values, statistical features, time-series change rate features, and moving average features. Each indicator in the fault prediction indicators is expanded according to the feature generation strategies to obtain the indicator set. When the feature generation strategy is based on the critical value difference (to determine the degree of abnormal deviation), the warning read / write error rate is 10%, the current read / write error rate is 9%, and the expanded feature is that there is still 1% buffer space remaining. When the feature generation strategy is based on the time-series change rate feature (to calculate the rate of change between adjacent time points to capture dynamic trends), the number of sectors reallocated from day one to day four is 10, 10, 10, and 50 respectively, and the expanded feature is that the growth rate accelerates on day four, indicating a significant deterioration trend. Taking the fault prediction indicator "Today's bad sector count = 2" as an example, the corresponding expanded indicators in the indicator set are as follows:

[0086] The weekly average number of bad sectors is 1.5 (statistical characteristic).

[0087] Today's bad sector growth rate = 33% (time-series change rate characteristic);

[0088] The difference between the critical value (5) and the critical value is 3 (critical value difference).

[0089] There may be high internal correlations among the indicators in the aforementioned indicator set, such as sequential read / write error rate and random read / write error rate, one of which can be selected; in addition, there may be cases where the distribution is relatively simple, such as when the business load is not high and the temperature remains at a constant state, remove indicators that are too highly correlated, have a simple distribution, and have low importance scores.

[0090] Next, the prediction model is set up using the Gradient Boosting Decision Tree (GDBT) algorithm, and iterated continuously. Each round of prediction has a residual between the actual value and the prediction. The next round is then based on this residual, and finally, the final result is output to obtain the failure risk probability of the storage unit. When the storage unit type is a Type I HDD main disk, the failure risk probability of the storage unit is converted into a health score, as shown in the following formula:

[0091]

[0092] Where θ1 is the failure risk probability of the HDD main storage. This is the health score of the HDD's main storage.

[0093] Optionally, the method, wherein obtaining the health score based on the failure risk probability includes:

[0094] A first health score is obtained based on the aforementioned failure risk probability;

[0095] If the first health score is lower than the second preset threshold, a first difference is obtained; wherein, the first difference is the first difference between the latency of multiple throughput collection points in the first curve and the latency in the standard curve; the standard curve is obtained by fitting the throughput and latency of a standard storage unit in historical data; the first curve is obtained by fitting the throughput and latency of the storage unit.

[0096] The average value of the first difference corresponding to the plurality of throughput collection points is taken to obtain the error amount;

[0097] Based on the error amount and the first health score, the health score corresponding to the storage unit is obtained.

[0098] In this embodiment, when the storage unit is a second type SSD system disk or cache disk, after obtaining the failure risk probability, a secondary judgment based on throughput and latency is needed to obtain the health score. The failure risk probability β2 of the SSD system disk or cache disk is converted into the first health score δ2 of the SSD system disk or cache disk, and the calculation method is the same as that for the first type. When the first health score δ2 is lower than a second preset threshold... In such cases, the storage unit will be added to the list of pending risk disks.

[0099] The throughput and latency of all storage units in the risk disk pending list are plotted as x and y axes, and connected to form a curve to obtain the first curve. Additionally, as... Figure 3 As shown, a regression algorithm is used to fit the throughput and latency of the entire SSD disk (i.e., standard storage unit) as datasets x and y, respectively, to generate a normal-failure edge curve (i.e., the standard curve). After fitting, the throughput and latency should be approximately convex functions with a proportional relationship. Figure 3 The diagram illustrates the case where a first difference is obtained through a throughput sampling point. The ordinate of the fitting point represents the delay of the standard curve, while the ordinate of the sampling point represents the delay of the storage unit. The difference between the ordinates of the two points is the first difference. The error is obtained using the following formula:

[0100]

[0101] Among them, S m The latency on the first curve (i.e., the latency of the corresponding storage unit) represents the throughput m, and T represents the latency on the standard curve (i.e., the latency of the standard storage unit) under the same throughput. Subtracting these two values ​​yields the first difference. The sum of the first differences across all valid throughput sampling points is then divided by the total number of points to obtain the error ρ3. The health score of the second type of storage unit, the SSD system disk or cache disk... The calculation formula is as follows:

[0102]

[0103] Here, λ is the adjustment coefficient, which is determined based on expert experience and the specific resource pool situation, and can be set to 10% by default. The first health score δ2 is dynamically adjusted by multiplying the error amount ρ3 and the adjustment coefficient λ, so that the actual influencing factors are reflected in the score.

[0104] Optionally, the method further includes:

[0105] If the first health score is not lower than the second preset threshold, the first health score is determined as the health score.

[0106] In this embodiment, the first health score δ2 is not lower than the second preset threshold. In the case of this, the first health score δ2 is determined as the health score. Right now

[0107] Optionally, the method, wherein step S10 includes:

[0108] Based on the hardware information of the storage unit, at least one first state and at least one first parameter are obtained, representing the growth rate of the parameter within a first preset period. The first state includes one or more of the following: storage unit state, battery state, and battery health state. The first parameter includes one or more of the following: number of read / write failures, number of predictable failures, and number of other failures.

[0109] If the first state is normal, the score is a first value; or if the first state is abnormal, the score is a second value.

[0110] If the growth rate of at least one of the first parameters does not exceed a third preset threshold, the sum of the scores of at least one of the first states is determined as the health score.

[0111] In this embodiment, when the storage unit is a third type, such as an array card, the calculation of the health score is divided into two layers: the first layer is a first value-second value judgment, typically setting the first value to 1 and the second value to 0. If any one of the following three first states changes, the score is judged to be the second value, which is 0; if there is no change, the score is the first value, which is 1.

[0112] (1) Array card state: Optimal (State: Optimal);

[0113] (2) Battery State: Optimal and isSOH Good: Yes;

[0114] The second layer needs to determine the growth rate of the following three parameters of the array card (i.e., the first parameter) within the first preset period:

[0115] (1) Media Error Count (number of read / write failures);

[0116] (2) Predictive Failure Count;

[0117] (3) Other Error Count.

[0118] If the growth rate of the first parameter does not exceed the third preset threshold, the sum of the scores of the first state is determined as the health score.

[0119] Optionally, the method further includes:

[0120] If at least one of the first parameters increases at a rate exceeding the third preset threshold within the first preset period, the health score is determined to be the second value.

[0121] In this embodiment, if the growth rate of any one or more of the three first parameters exceeds the third preset threshold, the health score of the array card is determined to be the second value, which is 0.

[0122] Optionally, in the method, wherein the storage unit is a first type of storage unit, step S20 includes:

[0123] If the health score is lower than the fourth preset threshold, the storage unit is classified as a faulty unit.

[0124] Obtain the total number of faulty units among the multiple storage units of the first type, and obtain the average of the health scores of the multiple storage units of the first type;

[0125] The risk coefficient is obtained based on the redundancy strategy of the storage pool nodes and the total number of faulty units;

[0126] The average health score of the storage units of the first type in the storage pool node is obtained based on the average of the risk coefficient and the health score.

[0127] In this embodiment, when the storage unit is a first type of storage unit, i.e., an HDD disk, the storage unit whose health score is lower than the fourth preset threshold is counted as the faulty unit. For example... Figure 4As shown, firstly, it is determined whether the storage pool node has any risky disks (i.e., faulty units). If there are no risky disks (i.e., the number of faulty units is 0), the risk level is determined to be no risk, with a coefficient of k0. If there are risky disks, it is determined whether the storage pool node is available. If the determination is no, the risk level is determined to be high risk, with a coefficient of k3. If the determination of whether the storage pool node is available is yes, the number of faulty disks (i.e., faulty units) is compared with the redundancy amount (determined according to the redundancy strategy). If the number is less than the redundancy amount, the risk level is determined to be low risk, with a coefficient of k1; if the number is equal to the redundancy amount, the risk level is determined to be medium risk, with a coefficient of k2; if the number is greater than the redundancy amount, the risk level is determined to be high risk, with a coefficient of k3. The product of the risk coefficient and the average of the health scores of all storage units of the first type is the average health score of the storage units of the first type. Where num1 is the number of storage units of the first type, and k is the risk coefficient.

[0128] Optionally, the method, wherein obtaining the risk coefficient based on the redundancy policy of the storage pool nodes and the total number of faulty units includes:

[0129] When the redundancy strategy of the storage pool node is the erasure coding redundancy strategy, the risk coefficient is obtained based on the total number of faulty units and the number of check blocks set in the erasure coding redundancy strategy.

[0130] When the redundancy strategy of the storage pool node is a multi-replica redundancy strategy, the risk coefficient is obtained based on the total number of faulty units, the location of the faulty units, and the number of replicas set in the multi-replica redundancy strategy.

[0131] In this embodiment, when the redundancy strategy of the storage pool node is an erasure coding redundancy strategy, the risk coefficient is determined based on the number of risk disks and the number of check blocks set in the erasure coding strategy:

[0132] The storage pool node has no risk disk, the risk level is determined to be no risk, and the risk coefficient is k0;

[0133] The number of risk disks in the storage pool node is greater than the number of check blocks, so the risk level is determined to be high risk, and the risk coefficient is k3.

[0134] The number of risk disks in the storage pool node is less than the number of check blocks, so the risk level is determined to be low risk, and the risk coefficient is k1.

[0135] The number of risk disks in the storage pool node is equal to the number of verification blocks, the risk level is determined to be medium risk, and the risk coefficient is k2.

[0136] For storage pools with a multi-replica redundancy strategy, the risk coefficient is determined based on the number of risky disks, the location of each risky disk, and the number of replicas set in the multi-replica strategy:

[0137] The storage pool node has no risk disk, and the risk level is determined to have a risk-free coefficient of k0.

[0138] If the number of storage pool nodes or racks containing the risk disk is greater than the number of replicas, the risk level is determined to be high risk, and the risk coefficient is k3.

[0139] If the number of storage pool nodes or racks containing the risk disk is less than the number of replicas, the risk level is determined to be low risk, and the risk coefficient is k1.

[0140] The number of storage pool nodes or racks containing the risk disk is equal to the number of replicas, the risk level is determined to be medium risk, and the risk coefficient is k2.

[0141] Optionally, in the method, wherein the storage unit is a second type of storage unit, step S20 includes:

[0142] The average health score of the multiple storage units of the second type is taken to obtain the average health score of the storage units of the second type in the storage pool node.

[0143] In this embodiment, for multiple storage units of the second type, namely SSDs, the average health score is the average of the health scores of the multiple SSDs. Wherein, num2 is the number of the storage units of the second type.

[0144] Optionally, in the method, wherein the storage unit is a third type of storage unit, step S20 includes:

[0145] The average health score of multiple storage units of the third type is taken to obtain the average health score of the third type of storage units in the storage pool node.

[0146] In this embodiment, for multiple storage units of the third type, i.e., RAID cards, the average health score is the average of the health scores of the multiple RAID cards. Wherein, num3 is the number of the storage units of the third type.

[0147] Optionally, the method, wherein step S30 includes:

[0148] The health status score is obtained by adding the products of the average health scores of multiple different types of storage units and their corresponding weight values.

[0149] In this embodiment, the health status score is obtained by adding the products of the average health scores of multiple storage units of different types and their corresponding weight values. When the number of storage units of the third type is 1, the formula for the health status score is as follows:

[0150]

[0151] In this system, β1 represents the weight of the first type of storage unit, HDD disks; β2 represents the weight of the second type of storage unit, SSD disks; and β3 represents the weight of the third type of storage unit, RAID cards. The weights of HDD disks, SSD disks, and RAID cards increase sequentially, approximately in a ratio of 1:5:10. The weights are adjusted according to the node's role. For example, the system disk (SSD disk) of a management storage pool node is more important than that of a regular data storage pool node; therefore, the β2 value is increased, and the β1 value is decreased. This is because a file system failure in a management storage pool node will render related data management functions unavailable, affecting the entire cluster. The weights reflect the different roles of each storage pool node.

[0152] Optionally, in the method, the hardware information includes one or more of the following:

[0153] Self-monitoring, analysis, and reporting technology information;

[0154] Hard drive alarm information;

[0155] Operations and maintenance event logs.

[0156] In this embodiment, software tools are used to collect SMART (Self-Monitoring, Analysis and Reporting Technology) information, hard disk alarm information, and operation and maintenance event logs, and valid data is selected from the hardware information as the fault prediction indicators.

[0157] It should be noted that the embodiments of the present invention take into account all hardware conditions of the network cloud cluster nodes, fully reflect the health status of the main disk, cache disk, system disk, and array card, and scientifically reflect the actual performance of the storage physical nodes through weights and risk coefficients. At the same time, a health score is calculated through a large artificial intelligence model, and the health score is corrected through I / O throughput and latency, avoiding excessive prevention that would lead to increased operation and maintenance costs.

[0158] like Figure 5As shown, to achieve the above objectives, embodiments of the present invention provide a storage pool failure risk prediction device, comprising:

[0159] The first acquisition module 501 is used to acquire the health score of each storage unit based on the hardware information of at least one storage unit included in the storage pool node.

[0160] The first calculation module 502 is used to calculate the average health score of different types of storage units in the storage pool node based on the health score of each storage unit.

[0161] The second acquisition module 503 is used to acquire the health status score of the storage pool node based on the average health score of multiple different types of storage units and the weight value corresponding to the storage unit of the corresponding type.

[0162] Optionally, the device further includes:

[0163] The first processing module is used to set the storage pool node to read-only state when the health status score is lower than a first preset threshold.

[0164] Optionally, in the aforementioned apparatus, the first acquisition module 501 includes:

[0165] The first acquisition unit is used to extract at least one fault prediction index from the hardware information, and to expand each fault prediction index according to a feature generation strategy to obtain an index set; wherein, the fault prediction index includes one or more of the following: number of reallocated sectors, read / write error rate, seek error rate, and number of rotational retries; the feature generation strategy includes one or more of the following: current value, critical value difference, statistical feature, time series change rate feature, and moving average feature;

[0166] The second acquisition unit is used to input the indicator set into the trained prediction model to obtain the failure risk probability; wherein, the prediction model is used to predict the failure risk probability of the storage unit based on the indicator set using the gradient spanning tree algorithm.

[0167] The third acquisition unit is used to acquire the health score based on the failure risk probability.

[0168] Optionally, in the aforementioned apparatus, the third acquisition unit comprises:

[0169] A first acquisition component is used to acquire a first health score based on the fault risk probability.

[0170] The second acquisition component is used to acquire a first difference when the first health score is lower than a second preset threshold; wherein, the first difference is the first difference between the latency of multiple throughput acquisition points in the first curve and the latency in the standard curve; the standard curve is obtained by fitting the throughput and latency of a standard storage unit in historical data; the first curve is obtained by fitting the throughput and latency of the storage unit.

[0171] The third acquisition component is used to take the average value of the first difference corresponding to the plurality of throughput acquisition points to obtain the error amount;

[0172] The fourth acquisition component is used to acquire the health score corresponding to the storage unit based on the error amount and the first health score.

[0173] Optionally, the device further includes:

[0174] The first determining module is used to determine the first health score as the health score if the first health score is not lower than the second preset threshold.

[0175] Optionally, in the aforementioned apparatus, the first acquisition module 501 includes:

[0176] The fourth acquisition unit is configured to acquire at least one first state and at least one first parameter within a first preset period based on the hardware information of the storage unit, wherein the first state includes one or more of the storage unit state, battery state, and battery health state; and the first parameter includes one or more of the number of read / write failures, the number of predictable failures, and the number of other failures.

[0177] The first processing unit is configured to obtain a first score value when the first state is normal; or to obtain a second score value when the first state is abnormal.

[0178] The first determining unit is configured to determine the sum of the scores of at least one of the first states as the health score, provided that the growth rate of at least one of the first parameters does not exceed a third preset threshold.

[0179] Optionally, the device further includes:

[0180] The second determining module is used to determine the health score as the second value when at least one of the first parameters has a growth rate exceeding the third preset threshold within the first preset period.

[0181] Optionally, in the aforementioned apparatus, the storage unit is a first type of storage unit, and the first computing module 502 includes:

[0182] The second processing unit is used to classify the storage unit as a faulty unit when the health score is lower than the fourth preset threshold.

[0183] The fifth acquisition unit is used to acquire the total number of faulty units among the multiple storage units of the first type, and to acquire the average health score of the multiple storage units of the first type;

[0184] The sixth acquisition unit is used to acquire a risk coefficient based on the redundancy policy of the storage pool nodes and the total number of faulty units;

[0185] The seventh acquisition unit is used to acquire the average health score of the storage units of the first type in the storage pool node based on the average of the risk coefficient and the health score.

[0186] Optionally, in the aforementioned apparatus, the sixth acquiring unit comprises:

[0187] The fifth acquisition component is used to acquire the risk coefficient based on the total number of faulty units and the number of check blocks set in the erasure coding redundancy strategy when the redundancy strategy of the storage pool node is the erasure coding redundancy strategy.

[0188] The sixth acquisition component is used to acquire the risk coefficient based on the total number of faulty units, the location of the faulty units, and the number of replicas set in the multi-replica redundancy strategy when the redundancy strategy of the storage pool node is a multi-replica redundancy strategy.

[0189] Optionally, in the aforementioned apparatus, the storage unit is a second type of storage unit, and the first computing module 502 includes:

[0190] The eighth acquisition unit is used to average the health scores of multiple storage units of the second type and acquire the average health score of the storage units of the second type in the storage pool node.

[0191] Optionally, in the aforementioned apparatus, the storage unit is a third type of storage unit, and the first computing module 502 includes:

[0192] The ninth acquisition unit is used to average the health scores of multiple storage units of the third type to obtain the average health score of the storage units of the third type in the storage pool node.

[0193] Optionally, in the aforementioned apparatus, the second acquisition module 503 includes:

[0194] The tenth acquisition unit is used to add the product of the average health score of multiple storage units of different types and the corresponding weight value to obtain the health status score.

[0195] Optionally, in the aforementioned apparatus, the hardware information includes one or more of the following:

[0196] Self-monitoring, analysis, and reporting technology information;

[0197] Hard drive alarm information;

[0198] Operations and maintenance event logs.

[0199] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0200] To achieve the above objectives, embodiments of the present invention provide an electronic device, including: a transceiver, a processor, a memory, and a program or instructions stored in the memory and executable on the processor; wherein, when the processor executes the program or instructions, it implements the memory pool failure risk prediction method as described above.

[0201] To achieve the above objectives, embodiments of the present invention provide a readable storage medium having a program or instructions stored thereon, wherein the program or instructions, when executed by a processor, implement the steps in the storage pool failure risk prediction method as described above.

[0202] To achieve the above objectives, embodiments of the present invention provide a computer program product, comprising computer instructions that, when executed by a processor, implement the steps of the memory pool failure risk prediction method as described above.

[0203] It should be further noted that the terminals described in this specification include, but are not limited to, smartphones, tablets, etc., and many of the functional components described are referred to as modules in order to emphasize the independence of their implementation.

[0204] In this embodiment of the invention, the module can be implemented in software so that it can be executed by various types of processors. For example, an identified executable code module may include one or more physical or logical blocks of computer instructions, which may be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but may include different instructions stored in different locations, which, when logically combined, constitute the module and achieve the module's intended purpose.

[0205] In practice, an executable code module can be a single instruction or many instructions, and can even be distributed across multiple different code segments, different programs, and across multiple memory devices. Similarly, operational data can be identified within the module and can be implemented in any suitable form and organized within any suitable type of data structure. This operational data can be collected as a single dataset or distributed across different locations (including different storage devices), and can exist, at least in part, solely as electronic signals within the system or network.

[0206] When a module can be implemented using software, considering the current level of hardware technology, modules that can be implemented in software can be implemented using hardware circuits by those skilled in the art to achieve the corresponding functions, without considering cost. These hardware circuits include conventional very-large-scale integrated circuits (VLSI) or gate arrays, as well as existing semiconductors such as logic chips and transistors, or other discrete components. Modules can also be implemented using programmable hardware devices, such as field-programmable gate arrays, programmable array logic, and programmable logic devices.

[0207] The exemplary embodiments described above are with reference to the accompanying drawings. Many different forms and embodiments are feasible without departing from the spirit and teachings of the invention. Therefore, the invention should not be construed as limiting the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention complete and convey the scope of the invention to those skilled in the art. In these drawings, component dimensions and relative dimensions may be exaggerated for clarity. The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, unless clearly indicated otherwise, the singular forms “a,” “an,” and “the” are intended to include all such forms. It will be further understood that the terms “comprising” and / or “including”, when used in this specification, indicate the presence of the stated features, integers, steps, operations, components, and / or elements, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, and / or groups thereof. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of the range and any subranges in between.

[0208] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for predicting storage pool failure risk, characterized in that, include: Based on the hardware information of at least one storage unit included in the storage pool node, obtain the health score of each storage unit; Calculate the average health score of different types of storage units in the storage pool node based on the health score of each storage unit; The health status score of the storage pool node is obtained based on the average health score of multiple different types of storage units and the weight value corresponding to the respective types of storage units.

2. The method according to claim 1, characterized in that, The method further includes: If the health status score is lower than a first preset threshold, the storage pool node is set to read-only mode.

3. The method according to claim 1, characterized in that, Based on the hardware information of at least one storage unit included in the storage pool node, obtain the health score of each storage unit, including: At least one fault prediction indicator is extracted from the hardware information, and each fault prediction indicator is expanded according to the feature generation strategy to obtain an indicator set; wherein, the fault prediction indicator includes one or more of the following: number of reassigned sectors, read / write error rate, seek error rate, and number of rotational retries; the feature generation strategy includes one or more of the following: current value, critical value difference, statistical feature, time series change rate feature, and moving average feature; The indicator set is input into the trained prediction model to obtain the failure risk probability; wherein, the prediction model is used to predict the failure risk probability of the storage unit based on the indicator set using the gradient spanning tree algorithm. The health score is obtained based on the probability of failure risk.

4. The method according to claim 3, characterized in that, The health score is obtained based on the probability of failure risk, including: A first health score is obtained based on the aforementioned failure risk probability; If the first health score is lower than the second preset threshold, a first difference is obtained; wherein, the first difference is the first difference between the latency of multiple throughput collection points in the first curve and the latency in the standard curve; the standard curve is obtained by fitting the throughput and latency of a standard storage unit in historical data; the first curve is obtained by fitting the throughput and latency of the storage unit. The average value of the first difference corresponding to the plurality of throughput collection points is taken to obtain the error amount; Based on the error amount and the first health score, the health score corresponding to the storage unit is obtained.

5. The method according to claim 4, characterized in that, The method further includes: If the first health score is not lower than the second preset threshold, the first health score is determined as the health score.

6. The method according to claim 1, characterized in that, Based on the hardware information of at least one storage unit included in the storage pool node, obtain the health score of each storage unit, including: Based on the hardware information of the storage unit, at least one first state and at least one first parameter are obtained, representing the growth rate of the parameter within a first preset period. The first state includes one or more of the following: storage unit state, battery state, and battery health state. The first parameter includes one or more of the following: number of read / write failures, number of predictable failures, and number of other failures. If the first state is normal, the score is a first value; or if the first state is abnormal, the score is a second value. If the growth rate of at least one of the first parameters does not exceed a third preset threshold, the sum of the scores of at least one of the first states is determined as the health score.

7. The method according to claim 6, characterized in that, The method further includes: If at least one of the first parameters has a growth rate exceeding the third preset threshold within the first preset period, the health score is determined to be the second value.

8. The method according to claim 1, characterized in that, The storage unit is a first type of storage unit. Based on the health score of each storage unit, the average health score of the first type of storage units in the storage pool node is calculated, including: If the health score is lower than the fourth preset threshold, the storage unit is classified as a faulty unit. Obtain the total number of faulty units among the multiple storage units of the first type, and obtain the average of the health scores of the multiple storage units of the first type; The risk coefficient is obtained based on the redundancy strategy of the storage pool nodes and the total number of faulty units; The average health score of the storage units of the first type in the storage pool node is obtained based on the average of the risk coefficient and the health score.

9. The method according to claim 8, characterized in that, Based on the redundancy policy of the storage pool nodes and the total number of faulty units, a risk coefficient is obtained, including: When the redundancy strategy of the storage pool node is the erasure coding redundancy strategy, the risk coefficient is obtained based on the total number of faulty units and the number of check blocks set in the erasure coding redundancy strategy. When the redundancy strategy of the storage pool node is a multi-replica redundancy strategy, the risk coefficient is obtained based on the total number of faulty units, the location of the faulty units, and the number of replicas set in the multi-replica redundancy strategy.

10. The method according to claim 1, characterized in that, The storage unit is a second type of storage unit. Based on the health score of each storage unit, the average health score of the second type of storage units in the storage pool node is calculated, including: The average health score of the multiple storage units of the second type is taken to obtain the average health score of the storage units of the second type in the storage pool node.

11. The method according to claim 1, characterized in that, The storage unit is a third type of storage unit. Based on the health score of each storage unit, the average health score of the third type of storage units in the storage pool node is calculated, including: The average health score of multiple storage units of the third type is taken to obtain the average health score of the third type of storage units in the storage pool node.

12. The method according to claim 1, characterized in that, The health status score of the storage pool node is obtained based on the average health score of multiple different types of storage units and the weight value corresponding to the respective type of storage unit, including: The health status score is obtained by adding the products of the average health scores of multiple different types of storage units and their corresponding weight values.

13. The method according to claim 1, characterized in that, The hardware information includes one or more of the following: Self-monitoring, analysis, and reporting technology information; Hard drive alarm information; Operations and maintenance event logs.

14. A device for predicting storage pool failure risk, characterized in that, include: The first acquisition module is used to acquire the health score of each storage unit based on the hardware information of at least one storage unit included in the storage pool node. The first calculation module is used to calculate the average health score of different types of storage units in the storage pool node based on the health score of each storage unit; The second acquisition module is used to acquire the health status score of the storage pool node based on the average health score of multiple different types of storage units and the weight value corresponding to the respective types of storage units.

15. An electronic device comprising: A transceiver, a processor, a memory, and a program or instructions stored in the memory and executable on the processor; characterized in that, when the processor executes the program or instructions, it implements the method for predicting storage pool failure risk as described in any one of claims 1-13.

16. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps in the method for predicting storage pool failure risk as described in any one of claims 1-13.

17. A computer program product, characterized in that, Includes computer instructions, which, when executed by a processor, implement the steps of the method for predicting storage pool failure risk as described in any one of claims 1-13.

Images