Disk array fault detection method and device

Abstract

The invention relates to the field of fault detection, in particular to a disk array fault detection method and device. The method mainly comprises the steps of calculating a prior probability of a fault according to a failure rate of a circuit module; determining a target signal influencing a working state in the disk array, collecting an original sample of the target signal, obtaining a secondary sample under a fault condition in the original sample, and calculating an original conditional probability of the corresponding target signal according to the secondary sample; establishing a multi-factor model containing all target signals, and calculating a fault threshold value by using the multi-factor model based on the original conditional probability; and in the running process of the disk array, collecting alarm values of the target signals when the alarm signals are generated, forming alarm samples by the alarm values in a specified time period, calculating the posterior probability corresponding to each alarm sample, and judging the working state of the disk array in the corresponding time period according to the fault threshold and the posterior probability. According to the method, the false early warning rate can be reduced, and the RAID reliability and availability are improved.

Description

Technical Field

[0001] The present invention relates to the field of fault detection, and in particular to a method and device for detecting a disk array fault. Background Art

[0002] Redundant Array of Independent Disks (RAID) composed of independent disks with redundancy capabilities, usually referred to as "disk array", has the characteristics of huge storage capacity, excellent read and write speed performance, data protection and repair, and "dual control" mechanism. RAID usually plays an important role in data storage and forwarding, and needs to have high availability. Therefore, various means are needed to improve the reliability and stability of RAID to ensure the normal operation of the RAID system.

[0003] In the prior art, RAID device fault detection is usually performed using a built-in test (BIT). BIT is a common fault detection method for electronic devices, which is generally achieved by adding a BIT circuit to the hardware circuit. The BIT circuit is an embedded microcontroller, which is mainly used to monitor the signals and status of electronic equipment. By monitoring the temperature, voltage, fan speed and other parameters of the hardware circuit, it ensures the stable operation of the device and provides remote management and control functions.

[0004] In the actual detection process, the monitoring signal of the BIT circuit should be set with a suitable threshold value. When the actual observation value of a certain observation signal exceeds the threshold value, the alarm mechanism of the BIT circuit is triggered. However, this alarm mechanism has the problem of false alarms, which is the so-called "false alarm rate". The "false alarm rate" is mainly caused by the following reasons: ① Sensor failure can lead to false alarms of temperature, voltage or fan speed; ② Electromagnetic interference in the surrounding environment may affect the transmission and accuracy of the signal, resulting in false triggering; ③ BIT software may have loopholes or errors, resulting in misjudgment of signals; ④ If the BIT itself or related hardware components fail, it may also cause false triggering of signals; ⑤ Improper setting of the monitoring threshold parameters of the BIT may lead to false triggering; ⑥ There may be problems in the communication between the BIT and the monitored device, resulting in signal loss or misjudgment. In the prior art, it is impossible to effectively distinguish between real alarms and false alarms. Therefore, when an alarm occurs, the RAID must be shut down for maintenance, resulting in suspension of data services and affecting the execution of data-related services.

[0005] In view of this, how to overcome the defects of the prior art and improve the accuracy of the existing RAID fault detection is a problem to be solved in the technical field. Summary of the invention

[0006] In view of the above defects or improvement needs of the prior art, the present invention effectively improves the accuracy of RAID fault detection by scientifically setting the fault threshold and solves the problem of excessively high false alarm rate of the equipment.

[0007] The embodiment of the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for detecting disk array faults, specifically: obtaining the failure rate of each circuit module in the disk array, and calculating the prior probability of the fault based on the failure rate of the circuit module; determining the target signal that affects the working state in the disk array, collecting the original samples of the target signal, obtaining the secondary samples under the fault condition from the original samples, and calculating the original conditional probability of the corresponding target signal based on the secondary samples; establishing a multi-factor model including all target signals, and calculating the fault threshold based on the original conditional probability using the multi-factor model; during the operation of the disk array, collecting the alarm values ​​of each target signal when an alarm signal is generated, forming an alarm sample with the alarm values ​​within a specified time period, calculating the posterior probability corresponding to each alarm sample, and determining the working state of the disk array within the corresponding time period based on the fault threshold and the posterior probability.

[0009] Preferably, the method of obtaining the failure rate of each circuit module in the disk array and calculating the prior probability of failure based on the failure rate of the circuit module specifically includes: decomposing the circuit modules in the disk array into electronic components and counting the types and quantities of electronic components in each circuit module; for each circuit module, obtaining the failure rate of each type of electronic components based on the circuit characteristics of the type of electronic components, and calculating the failure rate of the circuit module based on the failure rate of each type of electronic components and the number of the type of electronic components; and taking the sum of the failure rates of all circuit modules as the total failure rate of the disk array, and taking the total failure rate as the prior probability of failure.

[0010] Preferably, the calculating the failure rate of the circuit module according to the failure rate of each type of electronic components and the number of the electronic components of this type specifically includes calculating the failure rate of each circuit module in the following manner:

[0011]

[0012] Among them, λ GS is the failure rate of the circuit module, N i is the number of electronic components of the i-th category, n is the total number of electronic component types, λ Gi is the failure rate of the i-th type of electronic components, π Qi is the universal quality factor of the i-th type of electronic components.

[0013] Preferably, the secondary samples under fault conditions are obtained from the original samples, and the original conditional probability of the corresponding target signal is calculated based on the secondary samples, specifically including: taking the disk array fault as a condition, screening the secondary samples from the original samples, and calculating the statistical parameters of the secondary samples; obtaining the abnormal working range of the target signal, calculating the probability density of the target signal in the abnormal working range according to the statistical parameters, and obtaining the original conditional probability under the fault condition according to the probability density.

[0014] Preferably, the statistical parameters include a mean and a variance, and the method of calculating the probability density of the target signal in the abnormal working range according to the statistical parameters specifically includes calculating the probability density in the following manner:

[0015]

[0016] Where P(x∈T) is the probability density of the value of x within the abnormal working interval T, a and b are the upper and lower limits of the abnormal working interval T respectively; μ is the mean of the quadratic sample, σ is the variance of the quadratic sample; x is the independent variable of the probability density function f(x∈T; μ,σ).

[0017] Preferably, when the target signal has multiple abnormal working intervals, the method of calculating the probability density of the target signal in the abnormal working interval based on statistical parameters also includes: calculating the probability density of the target signal in each abnormal working interval, and taking the sum of the probability densities in all abnormal working intervals as the probability density of the target signal in the abnormal working interval.

[0018] Preferably, the method of calculating the fault threshold using a multi-factor model based on the original conditional probability specifically includes: taking each target signal located in an abnormal working range as a factor of the fault occurrence, substituting the original conditional probability corresponding to each factor into the Bayesian model, calculating the posterior conditional probability including all factors, and taking the posterior conditional probability as the fault threshold;

[0019] Among them, the posterior conditional probability of all factors is calculated according to the following formula:

[0020]

[0021] Among them, P(A|B) is the posterior conditional probability including all factors, N is the total number of target signals, event A indicates that the disk array fails, event Bi indicates that the i-th target signal is in the abnormal working range, P(B i |A) represents the original conditional probability that the i-th target signal is in the abnormal working interval when the disk array fails, P(B i|~A) represents the original conditional probability that the i-th target signal is located in the abnormal working interval when the disk array has not failed, P(A) represents the original conditional probability that the disk array has failed, and P(~A) represents the original conditional probability that the disk array has not failed.

[0022] Preferably, the calculation of the posterior probability corresponding to each alarm sample specifically includes: obtaining the abnormal working interval corresponding to the alarm signal, calculating the real-time conditional probability of the alarm sample in the abnormal working interval, and calculating the posterior probability of the disk array failure when the alarm signal is generated based on the real-time conditional probability.

[0023] In a second aspect, the present invention provides a disk array fault detection device, specifically comprising: at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the disk array fault detection method in the first aspect.

[0024] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method provided in the first aspect.

[0025] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method provided in the first aspect.

[0026] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes the method provided in the first aspect.

[0027] Compared with the prior art, the beneficial effects of the present invention are: monitoring the target signals related to the faults on the RAID hardware circuit, collecting and establishing data samples of various target signals in the initial stage, obtaining the prior probability and original conditional probability of the device, and then calculating the fault threshold. In the equipment operation monitoring stage, statistical analysis of the changes in the target signals is performed again to obtain the posterior probability of RAID device failure. For the faults exceeding the threshold, the hidden dangers are eliminated by analyzing the causes and repairing them. This proactive preventive maintenance method avoids false warnings and improves the reliability and availability of RAID. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 A flow chart of a method for detecting a disk array failure provided by an embodiment of the present invention;

[0030] Figure 2 A flow chart of another method for detecting disk array failures provided by an embodiment of the present invention;

[0031] Figure 3 A flow chart of another method for detecting disk array failures provided by an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of controller module failure rate data in a certain actual scenario of an embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of the statistical analysis results of target signals in a certain actual scenario of an embodiment of the present invention;

[0034] Figure 6 This is a schematic diagram of the calculation result of the original conditional probability in a certain actual scenario of an embodiment of the present invention;

[0035] Figure 7 A schematic diagram of the structure of a disk array fault detection device provided by an embodiment of the present invention;

[0036] The reference numerals are as follows:

[0037] 11: processor; 12: memory. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0040] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0041] Embodiment 1:

[0042] In order to effectively reduce the RAID downtime caused by the "false alarm rate" of the BIT circuit, this embodiment uses BIT to collect the status information of the target signal, and updates the estimated probability of RAID device failure based on the collected information. Once this probability is greater than the set threshold, preventive maintenance work is started to eliminate potential fault hazards.

[0043] like Figure 1 As shown, the method for detecting disk array failure provided by the embodiment of the present invention has the following specific steps:

[0044] Step 101: Obtain the failure rate of each circuit module in the disk array, and calculate the prior probability of failure based on the failure rates of all circuit modules.

[0045] In this embodiment, the original failure probability of RAID in general use scenarios is used as the prior probability of RAID failure. In specific implementation, the original failure probability can be estimated by predicting the failure rate of electronic components. Since there are many electronic components in RAID, in order to facilitate later data processing, the electronic equipment can be decomposed into various circuit modules, and the types and quantities of electronic components in each circuit module are counted. The failure rate of the circuit module is calculated according to the different circuit characteristics of different electronic components. The total failure rate of RAID is obtained by adding the failure rates of various circuit modules, that is, the prior probability of RAID failure.

[0046] Step 102: determine the target signal that affects the working state in the disk array, collect the original samples of the target signal, obtain the secondary samples under the fault condition from the original samples, and calculate the original conditional probability of the corresponding target signal according to the secondary samples.

[0047] By analyzing the device operation mode, hardware parameter performance and known experience, all target signals that affect the RAID operation are determined, the abnormal operation range of the target signal is determined, the original sample of the target signal is collected through the BIT circuit, and the secondary sample is screened out based on the condition that the RAID operation state is abnormal or fails, and the statistical parameters such as the mean and variance of the secondary sample are calculated. The probability density function in the abnormal operation range is obtained to obtain the conditional probability of the target signal. In this embodiment, the conditional probability calculated by the original sample is used as the original conditional probability of the RAID operation state being abnormal or failing.

[0048] Step 103: Establish a multi-factor model including all target signals, and use the multi-factor model to calculate the fault threshold based on the original conditional probability.

[0049] After obtaining the original conditional probabilities of all target signals, a multi-factor model is constructed based on the original conditional probabilities and other associated parameters, and the posterior probability of RAID failure under the joint conditions is calculated through the multi-factor model, and the posterior probability is used as the failure threshold. In actual implementation, the multi-factor model can use a multi-factor Bayesian model or other models that can calculate the posterior probability.

[0050] Step 104: During the operation of the disk array, the alarm values ​​of each target signal when an alarm signal is generated are collected, an alarm sample is generated according to the alarm value and the original sample, the posterior probability corresponding to each alarm sample is calculated, and the working state of the disk array is determined according to the fault threshold and the posterior probability.

[0051] After obtaining the fault threshold, actual fault detection can be performed during the actual operation of the RAID. Monitor the status values ​​of each target signal in real time, and continuously collect the alarm values ​​when the target signal is in the abnormal range. In a fixed time period, the collected alarm values ​​are combined into alarm samples, and the real-time conditional probability of the target signal is calculated based on the alarm samples. Furthermore, it is also necessary to calculate the current posterior probability of the target signal based on the real-time conditional probability, and then compare it with the fault threshold. If it is greater than the fault threshold, it can be considered that there is a major fault hidden danger, and it is necessary to shut down for maintenance immediately, determine the fault point, replace or repair the module, and eliminate the hidden danger of failure.

[0052] After steps 101 to 104 provided in this embodiment, fault detection can be performed on the RAID based on the fault thresholds of multiple factors to reduce the false alarm rate.

[0053] In actual implementation, the prior probability P(A) of failure of electronic equipment such as disk arrays can generally be obtained from the theoretical failure rate of electronic components. Figure 2 As shown, the failure rate of each circuit module in the disk array can be obtained in the following way, and the prior probability of failure can be calculated based on the failure rates of all circuit modules.

[0054] Step 201: Decompose the circuit modules in the disk array into electronic components, and count the types and quantities of the electronic components in each circuit module.

[0055] In actual RAID device hardware, electronic components usually exist in the form of circuit modules. The coupling of electronic components in each circuit module is high, and the coupling between circuit modules is low. If any electronic component in each circuit module fails, it will cause the failure of the circuit module in which it is located, but it will not necessarily cause the failure of other circuit modules. Therefore, when estimating the failure probability, the failure probability of a certain electronic component is highly correlated with the failure probability of the circuit module in which it is located, and is less correlated with the failure probability of other circuit modules. On the other hand, since the characteristics of each type of electronic component are relatively similar, the failure rate of each type of electronic component in the same working scenario is also similar. In actual implementation, in order to simplify the calculation of the failure probability, the failure probability can be calculated in units of circuit modules, the equipment hardware and electronic component information in each circuit module is collected, and the types and quantities of electronic components in each circuit module are counted, so as to facilitate the subsequent calculation of the failure rate of the circuit module based on the failure rate of each type of electronic component, and then calculate the prior probability of RAID failure.

[0056] Step 202: For each circuit module, the failure rate of each type of electronic component is obtained according to the circuit characteristics of the electronic component, and the failure rate of the circuit module is calculated according to the failure rate of each type of electronic component and the number of the electronic components of the type.

[0057] In this embodiment, a component counting method is used to estimate the failure rate of each circuit module in the RAID according to different circuit characteristics of different electronic components.

[0058] In the specific implementation, the failure rate of each circuit module is calculated using the following method:

[0059]

[0060] Among them, λ GS is the failure rate of the circuit module, N i is the number of electronic components of the i-th category, n is the total number of electronic component types, λ Gi is the failure rate of the i-th type of electronic components, π Qi is the universal quality factor of the i-th type of electronic components.

[0061] Step 203: The sum of the failure rates of all circuit modules is taken as the total failure rate of the disk array, and the total failure rate is taken as the priori probability of the fault.

[0062] The total failure rate of the RAID is obtained by adding up the failure rates of each circuit module. The total failure rate is the prior probability P(A) of a failure when the working environment of the RAID does not change, and the probability P(~A) of a normal state.

[0063] After steps 201 to 203 provided in this embodiment, the prior probability of RAID failure can be obtained.

[0064] In the actual working scenario of RAID, the actual working environment of each RAID may not be the same as the working environment during the electronic component testing. Therefore, it is also necessary to update the RAID failure through the failure probability data obtained through actual testing. After obtaining the prior probability of failure through the theoretical failure rate of electronic components, it is also necessary to debug, test and copy each RAID device to obtain the original conditional probability of RAID failure in the corresponding working environment. In actual implementation, such as Figure 3 , the secondary samples under fault conditions can be obtained from the original samples in the following way, and the original conditional probability of the corresponding target signal can be calculated based on the secondary samples.

[0065] Step 301: Taking disk array failure as a condition, secondary samples are obtained by screening the original samples, and statistical parameters of the secondary samples are calculated.

[0066] Conditional probability means: when event A occurs, the probability of the occurrence of event B associated with it. In this embodiment, the failure of RAID is regarded as event A, and the abnormal state of the target signal is regarded as event B. In order to obtain the original conditional probability, it is necessary to collect the real-time status data of the target signal during the debugging, testing and copying stages, and establish the original sample based on the data of the target signal. After obtaining the original sample, the original sample is filtered with the failure (i.e., the occurrence of event A) as the condition, and the original sample when the failure occurs is filtered out as the secondary sample. In order to perform probability calculation, it is also necessary to perform statistical analysis on the data of the secondary sample to obtain the corresponding statistical parameters. In this embodiment, the main statistical parameters used include the mean and variance of the secondary sample.

[0067] Step 302: Obtain an abnormal working range of a target signal, calculate a probability density of the target signal in the abnormal working range according to statistical parameters, and obtain an original conditional probability under a fault condition according to the probability density.

[0068] According to the statistical parameters of the secondary samples, the probability distribution of the data in the secondary samples is analyzed. According to the probability distribution characteristics, the probability density function is used to calculate the original conditional probability P(B|A) that the target signal is in a specific state (that is, event B occurs) when the equipment fails (that is, event A occurs), and the original conditional probability P(B|~A) that this feature occurs when the equipment is normal (that is, event A does not occur).

[0069] In a specific implementation, the probability density is calculated using the following method:

[0070]

[0071] Where P(x∈T) is the probability density of the value of x within the abnormal working interval T, a and b are the upper and lower limits of the abnormal working interval T respectively; μ is the mean of the quadratic sample, σ is the variance of the quadratic sample; x is the independent variable of the probability density function f(x∈T; μ,σ).

[0072] When the target signal has only one abnormal working interval, the probability density in the abnormal working interval is the original conditional probability. When the target signal has multiple abnormal working intervals, it is necessary to comprehensively calculate the probability densities of the multiple abnormal working intervals, and the sum of the probability densities in all abnormal working intervals is used as the original conditional density. Specifically, the probability density of the target signal in each abnormal working interval is calculated, and the sum of the probability densities in all abnormal working intervals is used as the probability density of the target signal in the abnormal working interval. In actual implementation, if the normal working range of the voltage of an electronic component is 3.0~3.6V, there are two abnormal working intervals of the voltage: abnormal working interval T1 range (-∞,3.0), lower limit a=-∞, upper limit b=3.0; abnormal working interval T2 range (3.6, +∞), lower limit a=3.6, upper limit b=+∞; then the probability density of the voltage x in the abnormal working interval in this scenario is: P(x∈T1)+P(x∈T2)=P(x∈(-∞,3.0))+P(x∈(3.6,+∞)) 。

[0073] After step 301 and step 302 provided in this embodiment, the original conditional probability of RAID failure can be obtained.

[0074] When performing fault detection, the fault threshold is a key factor. If the fault threshold is too low, the posterior probability of the device will frequently exceed the fault threshold, and the device will be shut down for maintenance, affecting the efficiency of the device; if the fault threshold is too high, the posterior probability of the device will hardly exceed the threshold, which will cover up the real hidden dangers and increase the possibility of failure. Electronic equipment failures are usually caused by the combined effect of multiple factors. When making fault judgments, it is necessary to consider multiple key factors such as circuits, voltages, and temperatures, rather than a single factor, in order to more comprehensively and accurately predict the possibility of disk array failures.

[0075] In this embodiment, during the debugging phase, target signal data related to faults, such as current, voltage, and temperature, of key components are collected. These data contain information about the disk array in various states, and can provide abundant original samples for equipment fault detection. The collected original samples are combined with empirical information to better construct a multi-factor Bayesian model. The posterior probability of considering multiple target signals is calculated based on the multi-factor Bayesian model, and the posterior probability is used as the fault threshold. Compared with the prior art that only uses the abnormal interval of a single target signal as the fault threshold, it can better fit the actual situation when the disk array is operating normally.

[0076] The target signal data collected in real time when the disk array is actually running can reflect the current operating status of the device. In this embodiment, the target signal alarm values ​​collected in real time within a time period are combined into alarm samples, and the posterior probability of the device failing in the current operating state is calculated using the alarm samples, which can dynamically track the state changes of the device and timely discover potential failure risks.

[0077] By comparing the real-time a posteriori probability under the current operating state and the fault threshold expressed in the form of probability, the possibility of fault is judged by the probability. If the probability is greater than the threshold, it means that the actual status of the surface equipment deviates from the normal operating state, that is, the possibility of failure is high and corresponding countermeasures need to be taken.

[0078] In this embodiment, each target signal being located in an abnormal working range is regarded as a factor of fault occurrence, the original conditional probability corresponding to each factor is substituted into the Bayesian model, the posterior conditional probability including all factors is calculated, and the posterior conditional probability is used as the fault threshold.

[0079] In specific implementation, the posterior conditional probability of all factors can be calculated according to the following formula:

[0080]

[0081] Where P(A|B) is the posterior conditional probability including all factors, N is the total number of target signals, event A indicates that the disk array fails, event Bi indicates that the i-th target signal is in the abnormal working range, for example: in a specific scenario, the target signals are the voltage of the CPU chip, the current of the FPGA chip and the temperature of the power chip, B1 indicates that the voltage of the first target signal CPU chip is in the abnormal range; B2 indicates that the current of the second target signal FPGA chip is in the abnormal range; B3 indicates that the temperature of the third target signal power chip is in the abnormal range, etc.; P(B i |A) represents the original conditional probability that the i-th target signal is in the abnormal working interval when the disk array fails, P(B i |~A) represents the original conditional probability that the i-th target signal is located in the abnormal working interval when the disk array has not failed, P(A) represents the original conditional probability that the disk array has failed, and P(~A) represents the original conditional probability that the disk array has not failed.

[0082] After the fault threshold is obtained, the fault detection of RAID can be performed according to the fault threshold. In daily use, when the value of the target signal is in the abnormal interval, the BIT circuit will alarm. In the prior art, each alarm will be regarded as a fault, resulting in a false alarm rate; in this embodiment, a single alarm is not directly regarded as a fault, but the posterior probability of the occurrence of the fault corresponding to a certain target signal is calculated based on all the alarm values ​​of the target signal within a time period. If the posterior probability is greater than the fault threshold, it indicates that the fault corresponding to the target signal may occur. Specifically, the BIT circuit monitors the state of the target signal in real time, obtains the alarm value when the target signal is in the abnormal interval, and forms an alarm sample with the obtained alarm value within a specified time period; the real-time conditional probability of the target signal is calculated based on the alarm sample of each target signal, and the current posterior probability of the target signal is calculated based on the real-time conditional probability.

[0083] Then compare the posterior probability with the fault threshold. If it is greater than the fault threshold, it can be considered that there is a major fault risk, and it is necessary to immediately shut down for maintenance, determine the fault point, replace or repair the module, and eliminate the hidden danger of failure. In actual implementation, as the RAID runs for a long time, the amount of alarm sample data will become richer and richer, and the accuracy of the posterior probability and even the credibility of fault detection will gradually improve.

[0084] In specific implementation, the accurate selection of target signals that play a key role in the normal operation of the disk array is a key factor in determining the threshold. In addition, the accuracy of various data, such as the determination of fault thresholds and the calculation of posterior probabilities, requires the help of a rich sample size. Generally speaking, the larger the number of samples, the closer the fault detection results are to reality. In addition, new data and actual verification results can be used to continuously optimize and adjust relevant parameters to improve the accuracy of fault detection. In practical applications, the judgment of the equipment fault state can be dynamically updated based on the characteristics of the target signals of the constantly emerging new states, combined with expert experience and knowledge of similar equipment, to continuously improve the accuracy of fault detection.

[0085] The disk array fault detection method provided in this embodiment monitors the target signal that plays a key role in the fault on the RAID hardware circuit, collects and establishes data samples of various target signals in the initial stage, obtains the prior probability and original conditional probability of the device, and then calculates the fault threshold. In the equipment operation monitoring stage, the changes in the target signal are statistically analyzed again to obtain the posterior probability of RAID device failure. For faults exceeding the threshold, the hidden dangers are eliminated by analyzing the cause and repairing. This proactive preventive maintenance method avoids false warnings and improves the reliability and availability of RAID.

[0086] Embodiment 2:

[0087] Based on the disk array fault detection method provided in Example 1, in some specific implementations, it can be implemented through the specific implementation of this embodiment. It can be understood that the specific implementation provided in this embodiment is only used to illustrate the specific implementation process of the method in Example 1 in some specific scenarios, and is not intended to limit the scope of protection.

[0088] In a real scenario, RAID mainly includes the following circuit modules: storage module, controller module, storage module, interface module, cache module, power module, management module and chassis. Among them, the failure rate of the controller module is as follows: Figure 4 shown.

[0089] The main electronic component in the controller module is the CPU, and the working environment is a normal temperature environment. The failure rate of the controller module is: CPU =ΣN(λ G π Q )=5.567452×10-6 / h.

[0090] The working environment of the storage module is a normal temperature environment, and the failure rate of the storage module is: SRM =ΣN(λ G π Q )=6.953788×10-6 / h.

[0091] Use the same method to collect information and calculate the failure rate of all circuit modules:

[0092] The failure rate of the interface module is 1.498584×10-6 / h;

[0093] The cache module failure rate is 1.8955×10-6 / h;

[0094] The failure rate of the power module is 1.100066×10-6 / h;

[0095] The failure rate of the management module is 1.743288×10-6 / h;

[0096] The chassis failure rate is 1.200158×10-6 / h.

[0097] Based on the above data, the overall expected failure rate of RAID devices is:

[0098] λ S =λ CPU +λ SRM +λ IFM +λ CCM +λ PWM +λ MGM +λ ETC =19.96×10-6 / h;

[0099] The prior probability of a RAID device is 19.96×10-6 / h≈0.00002.

[0100] That is, P(A)=0.00002, then P(~A)=0.99998.

[0101] By using the BIT monitoring circuit, real-time data of various RAID status signals can be collected. The following takes the core voltage value of the CPU chip in a certain model of RAID device as an example to calculate its original conditional probability.

[0102] In this embodiment, the core operating voltage is used as the target signal. According to the relevant chip manual, the core operating voltage of the CPU chip is 3.3v, and the normal working voltage range is 3.0v~3.6v. Event A indicates that the RAID fails, and event B1 indicates that the core operating voltage of the CPU chip is in an abnormal working range (that is, greater than 3.6v, or less than 3.0v). The original samples of the core operating voltage of the CPU chip during the debugging, testing and copying stages of the equipment are sorted. Taking the RAID working abnormality as the condition, 500 groups of data are selected as target samples, and statistical analysis is performed on the target samples. The analysis results are as follows: Figure 5 As shown, the mean μ of the sample data of the core operating voltage of the CPU chip is 3.3183V, and the variance σ is 1.5.

[0103] When a RAID failure occurs, the conditional probability P(B1|A) that the core operating voltage of the CPU chip is in the abnormal operating range can be achieved by integrating the function within the range.

[0104] To calculate the probability of the abnormal working interval x<p, it is necessary to integrate the probability density function f(x:μ,σ) from -∞ to p, that is:

[0105]

[0106] To calculate the probability in the interval x>q, it is necessary to integrate the probability density function f(x:μ,σ) from q to +∞, that is:

[0107]

[0108] When p=3, q=3.6, the mean and variance obtained by statistical analysis are substituted into the above formula for calculation, and P(x<3)=0.3109, P(x>3.6)=0.3306 are obtained. Then, when RAID fails, the original conditional probability that the core operating voltage of the CPU chip is in the abnormal operating range is P(B1|A)=P(x<3)+P(x>3.6)=0.6415.

[0109] It can be seen that when the RAID fails, the original conditional probability that the CPU chip core operating voltage is in the normal range (3.0v~3.6v) is P(B1|~A)=1-0.6415=0.3585

[0110] In this scenario, there are five factors affecting the probability of failure, which are the original conditional probabilities of B1 to B5, where B2 indicates that the CPU chip ambient temperature is in an abnormal range, B3 indicates that the FPGA operating voltage is in an abnormal range, B4 indicates that the FPGA chip ambient temperature is in an abnormal range, and B5 indicates that the fan speed is in an abnormal range. Using the above method of calculating conditional probabilities, the original conditional probabilities of the five influencing factors B1 to B5 are obtained as follows: Figure 6 shown.

[0111] Substitute the original conditional probability of each factor into the formula of the Bayesian model: P(A|B)=P(B|A)*P(A) / P(B).

[0112] It is known that P(A)=0.00002, so P(~A)=0.99998.

[0113] For multiple factors B1 to B5, the expression of the numerator is:

[0114] P(B|A)×P(A)=P(B1|A)×P(A)+P(B2|A)×P(A)+P(B3|A)×P(A)+P(B4|A)×P(A)+P(B5|A)×P(A)=47.36×10 -6 .

[0115] For multiple factors B1 to B5:

[0116] P(B1)=P(B1|A)×P(A)+P(B1|~A)×P(~A)=0.6415×0.00002+0.3585×0.99998=0.3585;

[0117] P(B2)=P(B2|A)×P(A)+P(B2|~A)×P(~A)=0.3824×0.00002+0.6176×0.99998=0.6176;

[0118] P(B3)=P(B3|A)×P(A)+P(B3|~A)×P(~A)=0.6195×0.00002+0.3805×0.99998;=0.3805

[0119] P(B4)=P(B4|A)×P(A)+P(B4|~A)×P(~A)=0.4662×0.00002+0.5338×0.99998=0.5338;

[0120] P(B5)=P(B5|A)*P(A)+P(B5|~A)*P(~A)=0.2583*0.00002+0.7417*0.99998=0.7417;

[0121] The expression of the denominator is: P(B)=P(B1)*P(B2)*P(B3)*P(B4)*P(B5)=0.0334;

[0122] Then P(A|B)=P(B|A)*P(A) / P(B)=0.001420=1420×10 -6 , which is the failure threshold of the RAID in this example.

[0123] In this example, during the operation of the RAID, the following target signals are monitored in real time through the BIT circuit: the CPU operating voltage and chip temperature, FPGA operating voltage, chip temperature and the number of fan revolutions inside the disk array chassis, and the alarm values ​​when the target signals are in the abnormal range are continuously collected. Within a fixed time period, the collected alarm values ​​are combined into alarm samples, and the real-time posterior probabilities of the above parameters are calculated respectively.

[0124] Through calculation, it can be obtained that within this time period, the real-time conditional probability of the CPU chip voltage failure is P(B|A)=0.9872, and the real-time conditional probability of the CPU chip voltage being normal is P(B|~A)=0.0128. Combined with the RAID prior probability P(A)=0.00002 and P(~A)=0.99998, the calculation formula of the Bayesian model is P(A|B)=P(B|A)*P(A) / [P(B|A)*P(A)+P(B|~A)*P(~A)]. After calculating that the CPU chip core working voltage is abnormal, the failure probability of RAID is 1540×10 -6 , which is greater than the failure threshold of 1420×10 -6 , it can be considered that the RAID needs to be shut down for maintenance immediately. Furthermore, according to the alarm value of the target signal, it can be analyzed that the CPU peripheral power supply chip is damaged, resulting in the CPU chip core power supply voltage being too high. After replacing the peripheral power supply chip, the fault is eliminated.

[0125] Embodiment 3:

[0126] Based on the disk array fault detection method provided in the above-mentioned embodiments 1 and 2, the present invention also provides a disk array fault detection device that can be used to implement the above-mentioned method, such as Figure 7 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The disk array fault detection equipment of this embodiment includes one or more processors 11 and a memory 12. Figure 7 A processor 11 is taken as an example.

[0127] The processor 11 and the memory 12 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0128] The memory 12 is a non-volatile computer-readable storage medium for a disk array fault detection method, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the disk array fault detection method in Embodiments 1 to 2. The processor 11 executes various functional applications and data processing of the disk array fault detection equipment by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, the disk array fault detection method in Embodiments 1 to 2 is implemented.

[0129] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11, and these remote memories may be connected to the processor 11 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] The program instructions / modules are stored in the memory 12. When executed by one or more processors 11, the disk array fault detection method in the above-mentioned embodiments 1 to 2 is executed. For example, the above-described Figures 1 to 3 The steps shown.

[0131] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Figure 7 A processor 11 in the embodiment 1 can enable the one or more processors to execute the disk array fault detection method in the embodiment 1 to the embodiment 2, for example, to execute the above described Figures 1 to 3 The steps shown.

[0132] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0133] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0134] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting a disk array failure, characterized in that: include: Obtain the failure rate of each circuit module in the disk array, and calculate the prior probability of failure based on the failure rate of the circuit module; Determine the target signal that affects the working state in the disk array, collect the original samples of the target signal, obtain the secondary samples under the fault condition from the original samples, and calculate the original conditional probability of the corresponding target signal based on the secondary samples; A multi-factor model including all target signals is established, and the fault threshold is calculated using the multi-factor model based on the original conditional probability; During the operation of the disk array, the alarm values ​​of each target signal when an alarm signal is generated are collected, the alarm values ​​within a specified time period are combined into an alarm sample, the posterior probability corresponding to each alarm sample is calculated, and the working status of the disk array within the corresponding time period is determined based on the fault threshold and the posterior probability.

2. The method for detecting disk array failure according to claim 1, characterized in that: The step of obtaining the failure rate of each circuit module in the disk array and calculating the prior probability of the fault according to the failure rate of the circuit module specifically includes: Decompose the circuit modules in the disk array into electronic components, and count the types and quantities of electronic components in each circuit module; For each circuit module, the failure rate of each type of electronic component is obtained according to the circuit characteristics of the electronic component, and the failure rate of the circuit module is calculated according to the failure rate of each type of electronic component and the number of the electronic components of the type; The sum of the failure rates of all circuit modules is taken as the total failure rate of the disk array, and the total failure rate is taken as the prior probability of failure.

3. The method for detecting disk array failure according to claim 2, characterized in that: The calculating the failure rate of the circuit module according to the failure rate of each type of electronic components and the number of the electronic components of the type specifically includes calculating the failure rate of each circuit module in the following manner: Among them, λ GS is the failure rate of the circuit module, N i is the number of electronic components of the i-th category, n is the total number of electronic component types, λ Gi is the failure rate of the i-th type of electronic components, π Qi is the universal quality factor of the i-th type of electronic components.

4. The method for detecting disk array failure according to claim 1, characterized in that: The step of obtaining secondary samples under fault conditions from the original samples and calculating the original conditional probability of the corresponding target signal according to the secondary samples specifically includes: Taking disk array failure as a condition, secondary samples are screened from the original samples, and the statistical parameters of the secondary samples are calculated; The abnormal working range of the target signal is obtained, the probability density of the target signal in the abnormal working range is calculated according to the statistical parameters, and the original conditional probability under the fault condition is obtained according to the probability density.

5. The method for detecting disk array failure according to claim 4, characterized in that: The statistical parameters include a mean and a variance, and the probability density of the target signal in the abnormal working range is calculated according to the statistical parameters, specifically including calculating the probability density using the following method: Where P(x∈T) is the probability density of the value of x within the abnormal working interval T, a and b are the upper and lower limits of the abnormal working interval T respectively; μ is the mean of the quadratic sample, σ is the variance of the quadratic sample; x is the independent variable of the probability density function f(x∈T; μ,σ).

6. The method for detecting disk array failure according to claim 4, characterized in that: When the target signal has multiple abnormal working intervals, the calculating the probability density of the target signal in the abnormal working interval according to the statistical parameters also includes: The probability density of the target signal in each abnormal working interval is calculated, and the sum of the probability densities in all abnormal working intervals is taken as the probability density of the target signal in the abnormal working interval.

7. The method for detecting disk array failure according to claim 1, characterized in that: The method of calculating the fault threshold using a multi-factor model based on the original conditional probability specifically includes: The abnormal working range of each target signal is regarded as a factor of fault occurrence. The original conditional probability corresponding to each factor is substituted into the Bayesian model to calculate the posterior conditional probability including all factors, and the posterior conditional probability is used as the fault threshold. Among them, the posterior conditional probability of all factors is calculated according to the following formula: Among them, P(A|B) is the posterior conditional probability including all factors, N is the total number of target signals, event A indicates that the disk array fails, event Bi indicates that the i-th target signal is in the abnormal working range, P(B i |A) represents the original conditional probability that the i-th target signal is in the abnormal working interval when the disk array fails, P(B i |~A) represents the original conditional probability that the i-th target signal is located in the abnormal working interval when the disk array has not failed, P(A) represents the original conditional probability that the disk array has failed, and P(~A) represents the original conditional probability that the disk array has not failed.

8. The method for detecting disk array failure according to claim 1, characterized in that: The alarm values ​​within the specified time period are combined into alarm samples, and the posterior probability corresponding to each alarm sample is calculated, specifically including: Monitor the state of the target signal in real time, obtain the alarm value when the target signal is in the abnormal range, and form the obtained alarm values ​​into alarm samples within a specified time period; The real-time conditional probability of each target signal is calculated according to the alarm sample of the target signal, and the current posterior probability of the target signal is calculated according to the real-time conditional probability.

9. A device for detecting disk array failure, characterized in that: The method comprises at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions executable by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for disk array fault detection according to any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the method for detecting disk array failures as described in any one of claims 1 to 8.

Images