A method and apparatus for disk array failure detection

Abstract

The present application relates to the field of fault detection, in particular to a kind of disk array fault detection method and device.Mainly include: according to the failure rate of circuit module, the prior probability of failure is calculated;Determine the target signal in disk array that affects working condition, collect the original sample of target signal, get the secondary sample under the condition of failure in original sample, calculate the original conditional probability of corresponding target signal according to secondary sample;Establish multi-factor model containing all target signals, calculate failure threshold using multi-factor model based on original conditional probability;In the running process of disk array, collect the alarm value of each target signal when generating alarm signal, form alarm sample by alarm value in specified time period, calculate the posterior probability corresponding to each alarm sample, determine the working condition of disk array in corresponding time period according to failure threshold and posterior probability.The present application can reduce false alarm rate, improve the reliability and availability of RAID.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of fault detection, in particular to a method and device for detecting faults of a disk array. BACKGROUND

[0002] A redundant array of independent disks (RAID) is usually referred to as a "disk array", which has the characteristics of large storage capacity, superior read-write speed performance, data protection and repair, and "dual control" mechanism. The RAID usually undertakes the important role of data storage and forwarding, and needs to have high availability. Therefore, various means are needed to improve the reliability and stability of the RAID to ensure the normal operation of the RAID system.

[0003] In the prior art, a built-in test (BIT) is usually used to detect faults of a RAID device. The BIT is a common means for detecting faults of electronic devices, which is generally realized by adding a BIT circuit on a hardware circuit. The BIT circuit is an embedded microcontroller, which is mainly used for monitoring signals and states of electronic devices, and ensures stable operation of the devices by monitoring parameters such as temperature, voltage and fan speed of the hardware circuit, and provides remote management and control functions.

[0004] In actual detection, the monitoring signal of the BIT circuit should be set with a proper threshold value, and when the actual observation value of a certain observation signal exceeds the threshold value, the alarm mechanism of the BIT circuit is triggered. However, this alarm mechanism has the problem of false alarm, that is, the so-called "false alarm rate". The "false alarm rate" is mainly caused by the following reasons: ① sensor failure, which can cause false temperature, voltage or fan speed; ② electromagnetic interference of the surrounding environment, which can affect the transmission and accuracy of the signal, causing false triggering; ③ BIT software may have bugs or errors, causing false signals; ④ BIT itself or the hardware components related thereto may fail, which can also cause false triggering of signals; ⑤ improper setting of the monitoring threshold parameter of the BIT, which can cause false triggering; ⑥ communication between the BIT and the monitored device may have problems, causing signal loss or false judgment. In the prior art, it is impossible to effectively distinguish between real alarms and false alarms, so when an alarm occurs, the RAID must be shut down for maintenance, causing the data service to be suspended and affecting the execution of data-related businesses.

[0005] Therefore, how to overcome the defects of the prior art and improve the accuracy of the existing RAID fault detection is a problem to be solved in the technical field. SUMMARY

[0006] In view of the above defects or improvement needs of the prior art, the present application effectively improves the RAID fault detection accuracy by scientific setting of the fault threshold, and solves the problem of excessively high false alarm rate of the device.

[0007] The embodiment of the present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a method for detecting faults of a disk array, which specifically comprises: obtaining the failure rate of each circuit module in the disk array, and calculating the prior probability of a fault according to the failure rate of the circuit module; determining a target signal affecting the working state in the disk array, collecting original samples of the target signal, obtaining secondary samples under the fault condition in the original samples, and calculating the original conditional probability of the corresponding target signal according to the secondary samples; establishing a multi-factor model containing all target signals, and calculating the fault threshold using the multi-factor model based on the original conditional probability; during the operation of the disk array, collecting the alarm values of each target signal when an alarm signal is generated, forming alarm samples by using the alarm values in a specified time period, calculating the posterior probability corresponding to each alarm sample, and determining the working state of the disk array in the corresponding time period according to the fault threshold and the posterior probability.

[0009] Preferably, the obtaining of the failure rate of each circuit module in the disk array and the calculation of the prior probability of a fault according to the failure rate of the circuit module specifically comprises: decomposing the circuit modules in the disk array into electronic components, and counting the types and quantities of electronic components in each circuit module; for each circuit module, obtaining the failure rate of each type of electronic component according to the circuit characteristics of the electronic component, and calculating the failure rate of the circuit module according to the failure rate of each type of electronic component and the quantity of the electronic component; summing up the failure rates of all circuit modules as the total failure rate of the disk array, and taking the total failure rate as the prior probability of a fault.

[0010] Preferably, the calculation of the failure rate of the circuit module according to the failure rate of each type of electronic component and the quantity of the electronic component specifically comprises using the following method to calculate the failure rate of each circuit module:

[0011]

[0012] wherein, λ GS is the failure rate of the circuit module, N i is the quantity of the i-th type of electronic component, n is the total number of types of electronic components, λ Gi is the failure rate of the i-th type of electronic component, π Qi is the general quality coefficient of the i-th type of electronic component.

[0013] Preferably, the secondary sample under the condition of the failure in the original sample is used to calculate the original conditional probability of the corresponding target signal, and specifically includes: filtering the secondary sample from the original sample under the condition of the disk array failure, and calculating the statistical parameters of the secondary sample; obtaining an abnormal working interval of the target signal, calculating the probability density of the target signal in the abnormal working interval according to the statistical parameters, and obtaining the original conditional probability under the failure condition according to the probability density.

[0014] Preferably, the statistical parameters include the mean and the variance, and the probability density of the target signal in the abnormal working interval is calculated according to the statistical parameters, and specifically includes that the probability density is calculated in the following manner:

[0015]

[0016] wherein P(x∈T) is the probability density of the value of x in the abnormal working interval T, a and b are upper and lower limits of the abnormal working interval T respectively; μ is the mean of the secondary sample, and σ is the variance of the secondary sample; x is the independent variable of the probability density function f(x∈T; μ, σ).

[0017] Preferably, when there are multiple abnormal working intervals of the target signal, the probability density of the target signal in the abnormal working interval is calculated according to the statistical parameters, and further includes: calculating the probability density of the target signal in each abnormal working interval, and taking the sum of the probability densities in all abnormal working intervals as the probability density of the target signal in the abnormal working interval.

[0018] Preferably, the failure threshold is calculated based on the original conditional probability using a multi-factor model, and specifically includes: taking each abnormal working interval of the target signal as a factor of the occurrence of the failure, taking the original conditional probability corresponding to each factor into the Bayesian model, calculating the posterior conditional probability of all factors, and taking the posterior conditional probability as the failure threshold.

[0019] wherein the posterior conditional probability of all factors is calculated according to the following formula:

[0020]

[0021] wherein P(A|B) is the posterior conditional probability of all factors, N is the total number of the target signals, event A represents the occurrence of the failure of the disk array, event Bi represents that the i th target signal is in the abnormal working interval, P(B i i) represents the original conditional probability that the i th target signal is in the abnormal working interval when the disk array fails, and P(B iP(A) represents the original condition probability of the disk array failure, and P(~A) represents the original condition probability of the disk array not failure.

[0022] Preferably, the calculating the posterior probability corresponding to each alarm sample comprises: obtaining an abnormal operation interval corresponding to the alarm signal, calculating a real-time condition probability of the alarm sample in the abnormal operation interval, and calculating the posterior probability of the disk array failure when the alarm signal is generated according to the real-time condition probability.

[0023] In a second aspect, the present application provides a device for detecting disk array failure, which comprises at least one processor and a memory, the at least one processor and the memory are connected through a data bus, the memory stores instructions executable by the at least one processor, and the instructions are used to complete the method for detecting disk array failure in the first aspect when executed by the processor.

[0024] In a third aspect, the present application further provides a non-volatile computer storage medium, which stores computer executable instructions, the computer executable instructions are executed by one or more processors, and are used to complete the method provided in the first aspect.

[0025] In a fourth aspect, a chip is provided, which comprises a processor and an interface, and is used to call and run a computer program stored in a memory to execute the method provided in the first aspect.

[0026] In a fifth aspect, a computer program product is provided, which comprises instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the method provided in the first aspect.

[0027] Compared with the prior art, the present application has the beneficial effects that: the target signals related to failure on the RAID hardware circuit are monitored, the data samples of various target signals are collected and established in the initial stage, the prior probability and the original condition probability of the equipment are obtained, and the failure threshold is calculated. In the equipment running monitoring stage, the change of the target signal is statistically analyzed again to obtain the posterior probability of the RAID equipment failure, the hidden danger is eliminated by analyzing the causes and repairing for the failure exceeding the threshold, the false early warning is avoided through the active preventive maintenance means, and the RAID reliability and availability are improved. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced. Obviously, the drawings described below are only some of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0029] Figure 1 A flow chart of a method for detecting disk array failure provided by the embodiment of the present application is shown in FIG. 4.

[0030] Figure 2 A flow chart of another method for detecting disk array failure provided by the embodiment of the present application is shown in FIG. 5.

[0031] Figure 3 A flow chart of another method for detecting disk array failure provided by the embodiment of the present application is shown in FIG. 6.

[0032] Figure 4 A diagram of controller module failure rate data in a certain actual scenario of the embodiment of the present application is shown in FIG. 7.

[0033] Figure 5 A diagram of target signal statistical analysis result in a certain actual scenario of the embodiment of the present application is shown in FIG. 8.

[0034] Figure 6 A diagram of original conditional probability calculation result in a certain actual scenario of the embodiment of the present application is shown in FIG. 9.

[0035] Figure 7 A diagram of an apparatus structure for detecting disk array failure provided by the embodiment of the present application is shown in FIG. 10.

[0036] In the drawings, the reference signs are as follows:

[0037] 11: processor; 12: memory. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and do not limit the present application.

[0039] The present application is a system architecture of a specific function system, so in the specific embodiments, the functional logical relationship of each structure module is mainly described, and the specific software and hardware implementation is not limited.

[0040] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict between them. The present application will be described in detail below in combination with the drawings and embodiments.

[0041] Embodiment 1:

[0042] In order to effectively reduce the RAID downtime maintenance caused by the false alarm rate of the BIT circuit, the embodiment uses the BIT to collect the state information of the target signal, updates the estimation of the failure probability of the RAID device according to the collected information, and starts the preventive maintenance work once the probability is greater than the set threshold, so as to eliminate the hidden trouble.

[0043] As shown in Figure 1 The method for detecting the failure of the disk array provided by the embodiment of the application comprises the following steps:

[0044] Step 101: Obtain the failure rate of each circuit module in the disk array, and calculate the prior probability of failure according to the failure rates of all the circuit modules.

[0045] In the embodiment, the original failure probability of the RAID in a general use scenario is taken as the prior probability of failure of the RAID. In a specific implementation, the original failure probability can be estimated by the failure rate prediction of electronic components. Since the number of electronic components in the RAID is large, in order to facilitate the subsequent data processing, the electronic devices can be decomposed into circuit modules, the types and quantities of electronic components in each circuit module are counted, the failure rates of the circuit modules are calculated according to the different circuit characteristics of different electronic components, and the failure rates of the circuit modules are added to obtain the total failure rate of the RAID, that is, the prior probability of failure of the RAID.

[0046] Step 102: Determine the target signal affecting the working state of the disk array, collect the original sample of the target signal, obtain the secondary sample in the failure condition from the original sample, and calculate the original conditional probability of the corresponding target signal according to the secondary sample.

[0047] All target signals affecting the working of the RAID are determined by analyzing the device operation mode, hardware parameter performance and known experience, the abnormal working interval of the target signal is determined, the original sample of the target signal is collected through the BIT circuit, the secondary sample is screened out under the condition that the working state of the RAID is abnormal or a failure occurs, the statistical parameters such as the mean and variance of the secondary sample are calculated, the probability density function in the abnormal working interval is obtained, and the conditional probability of the target signal is obtained. In the embodiment, the conditional probability calculated through the original sample is taken as the original conditional probability of the abnormal working state or failure of the RAID.

[0048] Step 103: Establish a multi-factor model containing all the target signals, and calculate the failure threshold using the multi-factor model based on the original conditional probability.

[0049] After obtaining the original conditional probability of all target signals, a multi-factor model is constructed according to the original conditional probability and other parameters associated therewith, and the posterior probability of RAID failure under the joint condition is calculated through the multi-factor model, and the posterior probability is taken as the failure threshold. In actual implementation, the multi-factor model can use a multi-factor Bayesian model, or other models capable of calculating the posterior probability.

[0050] Step 104: During the operation of the disk array, collect the alarm values of each target signal when the alarm signal is generated, generate alarm samples according to the alarm values and original samples, calculate the posterior probability corresponding to each alarm sample, and determine the working state of the disk array according to the failure threshold and the posterior probability.

[0051] After obtaining the failure threshold, actual failure detection can be performed in the actual operation of the RAID. The state values of each target signal are monitored in real time, and the alarm values when the target signal is in the abnormal interval are continuously collected. In a fixed time period, the collected alarm values are combined to form alarm samples, and the real-time conditional probability of the target signal is calculated according to the alarm samples. Further, the current posterior probability of the target signal needs to be calculated according to the real-time conditional probability, and then compared with the failure threshold. If it is greater than the failure threshold, it is considered that there is a major failure hidden danger, and immediate shutdown maintenance is required to determine the failure point, replace or repair the module, and eliminate the hidden danger of failure.

[0052] After the steps 101-104 provided in this embodiment, the RAID can be detected for failure based on the multi-factor failure threshold, so as to reduce the false alarm rate.

[0053] In actual implementation, the prior probability P(A) of the failure of the electronic device such as the disk array can be obtained from the theoretical failure rate of the electronic components. As shown in FIG. 2, the failure rate of each circuit module in the disk array can be obtained in the following manner, and the prior probability of failure is calculated according to the failure rates of all circuit modules. Figure 2

[0054] Step 201: The circuit modules in the disk array are decomposed into electronic components, and the types and quantities of electronic components in each circuit module are counted.

[0055] ​In actual RAID device hardware, electronic components usually exist in the form of circuit modules, the electronic components in each circuit module are highly coupled, and the coupling between circuit modules is low; any failure of an electronic component in each circuit module will cause the failure of the circuit module, but does not necessarily cause the failure of other circuit modules. Therefore, when estimating the failure probability, the failure probability of a certain electronic component is highly related to the failure probability of the circuit module in which it is located, and is less related to the failure probability of other circuit modules. On the other hand, because the characteristics of each type of electronic component are relatively close, the failure rates of each type of electronic component under the same working scenario are also close. In actual implementation, in order to simplify the calculation of the failure probability, the failure probability can be calculated in units of circuit modules, the device hardware and electronic components in each circuit module are collected, and the types and quantities of electronic components of each circuit module are counted, so as to subsequently calculate the failure rate of the circuit module according to the failure rate of each type of electronic component, and further calculate the prior probability of RAID failure.

[0056] Step 202: For each circuit module, obtain the failure rate of each type of electronic component according to the circuit characteristics of the electronic component, and calculate the failure rate of the circuit module according to the failure rate of each type of electronic component and the number of the electronic component.

[0057] In this embodiment, the component counting method is used to predict the failure rate of each circuit module in the RAID according to the different circuit characteristics of different electronic components.

[0058] In specific implementation, the failure rate of each circuit module is calculated in the following manner:

[0059]

[0060] Wherein, λ GS is the failure rate of the circuit module, N i is the number of the i-th type of electronic component, n is the total number of electronic component types, λ Gi is the failure rate of the i-th type of electronic component, π Qi is the general quality coefficient of the i-th type of electronic component.

[0061] Step 203: Sum the failure rates of all circuit modules as the total failure rate of the disk array, and take the total failure rate as the prior probability of failure.

[0062] The sum of the failure rates of each circuit module is the total failure rate of the RAID, and the total failure rate is the prior probability P(A) of failure when the working environment of the RAID does not change, and the probability P(~A) of normal state.

[0063] After the steps 201-203 provided in this embodiment, the prior probability of RAID failure can be obtained.

[0064] In real-world RAID scenarios, the actual operating environment of each RAID unit may differ from the environment in which the electronic components are tested. Therefore, it is necessary to update the RAID fault information using failure probability data obtained from actual testing. After obtaining the prior probability of failure through the theoretical failure rate of electronic components, each RAID device needs to be debugged, tested, and subjected to stress tests to obtain the original conditional probability of RAID failure under the corresponding operating environment. In practical implementation, such as... Figure 3 The secondary samples under fault conditions can be obtained from the original samples in the following way, and the original conditional probability of the corresponding target signal can be calculated based on the secondary samples.

[0065] Step 301: Using disk array failure as a condition, select secondary samples from the original samples and calculate the statistical parameters of the secondary samples.

[0066] Conditional probability represents the probability that event B, which is associated with event A, will occur when event A occurs. In this embodiment, RAID failure is taken as event A, and the abnormal state of the target signal is taken as event B. To obtain the original conditional probability, real-time status data of the target signal needs to be collected during debugging, testing, and stress testing phases, and original samples need to be established based on the target signal data. After obtaining the original samples, the original samples are then filtered based on the occurrence of failure (i.e., the occurrence of event A) as the condition, and the original samples at which failure occurs are selected as secondary samples. To perform probability calculations, statistical analysis of the secondary sample data is also required to obtain the corresponding statistical parameters. In this embodiment, the main statistical parameters used include the mean and variance of the secondary samples.

[0067] Step 302: Obtain the abnormal operating range of the target signal, calculate the probability density of the target signal within the abnormal operating range based on statistical parameters, and obtain the original conditional probability under fault conditions based on the probability density.

[0068] Based on the statistical parameters of the quadratic sample, the probability distribution of the data in the quadratic sample is analyzed. Based on the characteristics of the probability distribution, the original conditional probability P(B|A) of the target signal being in a specific state (i.e., event B) when the equipment fails (i.e., event A occurs) is calculated using the probability density function, as well as the original conditional probability P(B|~A) of the characteristic occurring when the equipment is normal (i.e., event A does not occur).

[0069] In practice, the probability density is calculated using the following method:

[0070]

[0071] Where P(x∈T) is the probability density of x being within the abnormal working interval T, a and b are the upper and lower limits of the abnormal working interval T, respectively; μ is the mean of the quadratic sample, σ is the variance of the quadratic sample; and x is the independent variable of the probability density function f(x∈T; μ,σ).

[0072] When the target signal has only one abnormal operating range, the probability density within that abnormal operating range is the original conditional probability. When the target signal has multiple abnormal operating ranges, the probability densities of all abnormal operating ranges need to be calculated together, and the sum of the probability densities within all abnormal operating ranges is taken as the original conditional density. Specifically, the probability density of the target signal within each abnormal operating range is calculated, and the sum of the probability densities within all abnormal operating ranges is taken as the probability density of the target signal within the abnormal operating range. In practical implementation, if the normal operating range of a certain electronic component's voltage is 3.0 to 3.6V, then there are two abnormal operating ranges: abnormal operating range T1 (-∞, 3.0), lower limit a = -∞, upper limit b = 3.0; abnormal operating range T2 (3.6, +∞), lower limit a = 3.6, upper limit b = +∞; then the probability density of voltage x in this scenario within the abnormal operating range is: P(x∈T1) + P(x∈T2) = P(x∈(-∞, 3.0)) + P(x∈(3.6, +∞)). 。

[0073] After steps 301-302 provided in this embodiment, the original conditional probability of RAID failure can be obtained.

[0074] When performing fault detection, the fault threshold is a critical factor. If the fault threshold is too low, the posterior probability of the device frequently exceeds the threshold, requiring downtime for maintenance and impacting equipment efficiency. If the fault threshold is too high, the posterior probability is unlikely to exceed the threshold, masking potential problems and increasing the likelihood of failure. Electronic equipment failures are usually caused by a combination of factors. Therefore, fault diagnosis requires considering multiple key factors such as circuitry, voltage, and temperature, rather than a single factor, to more comprehensively and accurately predict the probability of disk array failures.

[0075] In this embodiment, during the commissioning phase, target signal data related to faults, such as current, voltage, and temperature, of key components are collected. This data contains information about the disk array under various states, providing rich raw samples for equipment fault detection. Combining the collected raw samples with empirical information allows for the construction of a better multi-factor Bayesian model. The posterior probability of considering multiple target signals is calculated based on the multi-factor Bayesian model, and this posterior probability is used as the fault threshold. Compared to existing technologies that only use the abnormal range of a single target signal as the fault threshold, this approach more closely reflects the actual situation during normal operation of the disk array.

[0076] The target signal data collected in real time during the actual operation of the disk array can reflect the current operating status of the device. In this embodiment, the target signal alarm values ​​collected in real time within a time period are composed of alarm samples. The alarm samples are used to calculate the posterior probability of a fault occurring in the device under the current operating state. This allows for dynamic tracking of device state changes and timely detection of potential fault risks.

[0077] By comparing the real-time posterior probability under the current operating state with the fault threshold expressed in probabilistic form, the probability of a fault is judged by the magnitude of the probability. If the probability is greater than the threshold, it indicates that the actual status of the equipment deviates from the normal operating state, that is, the probability of a fault is high, and corresponding countermeasures need to be taken.

[0078] In this embodiment, each target signal located in the abnormal working range is regarded as a factor for the occurrence of fault. The original conditional probability corresponding to each factor is substituted into the Bayesian model to calculate the posterior conditional probability containing all factors, and the posterior conditional probability is used as the fault threshold.

[0079] In practice, the posterior conditional probabilities of all factors can be calculated using the following formula:

[0080]

[0081] Where P(A|B) is the posterior conditional probability including all factors, N is the total number of target signals, event A represents a disk array failure, event Bi represents the i-th target signal being in an abnormal operating range. For example, in a specific scenario, the target signals are, in order, the voltage of the CPU chip, the current of the FPGA chip, and the temperature of the power supply chip. B1 indicates that the first target signal, the voltage of the CPU chip, is in an abnormal range; B2 indicates that the second target signal, the current of the FPGA chip, is in an abnormal range; B3 indicates that the third target signal, the temperature of the power supply chip, is in an abnormal range, and so on. i |A) represents the original conditional probability that the i-th target signal is located in the abnormal operating range when the disk array fails, P(B) i |~A) represents the original conditional probability that the i-th target signal is in the abnormal working range when the disk array is not faulty, P(A) represents the original conditional probability that the disk array is faulty, and P(~A) represents the original conditional probability that the disk array is not faulty.

[0082] Once the fault threshold is obtained, RAID fault detection can be performed based on it. During daily use, when the value of a target signal is in an abnormal range, the BIT circuit will issue an alarm. In existing technologies, each alarm is treated as a fault, resulting in a false alarm rate. In this embodiment, instead of directly considering a single alarm as a fault, the posterior probability of a fault occurring for a target signal is calculated based on all alarm values ​​of that target signal within a time period. If the posterior probability is greater than the fault threshold, it indicates that a fault corresponding to that target signal may have occurred. Specifically, the BIT circuit monitors the state of the target signal in real time, obtains the alarm values ​​when the target signal is in an abnormal range, and forms alarm samples from the obtained alarm values ​​within a specified time period. The real-time conditional probability of each target signal is calculated based on its alarm samples, and the current posterior probability of the target signal is calculated based on the real-time conditional probability.

[0083] The posterior probability is then compared with the fault threshold. If it exceeds the threshold, a significant potential fault is identified, requiring immediate shutdown for maintenance to pinpoint the fault location and replace or repair the module to eliminate the risk of failure. In practice, as the RAID system continues to run, the amount of alarm sample data becomes increasingly rich, gradually improving the accuracy of the posterior probability and the reliability of fault detection.

[0084] In practical implementation, accurately selecting the target signal that plays a crucial role in the normal operation of the disk array is a key factor in determining the threshold. Furthermore, the accuracy of various data, such as the determination of the fault threshold and the calculation of the posterior probability, requires a large sample size. Generally, the larger the sample size, the closer the fault detection results are to reality. In addition, new data and actual verification results can be used to continuously optimize and adjust relevant parameters to improve the accuracy of fault detection. In practical applications, the judgment of equipment fault status can be dynamically updated based on the characteristics of target signals in constantly emerging new states, combined with expert experience and knowledge of similar equipment, thereby continuously improving the accuracy of fault detection.

[0085] The disk array fault detection method provided in this embodiment monitors target signals on the RAID hardware circuitry that play a key role in influencing faults. In the initial stage, data samples of various target signals are collected and established to obtain the prior probability and original conditional probability of the device, and then a fault threshold is calculated. During the device operation monitoring phase, changes in the target signals are statistically analyzed again to obtain the posterior probability of RAID device faults. For faults exceeding the threshold, the causes are analyzed and repairs are carried out to eliminate potential risks. This proactive preventative maintenance approach avoids false alarms and improves RAID reliability and availability.

[0086] Example 2:

[0087] The disk array fault detection method provided in Embodiment 1 can be implemented in certain specific embodiments using the specific implementation methods described in this embodiment. It is understood that the specific implementation methods provided in this embodiment are only used to illustrate the specific implementation process of the method in Embodiment 1 in certain specific scenarios and are not intended to limit the scope of protection.

[0088] In a real-world scenario, RAID primarily comprises the following circuit modules: storage module, controller module, interface module, cache module, power module, management module, and chassis. The failure rate of the controller module is as follows: Figure 4 As shown.

[0089] The main electronic component in the controller module is the CPU, and it operates in a normal temperature environment. The failure rate of the controller module is: λ CPU =ΣN(λ G π Q = 5.567452 × 10⁻⁶ / h.

[0090] The storage module operates in a normal temperature environment, and its failure rate is: λ SRM =ΣN(λ G π Q = 6.953788 × 10⁻⁶ / h.

[0091] Information was collected using the same method, and the failure rate of all circuit modules was calculated:

[0092] The interface module failure rate is 1.498584×10⁻⁶ / h;

[0093] The cache module failure rate is 1.8955×10⁻⁶ / h;

[0094] The power module failure rate is 1.100066×10⁻⁶ / h;

[0095] The failure rate of the management module is 1.743288×10⁻⁶ / h;

[0096] The chassis failure rate is 1.200158×10-6 / h.

[0097] Based on the data above, the overall expected failure rate of RAID devices is:

[0098] λ S =λ CPU +λ SRM +λ IFM +λ CCM +λ PWM +λ MGM +λ ETC =19.96×10⁻⁶ / h;

[0099] The prior probability of a RAID device is 19.96×10⁻⁶ / h≈0.00002.

[0100] That is, if P(A) = 0.00002, then P(~A) = 0.99998.

[0101] Using the BIT monitoring circuit, real-time data of various RAID status signals can be collected. The following example uses the core voltage value of the CPU chip in a certain type of RAID device to calculate its original conditional probability.

[0102] In this embodiment, the core operating voltage is used as the target signal. Referring to the relevant chip datasheet, the core operating voltage of this CPU chip is 3.3V, and its normal operating voltage range is 3.0V to 3.6V. Event A represents a RAID failure, and event B1 represents the CPU chip's core operating voltage being in an abnormal operating range (i.e., greater than 3.6V or less than 3.0V). Raw samples of the CPU chip's core operating voltage during the debugging, testing, and stress testing phases of the device were collected. Using RAID malfunction as a condition, 500 sets of data were selected as target samples, and statistical analysis was performed on these target samples. The analysis results are as follows: Figure 5 As shown, the mean μ of the sample data for the core operating voltage of this CPU chip is 3.3183V, and the variance σ is 1.5.

[0103] When a RAID failure occurs, the conditional probability P(B1|A) that the CPU chip core operating voltage is in an abnormal operating range can be realized by integrating a function within that range.

[0104] Calculating the probability of operating in the abnormal interval x < p requires integrating the probability density function f(x:μ,σ) from -∞ to p, i.e.:

[0105]

[0106] To calculate the probability in the interval x > q, we need to integrate the probability density function f(x: μ, σ) from q to +∞, that is:

[0107]

[0108] When p = 3 and q = 3.6, substituting the mean and variance obtained from the statistical analysis into the above formula, we get P(x<3) = 0.3109 and P(x>3.6) = 0.3306. Therefore, the original conditional probability that the CPU chip core operating voltage is in the abnormal operating range when the RAID fails is P(B1|A) = P(x<3) + P(x>3.6) = 0.6415.

[0109] Therefore, when the RAID fails, the primitive conditional probability that the CPU chip core operating voltage is within the normal range (3.0V~3.6V) is P(B1|~A)=1-0.6415=0.3585

[0110] In this scenario, there are five influencing factors on the probability of a fault occurring, represented by the original conditional probabilities B1 to B5. B2 represents the CPU chip's ambient temperature being in an abnormal range, B3 represents the FPGA's operating voltage being in an abnormal range, B4 represents the FPGA chip's ambient temperature being in an abnormal range, and B5 represents the fan speed being in an abnormal range. Using the method described above for calculating conditional probabilities, the original conditional probabilities of the five influencing factors B1 to B5 are obtained as follows: Figure 6 As shown.

[0111] Substituting the original conditional probability of each factor into the formula of the Bayesian model: P(A|B)=P(B|A)*P(A) / P(B).

[0112] Given that P(A) = 0.00002, then P(~A) = 0.99998.

[0113] For multiple factors B1 to B5, the molecule expression is:

[0114] P(B|A)×P(A)=P(B1|A)×P(A)+P(B2|A)×P(A)+P(B3|A)×P(A)+P(B4|A)×P(A)+P(B5|A)×P(A)=47.36×10 -6 .

[0115] For multiple factors B1 to B5:

[0116] P(B1)=P(B1|A)×P(A)+P(B1|~A)×P(~A)=0.6415×0.00002+0.3585×0.99998=0.3585;

[0117] P(B2)=P(B2|A)×P(A)+P(B2|~A)×P(~A)=0.3824×0.00002+0.6176×0.99998=0.6176;

[0118] P(B3)=P(B3|A)×P(A)+P(B3|~A)×P(~A)=0.6195×0.00002+0.3805×0.99998;=0.3805

[0119] P(B4)=P(B4|A)×P(A)+P(B4|~A)×P(~A)=0.4662×0.00002+0.5338×0.99998=0.5338;

[0120] P(B5)=P(B5|A)*P(A)+P(B5|~A)*P(~A)=0.2583*0.00002+0.7417*0.99998=0.7417;

[0121] The expression for the denominator is: P(B) = P(B1) * P(B2) * P(B3) * P(B4) * P(B5) = 0.0334;

[0122] Then P(A|B)=P(B|A)*P(A) / P(B)=0.001420=1420×10 -6 This is the RAID fault threshold in this example.

[0123] In this example, during the operation of RAID, the following target signals are monitored in real time through the BIT circuit: CPU operating voltage and chip temperature, FPGA operating voltage, chip temperature and the fan speed inside the disk array chassis. Alarm values ​​when the target signals are in abnormal ranges are continuously collected. Within a fixed time period, the collected alarm values ​​are combined into alarm samples, and the real-time posterior probability of the above parameters is calculated respectively.

[0124] Calculations show that, within this time period, the real-time conditional probability of a CPU chip voltage failure is P(B|A) = 0.9872, and the real-time conditional probability of a normal CPU chip voltage is P(B|~A) = 0.0128. Combining this with the RAID prior probabilities P(A) = 0.00002 and P(~A) = 0.99998, and using the Bayesian model formula P(A|B) = P(B|A)*P(A) / [P(B|A)*P(A) + P(B|~A)*P(~A)], the probability of RAID failure after the CPU chip core operating voltage becomes abnormal is calculated to be 1540 × 10⁻⁶. -6 1420×10 greater than the fault threshold -6 If this is the case, the RAID system needs immediate shutdown and maintenance. Furthermore, based on the alarm value of the target signal, it can be analyzed that the CPU's core power supply voltage is too high due to a damaged peripheral power supply chip. Replacing the peripheral power supply chip resolves the problem.

[0125] Example 3:

[0126] Based on the disk array fault detection methods provided in Embodiments 1 and 2 above, the present invention also provides a disk array fault detection device that can be used to implement the above methods, such as... Figure 7 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The disk array fault detection equipment of this embodiment includes one or more processors 11 and a memory 12.Figure 7 Take a processor 11 as an example.

[0127] Processor 11 and memory 12 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0128] The memory 12, as a non-volatile computer-readable storage medium for disk array fault detection, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the disk array fault detection methods in Embodiments 1 and 2. The processor 11 executes various functional applications and data processing of the disk array fault detection equipment by running the non-volatile software programs, instructions, and modules stored in the memory 12, thereby implementing the disk array fault detection methods of Embodiments 1 and 2.

[0129] Memory 12 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 12 may optionally include memory remotely located relative to processor 11, which can be connected to processor 11 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] The program instructions / modules are stored in memory 12. When executed by one or more processors 11, they perform the disk array fault detection methods described in Embodiments 1 and 2 above, for example, performing the methods described above. Figures 1 to 3 The steps shown.

[0131] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 7 One of the processors 11 can enable the one or more processors to execute the disk array fault detection method of Embodiments 1 to 2, for example, to perform the above-described... Figures 1 to 3 The steps shown.

[0132] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0133] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0134] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting disk array faults, characterized in that, include: Obtain the failure rate of each circuit module in the disk array, and calculate the prior probability of failure based on the failure rate of the circuit module; Identify the target signals affecting the working state in the disk array, collect the original samples of the target signals, obtain secondary samples under fault conditions from the original samples, and calculate the original conditional probability of the corresponding target signals based on the secondary samples. A multi-factor model incorporating all target signals is established, and the fault threshold is calculated using the multi-factor model based on the original conditional probabilities. This includes: treating each target signal's location within an abnormal operating range as a factor in fault occurrence; substituting the original conditional probability corresponding to each factor into the Bayesian model to calculate the posterior conditional probability encompassing all factors; and using the posterior conditional probability as the fault threshold. The posterior conditional probabilities of all factors are calculated according to the following formula: ； in, Let N be the posterior conditional probability that includes all factors, where N is the total number of target signals, event A represents a disk array failure, and event Bi represents the i-th target signal being in the abnormal operating range. This represents the raw conditional probability that the i-th target signal is located in the abnormal operating range when the disk array fails. This represents the raw conditional probability that the i-th target signal is located in the abnormal operating range when the disk array is not faulty. This represents the raw conditional probability of a disk array failure. This represents the raw conditional probability that the disk array has not failed; During the operation of the disk array, the alarm values ​​of each target signal when an alarm signal is generated are collected, the alarm values ​​within a specified time period are combined into an alarm sample, the posterior probability corresponding to each alarm sample is calculated, and the working status of the disk array within the corresponding time period is determined based on the fault threshold and the posterior probability.

2. The method for detecting disk array faults according to claim 1, characterized in that, The process of obtaining the failure rate of each circuit module in the disk array and calculating the prior probability of failure based on the failure rate of the circuit modules specifically includes: The circuit modules in the disk array are broken down into electronic components, and the types and quantities of electronic components in each circuit module are counted. For each circuit module, the failure rate of each type of electronic component is obtained based on the circuit characteristics of that type of electronic component, and the failure rate of the circuit module is calculated based on the failure rate of each type of electronic component and the number of that type of electronic component. The sum of the failure rates of all circuit modules is taken as the total failure rate of the disk array, and the total failure rate is taken as the prior probability of failure.

3. The method for detecting disk array faults according to claim 2, characterized in that, The calculation of the failure rate of the circuit module based on the failure rate of each type of electronic component and the number of such electronic components specifically includes calculating the failure rate of each circuit module using the following method: ； in, The failure rate of the circuit module. Let i be the number of electronic components of type i. This represents the total number of types of electronic components. Let be the failure rate of the i-th type of electronic component. Let be the general quality coefficient for the i-th type of electronic component.

4. The method for detecting disk array faults according to claim 1, characterized in that, The process of obtaining secondary samples under fault conditions from the original samples and calculating the original conditional probability of the corresponding target signal based on the secondary samples specifically includes: Using disk array failure as a condition, secondary samples are obtained by filtering from the original samples, and the statistical parameters of the secondary samples are calculated. Obtain the abnormal operating range of the target signal, calculate the probability density of the target signal within the abnormal operating range based on statistical parameters, and obtain the original conditional probability under fault conditions based on the probability density.

5. The method for detecting disk array faults according to claim 4, characterized in that, The statistical parameters include mean and variance. Calculating the probability density of the target signal within the abnormal operating range based on the statistical parameters specifically includes calculating the probability density using the following method: ； in, Let x be the probability density of the value of x within the abnormal working interval T, and a and b be the upper and lower limits of the abnormal working interval T, respectively. The mean of the two samples is denoted as . Let x be the variance of the quadratic sample; x is the probability density function. The independent variable.

6. The method for detecting disk array faults according to claim 4, characterized in that, When the target signal has multiple abnormal operating intervals, the step of calculating the probability density of the target signal within the abnormal operating intervals based on statistical parameters further includes: Calculate the probability density of the target signal in each abnormal working interval, and use the sum of the probability densities in all abnormal working intervals as the probability density of the target signal in the abnormal working interval.

7. The method for detecting disk array faults according to claim 1, characterized in that, The step of forming alarm samples from alarm values ​​within a specified time period and calculating the posterior probability corresponding to each alarm sample specifically includes: The system monitors the status of the target signal in real time, obtains the alarm value when the target signal is in an abnormal range, and forms an alarm sample from the obtained alarm values ​​within a specified time period. Calculate the real-time conditional probability of each target signal based on the alarm sample of that target signal, and then calculate the current posterior probability of that target signal based on the real-time conditional probability.

8. A device for detecting disk array faults, characterized in that: The method includes at least one processor and a memory, which are connected via a data bus. The memory stores instructions that can be executed by the at least one processor, which, after being executed by the processor, are used to perform the disk array fault detection method according to any one of claims 1-7.

9. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions that are executed by one or more processors to perform the disk array fault detection method as described in any one of claims 1-7.

Images