A fault detection method, device, medium, and program product

Abstract

The application discloses a kind of fault detection method, equipment, medium and program product in computer technology field.The present application is based on the current use parameters such as current operating parameter and current structure parameter of computing device, then combines its corresponding reference data information, fluctuation data information and importance, obtains equipment health information by evaluation, finally determines equipment failure risk based on equipment health information, whereby a variety of equipment parameters determine equipment failure risk, not only through equipment parameter is associated with equipment operating state, but equipment structure parameter can reflect the degree of equipment new and old, improve the fault detection accuracy of computing device, can detect the physical loss condition such as component aging in computing device, reduce false alarm rate, conducive to timely operation and management of computing device.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a fault detection method, device, medium, and program product. Background Technology

[0002] Currently, computing devices typically undertake core tasks in high-performance computing, such as performing artificial intelligence calculations. To prevent computing devices from malfunctioning due to prolonged high-load operation, temperature threshold alarm mechanisms can be used for management and maintenance. For example, an alarm can be triggered when the temperature exceeds 85°C, allowing technicians to reduce the load on the computing device in a timely manner and lower the probability of failure. However, this solution cannot detect the aging of components in the computing device, such as the slow degradation of capacitors; moreover, the false alarm rate is relatively high, which is not conducive to timely maintenance of the computing device and may also affect task operation.

[0003] Therefore, how to improve the accuracy of fault detection in computing devices is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a fault detection method, device, medium and program product to improve the accuracy of fault detection of computing devices.

[0005] In a first aspect, the present invention provides a fault detection method, comprising: collecting current operating parameters of a computing device; the current operating parameters include: current running parameters and current structural parameters; determining, based on historical usage information of the computing device, baseline data information and fluctuation data information corresponding to the current operating parameters; determining, based on the fault correlation and permissible fluctuation range of the current operating parameters, the importance of the current operating parameters; assessing and obtaining device health information based on the current operating parameters, baseline data information, fluctuation data information, and importance; and determining the device fault risk based on the device health information.

[0006] Secondly, the present invention provides a fault detection device, comprising: a data acquisition module for acquiring current operating parameters of a computing device; the current operating parameters include current running parameters and current structural parameters; a first determination module for determining baseline data information and fluctuation data information corresponding to the current operating parameters based on historical usage information of the computing device; a second determination module for determining the importance of the current operating parameters based on the fault correlation and permissible fluctuation range of the current operating parameters; an evaluation module for evaluating and obtaining device health information based on the current operating parameters, baseline data information, fluctuation data information, and importance; and a detection module for determining the device fault risk based on the device health information.

[0007] Thirdly, the present invention provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the aforementioned fault detection method.

[0008] Fourthly, the present invention provides a non-volatile storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned fault detection method.

[0009] Fifthly, the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed fault detection method.

[0010] As can be seen from the above scheme, the present invention provides a fault detection method, including: collecting the current usage parameters of a computing device; the current usage parameters include: current operating parameters and current structural parameters; determining the baseline data information and fluctuation data information corresponding to the current usage parameters based on the historical usage information of the computing device; determining the importance of the current usage parameters based on the fault correlation and fluctuation allowable range of the current usage parameters; evaluating the device health information based on the current usage parameters, baseline data information, fluctuation data information, and importance; and determining the device fault risk based on the device health information.

[0011] Therefore, the beneficial effects of this invention are as follows: Based on the current operating parameters and structural parameters of the computing device, and combined with historical usage information, the baseline data and fluctuation data corresponding to the current operating parameters are determined. Then, based on the fault correlation and permissible fluctuation range of the current operating parameters, the importance of the current operating parameters is determined. Based on the current operating parameters, baseline data, fluctuation data, and importance, device health information is assessed. Based on this device health information, the device failure risk can be determined. This scheme uses the current operating parameters and structural parameters of the computing device as a foundation, then combines their corresponding baseline data, fluctuation data, and importance to assess device health information. Finally, based on this device health information, the device failure risk can be determined. This method determines device failure risk using multiple device parameters, not only linking device operating status through device parameters but also reflecting the age of the device through structural parameters. This improves the accuracy of fault detection in computing devices, enabling the detection of physical wear and tear such as component aging, reducing false alarm rates, and facilitating timely operation and maintenance of computing devices.

[0012] Correspondingly, the fault detection device, equipment, medium, and program product provided by the present invention also have the above-mentioned technical effects. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0014] Figure 1 This is a flowchart of a fault detection method disclosed in this invention;

[0015] Figure 2 This is a flowchart of the second fault detection method disclosed in this invention;

[0016] Figure 3 This invention discloses a flowchart for calculating a health score.

[0017] Figure 4 This is a schematic diagram of a fault detection device disclosed in this invention;

[0018] Figure 5 This is a schematic diagram of an electronic device disclosed in this invention;

[0019] Figure 6 A server structure diagram provided by the present invention;

[0020] Figure 7 This invention provides a terminal structure diagram. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0022] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0023] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Currently, computing devices typically undertake core tasks in high-performance computing, such as performing artificial intelligence calculations. To prevent computing devices from malfunctioning due to prolonged high-load operation, temperature threshold alarm mechanisms can be used for management and maintenance. For example, an alarm can be triggered when the temperature exceeds 85°C, allowing technicians to reduce the load on the computing device in a timely manner and lower the probability of failure. However, this solution cannot detect the aging of components in the computing device, such as the slow degradation of capacitors; moreover, the false alarm rate is relatively high, which is not conducive to timely maintenance of the computing device and can also affect task operation. This invention provides a fault detection scheme that can improve the accuracy of fault detection for computing devices.

[0025] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a fault detection method, including:

[0026] S101. Collect the current operating parameters of the computing device; the current operating parameters include: current running parameters and current structural parameters.

[0027] In this embodiment, the computing device may be a GPU, TPU (Tensor Processing Unit), etc. To confirm the real-time operating status of the computing device, the current usage parameters may include current operating parameters; these parameters may include at least one of the following: current temperature, current voltage, error count, clock skew, and memory utilization. The current temperature, current voltage, and error count reflect the actual operating status of the computing device, while the clock skew and memory utilization reflect its actual performance. Correspondingly, to confirm the hardware wear and tear of the computing device, the current usage parameters may include current structural parameters, which may be board deformation parameters. In one implementation, collecting the current usage parameters of the computing device includes: collecting at least one of the following: current temperature, current voltage, error count, clock skew, and memory utilization, as current operating parameters; and collecting the board deformation parameters of the computing device, as current structural parameters.

[0028] S102. Based on the historical usage information of the computing device, determine the baseline data information and fluctuation data information corresponding to the current usage parameters.

[0029] It should be noted that there is more than one currently used parameter, and each currently used parameter corresponds to its own baseline data and fluctuation data. Specifically, the currently used parameters include the current operating parameters, which each have their own baseline data, fluctuation data, fault correlation, permissible fluctuation range, and importance; the current structural parameters also have their own baseline data, fluctuation data, fault correlation, permissible fluctuation range, and importance. For example, when the current operating parameters are the current temperature and the current voltage, the current temperature corresponds to its own baseline data, fluctuation data, fault correlation, permissible fluctuation range, and importance; the current voltage also corresponds to its own baseline data, fluctuation data, fault correlation, permissible fluctuation range, and importance. Correspondingly, when the current structural parameter is a board deformation parameter, the board deformation parameter corresponds to its own baseline data, fluctuation data, fault correlation, permissible fluctuation range, and importance.

[0030] In this embodiment, the historical usage information of the computing device includes: each historical usage time and the corresponding current usage parameters. For example, the historical usage information includes: the current temperature, current voltage, and board deformation parameters collected every hour after the computing device is put into use. In one implementation, the historical usage information includes: each historical usage time and the corresponding current usage parameters; accordingly, based on the current usage parameters and a preset first weight at the first usage time (e.g., the first hour after the computing device is put into use), and the current usage parameters and a preset second weight at the second usage time (e.g., the second hour after the computing device is put into use), baseline data information is calculated; the first usage time is before the second usage time. The sum of the preset first weight and the preset second weight is 1, and the preset first weight is less than the preset second weight. For example, if the first usage time is the first day after the computing device goes online (i.e., 24 hours after going online), and the second usage time is the 7th day after the computing device goes online (i.e., 168 hours after going online), and the usage parameter at the corresponding time is temperature, then the baseline data information = D1 × preset first weight + D7 × preset second weight; D1 is the current usage parameter (temperature) at the first usage time, with a value of 73℃; D7 is the current usage parameter (temperature) at the second usage time, with a value of 75℃; the preset first weight is 0.3, and the preset second weight is 0.7, then the baseline data information u k =0.3×73+0.7×75=21.9+52.5=74.4℃.

[0031] In one example, a target time period (e.g., the last 24 hours, the last 72 hours, etc.) is determined based on historical usage times. The dispersion of the current usage parameters within the target time period is used as fluctuation data information; this dispersion can be represented by standard deviation or variance. For example, the standard deviation or variance of the current temperature within the target time period can be calculated as fluctuation data information of the current temperature; the standard deviation or variance of the current voltage within the target time period can be calculated as fluctuation data information of the current voltage.

[0032] It should be noted that the historical usage information of computing devices can be updated on a time basis. For example, usage information within the most recent period (such as the most recent month) can always be selected for the importance calculation.

[0033] S103. Determine the importance of the current usage parameters based on their fault correlation and permissible fluctuation range.

[0034] It should be noted that the fault correlation of each currently used parameter is determined based on its historical data. In one implementation, historical fault data and historical used parameters are parsed from the operating log of the computing device; the correlation coefficient between the historical fault data and the historical used parameters is determined; and the fault correlation is represented by the correlation coefficient. The historical fault data includes parameters such as fault indicators, temperature, and voltage at the time of the fault. By analyzing the relationship between the fault indicators and parameters such as temperature and voltage at the time of the fault, the corresponding correlation coefficient can be determined. Specifically, the Pearson correlation coefficient between the fault indicators and parameters such as temperature and voltage at the time of the fault can be analyzed as the corresponding fault correlation. Of course, other methods can also be used to calculate the correlation coefficient. Therefore, in one implementation, determining the correlation coefficient between historical fault data and historical used parameters includes selecting at least one of a variety of preset correlation measurement methods and calculating the correlation coefficient between historical fault data and historical used parameters. The various correlation measurement methods include: Pearson correlation coefficient, partial correlation coefficient, mutual information, etc.

[0035] In this embodiment, the permissible fluctuation range is used to represent the allowed value range of various parameters. For example, when the GPU temperature is within the range [a, b], it is considered that the GPU is running stably and is unlikely to malfunction due to temperature anomalies. Therefore, the range [a, b] can be considered the permissible fluctuation range of the temperature. In one implementation, the board structure safety threshold and the parameter permissible range (i.e., the permissible fluctuation range) of the currently used parameters are read from the applicable technical specification document of the computing device; the endpoint difference of the parameter permissible range is calculated; the ratio of the endpoint difference to the board structure safety threshold is used to represent the dimensionless quantity of the permissible fluctuation range, so that the importance can be calculated using the permissible fluctuation range. Referring to this implementation, the parameter permissible range corresponding to each parameter can be obtained by querying the technical specification document. The permissible fluctuation range corresponding to each parameter can be seen in Table 1.

[0036] Table 1

[0037]

[0038] In one implementation, the importance of the currently used parameters is determined based on their fault correlation and fluctuation tolerance range. This includes: reading the board structure safety threshold from the applicable technical specifications of the computing device; calculating the endpoint difference of the fluctuation tolerance range; using the ratio of the endpoint difference to the board structure safety threshold as the dimensionless quantity of the fluctuation tolerance range; and using the product of the fault correlation and the dimensionless quantity of the fluctuation tolerance range as the importance. To avoid the importance value being too large or too small, in one implementation, a preset limit range can be used to constrain the value of the importance, and the sum of the importance of all parameters involved in the calculation is 1. The preset limit range is [0.05, 0.35]; the minimum value of 0.05 is used to ensure that key parameters are not ignored; the maximum value of 0.35 is used to prevent a single parameter from dominating. Historical data shows that when the importance is below 0.05, the impact of the corresponding parameter on the device health information is negligible; when it is above 0.35, the impact of the corresponding parameter on the device health information is too large.

[0039] S104. Based on current usage parameters, baseline data information, fluctuation data information, and importance, the equipment health information is assessed.

[0040] As mentioned earlier, the current temperature, current voltage, error count, clock skew, memory utilization, and board deformation parameters all have their own corresponding baseline data. k Fluctuation data information δ k Fault correlation c k Dimensionless quantity within the permissible range of fluctuation and the degree of importance w k If all six parameters mentioned above are used in the calculation, then k = 1, 2, 3, 4, 5, 6.

[0041] In one embodiment, based on the current usage parameters, reference data information, fluctuation data information, and importance level, device health information is evaluated, including: calculating the device health information using the first formula; the first formula is: ; where, H represents the device health information, w k represents the importance level, P k represents the current usage parameters, μ k represents the reference data information, δ k represents the fluctuation data information, and k represents the type of the current usage parameters. If all six of the above parameters are involved in the calculation, then subtracting the sum of the penalties of the six parameters from 100 can obtain the device health information.

[0042] S105. Determine the device failure risk based on the device health information.

[0043] In one embodiment, determining the device failure risk based on the device health information includes: calculating the instantaneous failure rate according to the preset basic failure rate, failure acceleration factor, and device health information; using the cumulative value of the instantaneous failure rate within a preset time period to represent the device failure risk. Among them, calculating the instantaneous failure rate according to the preset basic failure rate, failure acceleration factor, and device health information includes: calculating the instantaneous failure rate using the second formula; the second formula is: λ = λ0 × exp[β × (100 - H)]; where, λ represents the instantaneous failure rate, λ0 represents the preset basic failure rate, β represents the failure acceleration factor, and H represents the device health information.

[0044] In one embodiment, it further includes: evaluating the device failure risk using a preset evaluation interval; determining a corresponding maintenance plan for the computing device according to the evaluation result. The preset evaluation interval includes: emergency maintenance strategy R > 9.0; planned maintenance strategy 7.0 < R ≤ 9.0; monitoring operation strategy 5.0 < R ≤ 7.0; normal operation strategy R ≤ 5.0. R represents the device failure risk. When R falls into the above interval, the computing device is managed according to the strategy (i.e., the maintenance plan) corresponding to the corresponding interval.

[0045] In this embodiment, based on the current operating parameters and structural parameters of the computing device, and combined with historical usage information, baseline data and fluctuation data are determined for the current operating parameters. Then, based on the fault relevance and permissible fluctuation range of the current operating parameters, their importance is determined. Based on the current operating parameters, baseline data, fluctuation data, and importance, device health information is assessed. Based on this device health information, the device failure risk can be determined. This scheme uses the current operating parameters and structural parameters of the computing device as a foundation, then combines their corresponding baseline data, fluctuation data, and importance to assess device health information. Finally, based on this device health information, the device failure risk can be determined. This approach uses multiple device parameters to determine device failure risk, not only linking device parameters to device operating status but also allowing structural parameters to reflect the device's age, improving the accuracy of fault detection. It can detect physical wear and tear such as component aging in the computing device, reducing false alarm rates and facilitating timely operation and maintenance of the computing device.

[0046] Please see Figure 2 The following section will use specific numerical values ​​to perform fault detection on GPU computing devices.

[0047] (1) Obtain GPU parameters through the following interface.

[0048] Temperature: GPU core temperature sensor (NVMLAPI); Voltage: Power management chip (IPMI protocol); ECC (Error Checking and Correcting) error: Memory controller counter (nvidia-smi command); Clock skew: Clock frequency monitor (DCGMAPI); Memory utilization: Memory bandwidth counter (CUDARuntimeAPI); Board deformation: PCB strain sensor (24-bit ADC sampling).

[0049] (2) Establish health benchmarks (i.e. benchmark data information).

[0050] Among them, the formula for calculating the benchmark value is μ. k =0.3×D1+0.7×D7, and its weight allocation is based on the analysis of the device aging curve. 30% of the weight on the first day is used to eliminate the impact of the initial unstable state of the new device; 70% of the weight on the 7th day is used to reflect the stable operating state. This ratio has been verified by aging tests on 200 GPUs (average error <2%).

[0051] The baseline value will be calculated below based on the specific data collected.

[0052] Temperature: The average 24-hour temperature on the first day, D1, was 73℃; the average 24-hour temperature on the 7th day, D7, was 75℃; μk =0.3×73+0.7×75=21.9+52.5=74.4℃.

[0053] Voltage: D1 = 1.21V, D7 = 1.20V, μ k =0.3×1.21+0.7×1.20=0.363+0.84=1.203V.

[0054] ECC error: D1=8 times / hour, D7=6 times / hour, μ k =0.3×8+0.7×6=2.4+4.2=6.6 times / hour.

[0055] Clock offset: D1=0.12%, D7=0.15%, μ k =0.3×0.12+0.7×0.15=0.036+0.105=0.141%.

[0056] Memory utilization: D1=58%, D7=62%, μ k =0.3×58+0.7×62=17.4+43.4=60.8%.

[0057] Circuit board deformation: D1=0.05mm, D7=0.07mm, μ k =0.3×0.05+0.7×0.07=0.015+0.049=0.064mm.

[0058] (3) Calculation of standard deviation.

[0059] Formula for calculating standard deviation: .

[0060] Temperature: The sum of the squared differences of the temperature data over the first 168 hours = ∑(hourly value - 74.4)² = 8307.52, variance = 8307.52 / (168-1) = 8307.52 / 167 = 49.745, δ k =√49.745=7.05℃.

[0061] Voltage: ∑(p) i -1.203)²=1025.34, variance=1025.34 / 167=6.139, δ k =√6.139=2.48mV.

[0062] ECC Error: ∑(p i -6.6)²=1302.4, variance=1302.4 / 167=7.8, δ k =√7.8=2.79 times / hour.

[0063] Clock offset: ∑(p i -0.141)²=0.385, variance=0.385 / 167=0.0023, δ k =√0.0023=0.048%.

[0064] Video memory utilization: ∑(p i -60.8)²=15482.6, variance=15482.6 / 167=92.77, δ k =√92.77=9.63%.

[0065] Board deformation: ∑(p i -0.064)²=0.0542, variance=0.0542 / 167=0.000324, δ k =√0.000324=0.018mm.

[0066] (4) Health score calculation.

[0067] ① The weight (importance) is calculated from two scientifically determined parameters: fault correlation and fluctuation tolerance range.

[0068] Fault correlation definition: The strength of the historical statistical correlation between parameter deviation and fault occurrence.

[0069] Calculation basis: Based on 500 sets of historical fault data (source: GPU operation logs of a data center from 2020-2023), Pearson correlation coefficient formula: ; Parameter deviation value = |real-time value - baseline value|; : Fault flag (0=normal, 1=fault); : Average parameter deviation; : Average value of fault indicators.

[0070] Please refer to Table 2 for the specific values ​​of the fault correlation of each parameter.

[0071] Table 2

[0072]

[0073] ② Definition of fluctuation coefficient (permissible fluctuation range): The ratio of the normal fluctuation range of the parameter (permissible parameter range) to the board structure safety threshold of 0.05mm. Both the normal fluctuation range of the parameter and the board structure safety threshold are determined by consulting the technical white paper (technical specification document) of the corresponding GPU, as shown in Table 1 above.

[0074] According to the weighting calculation formula: Calculate the weights of each parameter.

[0075] Calculate each parameter Product: Temperature: 0.78 × 560 = 436.8; Voltage: 0.65 × 2.4 = 1.56; ECC error: 0.72 × 1000 = 720.0; Clock skew: 0.68 × 20 = 13.6; Memory utilization: 0.61 × 2000 = 1220.0; Board deformation: 0.59 × 16 = 9.44.

[0076] Calculate the sum of the products: =436.8+1.56+720.0+13.6+1220.0+9.44=2401.4.

[0077] Calculate the normalized weights: W_temperature = 436.8 / 2401.4 = 0.1819; W_voltage = 1.56 / 2401.4 = 0.0006; W_ECC = 720.0 / 2401.4 = 0.2998; W_clock = 13.6 / 2401.4 = 0.0057; W_video memory = 1220.0 / 2401.4 = 0.5080; W_deformation = 9.44 / 2401.4 = 0.0039.

[0078] Then, the minimum weight threshold of 0.05 and the maximum weight limit of 0.35 are used to constrain the aforementioned normalized weights to obtain the final weight values ​​for each parameter. The final sum of the weights of all parameters is kept at 1.0.

[0079] The final weight allocation can be found in Table 3.

[0080] Table 3

[0081]

[0082] As can be seen, the final weights of each parameter are: Temperature: 0.18; Voltage: 0.05; ECC error: 0.30; Clock skew: 0.05; Memory utilization: 0.35; Board deformation: 0.07; Total: 0.18+0.05+0.30+0.05+0.35+0.07=1.00.

[0083] Please see Figure 3 If the real-time parameter values ​​collected are: P_temperature = 76.2℃; P_voltage = 1.19V; P_ECC = 15 times / hour; P_clock = 0.18%; P_memory = 68%; P_deformation = 0.082mm.

[0084] The health score (device health information) calculation process includes: deviation D k =|P k -μ k | / (3δ k ).

[0085] D_temperature = |76.2-74.4| / (3×7.05) = 1.8 / 21.15 = 0.0851.

[0086] D_voltage = |1.19-1.203| / (3×0.00248) = 0.013 / 0.00744 = 1.7473.

[0087] D_ECC=|15-6.6| / (3×2.79)=8.4 / 8.37=1.0033.

[0088] D_clock = |0.18 - 0.141| / (3 × 0.048) = 0.039 / 0.144 = 0.2708.

[0089] D_video memory = |68-60.8| / (3×9.63) = 7.2 / 28.89 = 0.2491.

[0090] D_deformation = |0.082-0.064| / (3×0.018) = 0.018 / 0.054 = 0.3333.

[0091] Penalty calculation formula: .

[0092] T_temperature = 0.18 × (0.0851)² = 0.18 × 0.00724 = 0.001303.

[0093] T_voltage = 0.05 × (1.7473)² = 0.05 × 3.05² = 0.152600.

[0094] T_ECC=0.30×(1.0033)²=0.30×1.0066=0.301980.

[0095] T_clock = 0.05 × (0.2708)² = 0.05 × 0.0733 = 0.003665.

[0096] T_video memory = 0.35 × (0.2491)² = 0.35 × 0.0620 = 0.021700.

[0097] T_deformation = 0.07 × (0.3333)² = 0.07 × 0.1111 = 0.007777.

[0098] Total penalty value: The corresponding health score .

[0099] (5) Fault prediction.

[0100] Determining the baseline failure rate: Accelerated life testing at 125°C; Sample size: 10 new GPUs of the same model; Mean time between failures: 1000 hours. .

[0101] Acceleration factor was determined using historical fault records; regression equation: lnλ=lnλ0+β×(100-H), and the acceleration factor β=0.0487≈0.05 was calculated.

[0102] Calculate the instantaneous failure rate: Health deviation = 100 - 51.0975 = 48.9025, λ = 0.001 × e^(0.05 × 48.9025) = 0.001 × e^(2.445125) = 0.001 × 11.532 = 0.011532 .

[0103] Therefore, the cumulative risk over 24 hours is: R = 0.011532 × 24 = 0.276768.

[0104] (6) Maintain decision-making.

[0105] According to the decision rule: 0.276768 < 5.0 → determined as: normal operation.

[0106] As can be seen, this embodiment adopts a three-layer architecture design, which includes three core modules: health assessment, fault prediction, and decision generation.

[0107] 1. The core mathematical formula for the health assessment module is: .

[0108] Parameter Description: H (Health Score): A quantitative value of the overall health status of the equipment, ranging from 0 to 100 points, derived from model calculation output. k (Parameter weights): Importance coefficients of each monitoring parameter, ranging from 0 to 1, derived from entropy weighting calculation. P k (Real-time parameter values): Real-time monitored values ​​from the sensor, with a continuous range, derived directly from the hardware sensor readings. μ k (Health baseline value): The central value of the normal state parameter of the equipment, a continuous variable with a range of values, derived from the calculation during the initialization phase. δ k (Standard deviation): The normal fluctuation range of the parameter, taking the range of positive real numbers, derived from historical data statistics. |P k -μ k | / (3δ k (Deviation coefficient): The degree of parameter abnormality, with a value range of ≥0, derived from real-time calculation.

[0109] 2. The core mathematical formula for the fault prediction module is: ; .

[0110] Parameter description: Instantaneous failure rate λ: Unit: Probability of failure per hour ( ), Physical meaning: Characterize the possibility of failure per unit time of the device in the current health state. Basic failure rate λ0: Unit: Probability of failure per hour ( ), Physical meaning: The inherent failure rate of the device in a brand-new and ideal working condition. Determination method: Obtained through accelerated life test. Acceleration factor β: Dimensionless parameter, Physical meaning: Characterize the influence intensity of health state deterioration on the failure rate, Determination method: Obtained through regression analysis of historical failure data. Health score H: Range: 0 - 100 points, Physical meaning: The quantitative evaluation value of the current health state of the device.

[0111] 3. The decision-making generation module determines the range to which R belongs according to the decision rule.

[0112] Decision rule: Emergency maintenance: R > 9.0; Planned maintenance: 7.0 < R ≤ 9.0; Monitoring operation: 5.0 < R ≤ 7.0; Normal operation: R ≤ 5.0.

[0113] It should be noted that the reference value can be updated monthly according to the following formula. The reference value update formula is: ; is the average value of the parameter in the current month from.

[0114] Correspondingly, the standard deviation is also updated monthly, and the update formula is: ; m is the sample size in the current month, is the parameter value in the current month.

[0115] Correspondingly, the weight is updated to: New weight ; is the fault correlation in the current month, is the fluctuation coefficient in the current month.

[0116] The triggering mechanism for updating the reference value, standard deviation and weight is: The system automatically executes at 00:00 on the 1st of each month; Data requirement: The sample size in the current month ≥ 432,000 (sampling per minute). The update steps include: (1) Calculate the average value of each parameter in the current month ; (2) Update the reference value according to the formula ; (3) Calculate the sum of squares in the current month ; (4) Update the standard deviation according to the formula ; (5) Recalculate and based on the data in the current month; (6) Update the weight according to the formula .

[0117] Furthermore, to make GPU health assessments more accurate, temperature, voltage, and ECC errors can be designated as mandatory core parameters; clock skew and memory utilization are optional performance parameters; and board deformation is an optional structural parameter.

[0118] This embodiment, based on a multi-parameter dynamic weight allocation mechanism and adaptive calibration throughout the equipment's lifecycle, can detect early degradation rates, reduce false alarm rates, and improve prediction accuracy; it can also reduce maintenance costs, decrease the probability of downtime due to malfunctions, and extend equipment lifespan.

[0119] The following describes a fault detection device provided by an embodiment of the present invention. The fault detection device described below can be referred to in conjunction with other embodiments described herein.

[0120] See Figure 4 As shown in the figure, an embodiment of the present invention discloses a fault detection device, including: a data acquisition module 401, used to acquire the current usage parameters of a computing device; the current usage parameters include: current operating parameters and current structural parameters; a first determination module 402, used to determine the baseline data information and fluctuation data information corresponding to the current usage parameters based on the historical usage information of the computing device; a second determination module 403, used to determine the importance of the current usage parameters based on the fault correlation and fluctuation allowable range of the current usage parameters; an evaluation module 404, used to evaluate and obtain device health information based on the current usage parameters, baseline data information, fluctuation data information, and importance; and a detection module 405, used to determine the device fault risk based on the device health information.

[0121] In one implementation, the acquisition module is specifically used to: acquire at least one of the following parameters of the computing device: current temperature, current voltage, error count, clock offset, and video memory utilization, as current operating parameters; and acquire the board deformation parameters of the computing device, as current structural parameters.

[0122] In one implementation, the historical usage information includes: each historical usage time and the corresponding current usage parameters; accordingly, baseline data information is calculated based on the current usage parameters and a preset first weight at the first usage time in each historical usage time, and the current usage parameters and a preset second weight at the second usage time in each historical usage time; the first usage time is before the second usage time; accordingly, a target time period is determined based on each historical usage time; the dispersion of the current usage parameters within the target time period is used as fluctuation data information.

[0123] In one implementation, historical fault data and historical usage parameters are parsed from the operating logs of the computing device; the correlation coefficient between the historical fault data and the historical usage parameters is determined; and the correlation coefficient is used to represent the fault correlation.

[0124] In one implementation, determining the correlation coefficient between historical fault data and historical usage parameters includes: selecting at least one from a variety of preset correlation measurement methods to calculate the correlation coefficient between historical fault data and historical usage parameters.

[0125] In one implementation, the board structure safety threshold and the parameter allowable range of the currently used parameters are read from the technical specification document applicable to the computing device; the endpoint difference of the parameter allowable range is calculated; and the dimensionless quantity of the fluctuation allowable range is represented by the ratio of the endpoint difference to the board structure safety threshold.

[0126] In one implementation, the second determining module is specifically used to: take the product of the fault correlation and the dimensionless quantity of the fluctuation allowable range as the importance.

[0127] In one embodiment, it further includes a constraint module for constraining the value of importance using a preset limit range.

[0128] In one implementation, the evaluation module is used to: calculate device health information using a first formula; the first formula is: Where H represents device health information, w k P indicates the degree of importance. k Indicates the currently used parameter, μ k Represents the baseline data information, δ k This indicates fluctuation data information.

[0129] In one implementation, the detection module is used to: calculate the instantaneous failure rate based on a preset basic failure rate, a failure acceleration factor, and equipment health information; and use the cumulative value of the instantaneous failure rate over a preset period of time to represent the equipment failure risk.

[0130] In one implementation, the detection module is used to: calculate the instantaneous failure rate using a second formula; the second formula is: λ=λ0×exp[β×(100-H)]; where λ represents the instantaneous failure rate, λ0 represents the preset basic failure rate, β represents the failure acceleration factor, and H represents the equipment health information.

[0131] In one embodiment, it further includes: a maintenance module, used to assess the risk of equipment failure using a preset assessment range; and to determine a corresponding maintenance plan for the computing device based on the assessment results.

[0132] For more detailed information on the working process of each module and unit in this embodiment, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0133] As can be seen, this embodiment provides a fault detection device that determines the risk of equipment failure by using multiple equipment parameters. It not only associates the equipment operating status with the equipment parameters, but also reflects the age of the equipment by the equipment structural parameters. This improves the accuracy of fault detection for computing equipment, can detect physical wear and tear such as aging of components in computing equipment, reduces false alarm rate, and is conducive to the timely operation and maintenance and management of computing equipment.

[0134] The present invention provides an electronic device according to an embodiment of the present invention. The electronic device described below can be referred to in conjunction with other embodiments described herein.

[0135] See Figure 5 As shown, an embodiment of the present invention discloses an electronic device, including: a memory 501 for storing a computer program; and a processor 502 for executing the computer program to implement the method disclosed in any of the above embodiments.

[0136] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: collecting the current usage parameters of the computing device; the current usage parameters include: current operating parameters and current structural parameters; determining the baseline data information and fluctuation data information corresponding to the current usage parameters based on the historical usage information of the computing device; determining the importance of the current usage parameters based on the fault correlation and fluctuation allowable range of the current usage parameters; evaluating the device health information based on the current usage parameters, baseline data information, fluctuation data information, and importance; and determining the device fault risk based on the device health information.

[0137] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: collecting at least one of the current temperature, current voltage, error count, clock offset, and video memory utilization of the computing device as current operating parameters; collecting the board deformation parameters of the computing device as current structural parameters.

[0138] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: calculate the baseline data information based on the current usage parameters and preset first weight at the first usage time in each historical usage time, and the current usage parameters and preset second weight at the second usage time in each historical usage time; the first usage time is before the second usage time.

[0139] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: determine the target time period based on each historical usage time; and use the dispersion of the current usage parameters within the target time period as fluctuation data information.

[0140] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: parsing historical fault data and historical usage parameters from the operation log of the computing device; determining the correlation coefficient between the historical fault data and the historical usage parameters; and using the correlation coefficient to represent the fault correlation.

[0141] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: select at least one from a variety of preset correlation measurement methods to calculate the correlation coefficient between historical fault data and historical usage parameters.

[0142] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: read the board structure safety threshold and the parameter allowable range of the currently used parameters from the technical specification file applicable to the computing device; calculate the endpoint difference of the parameter allowable range; and use the ratio of the endpoint difference to the board structure safety threshold to represent the dimensionless quantity of the fluctuation allowable range.

[0143] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: taking the product of the fault correlation and the dimensionless quantity of the fluctuation tolerance range as the importance.

[0144] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: calculate the instantaneous failure rate based on the preset basic failure rate, the failure acceleration factor, and the equipment health information; and use the cumulative value of the instantaneous failure rate over a preset time period to represent the equipment failure risk.

[0145] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: a maintenance module, used to assess the risk of equipment failure using a preset assessment interval; and to determine a corresponding maintenance plan for the computing device based on the assessment results.

[0146] Furthermore, embodiments of the present invention also provide an electronic device. The aforementioned electronic device can be, for example,... Figure 6 The server shown can also be as follows: Figure 7 The terminal shown. Figure 6 and Figure 7 These are all structural diagrams of an electronic device according to an exemplary embodiment, and the content in the diagrams should not be considered as any limitation on the scope of the invention.

[0147] Figure 6This is a schematic diagram of a server structure provided in an embodiment of the present invention. The server may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory stores a computer program, which is loaded and executed by the processor to implement the relevant steps in fault detection disclosed in any of the foregoing embodiments.

[0148] In this embodiment, the power supply is used to provide operating voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this invention, and is not specifically limited here; the input / output interface is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0149] In addition, the memory, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system, computer programs and data, etc., and the storage method can be temporary storage or permanent storage.

[0150] The operating system manages and controls the various hardware devices and computer programs on the server to enable the processor to perform operations and processes on the data in the memory. It can be Windows Server, Netware, Unix, Linux, etc. In addition to computer programs capable of performing the fault detection method disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The data may include application update information and application developer information.

[0151] Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. The terminal may include, but is not limited to, a smartphone, tablet computer, laptop computer, or desktop computer.

[0152] Typically, the terminal in this embodiment includes a processor and a memory.

[0153] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor can be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and coprocessors. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which handles computational operations related to machine learning.

[0154] The memory may include one or more computer non-volatile storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory is used to store at least the following computer program, which, after being loaded and executed by the processor, is capable of implementing the relevant steps in the fault detection method executed by the terminal side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory may also include operating systems and data, and the storage method may be temporary or permanent storage. The operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, application update information.

[0155] In some embodiments, the terminal may further include a display screen, an input / output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0156] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than illustrated.

[0157] The following describes a non-volatile storage medium provided by an embodiment of the present invention. The non-volatile storage medium described below can be referred to in conjunction with other embodiments described herein.

[0158] A non-volatile storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements the fault detection method disclosed in the foregoing embodiments. The non-volatile storage medium is a computer-readable non-volatile storage medium, which, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon include an operating system, computer programs, and data, and the storage method can be temporary storage or permanent storage.

[0159] The following describes a computer program product provided by an embodiment of the present invention. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0160] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed fault detection method.

[0161] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the steps in any of the above embodiments.

[0162] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0163] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of non-volatile storage medium known in the art.

[0164] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only intended to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fault detection method, characterized in that, include: Collect the current operating parameters of the computing device; The currently used parameters include: current operating parameters and current structure parameters; Based on the historical usage information of the computing device, determine the baseline data information and fluctuation data information corresponding to the current usage parameters; The importance of the current usage parameters is determined based on their fault relevance and permissible fluctuation range. Based on the current usage parameters, the baseline data information, the fluctuation data information, and the importance level, the device health information is assessed and obtained. The risk of equipment failure is determined based on the equipment health information. The determination of the importance of the current usage parameters based on their fault relevance and permissible fluctuation range includes: The board structure safety threshold is obtained from the applicable technical specification document of the computing device; Calculate the endpoint difference of the permissible fluctuation range; The ratio of the endpoint difference to the board structure safety threshold is used as the dimensionless quantity of the fluctuation allowable range; The importance is determined by multiplying the fault correlation by a dimensionless quantity of the fluctuation tolerance range. The device health information is assessed based on the current usage parameters, the baseline data information, the fluctuation data information, and the importance level, including: The health information of the device is calculated using the first formula; the first formula is: ； Where H represents the device health information, w k P indicates the degree of importance. k Indicates the currently used parameter, μ k The reference data information, δ k This refers to the fluctuation data information.

2. The method according to claim 1, characterized in that, The current usage parameters of the collected computing device include: Collect at least one of the following parameters of the computing device: current temperature, current voltage, error count, clock skew, and video memory utilization, as the current operating parameter; The board deformation parameters of the computing device are collected and used as the current structural parameters.

3. The method according to claim 1, characterized in that, The historical usage information includes: each historical usage time and the corresponding current usage parameters; Accordingly, based on the historical usage information of the computing device, the reference data information corresponding to the current usage parameters is determined, including: calculating the reference data information based on the current usage parameters and preset first weight at the first usage time in each historical usage time, and the current usage parameters and preset second weight at the second usage time in each historical usage time; the first usage time is before the second usage time; Accordingly, based on the historical usage information of the computing device, the fluctuation data information corresponding to the current usage parameters is determined, including: determining a target time period based on each historical usage time; and using the dispersion of the current usage parameters within the target time period as the fluctuation data information.

4. The method according to claim 1, characterized in that, Also includes: Historical fault data and historical usage parameters are obtained by parsing the operation logs of the computing device; Determine the correlation coefficient between the historical fault data and the historical usage parameters; The correlation coefficient is used to represent the fault correlation.

5. The method according to claim 4, characterized in that, Determining the correlation coefficient between the historical fault data and the historical usage parameters includes: Select at least one of the preset correlation measurement methods to calculate the correlation coefficient between the historical fault data and the historical usage parameters.

6. The method according to claim 1, characterized in that, Also includes: The permissible range of fluctuations for the current usage parameters is obtained from the applicable technical specifications document of the computing device.

7. The method according to claim 1, characterized in that, Also includes: The value of the importance level is constrained by a preset limit range.

8. The method according to any one of claims 1 to 7, characterized in that, Determining equipment failure risk based on the aforementioned equipment health information includes: The instantaneous failure rate is calculated based on the preset basic failure rate, the failure acceleration factor, and the device health information. The cumulative value of the instantaneous failure rate over a preset time period represents the equipment failure risk.

9. The method according to claim 8, characterized in that, Based on the preset baseline failure rate, failure acceleration factor, and the device health information, the instantaneous failure rate is calculated, including: The instantaneous failure rate is calculated using the second formula; the second formula is: λ=λ0×exp[β×(100-H)]; Wherein, λ represents the instantaneous failure rate, λ0 represents the preset basic failure rate, β represents the failure acceleration factor, and H represents the device health information.

10. The method according to any one of claims 1 to 7, characterized in that, Also includes: The equipment failure risk is assessed using a preset assessment range; Based on the assessment results, a corresponding maintenance plan will be determined for the computing device.

11. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in any one of claims 1 to 10.

12. A non-volatile storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 10.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1 to 10.

Images