Hard disk failure detection method and electronic device

Abstract

This application discloses a hard drive fault detection method and electronic device. The method obtains hard drive fault detection data, hard drive usage data, and hard drive temperature data corresponding to the hard drive under test within a preset time period; performs periodic time series feature extraction on the hard drive usage data and hard drive temperature data to obtain the time series features corresponding to the hard drive usage data and hard drive temperature data, respectively; fuses the hard drive fault detection data and the time series features to obtain a hard drive feature vector; and obtains a hard drive fault detection result for the hard drive under test based on the hard drive feature vector and a target hard drive fault detection model, which is pre-trained. This solution enables the model to simultaneously capture the hard drive's internal fault data, external load, and temperature fluctuations, comprehensively quantifying the impact of usage and environmental factors on hard drive health, effectively improving the accuracy of hard drive fault diagnosis and enabling operations and maintenance personnel to perform timely hard drive maintenance.

Description

Technical Field

[0001] The present application relates to the technical field of hard disk detection, and in particular to a hard disk fault detection method and electronic equipment. Background Art

[0002] Currently, hard drive failure prediction is mostly based on the detection of the hard drive's Self-Monitoring, Analysis, and Reporting Technology (SMART) indicators. These indicators can be compared with thresholds, or machine learning models can be introduced for feature processing and detection. However, in actual hard drive use, hard drive failures are often closely related to usage and environmental factors. Therefore, if fault diagnosis is based solely on SMART indicators without considering the impact of usage and environmental factors, the fault diagnosis results may be biased. Summary of the Invention

[0003] The present application provides a hard disk fault detection method and electronic device to at least solve the problem that fault judgment is performed only through SMART indicators without considering the influence of usage and environmental factors, which may lead to certain deviations in the fault judgment results.

[0004] The present application provides a hard disk fault detection method, comprising: obtaining hard disk fault detection data, hard disk usage data, and hard disk temperature data corresponding to a hard disk to be tested within a preset time period;

[0005] Extract periodic time series features from the hard disk usage data and hard disk temperature data to obtain the time series features corresponding to the hard disk usage data and hard disk temperature data respectively;

[0006] Fuse hard drive fault detection data and time series features to obtain hard drive feature vectors;

[0007] A hard disk fault detection result of the hard disk to be tested is obtained according to the hard disk feature vector and the target hard disk fault detection model, where the target hard disk fault detection model is obtained in advance through model training.

[0008] The present application also provides a hard disk fault detection device, comprising: an acquisition module for acquiring hard disk fault detection data, hard disk usage data, and hard disk temperature data corresponding to the hard disk to be tested within a preset time period;

[0009] A feature extraction module is used to extract periodic time series features from the hard disk usage rate data and the hard disk temperature data to obtain the time series features corresponding to the hard disk usage rate data and the hard disk temperature data respectively;

[0010] Feature fusion module, used to fuse hard disk fault detection data and time series features to obtain hard disk feature vectors;

[0011] The processing module is used to obtain a hard disk fault detection result of the hard disk to be tested according to the hard disk feature vector and the target hard disk fault detection model, where the target hard disk fault detection model is obtained in advance through model training.

[0012] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned hard disk failure detection methods when executing the computer program.

[0013] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned hard disk failure detection methods are implemented.

[0014] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned hard disk failure detection methods when executed by a processor.

[0015] Through this application, hard drive fault detection data, hard drive usage data, and hard drive temperature data corresponding to the hard drive under test within a preset time period are obtained; periodic time series feature extraction is performed on the hard drive usage data and hard drive temperature data to obtain the time series features corresponding to the hard drive usage data and hard drive temperature data, respectively; the hard drive fault detection data and the time series features are fused to obtain a hard drive feature vector; and the hard drive fault detection result of the hard drive under test is obtained based on the hard drive feature vector and the target hard drive fault detection model, which is pre-trained. In this solution, in addition to traditional hard drive fault detection data, hard drive usage data and hard drive temperature data are also obtained. After time series processing of the above data, hard drive faults are detected using a pre-trained model. This allows the model to simultaneously capture the hard drive's internal fault data, external load, and temperature fluctuations, comprehensively quantifying the impact of usage and environmental factors on hard drive health, effectively improving the accuracy of hard drive fault diagnosis, and allowing operation and maintenance personnel to perform hard drive maintenance in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 A process of a hard disk fault detection method provided in an embodiment of the present application Figure 1 ;

[0018] Figure 2A process of a hard disk fault detection method provided in an embodiment of the present application Figure 2 ;

[0019] Figure 3 A structural diagram of a hard disk failure detection device provided in an embodiment of the present application;

[0020] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0023] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being more preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0024] A hard drive's Self-Monitoring, Analysis, and Reporting Technology (SMART) metrics are important tools for assessing drive health and predicting potential failures. SMART metrics monitor drive reliability, predict failures, and perform various self-tests, helping users identify potential issues and take action to avoid data loss and system crashes.

[0025] Currently, hard drive failure detection is mainly based on the comparison of SMART indicators and thresholds, or achieved through traditional machine learning methods.

[0026] Among them, directly setting static thresholds for SMART indicators and issuing alarms when exceeding the thresholds is simple to implement and has low overhead. However, once the thresholds are set, it is difficult to dynamically adjust them according to actual load, equipment aging, or environmental changes, which can easily lead to false alarms or missed alarms, as well as an alarm when a single indicator exceeds the threshold. It is impossible to capture the interactive relationship between multiple SMART indicators or environmental factors.

[0027] In recent years, algorithms such as XGBoost, LightGBM, and CatBoost have been widely used for hard drive failure prediction. They can better handle nonlinear relationships between features. Alternatively, the SMART indicator can be input into a fully connected neural network through a multilayer perceptron (MLP) to predict failure probability. Alternatively, recurrent neural networks (RNN) and long short-term memory networks (LSTM) can be used to model time series features such as SMART to capture temporal dependencies and trends. However, most of the above methods only use the SMART indicator as input and have high requirements for massive labeled data. Hard drive failure samples are usually scarce and unbalanced, making it difficult to intuitively reveal the physical meaning of each feature. This makes it difficult for operation and maintenance personnel to trust their prediction results and is not conducive to failure mechanism analysis.

[0028] To address the above-mentioned technical problems, an embodiment of the present application provides a hard drive fault detection method and electronic device. These methods obtain hard drive fault detection data, hard drive usage data, and hard drive temperature data corresponding to a hard drive under test within a preset time period; perform periodic time series feature extraction on the hard drive usage data and hard drive temperature data to obtain time series features corresponding to the hard drive usage data and hard drive temperature data, respectively; fuse the hard drive fault detection data and the time series features to obtain a hard drive feature vector; and obtain a hard drive fault detection result for the hard drive under test based on the hard drive feature vector and a target hard drive fault detection model, which is pre-trained. In this solution, in addition to traditional hard drive fault detection data, hard drive usage data and hard drive temperature data are also obtained. After time series processing of these data, hard drive faults are detected using a pre-trained model. This allows the model to simultaneously capture the hard drive's internal fault data, external load, and temperature fluctuations, comprehensively quantifying the impact of usage and environmental factors on hard drive health, effectively improving the accuracy of hard drive fault diagnosis and enabling operations and maintenance personnel to perform hard drive maintenance in a timely manner.

[0029] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0030] like Figure 1 As shown, Figure 1 This is a flowchart of a hard disk failure detection method provided in an embodiment of the present application. The method may include the following steps:

[0031] 101. Obtain hard disk fault detection data, hard disk usage data, and hard disk temperature data corresponding to the hard disk to be tested within a preset time period.

[0032] In an embodiment of the present application, the preset time period can be a custom data collection period, such as one day, one week, 15 hours, etc., without specific limitation. The hard disk usage data can be represented by the percentage of the hard disk's I / O request busy time to the total time, that is, the proportion of time the hard disk has I / O operations. Specifically, it can be obtained through a hard disk I / O monitoring tool (such as iostat in the Linux system), that is, the value of the %util column output by iostat-x in the Linux system is obtained by checking the value of the hard disk usage data. The value range of 0-100%, 0 means that the hard disk is currently completely idle, and 100% means that the hard disk is currently fully saturated and has been continuously processing requests. The hard disk temperature data can be obtained directly through the hard disk's built-in temperature sensor or an external temperature monitoring tool (such as lm-sensors, Sysctl, etc.).

[0033] During the operation of the hard disk, the hard disk usage data and hard disk temperature data may change in real time. Therefore, within a preset time period, data can be collected according to a certain period. For example, within a day, the current usage data and temperature data are collected once every hour, or once every half hour, etc. In other words, the hard disk usage data and hard disk temperature data corresponding to the hard disk to be tested at multiple times within the preset time period can be obtained.

[0034] It should be noted that hard disk fault detection data can be detection data used to indicate various possible faults that may occur during the operation of the hard disk, that is, it can be SMART indicators, which can specifically include: reallocated sector count (Reallocated Sectors Count), spindle spin retry count (Spin Retry Count), end-to-end error detection count (End-to-End Error detection count), reported uncorrectable errors (Reported Uncorrect), current pending sector count (Current Pending Sector Count), offline uncorrectable errors (Offline Uncorrectable Sector Count), media wear index (Media Wearout Indicator), etc. The above data can be directly read through standard hard disk monitoring tools (such as smartctl in Linux system).

[0035] In some embodiments, after obtaining the hard disk fault detection data, hard disk usage data and hard disk temperature data of the hard disk to be tested, the hard disk fault detection data, hard disk usage data and hard disk temperature data can be preprocessed. The preprocessing may include: data cleaning and denoising, normalization processing, timing processing, etc.

[0036] Among them, data cleaning and denoising: can remove missing value data. If the data of SMART indicators, usage rate, temperature, etc. cannot be obtained, they will be deleted uniformly; outlier detection, through statistical methods (such as Z-Score) to detect and remove outliers; any abnormal data in the hard disk SMART indicators (such as extremely high or low values) will be deleted uniformly.

[0037] Normalization: Standardize all hard drive SMART indicators, usage data, and temperature data. Convert data of different magnitudes (such as temperature in degrees Celsius, usage percentage, and SMART indicator values) to a unified scale for normalization. This is achieved using the following formula:

[0038]

[0039] in, The original data (the original hard disk SMART indicators, usage data and temperature data) Normalized data (data obtained by normalizing the original hard disk SMART indicators, usage data, and temperature data). The minimum value of the data (the minimum value of the originally obtained hard disk SMART indicators, the minimum value of the originally obtained usage data, and the minimum value of the originally obtained temperature data). The maximum value of the data (the maximum value of the originally obtained hard disk SMART indicators, the maximum value of the originally obtained usage data, and the maximum value of the originally obtained temperature data).

[0040] Time series processing: For time series data such as hard disk usage and hard disk temperature, the data is divided into hourly time windows in chronological order to facilitate subsequent periodic analysis.

[0041] 102. Perform periodic time series feature extraction on the hard disk usage rate data and the hard disk temperature data to obtain time series features corresponding to the hard disk usage rate data and the hard disk temperature data, respectively.

[0042] In an embodiment of the present application, since the hard disk usage data and the hard disk temperature data will change in real time during the hard disk operation process, and there may be certain periodic change patterns, the hard disk usage data and the hard disk temperature data can be periodically analyzed respectively to determine the periodic change trends of the hard disk usage data and the hard disk temperature data, thereby extracting periodic time series features.

[0043] It should be noted that the periodic time series features of the hard disk usage data and hard disk temperature data can be extracted through Fourier transform. Fourier transform is a common method for analyzing periodic signals. It converts time domain features into frequency domain features, thereby extracting the corresponding time series features.

[0044] It should be noted that periodic time series feature extraction can be performed on the hard disk usage data to obtain the time series features corresponding to the hard disk usage data, and then periodic time series feature extraction can be performed on the hard disk temperature data to obtain the time series features corresponding to the hard disk temperature data.

[0045] It should be noted that, since the acquired hard disk usage rate data and hard disk temperature data may be data corresponding to multiple moments within a preset time period, when performing periodic time series feature extraction, periodic time series feature extraction may be performed on the hard disk usage rate data and hard disk temperature data corresponding to all moments within the preset time period. For example, if hard disk usage rate data and hard disk temperature data corresponding to 10 moments are collected in one day, periodic time series feature extraction may be performed on the 10 hard disk usage rate data, and periodic time series feature extraction may be performed on the 10 hard disk temperature data, thereby obtaining a time series feature corresponding to the hard disk usage rate data and a time series feature corresponding to the hard disk temperature data; of course, it is also possible to It is possible to first divide multiple moments within a preset time period, and then perform periodic time series feature extraction on the hard disk usage data and hard disk temperature data corresponding to the divided multiple groups of moments. For example, if hard disk usage data and hard disk temperature data corresponding to 50 moments are collected in one day, then the 50 collection moments are first divided into five groups in sequence, and each group includes 10 consecutive collection moments. Periodic time series feature extraction is performed on the 10 hard disk usage data in each group, and periodic time series feature extraction is performed on the 10 hard disk temperature data in each group, so that five time series features corresponding to the hard disk usage data and five time series features corresponding to the hard disk temperature data can be obtained; this is not specifically limited in the embodiments of the present application.

[0046] 103. Fuse the hard disk fault detection data and the time series features to obtain a hard disk feature vector.

[0047] In an embodiment of the present application, after obtaining the timing characteristics corresponding to the hard disk usage data and the hard disk temperature data respectively, the timing characteristics and the hard disk usage data can be fused to obtain a hard disk feature vector. That is to say, the hard disk feature vector can include: hard disk fault detection data within a preset time period, the timing characteristics corresponding to the hard disk usage data, and the timing characteristics corresponding to the hard disk temperature data.

[0048] 104. Obtain a hard disk fault detection result of the hard disk to be tested according to the hard disk feature vector and the target hard disk fault detection model.

[0049] In an embodiment of the present application, after obtaining the hard disk feature vector, the hard disk feature vector can be input into the target hard disk fault detection model, thereby obtaining the hard disk fault detection result of the hard disk to be tested output by the target hard disk fault detection model. The hard disk fault detection result can be used to indicate whether the hard disk to be tested has a fault.

[0050] It should be noted that the target hard disk fault detection model can be obtained in advance through model training. During the training of the target hard disk fault detection model, the fault detection data, hard disk usage data and hard disk temperature data corresponding to a large number of faulty hard disks and normal hard disks can be processed, and then the model can be trained using preset fault marks and normal marks.

[0051] In an embodiment of the present application, in addition to traditional hard disk fault detection data, hard disk usage data and hard disk temperature data are also obtained. After the above data is time-series processed, hard disk faults are detected through a pre-trained model, so that the model can simultaneously capture the hard disk's internal fault data, external load and temperature fluctuations, and comprehensively quantify the impact of usage and environmental factors on hard disk health, effectively improving the accuracy of hard disk fault diagnosis, so that operation and maintenance personnel can maintain the hard disk in a timely manner.

[0052] like Figure 2 As shown, Figure 2 Another flowchart of a hard disk failure detection method provided in an embodiment of the present application, the method may include the following steps:

[0053] 201. Obtain training sample data at multiple moments.

[0054] In an embodiment of the present application, a large amount of training sample data is required when pre-training the model. Since the target hard disk fault detection model finally obtained is to detect whether the hard disk is faulty, it is necessary to obtain sample data of normal hard disks and sample data of faulty hard disks. In addition, in actual applications, the fault detection data, usage data and temperature data of the hard disk are taken into consideration. Therefore, in the model training process, it is necessary to learn the correlation between the fault detection data, usage data and temperature data and whether the hard disk is faulty. That is to say, the training sample data may include: hard disk fault detection training data, hard disk usage training data, hard disk temperature training data and hard disk fault mark. The hard disk fault mark is used to indicate whether the hard disk has a fault. The sample of the faulty hard disk can be marked as 1, and the sample of the normal hard disk can be marked as 0. The hard disk fault mark can be derived from the real operation and maintenance fault record of the hard disk.

[0055] It should be noted that for each training hard disk, training sample data at multiple time points can be obtained. That is to say, each training hard disk obtains hard disk fault detection training data corresponding to multiple time points, hard disk usage training data corresponding to multiple time points, and hard disk temperature training data corresponding to multiple time points, as well as a hard disk fault mark for the training hard disk.

[0056] In some embodiments, the training sample data can be encapsulated into the DMatrix data format of XGBoost to improve computing performance.

[0057] 202. Train a preset model using training sample data at multiple moments to obtain a target hard disk fault detection model.

[0058] In an embodiment of the present application, a preset model can be trained using training sample data at multiple moments to obtain a target hard disk fault detection model.

[0059] In some embodiments, since the usage data and temperature data of the hard disk may have certain periodic change patterns during operation, in order to learn their change patterns, the preset model is trained with training sample data at multiple moments to obtain the target hard disk fault detection model, which may specifically include: extracting periodic time series features from the hard disk usage training data and hard disk temperature training data at multiple moments to obtain the time series training features corresponding to the hard disk usage training data and the hard disk temperature training data respectively; fusing the hard disk fault detection training data and the time series training features to obtain a hard disk training feature vector; and training the preset model with the hard disk training feature vector and the hard disk fault mark to obtain the target hard disk fault detection model.

[0060] It should be noted that, periodic time series feature extraction can be performed on the hard disk usage training data at multiple moments of each training hard disk to obtain the time series training features corresponding to the hard disk usage training data at multiple moments; periodic time series feature extraction can be performed on the hard disk temperature training data at multiple moments of each training hard disk to obtain the time series training features corresponding to the hard disk temperature training data at multiple moments; then the hard disk fault detection training data corresponding to each training hard disk, the time series training features corresponding to the hard disk usage training data at multiple moments and the time series training features corresponding to the hard disk temperature training data at multiple moments are fused to obtain the hard disk training feature vector corresponding to each training hard disk; at this time, each training hard disk will correspond to a hard disk training feature vector and a hard disk fault mark, and the hard disk training feature vector and the hard disk fault mark are associated. By training the preset model through the hard disk training feature vectors and hard disk fault marks of all training hard disks, the target hard disk fault detection model can be obtained.

[0061] During the training process, the preset model is trained with the hard disk training feature vector as input and the hard disk fault mark as output.

[0062] In some embodiments, when performing periodic time series feature extraction, since the hard disk usage training data and the hard disk temperature training data can be considered as discrete time domain data, in order to better understand their laws, frequency domain processing can be performed, and periodic time series feature extraction is performed on the hard disk usage training data and the hard disk temperature training data at multiple moments to obtain the time series training features corresponding to the hard disk usage training data and the hard disk temperature training data, respectively. Specifically, it can include: performing discrete Fourier transform on the hard disk usage training data and the hard disk temperature training data at multiple moments to obtain the frequency domain signals corresponding to the hard disk usage training data and the hard disk temperature training data, respectively; and determining the time series training features corresponding to the hard disk usage training data and the hard disk temperature training data, respectively, based on the frequency domain signals.

[0063] It should be noted that discrete Fourier transform is a common method for analyzing periodic signals. For hard disk usage rate training data and hard disk temperature training data, discrete Fourier transform can analyze the periodic characteristics between their data and the periodic trend of data fluctuations.

[0064] Discrete Fourier transforms can be performed on the hard disk usage training data and hard disk temperature training data. The hard disk usage training data at multiple moments is represented as a time series x(t), where t represents the moment and x(t) represents the value of the hard disk usage training data at that moment. For N hard disk usage training data x(t) collected at N moments, where t = 0, 1, 2, ..., N-1, the discrete Fourier transform can be expressed using the following formula:

[0065]

[0066] Among them, x(t) represents the t-th hard disk usage training data, represents the kth frequency domain signal after discrete Fourier transform, N is the number of samples of hard disk usage training data, and j represents the imaginary unit.

[0067] This discrete Fourier transform formula decomposes the time-domain signal x(t) into a series of sinusoidal waves of different frequencies. The resulting X(k) represents the amplitude and phase information of each frequency component in the signal. The modulus |X(k)| of the frequency-domain signal X(k) represents the amplitude of that frequency component, while the phase arg(X(k)) represents the phase of that frequency component.

[0068] Similarly, the hard disk temperature training data at multiple moments is also processed using the above discrete Fourier transform formula to obtain the frequency domain signals of the hard disk usage training data and the frequency domain signals of the hard disk temperature training data. Based on the frequency domain signals, the time series training features of the hard disk usage training data and the hard disk temperature training data can be extracted.

[0069] In some embodiments, discrete Fourier transform can be used to convert data acquired at multiple moments into frequency domain signals, thereby facilitating the capture of periodic features between data and periodic trends in data fluctuations, and improving the accuracy of model training and fault diagnosis.

[0070] In some embodiments, the timing training features corresponding to the hard disk usage training data and the hard disk temperature training data are determined based on the frequency domain signal, which may specifically include: determining the maximum amplitude and the minimum amplitude based on the frequency domain signal; determining the amplitude data based on the maximum amplitude and the minimum amplitude; determining the periodic data based on the moment corresponding to the maximum amplitude and the moment corresponding to the minimum amplitude; and obtaining the timing training features based on the amplitude data and the periodic data.

[0071] In the frequency domain signal after discrete Fourier transform, the frequency component with the largest amplitude can be determined. Usually, this frequency component can correspond to the main period of the calculated data. If the frequency is f, then the periodic data is T=1 / f; the amplitude of each frequency component represents the contribution of the frequency component in the signal. The amplitude of the main frequency is usually relatively large, indicating that it is the most important periodic component in the signal. Therefore, it can be seen that it is necessary to determine the maximum amplitude and minimum amplitude in the frequency domain signal, and then determine the difference between the maximum amplitude and the minimum amplitude as the amplitude data; the difference between the moment corresponding to the maximum amplitude and the moment corresponding to the minimum amplitude is determined as the periodic data, and the amplitude data and periodic data can be considered as time series training features.

[0072] By determining the amplitudes of the frequency domain signals of the hard disk usage training data and the frequency domain signals of the hard disk temperature training data respectively, the first maximum amplitude and the first minimum amplitude in the frequency domain signal of the hard disk usage training data can be determined, and the difference between the first maximum amplitude and the first minimum amplitude is determined as the amplitude data of the hard disk usage training data, and the difference between the first moment corresponding to the first maximum amplitude and the second moment corresponding to the first minimum amplitude is determined as the period data of the hard disk usage training data, thereby obtaining the amplitude data and period data of the hard disk usage training data, which constitute the time series training features of the hard disk usage training data; similarly for the hard disk temperature training data, the second maximum amplitude and the second minimum amplitude in the frequency domain signal of the hard disk temperature training data are determined, and the difference between the second maximum amplitude and the second minimum amplitude is determined as the amplitude data of the hard disk temperature training data, and the difference between the third moment corresponding to the second maximum amplitude and the fourth moment corresponding to the second minimum amplitude is determined as the period data of the hard disk temperature training data, thereby obtaining the amplitude data and period data of the hard disk temperature training data, which constitute the time series training features of the hard disk temperature training data.

[0073] In some embodiments, during the model training process, the trained model usually needs to be verified and continuously optimized. Therefore, the large amount of sample data obtained will not all participate in the model training, and a part of them will be used for model verification. The preset model is trained with the training sample data at multiple times to obtain the target hard disk fault detection model. Specifically, it can include: dividing the training sample data at multiple times into a training data set and a verification data set; training the preset model with the data set, and verifying the trained model with the verification data set to obtain the target hard disk fault detection model.

[0074] It should be noted that, generally speaking, the training data set will account for the majority of the training sample data, while the validation data set is a minority, for example: the training data set is 80%, the validation data set is 20%; or, the training data set is 70%, the validation data set is 30%, etc. The preset model is trained with the training data set to obtain a trained model, and then the trained model is verified with the validation data set. Specifically, the hard disk training feature vector obtained by processing the hard disk fault detection training data, hard disk usage training data, and hard disk temperature training data in the validation data set can be input into the trained model to obtain the detection result output by the trained model. The detection result is compared with the hard disk fault mark in the validation data set. If they are consistent, it means that the trained model has detected correctly, and the trained model can be determined as the target hard disk fault detection model. If there are many inconsistencies, it means that the trained model still has errors and needs to be further optimized, and the trained model will continue to be trained.

[0075] In some embodiments, by dividing the training data set and the validation data set, it can be ensured that the trained model is data-validated, effectively improving the accuracy of model training and the accuracy of hard disk fault diagnosis.

[0076] In some embodiments, hyperparameters can be set during model training. These hyperparameters may include: the maximum tree depth (max_depth) to control model complexity; the learning rate (learning_rate) to control weight reduction for each tree; the sample sampling ratio (subsample) when training each tree; the feature sampling ratio (colsample_bytree) when training each tree; the minimum loss drop required for node splitting (gamma); and the regularization coefficients (lambda, alpha). These hyperparameters can be used to constrain the model during training to improve the accuracy of the model's generalization capabilities.

[0077] In some embodiments, the model training process will continuously improve the fitting accuracy. However, overfitting will reduce the generalization ability of the model and consume a lot of resources. Therefore, the model training process can be restricted. The trained model is verified through a verification data set to obtain a target hard disk fault detection model. Specifically, it can include: verifying the trained model through a verification data set to obtain an evaluation index of the trained model; if the evaluation index meets the early stopping hyperparameter condition, the trained model is determined as the target hard disk fault detection model.

[0078] During model validation, you can call xgb.train() to monitor eval_metri metrics (such as AUC and logloss) and compare them to the early stopping hyperparameter conditions. If the early stopping hyperparameter conditions are met, the model is considered trained and can be used as the target hard drive fault detection model. The early stopping hyperparameter condition can be the early stopping hyperparameter threshold (early_stopping_rounds) determined based on historical experience.

[0079] In some embodiments, the early stopping hyperparameter condition is determined based on the loss of the trained model, so that the change in the evaluation index of the trained model is less than a preset threshold.

[0080] During the training process of the model, the loss value will gradually decrease and the convergence curve will gradually flatten out. At this time, it can be considered that the model performance has reached the optimal level and the model training can be stopped. In order to prevent the model from overfitting, the hyperparameters and early stopping strategy can be combined to set the early stopping hyperparameter conditions to limit the performance of the model. During the cross-training and verification process of the model, the hyperparameters of the model will continue to change, and the performance of the model will gradually be optimized. Of course, after optimization to a certain extent, it will tend to flatten out. In other words, the change in the evaluation index of the model will become smaller and smaller until it is less than the preset threshold. Therefore, the early stopping hyperparameter condition can be determined according to the loss value of the model. In order to make the change in the evaluation index of the model less than the preset threshold, the performance is optimized.

[0081] In some embodiments, by setting hyperparameters and early stopping conditions, it is possible to ensure that model training does not overfit and the generalization ability of the model is guaranteed.

[0082] In some embodiments, the evaluation indicators may also include ROC AUC, precision, recall, etc., and ROC curves and Precision-Recall curves may be drawn to evaluate the detection capability of the model.

[0083] 203. Obtain hard disk fault detection data, hard disk usage data, and hard disk temperature data corresponding to the hard disk to be tested within a preset time period.

[0084] 204. Perform periodic time series feature extraction on the hard disk usage rate data and the hard disk temperature data to obtain time series features corresponding to the hard disk usage rate data and the hard disk temperature data, respectively.

[0085] 205. Fuse the hard disk fault detection data and the time series features to obtain a hard disk feature vector.

[0086] 206. Obtain a hard disk fault detection result of the hard disk to be tested according to the hard disk feature vector and the target hard disk fault detection model.

[0087] In the embodiment of the present application, for the description of steps 203 to 206 , please refer to the detailed description of steps 101 to 104 in the above embodiment, which will not be repeated in the embodiment of the present application.

[0088] 207. When the hard disk fault detection result indicates that the hard disk to be tested has a fault, output a fault warning message.

[0089] In an embodiment of the present application, the hard disk fault detection result can be used to indicate whether the hard disk to be tested has a fault. Therefore, if the hard disk fault detection result indicates that the hard disk to be tested has a fault, then in order to enable the staff to repair the hard disk in time, a fault warning information can be output. The fault warning information is used to indicate that the hard disk to be tested may have a fault.

[0090] In some embodiments, fault warning information is output based on the hard disk fault detection results, which allows operation and maintenance personnel to have sufficient time to locate and intervene in the fault, and to maintain the faulty hard disk in a timely manner, significantly reducing data loss and system downtime caused by hard disk failure, thereby improving business continuity and reliability.

[0091] In some embodiments, in addition to detecting whether the hard disk to be tested is currently faulty, the target hard disk fault detection model can also predict whether the hard disk to be tested will fail in the future, because the hard disk usage data and hard disk temperature data are periodic time series data and may have certain patterns. The target hard disk fault detection model will learn the time series characteristics of the hard disk usage data and hard disk temperature data during the training process. Therefore, in actual applications, after the hard disk feature vector is input into the target hard disk fault detection model, the target hard disk fault detection model can analyze the time series patterns of the current hard disk usage data and hard disk temperature data contained in the hard disk feature vector and make predictions, so that fault warnings can be issued in advance for hard disks that may fail. The warning information can be notified to relevant operation and maintenance personnel via email, text message or management platform, so that the operation and maintenance personnel have sufficient time to locate and intervene in the fault, significantly reducing data loss and system downtime caused by hard disk failure, thereby improving business continuity and reliability.

[0092] 208. Obtain actual failure results of the hard disk under test.

[0093] In an embodiment of the present application, since the hard disk fault detection result is obtained through the target hard disk fault detection model and is not the actual detection result, the hard disk fault detection result may not necessarily be accurate. In this case, staff are required to perform manual detection on the hard disk to be tested to obtain the actual fault result, which is used to indicate whether the hard disk to be tested actually has a fault.

[0094] 209. Update the hard disk failure mark of the hard disk to be tested according to the actual failure result.

[0095] In an embodiment of the present application, after obtaining the hard disk fault detection result of the hard disk to be tested through the target hard disk fault detection model, a hard disk fault mark can be set for the hard disk to be tested. After determining the actual fault result, the hard disk fault mark of the hard disk to be tested can be updated according to the actual fault result. If it is faulty, it is marked as 1, and if it is normal, it is marked as 0.

[0096] 210. Optimize the target hard disk fault detection model based on the hard disk fault detection data, hard disk usage data, hard disk temperature data, and the updated hard disk fault mark.

[0097] In the embodiment of the present application, since the actual fault result is obtained by manual detection by the staff, the accuracy of the actual fault result will be very high, so the target hard disk fault detection model can be optimized according to the actual fault result. Specifically, the target hard disk fault detection model can be optimized according to the hard disk fault detection data, hard disk usage data, hard disk temperature data and updated hard disk fault mark of the hard disk to be tested.

[0098] It should be noted that the optimization of the target hard disk fault detection model can be carried out according to a certain period, for example: every month, based on the hard disk fault detection data, hard disk usage data, hard disk temperature data and updated hard disk fault marks of all hard disks within this month, the target hard disk fault detection model is incrementally optimized or fully optimized.

[0099] In some embodiments, by periodically optimizing the model, a feedback mechanism and a model optimization cycle are implemented to continuously improve the accuracy of model diagnosis, while also improving the adaptability and predictive capabilities of the model in different hard disks and environments.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0101] like Figure 3 As shown, an embodiment of the present application further provides a hard disk fault detection device, which may include:

[0102] An acquisition module 301 is configured to acquire hard disk fault detection data, hard disk usage data, and hard disk temperature data corresponding to the hard disk to be tested within a preset time period;

[0103] The feature extraction module 302 is used to extract periodic time series features from the hard disk usage data and the hard disk temperature data to obtain time series features corresponding to the hard disk usage data and the hard disk temperature data respectively;

[0104] Feature fusion module 303, used to fuse hard disk fault detection data and time series features to obtain a hard disk feature vector;

[0105] The processing module 304 is used to obtain a hard disk fault detection result of the hard disk to be tested according to the hard disk feature vector and the target hard disk fault detection model. The target hard disk fault detection model is obtained in advance through model training.

[0106] In some embodiments, the processing module 304 is further configured to output fault warning information when the hard disk fault detection result indicates that the hard disk to be tested has a fault.

[0107] In some embodiments, the acquisition module 301 is further configured to acquire training sample data at multiple moments, the training sample data including: hard disk fault detection training data, hard disk usage training data, hard disk temperature training data, and hard disk fault flags, where the hard disk fault flags are used to indicate whether a hard disk has a fault.

[0108] The processing module 304 is further configured to train a preset model using training sample data at multiple moments to obtain a target hard disk fault detection model.

[0109] In some embodiments, the feature extraction module 302 is specifically configured to extract periodic time series features from the hard disk usage rate training data and the hard disk temperature training data at multiple moments, and obtain time series training features corresponding to the hard disk usage rate training data and the hard disk temperature training data, respectively.

[0110] The feature fusion module 303 is specifically used to fuse the hard disk fault detection training data and the time series training features to obtain a hard disk training feature vector;

[0111] The processing module 304 is specifically configured to train a preset model using hard disk training feature vectors and hard disk fault markers to obtain a target hard disk fault detection model.

[0112] In some embodiments, the processing module 304 is specifically configured to perform discrete Fourier transform on the hard disk usage rate training data and the hard disk temperature training data at multiple moments to obtain frequency domain signals corresponding to the hard disk usage rate training data and the hard disk temperature training data, respectively.

[0113] The processing module 304 is specifically configured to determine the time series training features corresponding to the hard disk usage rate training data and the hard disk temperature training data, respectively, based on the frequency domain signal.

[0114] In some embodiments, the processing module 304 is specifically configured to determine a maximum amplitude and a minimum amplitude based on the frequency domain signal;

[0115] The processing module 304 is specifically configured to determine amplitude data according to the maximum amplitude and the minimum amplitude;

[0116] The processing module 304 is specifically configured to determine periodic data based on the time corresponding to the maximum amplitude and the time corresponding to the minimum amplitude;

[0117] The processing module 304 is specifically configured to obtain time series training features based on the amplitude data and the period data.

[0118] In some embodiments, the processing module 304 is specifically configured to divide the training sample data at multiple moments into a training data set and a validation data set;

[0119] The processing module 304 is specifically configured to train a preset model using a data set, and verify the trained model using a verification data set, to obtain a target hard disk fault detection model.

[0120] In some embodiments, the processing module 304 is specifically configured to verify the trained model using a validation dataset to obtain evaluation indicators of the trained model;

[0121] The processing module 304 is specifically configured to determine the trained model as the target hard disk fault detection model if the evaluation index satisfies the early stopping super-parameter condition.

[0122] In some embodiments, the early stopping hyperparameter condition is determined based on the loss of the trained model, so that the change in the evaluation index of the trained model is less than a preset threshold.

[0123] In some embodiments, the acquisition module 301 is further configured to obtain an actual failure result of the hard disk under test;

[0124] The processing module 304 is further configured to update the hard disk fault flag of the hard disk to be tested according to the actual fault result;

[0125] The processing module 304 is further configured to optimize the target hard disk fault detection model according to the hard disk fault detection data, the hard disk usage rate data, the hard disk temperature data and the updated hard disk fault mark.

[0126] In the embodiments of the present application, the description of the features in the embodiments corresponding to the hard disk fault detection device can refer to the relevant description of the embodiments corresponding to the hard disk fault detection method, and will not be repeated here.

[0127] like Figure 4 As shown, an embodiment of the present application further provides an electronic device, including a memory 401 and a processor 402, wherein the memory 401 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above-mentioned hard disk fault detection method embodiments.

[0128] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned hard disk failure detection method embodiments when running.

[0129] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0130] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned hard disk failure detection method embodiments are implemented.

[0131] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned hard disk failure detection method embodiments.

[0132] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] The above is a detailed introduction to the process monitoring of a storage system provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core ideas of this application. It should be pointed out that, for those skilled in the art, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A hard disk failure detection method, characterized in that: The method comprises: Obtain hard disk fault detection data, hard disk usage data, and hard disk temperature data corresponding to the hard disk to be tested within a preset time period; Extracting periodic time series features from the hard disk usage data and the hard disk temperature data to obtain time series features corresponding to the hard disk usage data and the hard disk temperature data respectively; Fusing the hard disk fault detection data and the time series features to obtain a hard disk feature vector; Acquire training sample data at multiple moments, the training sample data including: hard disk fault detection training data, hard disk usage training data, hard disk temperature training data, and hard disk fault flags, the hard disk fault flags being used to indicate whether a hard disk has a fault; The preset model is trained using the training sample data at the multiple moments to obtain a target hard disk fault detection model; the target hard disk fault detection model is determined based on the time series training features obtained after periodic time series feature extraction of the training sample data at the multiple moments, and the time series training features are determined based on the amplitude data and period data corresponding to the frequency domain signal obtained after discrete Fourier transform of the training sample data at the multiple moments; the amplitude data is the difference between the maximum amplitude and the minimum amplitude in the frequency domain signal, and is used to represent the contribution of the frequency component in the frequency domain signal; the period data is the difference between the moments corresponding to the maximum amplitude and the minimum amplitude, and is used to represent the frequency of the frequency domain signal; A hard disk fault detection result of the hard disk to be tested is obtained according to the hard disk feature vector and the target hard disk fault detection model, wherein the target hard disk fault detection model is obtained in advance through model training.

2. The method according to claim 1, characterized in that The method of training a preset model using the training sample data at the plurality of moments to obtain the target hard disk fault detection model includes: Performing periodic time series feature extraction on the hard disk usage rate training data and the hard disk temperature training data at the multiple moments to obtain time series training features corresponding to the hard disk usage rate training data and the hard disk temperature training data respectively; Fusing the hard disk fault detection training data and the time series training features to obtain a hard disk training feature vector; The preset model is trained using the hard disk training feature vector and the hard disk fault mark to obtain the target hard disk fault detection model.

3. The method according to claim 2, characterized in that The periodic time series feature extraction is performed on the hard disk usage rate training data and the hard disk temperature training data at the multiple moments to obtain the time series training features corresponding to the hard disk usage rate training data and the hard disk temperature training data, respectively, including: Performing discrete Fourier transform on the hard disk usage rate training data and the hard disk temperature training data at the multiple moments to obtain frequency domain signals corresponding to the hard disk usage rate training data and the hard disk temperature training data respectively; Determine, according to the frequency domain signal, the time series training features corresponding to the hard disk usage rate training data and the hard disk temperature training data, respectively.

4. The method according to claim 3, characterized in that Determining, based on the frequency domain signal, the time series training features corresponding to the hard disk usage rate training data and the hard disk temperature training data, respectively, includes: Determining a maximum amplitude and a minimum amplitude according to the frequency domain signal; determining amplitude data according to the maximum amplitude and the minimum amplitude; determining periodic data according to a time corresponding to the maximum amplitude and a time corresponding to the minimum amplitude; The time series training feature is obtained according to the amplitude data and the period data.

5. The method according to claim 1, wherein The method of training a preset model using the training sample data at the plurality of moments to obtain the target hard disk fault detection model includes: Dividing the training sample data at the multiple moments into a training data set and a validation data set; The preset model is trained using the data set, and the trained model is verified using the verification data set to obtain the target hard disk fault detection model.

6. The method according to claim 5, characterized in that The method of verifying the trained model using the verification data set to obtain the target hard disk fault detection model includes: Validating the trained model using the validation data set to obtain evaluation indicators for the trained model; If the evaluation index satisfies the early stopping super-parameter condition, the trained model is determined as the target hard disk fault detection model.

7. The method according to claim 6, characterized in that The early stopping hyperparameter condition is determined based on the loss of the trained model, so that the change in the evaluation index of the trained model is less than a preset threshold.

8. The method according to claim 1, characterized in that After obtaining the hard disk fault detection result of the hard disk to be tested according to the hard disk feature vector and the target hard disk fault detection model, the method further includes: Obtaining actual failure results of the hard disk to be tested; updating the hard disk fault flag of the hard disk to be tested according to the actual fault result; The target hard disk fault detection model is optimized according to the hard disk fault detection data, the hard disk usage rate data, the hard disk temperature data and the updated hard disk fault mark.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the hard disk failure detection method according to any one of claims 1 to 8 when executing the computer program.

Images