Hard disk fault detection method and device
By combining the target supervision learning integration module and the target gated cycle module, the problem of misjudgment of hard disk health status in the prior art is solved, and the accuracy of fault prediction and the accuracy of hard disk failure detection are improved.
Patent Information
- Application Number
- CN202410467867.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-18
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-04-18
AI Technical Summary
Existing hard disk failure detection technology can easily cause misjudgment of the hard disk health status, resulting in low prediction accuracy.
The combination model of the target supervised learning integration module and the target gating cycle module is adopted to select and evaluate the hard disk attribute data, filter out the high-important features through the multi-model fusion module, and use the GRU model to detect faults to output hard disk failures. Test results.
By filtering high-quality hard disk selection data, the impact of data dimensions is reduced, the accuracy of fault prediction is improved, and the misjudgment rate is reduced.
Smart Images

Figure CN118193273B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer storage technology, and in particular to a hard disk fault detection method and device. Background Art
[0002] With the development of technologies such as big data and cloud computing, the amount of information has increased rapidly. Many Internet companies such as Microsoft, Google, and Alibaba have established large data centers to provide users with storage systems with huge storage capacity. However, with the continuous increase in the number of users and the sudden increase in storage scale, various failures will occur in data centers.
[0003] The core of the storage system is the storage medium. Currently, most enterprise data centers still use storage systems based on hard disks. As a key hardware device in large storage systems, hard disks often cause serious accidents due to their physical characteristics. At the very least, the data services provided by the data center will be unavailable. At worst, the stored data may be permanently lost, causing huge losses to users and enterprises.
[0004] Most existing hard disk fault detection technologies use a single model to process the SMART attribute data of the hard disk itself. This process needs to trace back to the data of the hard disk attribute time series. The SMART attributes of the time series will cause the gradient vanishing and gradient exploding phenomena, making it difficult to accurately identify the SMART attribute data, which can easily lead to misjudgment of the hard disk health status and low prediction accuracy. Summary of the invention
[0005] The present invention provides a hard disk fault detection method and device, which are used to solve the technical problem that the existing hard disk fault detection technology easily causes misjudgment of the hard disk health status, resulting in low prediction accuracy.
[0006] A first aspect of the present invention provides a hard disk failure detection method, comprising:
[0007] Acquire multiple hard disk attribute data to be detected, and input each of the hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module;
[0008] Using the target supervised learning integrated module to perform feature selection on each of the hard disk attribute data to be detected, and generate a plurality of hard disk selection data;
[0009] Performing feature evaluation on each of the hard disk selection data through a preset multi-model fusion module and the target supervised learning integration module, and outputting an importance evaluation score corresponding to each of the hard disk selection data;
[0010] The importance evaluation scores are sorted in descending order, the hard disk selection data of the first preset number of digits are input into the target gated cycle module for fault detection, and the hard disk fault detection result is output.
[0011] Optionally, the target supervised learning integration module includes a data preprocessing layer, a feature conversion layer and a feature selection mechanism layer; the step of using the target supervised learning integration module to perform feature selection on each of the hard disk attribute data to be detected to generate a plurality of hard disk selection data includes:
[0012] Preprocessing the hard disk attribute data to be detected by the data preprocessing layer to generate multiple target hard disk attribute data to be detected;
[0013] Inputting the attribute data of each target hard disk to be detected into the feature conversion layer for data enhancement, and outputting a plurality of enhanced hard disk attribute data;
[0014] A feature selection mechanism layer is used to perform feature screening on each of the enhanced hard disk attribute data to determine a plurality of hard disk selection data.
[0015] Optionally, the step of performing feature evaluation on each of the hard disk selection data by presetting the multi-model fusion module and the target supervised learning integration module and outputting the importance evaluation score corresponding to each of the hard disk selection data includes:
[0016] The target supervised learning integration module and the preset decision tree are model-fused by a preset multi-model fusion module to generate a supervised learning decision tree fusion module;
[0017] The supervised learning decision tree fusion module is used to perform feature evaluation on each of the hard disk selection data, and an importance evaluation score corresponding to each of the hard disk selection data is output.
[0018] Optionally, the target gated loop module includes a convolution layer, an activation function layer, a maximum pooling layer, a gated loop submodule, a fully connected layer and a target detection layer; the step of inputting the hard disk selection data of the first preset number of digits into the target gated loop module for fault detection and outputting the hard disk fault detection result includes:
[0019] The hard disk selection data of the first preset number of bits are used as target hard disk attribute data, and are input into the convolution layer for linear transformation, and hard disk feature data corresponding to each target hard disk attribute data is output;
[0020] Using an activation function layer to perform nonlinear transformation on each of the hard disk feature data to generate a plurality of hard disk delinearization data;
[0021] Performing maximum pooling on each of the hard disk delinearized data through a maximum pooling layer to generate a plurality of hard disk pooled data;
[0022] Inputting each of the hard disk pooling data into the gated loop submodule for feature adjustment, and outputting a plurality of hard disk adjustment feature data and a target hidden state corresponding to each of the hard disk adjustment feature data;
[0023] Using a fully connected layer to integrate the hard disk adjustment feature data and the target hidden state to generate a plurality of hard disk integration data;
[0024] The target detection layer performs fault diagnosis on the integrated data of each hard disk and outputs the hard disk fault detection result.
[0025] Optionally, the gated cycle submodule includes a tanh activation function layer, an update gate, and a reset gate; the step of inputting each of the hard disk pooling data into the gated cycle submodule for feature adjustment, and outputting a plurality of hard disk adjustment feature data and a target hidden state corresponding to each of the hard disk adjustment feature data includes:
[0026] A tanh activation function layer is used to perform nonlinear mapping on each of the hard disk pooling data, and a hidden state at a historical moment corresponding to each of the hard disk pooling data is output;
[0027] Inputting the hidden states of each historical moment and the hard disk pooling data into an update gate for state update, and outputting a plurality of initial hidden states and a plurality of hard disk mapping data;
[0028] The initial hidden states and the hard disk mapping data are reset through a reset gate to generate a plurality of hard disk adjustment feature data and a plurality of target hidden states.
[0029] Optionally, before the step of acquiring a plurality of hard disk attribute data to be detected and inputting each of the hard disk attribute data to be detected into a preset hard disk fault detection model, the step includes:
[0030] Acquire a plurality of first hard disk attribute data to be trained and a plurality of second hard disk attribute data to be trained;
[0031] Using each of the first hard disk attribute data to be trained to perform feature selection on the initial supervised learning integration module to generate a plurality of first hard disk selection data to be trained;
[0032] Selecting data from each of the first hard disks to be trained, calculating a first target loss value, and determining whether the first target loss value converges;
[0033] If the first target loss value converges, the trained initial supervised learning integrated module is used as the target supervised learning integrated module;
[0034] Based on the target supervised learning integrated module, the initial gated cycle module is trained using each of the second hard disk attribute data to be trained to obtain the trained target gated cycle module;
[0035] The preset hard disk fault detection model is constructed by using the target supervised learning integration module and the target gated loop module.
[0036] Optionally, the step of performing model training on the initial gated cycle module based on the target supervised learning integrated module using each of the second hard disk attribute data to be trained to obtain the trained target gated cycle module includes:
[0037] Inputting each of the second hard disk attribute data to be trained into the target supervised learning integration module for feature selection to generate a plurality of second hard disk selection data to be trained;
[0038] Performing feature evaluation on each of the second hard disk selection data to be trained by using the preset multi-model fusion module and the target supervised learning integration module, and outputting a plurality of importance evaluation scores to be trained;
[0039] Sorting the importance evaluation scores of the hard disks to be trained in descending order, inputting the first preset number of bits of the second hard disk selection data to be trained into the initial gated cycle module for fault detection, and outputting the fault detection result of the hard disk to be trained;
[0040] Using the hard disk failure detection result to be trained, calculating a second target loss value, and determining whether the second target loss value converges;
[0041] If the second target loss value converges, the trained initial gated recurrent module is used as the target gated recurrent module.
[0042] A second aspect of the present invention provides a hard disk fault detection device, comprising:
[0043] An acquisition module, used for acquiring multiple hard disk attribute data to be detected, and inputting each of the hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module;
[0044] An adopting module, used for adopting the target supervised learning integrated module to perform feature selection on each of the hard disk attribute data to be detected, and generate a plurality of hard disk selection data;
[0045] An output module, used to perform feature evaluation on each of the hard disk selection data through a preset multi-model fusion module and the target supervised learning integration module, and output an importance evaluation score corresponding to each of the hard disk selection data;
[0046] The sorting module is used to sort the importance evaluation scores in descending order, input the hard disk selection data of the first preset number of digits into the target gated cycle module for fault detection, and output the hard disk fault detection result.
[0047] A third aspect of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction, when executed by a processor, implements the steps of the hard disk failure detection method as described in any one of the above items.
[0048] A fourth aspect of the present invention provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of any of the hard disk failure detection methods described above.
[0049] It can be seen from the above technical solutions that the present invention has the following advantages:
[0050] The above technical scheme of the present invention provides a hard disk fault detection method, firstly, obtaining multiple hard disk attribute data to be detected, and inputting each hard disk attribute data to be detected into a preset hard disk fault detection model, the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module; the target supervised learning integration module is used to perform feature selection on each hard disk attribute data to be detected, and generate multiple hard disk selection data; the preset multi-model fusion module and the target supervised learning integration module are used to perform feature evaluation on each hard disk selection data, and the importance evaluation score corresponding to each hard disk selection data is output; each importance evaluation score is sorted in descending order, and the hard disk selection data of the first preset number of digits is input into the target gated loop module for fault detection, and the hard disk fault detection result is output; based on the above scheme, the target supervised learning integration module and the preset multi-model fusion module are used to screen each hard disk attribute data to be detected, and the target gated loop module is used to perform fault detection on the screened hard disk selection data, so that high-quality hard disk selection data can be screened out, the influence of data dimension is reduced, and the accuracy of fault prediction is further improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0052] Figure 1 A flowchart of a hard disk failure detection method provided by an embodiment of the present invention;
[0053] Figure 2A schematic diagram of a process for preprocessing hard disk attribute data to be detected provided by an embodiment of the present invention;
[0054] Figure 3 A flowchart of another hard disk failure detection method provided by an embodiment of the present invention;
[0055] Figure 4 A schematic diagram of the process of training the initial supervised learning integrated module provided by an embodiment of the present invention;
[0056] Figure 5 A schematic diagram of iterative changes in the total loss function under a training network provided by an embodiment of the present invention;
[0057] Figure 6 A schematic diagram of the change in accuracy under different features provided by an embodiment of the present invention;
[0058] Figure 7 A schematic diagram of the accuracy of fault category recognition with different numbers of features provided in an embodiment of the present invention;
[0059] Figure 8 A schematic diagram of the training process of the initial supervised learning integration module and the initial GRU model provided in an embodiment of the present invention;
[0060] Fig. 9 A structural block diagram of a hard disk failure detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0061] The embodiments of the present invention provide a hard disk fault detection method and device, which are used to solve the technical problem that the existing hard disk fault detection technology easily causes misjudgment of the hard disk health status, resulting in low prediction accuracy.
[0062] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] See also Figure 1 , Figure 1 A flowchart of a hard disk failure detection method provided by an embodiment of the present invention.
[0064] The present invention provides a hard disk fault detection method, comprising:
[0065] Step 101 : Acquire multiple hard disk attribute data to be detected, and input each hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module.
[0066] The pre-installed hard disk fault detection model is a neural network that integrates a target supervised learning integration module and a target gated loop module. The target supervised learning integration module is a LightGBM (Light Gradient Boosting Machine) model. As a gradient boosting framework, LightGBM has an efficient training speed and excellent performance, and can effectively process large-scale data sets. Through feature screening by LightGBM, the most representative features can be effectively extracted, thereby reducing the impact of data dimension and noise, and providing a better data foundation for the training of subsequent models.
[0067] The target gated recurrent module is the GRU (Gated Recurrent Unit) model. As a recurrent neural network structure, the GRU model has strong representation and memory capabilities when processing sequence data. GRU can better capture the temporal relationship and long-term dependency between data, and has high applicability for fault category recognition tasks. By combining the LightGBM and GRU models, we can fully utilize the high-quality features screened by LightGBM and input them into the GRU model for further learning and prediction. This combination of feature extraction and sequence modeling enables the model to achieve better performance in fault diagnosis tasks.
[0068] The multiple hard disk attribute data to be detected are hard disk SMART attribute data collected in the hard disk monitoring system, which include the operation status and performance indicators of the hard disk, such as read and write error rate, startup time, remapped sector count, etc. These are important indicators for evaluating the health status of the hard disk.
[0069] In this embodiment, multiple hard disk attribute data to be detected are obtained, and each hard disk attribute data to be detected is input into a preset hard disk fault detection model, which includes a target supervised learning integration module and a target gated loop module.
[0070] Step 102: Use the target supervised learning integration module to perform feature selection on the attribute data of each hard disk to be detected, and generate multiple hard disk selection data.
[0071] The target supervised learning integrated module includes data preprocessing layer, feature conversion layer and feature selection mechanism layer.
[0072] Please note that Figure 2After collecting multiple hard disk attribute data to be detected, data cleaning, i.e., preprocessing, is performed on each hard disk attribute data to be detected to ensure the quality and reliability of the data used, i.e., preprocessing is performed on each hard disk attribute data to be detected through a data preprocessing layer to generate multiple target hard disk attribute data to be detected. Specifically, first, missing values are removed from each hard disk attribute data to be detected. This process can identify and process missing items in the data. Possible measures include deleting records with missing values, filling missing values, or using models to predict missing values. Next, outlier processing is performed on each hard disk attribute data to be detected that has undergone the missing value removal processing process. This process can detect and correct or delete values that do not conform to other data patterns, which may be caused by errors or other atypical events. Finally, the hard disk attribute data to be detected that have undergone the missing value removal processing and the outlier processing process are normalized. This process can normalize the data to a specific range or distribution, which can eliminate the influence of different dimensions and magnitudes, making the data more suitable for model processing.
[0073] Furthermore, the attribute data of each target hard disk to be detected is input into the feature conversion layer for data enhancement, and a plurality of enhanced hard disk attribute data is output; the feature selection mechanism layer is used to perform feature screening on each enhanced hard disk attribute data, and a plurality of hard disk selection data are determined.
[0074] In this embodiment, a target supervised learning integration module is used to perform feature selection on each hard disk attribute data to be detected, and a plurality of hard disk selection data are generated.
[0075] Step 103: Perform feature evaluation on each hard disk selection data through a preset multi-model fusion module and a target supervised learning integration module, and output an importance evaluation score corresponding to each hard disk selection data.
[0076] It should be noted that in order to screen out the most critical features for predicting the health status of hard disks from numerous SMART attribute data, the present invention adopts LightGBM, an efficient machine learning algorithm based on the gradient boosting framework, which not only has a fast training speed but also has outstanding effects in processing large-scale data. Specifically, the target supervised learning integration module and the preset decision tree are integrated through the preset multi-model fusion module to generate a supervised learning decision tree fusion module, and the supervised learning decision tree fusion module is used to perform feature evaluation on each hard disk selection data, output the importance evaluation score corresponding to each hard disk selection data, and sort each importance evaluation score in descending order. The descending sorting results are shown in Table 1; the supervised learning decision tree fusion module combines the decision tree algorithm and the LightGBM algorithm to perform feature evaluation on the hard disk selection data. The feature importance (importance evaluation score) of the SMART attributes of the disk is modeled, and the importance evaluation scores are sorted in descending order. The most appropriate features are selected to prepare for subsequent fault prediction, which can reduce the complexity of the model and improve the speed and accuracy of the prediction. The feature importance scoring combined with the LightGBM algorithm and the decision tree algorithm can identify the most useful SMART attributes for hard disk health status prediction, laying the foundation for the next step of GRU model fault detection and model training; wherein, the number of pre-set bits can be set as needed, and no specific limitation is made here. For example, the number of hard disk selection data is 10, and the number of pre-set bits is set to 5. Then, according to the descending order of the importance evaluation scores, the top five hard disk selection data are selected as the input of the target gated loop module.
[0077] Table 1 Importance ranking results of preset decision tree and LightGBM models
[0078]
[0079]
[0080] In this embodiment, feature evaluation is performed on each hard disk selection data through a preset multi-model fusion module and a target supervised learning integration module, and an importance evaluation score corresponding to each hard disk selection data is output.
[0081] Step 104 , sorting the importance evaluation scores in descending order, inputting the hard disk selection data of the first preset number of digits into the target gated cycle module for fault detection, and outputting the hard disk fault detection result.
[0082] The target gated recurrent module is a GRU (Gated Recurrent Unit) model. The gated recurrent unit (GRU) is a special recurrent neural network (RNN) that can solve the gradient vanishing problem that may occur when RNN processes long sequences. The GRU in this application includes a convolutional layer, an activation function layer, a maximum pooling layer, a gated recurrent submodule, a fully connected layer, and a target detection layer. The gated recurrent submodule contains two gates: an update gate and a reset gate. These gates can control the flow of information, enabling the model to learn different patterns at different time steps.
[0083] It should be noted that the SMART attribute data of the hard disk after feature selection (each hard disk selection data) is selected as the input feature. Specifically, the hard disk selection data of the preset number of digits is used as the target hard disk attribute data, and is input into the convolution layer for feature extraction, and the hard disk feature data corresponding to each target hard disk attribute data is output; the activation function layer is used to perform nonlinear transformation on the feature data of each hard disk to generate a plurality of hard disk delinearization data; the maximum pooling layer is used to perform maximum pooling on the delinearization data of each hard disk to generate a plurality of hard disk pooling data; the pooling data of each hard disk is input into the gated loop submodule for feature adjustment, and a plurality of hard disk adjustment feature data and the target hidden state corresponding to each hard disk adjustment feature data are output; the fully connected layer is used to perform feature integration on the adjustment feature data of each hard disk and each target hidden state to generate a plurality of hard disk integration data; the target detection layer is used to perform fault diagnosis on the integrated data of each hard disk, and the hard disk fault detection result (fault state of the hard disk device) is output.
[0084] In this embodiment, the importance evaluation scores are sorted in descending order, the hard disk selection data of the first preset number of digits are input into the target gate control cycle module for fault detection, and the hard disk fault detection result is output.
[0085] In an embodiment of the present invention, the present invention provides a hard disk fault detection method, firstly obtaining a plurality of hard disk attribute data to be detected, and inputting each hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model comprises a target supervised learning integration module and a target gated loop module; the target supervised learning integration module is used to perform feature selection on each hard disk attribute data to be detected, and a plurality of hard disk selection data are generated; the preset multi-model fusion module and the target supervised learning integration module are used to perform feature evaluation on each hard disk selection data, and an importance evaluation score corresponding to each hard disk selection data is output; each importance evaluation score is sorted in descending order, and the hard disk selection data of the first preset number of digits is input into the target gated loop module for fault detection, and a hard disk fault detection result is output; based on the above scheme, the target supervised learning integration module and the preset multi-model fusion module are used to screen each hard disk attribute data to be detected, and the target gated loop module is used to perform fault detection on the screened hard disk selection data, so that high-quality hard disk selection data can be screened out, the influence of data dimension is reduced, and the accuracy of fault prediction is further improved.
[0086] See also Figure 3 , Figure 3 A flowchart of another hard disk failure detection method provided by an embodiment of the present invention.
[0087] The present invention provides a hard disk fault detection method, comprising:
[0088] Step 301: Obtain a plurality of first hard disk attribute data to be trained and a plurality of second hard disk attribute data to be trained.
[0089] The multiple first hard disk attribute data to be trained and the multiple second hard disk attribute data to be trained are all hard disk SMART attribute data used for model training, which include the operating status and performance indicators of the hard disk, such as read and write error rate, startup time, remapped sector count, etc.
[0090] In this embodiment, a plurality of first hard disk attribute data to be trained and a plurality of second hard disk attribute data to be trained are obtained.
[0091] Step 302: Use the first hard disk attribute data to be trained to perform feature selection on the initial supervised learning integration module to generate multiple first hard disk selection data to be trained.
[0092] In this embodiment, each first hard disk attribute data to be trained is used to perform feature selection on the initial supervised learning integration module to generate a plurality of first hard disk selection data to be trained.
[0093] Step 303: Use the first hard disks to be trained to select data, calculate the first target loss value, and determine whether the first target loss value converges.
[0094] In this embodiment, each first hard disk to be trained is used to select data, calculate the first target loss value, and determine whether the first target loss value converges.
[0095] Step 304: If the first target loss value converges, the trained initial supervised learning integrated module is used as the target supervised learning integrated module.
[0096] Please note that Figure 4 , multiple first hard disk attribute data to be trained are input into the initial LightGBM model (initial supervised learning integrated module) for training. The initial LightGBM model tries to find the best way to distinguish healthy disks from disks that are about to fail by learning patterns in the data. During the training process, the initial LightGBM model will output multiple first hard disk selection data to be trained, and each first hard disk selection data to be trained is used to calculate the first target loss value. If the first target loss value converges, it indicates that the initial LightGBM model has been trained, and it is used as the target supervised learning integrated module; if the first target loss value does not converge, the hyperparameters of the initial LightGBM model are updated according to the first target loss value to obtain an intermediate supervised learning integrated module, and each first hard disk attribute data to be trained is input into the intermediate supervised learning integrated module for training until the first target loss value converges, and the intermediate supervised learning integrated module determined when the first target loss value converges is used as the target supervised learning integrated module; wherein, the calculation process of the first target loss value using each first hard disk selection data to be trained can refer to the calculation process of the prior art, and is not specifically limited here.
[0097] Furthermore, the built-in feature importance scoring mechanism of the target supervised learning integration module is used in combination with the preset decision tree to evaluate the contribution to the model performance, that is, the importance of multiple second hard disk attribute data to be trained is evaluated and sorted, and features with a preset number of digits of high importance are selected from them (second hard disk selection data to be trained), and the selected second hard disk selection data to be trained are used as the screened feature set to train the initial GRU model.
[0098] In this embodiment, the present invention also supplements another training process of the initial supervised learning integration module, which is specifically: randomly divide the data set into three parts: 70% as a training set, 15% as a validation set, and 15% as a test set; set the LightGBM parameters: objective: set to binary; learning_rate: initially set to 0.1; num_leaves: set to 31, which is the maximum number of leaves; max_depth: set to -1, indicating no depth limit, but actually constrained by num_leaves; metric: use binary_logloss and auc as evaluation indicators. Training model: Use the training set data to train the initial LightGBM model. Use the lightgbm.train() function in Python to pass in the training data, model parameters, and early stopping conditions; set early_stopping_rounds = 50, which means that if the performance of the validation set does not improve within 50 consecutive iterations, stop training. Early stopping mechanism: specify the valid_sets parameter as the validation set in lightgbm.train(); during training, the model will evaluate the performance on the validation set after each round of iteration. If the performance on the validation set does not improve within the specified number of consecutive iterations, the early stopping mechanism is triggered and training is terminated.
[0099] Step 305 : Based on the target supervised learning integrated module, the initial gated loop module is trained using the attribute data of each second hard disk to be trained to obtain a trained target gated loop module.
[0100] The initial gated loop module is the initial GRU model. The process of building the initial GRU model is as follows: first, establish a Python environment, then install the TensorFlow or PyTorch library to complete the numerical calculations. In TensorFlow and PyTorch, the initial GRU model is built by calling the built-in GRU function; Python is a widely used high-level programming language suitable for various applications, including data analysis and machine learning; in PyTorch, nn.GRU is used to complete the call; in TensorFlow, tf.keras.layers.GRU is used to complete the subsequent experiments.
[0101] Further, step 305 may include the following sub-steps:
[0102] S51, inputting each second hard disk attribute data to be trained into a target supervised learning integration module for feature selection, and generating a plurality of second hard disk selection data to be trained;
[0103] S52, performing feature evaluation on each second hard disk selection data to be trained through a preset multi-model fusion module and a target supervised learning integration module, and outputting a plurality of importance evaluation scores to be trained;
[0104] S53, sorting the importance evaluation scores of the hard disks to be trained in descending order, inputting the first preset number of bits of the second hard disk selection data to be trained into the initial gated loop module for fault detection, and outputting the fault detection result of the hard disks to be trained;
[0105] S54, using the hard disk failure detection result to be trained, calculating the second target loss value, and determining whether the second target loss value converges;
[0106] S55. If the second target loss value converges, the trained initial gated recurrent module is used as the target gated recurrent module.
[0107] S55. If the second target loss value converges, the trained initial gated recurrent module is used as the target gated recurrent module.
[0108] It should be noted that the trained target supervised learning integration module is combined with the second hard disk selection data to be trained for the first preset number of digits output by the preset decision tree as the training data of the initial gated cycle module, the initial gated cycle module performs fault detection on the second hard disk selection data to be trained for the first preset number of digits, outputs the fault detection result of the hard disk to be trained, uses the fault detection result of the hard disk to be trained to calculate the second target loss value, and determines whether the second target loss value converges; if the second target loss value converges, the trained initial gated cycle module is used as the target gated cycle module; if the second target loss value does not converge, the second target loss value is used to update the hyperparameters of the initial gated cycle module to obtain an intermediate gated cycle module, the second hard disk selection data to be trained for the first preset number of digits is input into the intermediate gated cycle module for training until the second target loss value converges, and the intermediate gated cycle module determined when the second target loss value converges is used as the target gated cycle module; wherein, the calculation of the second target loss value using the fault detection result of the hard disk to be trained can refer to the calculation process of the prior art, and is not specifically limited here.
[0109] Furthermore, in the training process of the above-mentioned initial gated loop module and the initial supervised learning integration module, selecting the appropriate loss function and optimizer is an important part of model training. The loss function should match the type of problem (for example, mean square error is commonly used for regression problems, and cross entropy loss is commonly used for classification problems). The optimizer is responsible for adjusting the network weights to minimize the loss function. Common optimizers include SGD, Adam, etc. The relevant components and steps are shown in Table 2 below. During the training process, the model will iteratively learn on the training set. This process includes forward propagation to calculate the prediction results, calculate the loss function value, and back propagate to update the model parameters. This process is repeated until the model reaches a certain accuracy on the training set or the loss drops to an acceptable level.
[0110] Table 2 Overview of components and steps for model training
[0111]
[0112]
[0113] For further information, see Figure 5 In the training process of machine learning models, one of the core methods to optimize model parameters is to minimize the loss function through iterative training. The loss function is used as a metric to measure the difference between the model's predicted value and the true value, and is a quantitative evaluation of the model's performance. The two-dimensional curve between the number of iterative training and the total loss function shows how the total loss changes as the number of training increases, such as Figure 5 As shown in the figure, the number of training times is represented on the horizontal axis, and the total loss is represented on the vertical axis. The training effects of four different network structures are discussed. Among them, one network structure integrates LightGBM and GRU for optimization, another only improves the long short-term memory neural network (LSTM) part, the third improves the recurrent neural network (RNN) part, and the last one uses the traditional decision tree algorithm. The results show that all network structures can reach a convergence state. Specifically, the network integrating LightGBM and GRU reaches the minimum value in the loss function, followed by the LSTM model. In contrast, the training process of the decision tree and RNN network shows more violent oscillations.
[0114] For further information, see Figure 6 After model training, four models under four different feature selection strategies were tested on a test set containing 188 samples, and the recognition accuracy of fault categories was statistically analyzed. Figure 6As shown in the figure, when no feature selection is performed, the performance of the four models is relatively poor, with an overall prediction accuracy of less than 78%, and there is a large error. However, when the decision tree is used for feature selection, the LSTM neural network and the overall prediction rate are significantly improved. Next, after feature screening based on LightGBM, the results show that under this feature, the total accuracy of the fault diagnosis algorithm of the GRU neural network reaches the highest value of 92.3%, indicating that it has the potential to be applied in practical scenarios.
[0115] For further information, see Figure 7 Based on the above results, it can be concluded that the two models, LightGBM and GRU, can complement each other to improve the prediction effect of fault category identification. In terms of feature selection, the application of decision trees provides important support for the performance improvement of LSTM neural networks, which improves the overall prediction accuracy. After LightGBM performs feature screening, the performance of the GRU neural network has been significantly improved, and the total accuracy of its diagnostic algorithm under this feature has reached 92.3%, which shows that it has good predictive ability in fault category identification. Therefore, the combination of the two models, LightGBM and GRU, can more comprehensively mine feature information, improve the accuracy of fault diagnosis, and provide reliable support for the application of actual scenarios.
[0116] Furthermore, based on the number of SMART attributes of the hard disk, this application explores the impact of different numbers of features on the accuracy of fault category recognition, and uses models such as decision tree, recurrent neural network (RNN), long short-term memory neural network (LSTM) and gated recurrent unit (GRU) for testing. The results show that with the increase in the number of features, the accuracy of fault category recognition gradually increases, and gradually decreases after reaching a peak, and finally stabilizes. This phenomenon may reflect that when the number of features is small, the model lacks sufficient information for effective fault category recognition, and as the number of features increases, the model can better capture the characteristics of the data, thereby improving the accuracy. However, when the number of features is too large, noise or redundant information may be introduced, resulting in a decrease in model performance. Therefore, selecting an appropriate number of features is crucial to maintaining the performance of the model. The research results provide an important reference for the design and optimization of fault diagnosis systems, which helps to improve the accuracy and reliability of fault diagnosis; at the same time, in the task of fault category recognition, the gated recurrent unit (GRU) model shows the best effect. This result is partly due to the complementary characteristics of the two models, LightGBM and GRU.
[0117] In this embodiment, the present invention also supplements another training process of the initial gated loop module, which is specifically: select the SMART attributes after feature selection as input features, build the architecture of the GRU model, and capture the long-term dependence of the SMART attribute time series. The relevant hyperparameters are set as follows: the input of the network is sensor data, and the output is the sensor fault category and fault parameters. The parameters of the network are set as follows: the convolution kernel size is 2x2, the number of convolution kernels is 4, the activation function selects the Relu function, the number of neurons in the loop layer is set to 60 and 100 respectively, the time step is set to 40 sampling moments, and random inactivation is set after each layer according to a probability of 30%. The loss function is a dynamic loss function, the initial learning rate is 0.001, the batch size is set to 64, the default number of training rounds is 100, and the data set is divided into 40% training sets and 60% test sets. At the same time, the improved network is trained according to the fixed loss function and the dynamic composite loss function respectively. The weight ratio of the main loss number to the auxiliary loss number was 3:1 for 15 rounds of training, followed by 1:3 for 15 rounds, and then 1:1 for the rest of the training until the end.
[0118] Step 306: Use the target supervised learning integration module and the target gated loop module to build a preset hard disk fault detection model.
[0119] In this embodiment, a target supervised learning integration module and a target gated loop module are used to construct a preset hard disk fault detection model.
[0120] Step 307 , obtaining multiple hard disk attribute data to be detected, and inputting each hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module.
[0121] In this embodiment, multiple hard disk attribute data to be detected are obtained, and each hard disk attribute data to be detected is input into a preset hard disk fault detection model, which includes a target supervised learning integration module and a target gated loop module.
[0122] Step 308: Use the target supervised learning integration module to perform feature selection on the attribute data of each hard disk to be detected, and generate multiple hard disk selection data.
[0123] The target supervised learning integrated module includes data preprocessing layer, feature conversion layer and feature selection mechanism layer.
[0124] Further, step 308 may include the following sub-steps:
[0125] S81, preprocessing the attribute data of each hard disk to be detected through a data preprocessing layer to generate multiple target hard disk attribute data to be detected;
[0126] S82, inputting the attribute data of each target hard disk to be detected into the feature conversion layer for data enhancement, and outputting a plurality of enhanced hard disk attribute data;
[0127] S83, using a feature selection mechanism layer to perform feature screening on each enhanced hard disk attribute data to determine a plurality of hard disk selection data.
[0128] In this embodiment, a target supervised learning integration module is used to perform feature selection on each hard disk attribute data to be detected, and a plurality of hard disk selection data are generated.
[0129] Step 309: perform feature evaluation on each hard disk selection data through a preset multi-model fusion module and a target supervised learning integration module, and output an importance evaluation score corresponding to each hard disk selection data.
[0130] Further, step 309 may include the following sub-steps:
[0131] S91, performing model fusion on the target supervised learning integration module and the preset decision tree through the preset multi-model fusion module to generate a supervised learning decision tree fusion module;
[0132] S92. Use a supervised learning decision tree fusion module to perform feature evaluation on each hard disk selection data, and output an importance evaluation score corresponding to each hard disk selection data.
[0133] In this embodiment, feature evaluation is performed on each hard disk selection data through a preset multi-model fusion module and a target supervised learning integration module, and an importance evaluation score corresponding to each hard disk selection data is output.
[0134] Step 310: sort the importance evaluation scores in descending order, input the hard disk selection data of the first preset number of digits into the target gated cycle module for fault detection, and output the hard disk fault detection result.
[0135] The target gated recurrent module includes a convolutional layer, an activation function layer, a maximum pooling layer, a gated recurrent submodule, a fully connected layer, and a target detection layer.
[0136] It should be noted that
[0137] Further, step 310 may include the following sub-steps:
[0138] S101, using the hard disk selection data of the first preset number of bits as target hard disk attribute data, and inputting them into the convolution layer for linear transformation, and outputting the hard disk feature data corresponding to each target hard disk attribute data;
[0139] S102, using an activation function layer to perform nonlinear transformation on the characteristic data of each hard disk to generate a plurality of hard disk delinearization data;
[0140] S103, performing maximum pooling on the delinearized data of each hard disk through a maximum pooling layer to generate multiple hard disk pooling data;
[0141] S104, inputting the pooled data of each hard disk into the gated loop submodule for feature adjustment, and outputting a plurality of hard disk adjustment feature data and a target hidden state corresponding to each hard disk adjustment feature data;
[0142] S105, using a fully connected layer to integrate the features of each hard disk adjustment feature data and each target hidden state to generate multiple hard disk integration data;
[0143] S106, performing fault diagnosis on the integrated data of each hard disk through the target detection layer, and outputting the hard disk fault detection result.
[0144] It should be noted that the convolutional layer receives the attribute data of each target hard disk and performs linear transformation to map the input vector to the dimensional space required by the model to obtain multiple hard disk feature data. The input vector can be expressed as x = {x1, x2, ..., x n}, where x is the set of target hard disk attribute data; x1 is the first SMART attribute (target hard disk attribute data), where each SMART attribute is connected to a neuron, and then the outputs of these neurons are weighted summed and nonlinearly transformed through the activation function. The input SMART attribute representation is mapped to the hidden layer in the model for subsequent processing and learning.
[0145] The gated loop submodule includes a tanh activation function layer, an update gate, and a reset gate.
[0146] Further, S104 may include the following sub-steps:
[0147] S1041, using a tanh activation function layer to perform nonlinear mapping on the pooled data of each hard disk, and outputting the hidden state of the historical moment corresponding to the pooled data of each hard disk;
[0148] S1042, inputting the hidden state of each historical moment and the pooled data of each hard disk into the update gate for state update, and outputting a plurality of initial hidden states and a plurality of hard disk mapping data;
[0149] S1043, resetting the states of each initial hidden state and each hard disk mapping data through a reset gate, generating a plurality of hard disk adjustment feature data and a plurality of target hidden states.
[0150] It should be noted that the update gate controls whether the input information at the current moment will be remembered and affect the subsequent output: its function is mainly reflected in: how much of the input information at the current moment will be retained and not forgotten. If the output of the update gate is close to 1, it means that most of the input information at the current moment will be retained. The output value of the update gate is determined by the input at the current moment and the hidden state at the previous moment. Specifically, the output of the update gate is determined by a sigmoid function, where the input includes the input at the current moment and the hidden state at the previous moment. The value output by the sigmoid function is between 0 and 1, indicating the importance of the input information at the current moment for updating the hidden state.
[0151] Furthermore, the reset gate is responsible for controlling whether to discard the information in the hidden state of the previous moment, and how the current input and the previous hidden state affect each other. Its functions include: deciding whether to ignore part of the information in the hidden state of the previous moment, which can help the model avoid the problem of gradient disappearance or gradient explosion, and thus better capture the long-term dependencies in sequence data. The output of the reset gate is also determined by a sigmoid function, whose input includes the current input and the hidden state of the previous moment. If the output of the reset gate is close to 0, it means that most of the information in the hidden state of the previous moment will be discarded; if it is close to 1, it means that the information of the hidden state of the previous moment will be completely retained.
[0152] In this embodiment, the importance evaluation scores are sorted in descending order, the hard disk selection data of the first preset number of digits are input into the target gate control cycle module for fault detection, and the hard disk fault detection result is output.
[0153] For example, see Figure 8 First, obtain sample data and perform feature importance training on the initial LightGBM model, that is, use the attribute data of each first hard disk to be trained to train the initial supervised learning integration module. After obtaining the target supervised learning integration module, screen and divide the sample data through the target supervised learning integration module to obtain a training data set and a test data set composed of the hard disk selection data to be trained. Use the training data set and the test data set to train the initial GRU model to obtain the target GRU model. Use the target GRU model and the target supervised learning integration module to build a preset hard disk fault detection model, and evaluate the performance indicators of the model.
[0154] As a comparison of technical effects, we can refer to the existing technology. The existing technology needs to trace back to the information of the hard disk attribute time series. The SMART attribute of the time series will cause the gradient to disappear and the gradient to explode. Secondly, the similarity screening and sorting between different features will also lead to the decline of model performance. At the same time, in the processing of large-scale storage device health monitoring data, the current technology shows two significant deficiencies: insufficient high-dimensional processing capabilities and unbalanced data sets. Specifically, in the practice of hard disk health status prediction, traditional machine learning methods encounter major challenges in the process of feature selection and model training. Especially when dealing with self-monitoring, analysis and reporting technology (SMART) attributes, these attributes often cover a lot of redundant or irrelevant information, which not only increases the computational burden of the model, but also may reduce the prediction performance.
[0155] Furthermore, given that the frequency of hard disk failures in actual operation is relatively low, the number of healthy hard disk samples in the resulting data set far exceeds the number of failed hard disk samples, resulting in a significant data imbalance. This imbalance makes it difficult for many supervised learning algorithms to effectively identify the characteristic patterns of failed hard disks, because the algorithms tend to optimize the prediction performance for the majority of categories. Therefore, the existing technology has obvious limitations in accurately predicting the health status of hard disks, and its related shortcomings that need to be noted are as follows:
[0156] 1. Inadequate processing of high-dimensional data: Current storage device health monitoring methods often face the difficulty of processing high-dimensional data. SMART attributes provide a wealth of indicators for predicting hard disk failures, but they also bring a huge dimensional space. In high-dimensional space, many traditional feature selection methods are inefficient, not only with high computational costs, but also prone to the curse of dimensionality, affecting the accuracy and interpretability of model predictions.
[0157] 2. Data imbalance: In hard disk health monitoring data sets, there is often a significant imbalance between samples of healthy hard disks and hard disks that are about to fail. When traditional methods are trained on such imbalanced data, they often lead to the model being biased towards the majority category (healthy hard disks) and ignoring the minority category (faulty hard disks), which seriously affects the accuracy and timeliness of fault prediction.
[0158] 3. Separation of feature screening and prediction model: Many existing hard disk health prediction methods perform feature selection and prediction modeling as independent steps. This separation may result in the selected features not being the most suitable for the prediction model, thus affecting the performance of the final prediction results.
[0159] In view of the above problems, the present invention proposes a hard disk fault detection method, which uses the LightGBM model for feature selection. LightGBM is an efficient algorithm based on gradient boosting. It uses a histogram-based algorithm, which can improve the training speed without reducing the accuracy. At the same time, its built-in feature selection mechanism effectively reduces the data dimension and improves the model performance, which can solve the problem of insufficient high-dimensional data processing; at the same time, the prediction accuracy is improved by combining LightGBM for feature selection and using the GRU model for time series prediction. The GRU model has significant advantages in processing time series data and can effectively capture long-term dependencies. Combined with the selected key features, this method is expected to enhance the detection ability of hard disks that are about to fail without sacrificing sensitivity. The impact of the dynamic changes in the threshold of the faulty disk on feature engineering is considered; the training effect of the SMART attributes integrated for different disk manufacturers; the introduction of a neural network that adapts to the disk time series improves the accuracy; considering the data imbalance problem between faulty disks and healthy disks, an adversarial neural network is introduced to increase the fault data set.
[0160] Furthermore, the feature selection capability of LightGBM is used to optimize the SMART attributes of the input, and the performance of the GRU model in processing long-time series data of hard disks is enhanced by reducing irrelevant and redundant information, and the problem of gradient vanishing or exploding is alleviated. The efficient feature selection of LightGBM can not only improve the training efficiency of the GRU model, but also enhance the accuracy of the model for fault prediction; to optimize the accuracy and efficiency of storage device health monitoring. With the help of the powerful feature selection function of LightGBM, key prediction indicators are screened out from complex SMART data, effectively reducing the data dimension and balancing the ratio of healthy and faulty disk samples. The streamlined data set provides high-quality input for the GRU model, enabling it to more accurately capture the dependencies between time series. This combined model, which combines efficient feature processing and advanced time series modeling, improves the fault detection rate (FDR) by 9%-18% compared with other algorithms, and controls the false warning rate to 0.08%, significantly improving the performance of hard disk fault prediction.
[0161] At the same time, through this fusion mechanism, the present invention aims to solve the limitations of traditional methods in hard disk failure prediction. Especially when dealing with high-dimensional and unbalanced data sets, the combination of LightGBM and GRU provides an efficient and effective solution. As a gradient boosting framework, LightGBM can handle large-scale data while maintaining the accuracy of the model; while the GRU neural network excels in capturing the temporal dynamics of sequence data. This comprehensive approach can significantly improve the ability to predict potential hard disk failures, thereby achieving a technological breakthrough in the field of storage device health monitoring.
[0162] In an embodiment of the present invention, the present invention provides a hard disk fault detection method, firstly obtaining a plurality of hard disk attribute data to be detected, and inputting each hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model comprises a target supervised learning integration module and a target gated loop module; the target supervised learning integration module is used to perform feature selection on each hard disk attribute data to be detected, and a plurality of hard disk selection data are generated; the preset multi-model fusion module and the target supervised learning integration module are used to perform feature evaluation on each hard disk selection data, and an importance evaluation score corresponding to each hard disk selection data is output; each importance evaluation score is sorted in descending order, and the hard disk selection data of the first preset number of digits is input into the target gated loop module for fault detection, and a hard disk fault detection result is output; based on the above scheme, the target supervised learning integration module and the preset multi-model fusion module are used to screen each hard disk attribute data to be detected, and the target gated loop module is used to perform fault detection on the screened hard disk selection data, so that high-quality hard disk selection data can be screened out, the influence of data dimension is reduced, and the accuracy of fault prediction is further improved.
[0163] See also Fig. 9 , Fig. 9 A structural block diagram of a hard disk failure detection device provided by an embodiment of the present invention.
[0164] The present invention provides a hard disk fault detection device, comprising:
[0165] The acquisition module 901 is used to acquire multiple hard disk attribute data to be detected, and input each hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module;
[0166] The module 902 is used to select the features of each hard disk attribute data to be detected by using the target supervised learning integration module to generate a plurality of hard disk selection data;
[0167] The output module 903 is used to perform feature evaluation on each hard disk selection data through a preset multi-model fusion module and a target supervised learning integration module, and output an importance evaluation score corresponding to each hard disk selection data;
[0168] The sorting module 904 is used to sort the importance evaluation scores in descending order, input the hard disk selection data of the first preset number of digits into the target gate control cycle module for fault detection, and output the hard disk fault detection result.
[0169] Furthermore, the target supervised learning integrated module includes a data preprocessing layer, a feature conversion layer and a feature selection mechanism layer; the module 902 is adopted, including:
[0170] The preprocessing submodule is used to preprocess the attribute data of each hard disk to be detected through the data preprocessing layer to generate multiple target hard disk attribute data to be detected;
[0171] The enhancement submodule is used to input the attribute data of each target hard disk to be detected into the feature conversion layer for data enhancement, and output a plurality of enhanced hard disk attribute data;
[0172] The screening submodule is used to perform feature screening on each enhanced hard disk attribute data by using a feature selection mechanism layer to determine a plurality of hard disk selection data.
[0173] Furthermore, the output module 903 includes:
[0174] A fusion submodule is used to perform model fusion on the target supervised learning integration module and the preset decision tree through a preset multi-model fusion module to generate a supervised learning decision tree fusion module;
[0175] The evaluation submodule is used to perform feature evaluation on each hard disk selection data by using a supervised learning decision tree fusion module, and output the importance evaluation score corresponding to each hard disk selection data.
[0176] Furthermore, the target gated loop module includes a convolutional layer, an activation function layer, a maximum pooling layer, a gated loop submodule, a fully connected layer and a target detection layer; the sorting module 904 includes:
[0177] A linear transformation submodule is used to use the hard disk selection data of the first preset number of bits as the target hard disk attribute data, and input it into the convolution layer for linear transformation, and output the hard disk feature data corresponding to each target hard disk attribute data;
[0178] A nonlinear transformation submodule is used to perform nonlinear transformation on the characteristic data of each hard disk by using an activation function layer to generate multiple hard disk delinearization data;
[0179] The pooling submodule is used to perform maximum pooling on the delinearized data of each hard disk through the maximum pooling layer to generate multiple hard disk pooling data;
[0180] The adjustment submodule is used to input the pooled data of each hard disk into the gated loop submodule for feature adjustment, and output a plurality of hard disk adjustment feature data and a target hidden state corresponding to each hard disk adjustment feature data;
[0181] An integration submodule, for integrating the features of each hard disk adjustment feature data and each target hidden state using a fully connected layer to generate multiple hard disk integration data;
[0182] The diagnosis submodule is used to perform fault diagnosis on the integrated data of each hard disk through the target detection layer and output the hard disk fault detection result.
[0183] Furthermore, the gated loop submodule includes a tanh activation function layer, an update gate, and a reset gate; the regulation submodule includes:
[0184] A mapping unit, used to perform nonlinear mapping on the pooled data of each hard disk using a tanh activation function layer, and output the hidden state of the historical moment corresponding to the pooled data of each hard disk;
[0185] An update unit, used for inputting the hidden state of each historical moment and the pooled data of each hard disk into the update gate for state update, and outputting a plurality of initial hidden states and a plurality of hard disk mapping data;
[0186] The reset unit is used to reset the state of each initial hidden state and each hard disk mapping data through a reset gate to generate multiple hard disk adjustment feature data and multiple target hidden states.
[0187] Optionally include:
[0188] The first module is used to obtain a plurality of first hard disk attribute data to be trained and a plurality of second hard disk attribute data to be trained;
[0189] The second module is used to perform feature selection on the initial supervised learning integration module by using the attribute data of each first hard disk to be trained, and generate a plurality of first hard disk selection data to be trained;
[0190] The third module is used to select data from each first hard disk to be trained, calculate a first target loss value, and determine whether the first target loss value converges;
[0191] The fourth module is used to use the trained initial supervised learning integration module as the target supervised learning integration module if the first target loss value converges;
[0192] The fifth module is used to train the initial gated cycle module based on the target supervised learning integration module using the attribute data of each second hard disk to be trained to obtain a trained target gated cycle module;
[0193] The sixth module is used to construct a pre-set hard disk fault detection model by using a target supervised learning integration module and a target gated loop module.
[0194] Optionally, the sixth module includes:
[0195] The first submodule is used to input the attribute data of each second hard disk to be trained into the target supervised learning integration module for feature selection, and generate a plurality of second hard disk selection data to be trained;
[0196] The second submodule is used to perform feature evaluation on each second hard disk selection data to be trained through a preset multi-model fusion module and a target supervised learning integration module, and output a plurality of importance evaluation scores to be trained;
[0197] The third submodule is used to sort the importance evaluation scores of the hard disks to be trained in descending order, input the second hard disk selection data of the first preset number of digits to be trained into the initial gated loop module for fault detection, and output the fault detection result of the hard disks to be trained;
[0198] The fourth submodule is used to calculate the second target loss value by using the hard disk failure detection result to be trained, and determine whether the second target loss value converges;
[0199] The fifth submodule is used to use the trained initial gated loop module as the target gated loop module if the second target loss value converges.
[0200] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules, sub-modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0201] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the hard disk failure detection method in any of the above embodiments are implemented.
[0202] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the hard disk failure detection method in any of the above embodiments when the computer program / instruction is executed by a processor.
[0203] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0204] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0205] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A hard disk failure detection method, characterized in that: include: Acquire multiple hard disk attribute data to be detected, and input each of the hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module; The target supervised learning integrated module is used to perform feature selection on each of the hard disk attribute data to be detected to generate multiple hard disk selection data; the target supervised learning integrated module includes a data preprocessing layer, a feature conversion layer and a feature selection mechanism layer; Performing feature evaluation on each of the hard disk selection data through a preset multi-model fusion module and the target supervised learning integration module, and outputting an importance evaluation score corresponding to each of the hard disk selection data; Sort the importance evaluation scores in descending order, input the hard disk selection data of the first preset number of digits into the target gated loop module for fault detection, and output the hard disk fault detection result; the target gated loop module includes a convolution layer, an activation function layer, a maximum pooling layer, a gated loop submodule, a fully connected layer and a target detection layer; The step of performing feature evaluation on each of the hard disk selection data by presetting the multi-model fusion module and the target supervised learning integration module, and outputting the importance evaluation score corresponding to each of the hard disk selection data, comprises: The target supervised learning integration module and the preset decision tree are model-fused by a preset multi-model fusion module to generate a supervised learning decision tree fusion module; The supervised learning decision tree fusion module is used to perform feature evaluation on each of the hard disk selection data, and an importance evaluation score corresponding to each of the hard disk selection data is output.
2. The hard disk failure detection method according to claim 1, characterized in that: The step of using the target supervised learning integrated module to perform feature selection on each of the hard disk attribute data to be detected to generate a plurality of hard disk selection data includes: Preprocessing the hard disk attribute data to be detected by the data preprocessing layer to generate multiple target hard disk attribute data to be detected; Inputting the attribute data of each target hard disk to be detected into the feature conversion layer for data enhancement, and outputting a plurality of enhanced hard disk attribute data; A feature selection mechanism layer is used to perform feature screening on each of the enhanced hard disk attribute data to determine a plurality of hard disk selection data.
3. The hard disk failure detection method according to claim 1, characterized in that: The step of inputting the hard disk selection data of the first preset number of bits into the target gated cycle module for fault detection and outputting the hard disk fault detection result includes: The hard disk selection data of the first preset number of bits are used as target hard disk attribute data, and are input into the convolution layer for linear transformation, and hard disk feature data corresponding to each target hard disk attribute data is output; Using an activation function layer to perform nonlinear transformation on each of the hard disk feature data to generate a plurality of hard disk delinearization data; Performing maximum pooling on each of the hard disk delinearized data through a maximum pooling layer to generate a plurality of hard disk pooled data; Inputting each of the hard disk pooling data into the gated loop submodule for feature adjustment, and outputting a plurality of hard disk adjustment feature data and a target hidden state corresponding to each of the hard disk adjustment feature data; Using a fully connected layer to integrate the hard disk adjustment feature data and the target hidden state to generate a plurality of hard disk integration data; The target detection layer performs fault diagnosis on the integrated data of each hard disk and outputs the hard disk fault detection result.
4. The hard disk failure detection method according to claim 3, characterized in that: The gated cycle submodule includes a tanh activation function layer, an update gate, and a reset gate; the step of inputting each of the hard disk pooling data into the gated cycle submodule for feature adjustment, and outputting a plurality of hard disk adjustment feature data and a target hidden state corresponding to each of the hard disk adjustment feature data includes: A tanh activation function layer is used to perform nonlinear mapping on each of the hard disk pooling data, and a hidden state at a historical moment corresponding to each of the hard disk pooling data is output; Inputting the hidden states of each historical moment and the hard disk pooling data into an update gate for state update, and outputting a plurality of initial hidden states and a plurality of hard disk mapping data; The initial hidden states and the hard disk mapping data are reset through a reset gate to generate a plurality of hard disk adjustment feature data and a plurality of target hidden states.
5. The hard disk failure detection method according to claim 1, characterized in that: Before the step of obtaining a plurality of hard disk attribute data to be detected and inputting each of the hard disk attribute data to be detected into the preset hard disk fault detection model, the method includes: Acquire a plurality of first hard disk attribute data to be trained and a plurality of second hard disk attribute data to be trained; Using each of the first hard disk attribute data to be trained to perform feature selection on the initial supervised learning integration module to generate a plurality of first hard disk selection data to be trained; Selecting data from each of the first hard disks to be trained, calculating a first target loss value, and determining whether the first target loss value converges; If the first target loss value converges, the trained initial supervised learning integrated module is used as the target supervised learning integrated module; Based on the target supervised learning integrated module, the initial gated cycle module is trained using each of the second hard disk attribute data to be trained to obtain the trained target gated cycle module; The preset hard disk fault detection model is constructed by using the target supervised learning integration module and the target gated loop module.
6. The hard disk failure detection method according to claim 5, characterized in that: The step of performing model training on the initial gated cycle module based on the target supervised learning integrated module using the second hard disk attribute data to be trained to obtain the trained target gated cycle module comprises: Inputting each of the second hard disk attribute data to be trained into the target supervised learning integration module for feature selection to generate a plurality of second hard disk selection data to be trained; Performing feature evaluation on each of the second hard disk selection data to be trained by using the preset multi-model fusion module and the target supervised learning integration module, and outputting a plurality of importance evaluation scores to be trained; Sorting the importance evaluation scores of the hard disks to be trained in descending order, inputting the first preset number of bits of the second hard disk selection data to be trained into the initial gated cycle module for fault detection, and outputting the fault detection result of the hard disk to be trained; Using the hard disk failure detection result to be trained, calculating a second target loss value, and determining whether the second target loss value converges; If the second target loss value converges, the trained initial gated recurrent module is used as the target gated recurrent module.
7. A hard disk fault detection device, characterized in that: include: An acquisition module, used for acquiring multiple hard disk attribute data to be detected, and inputting each of the hard disk attribute data to be detected into a preset hard disk fault detection model, wherein the preset hard disk fault detection model includes a target supervised learning integration module and a target gated loop module; An adopting module is used to adopt the target supervised learning integration module to perform feature selection on each of the hard disk attribute data to be detected, and generate multiple hard disk selection data; the target supervised learning integration module includes a data preprocessing layer, a feature conversion layer and a feature selection mechanism layer; An output module, used to perform feature evaluation on each of the hard disk selection data through a preset multi-model fusion module and the target supervised learning integration module, and output an importance evaluation score corresponding to each of the hard disk selection data; A sorting module, used to sort the importance evaluation scores in descending order, input the hard disk selection data of the first preset number of digits into the target gated cycle module for fault detection, and output the hard disk fault detection result; The target gated loop module includes a convolutional layer, an activation function layer, a maximum pooling layer, a gated loop submodule, a fully connected layer and a target detection layer; The output module comprises: A fusion submodule, used to perform model fusion on the target supervised learning integration module and the preset decision tree through a preset multi-model fusion module to generate a supervised learning decision tree fusion module; The evaluation submodule is used to use the supervised learning decision tree fusion module to perform feature evaluation on each hard disk selection data, and output the importance evaluation score corresponding to each hard disk selection data.
8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the hard disk failure detection method as described in any one of claims 1-6 are implemented.
9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the hard disk failure detection method as described in any one of claims 1-6 are implemented.
