Prediction method and device for disk fault, storage medium and computer program product
Through bidirectional long and short-term memory network model and feature sensitivity analysis, the problem of low prediction accuracy of existing disk failures is solved, and higher prediction accuracy and robustness are achieved, and the maximum attribute of disk state correlation is reasonably identified and the redundant attributes are reduced. Influence.
Patent Information
- Application Number
- CN202411178441.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-13
AI Technical Summary
The prediction accuracy of existing disk failure prediction methods is low, making it difficult to accurately capture the timing and nonlinear characteristics of disk running data, resulting in the inability to effectively identify disk failures.
Using a bidirectional long and short-term memory network model combined with feature sensitivity analysis, a variable time series data set is constructed by obtaining multiple attribute parameters of the disk, the sensitivity index of each sub-eigenvector is calculated, and the main feature subset and secondary feature subset are divided. And input them into the bidirectional long and short-term memory network model for prediction.
It improves the accuracy and robustness of disk failure prediction, reasonably judges the maximum attributes of disk state correlation, separates the influence of irrelevant redundant attributes, and reduces the inaccuracy of prediction results.
Smart Images
Figure CN119132375A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the computer field, and specifically, to a disk failure prediction method and device, a storage medium, and a computer program product. Background Art
[0002] Disks are the key cornerstone of data storage. Once a disk fails, it is likely to cause the loss of stored data, causing immeasurable losses to individuals and enterprises. Disk failure prediction technology can predict disk failures in advance through in-depth mining and analysis of hard disk performance data, thereby avoiding data loss and business interruption, and ensuring data availability and system stability.
[0003] Traditional disk failure prediction methods are mostly based on simple linear strategies or shallow machine learning models. When dealing with complex and changeable disk operation data, these methods often have difficulty accurately capturing its inherent temporal and nonlinear characteristics, thus limiting the prediction accuracy. As a result, the above disk failure prediction methods are affected by irrelevant attributes and cannot achieve reasonable and rapid identification and judgment.
[0004] With regard to the problem of low prediction accuracy of disk failure prediction solutions in related technologies, no effective solution has been proposed so far. Summary of the invention
[0005] The embodiments of the present application provide a disk failure prediction method and device to at least solve the problem of low prediction accuracy of disk failure prediction solutions in existing methods.
[0006] According to an embodiment of the present application, a disk failure prediction method is provided, comprising: obtaining multiple attribute parameters of a target disk, constructing a variable time series data set based on the multiple attribute parameters, and converting the variable time series data set to obtain an input feature vector set; calculating the sensitivity index of each sub-feature vector in the input feature vector set; sorting the features in the input feature vector set according to the sensitivity index, and dividing the input feature vector set into a primary feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the primary feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset; and inputting the primary feature vector set and the secondary feature vector set into a target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty.
[0007] In an exemplary embodiment, the main feature vector set and the secondary feature vector set are respectively input into a target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, including: initializing the target bidirectional long short-term memory network model, determining the first weight and number of layers corresponding to the initial setting of the model; using the main feature vector set as the input of the first layer sub-network in the model, and calculating the first output feature corresponding to the main feature vector set as input according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model; connecting the first output feature with the secondary feature vector set as the input of the second layer sub-network in the model, and again calculating the first output feature according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model; The forward propagation algorithm corresponding to the target bidirectional long short-term memory network model is used to calculate to obtain the second output feature; the above process of determining the output feature is executed repeatedly until the target output feature that meets the numerical value corresponding to the number of layers is obtained; wherein the number of neurons contained in each sub-network is less than or equal to the first weight; the target output feature is flattened to obtain a target feature vector; the target feature vector is synchronized to the fully connected layer of the target bidirectional long short-term memory network model for classification processing to obtain a prediction confidence value for indicating a disk failure; the prediction confidence value is compared with the prediction confidence threshold; and a prediction result of whether the target disk is faulty is generated according to the comparison result.
[0008] In an exemplary embodiment, a prediction result of whether the target disk is faulty is generated based on the comparison result, including: when it is determined that the prediction confidence value is greater than the prediction confidence threshold, determining to generate a first prediction result that the target disk is a faulty disk; when it is determined that the prediction confidence value is less than or equal to the prediction confidence threshold, determining to generate a second prediction result that the target disk is a healthy disk.
[0009] In an exemplary embodiment, before the main feature vector set and the secondary feature vector set are respectively input into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, the method also includes: obtaining historical fault status information of the target disk, and a historical main feature subset and a historical secondary feature subset corresponding to the historical fault status information; using the historical main feature subset and the historical secondary feature subset as model input, and using the historical fault status information as model output, performing model training on a preset bidirectional long short-term memory network model to obtain a target bidirectional long short-term memory network model.
[0010] In an exemplary embodiment, after model training is performed on a preset bidirectional long short-term memory network model, the method further includes: updating the weights of each sub-network layer and the fully connected layer deviation in the preset bidirectional long short-term memory network model through a back-propagation algorithm to obtain a second weight and a target deviation; when it is determined that the second weight is greater than the first weight and the target deviation is less than the preset deviation, error convergence is determined, and the trained target bidirectional long short-term memory network model is output.
[0011] In an exemplary embodiment, multiple attribute parameters of a target disk are obtained, and a variable time series data set is constructed based on the multiple attribute parameters, including: obtaining input data transmitted by the target disk within a preset time period; identifying attribute parameters associated with the input data to obtain the multiple attribute parameters; determining a time change interval corresponding to each of the multiple attribute parameters; and performing combined data normalization processing according to a preset arrangement order of the attribute parameters and the time change interval to obtain a variable time series data set.
[0012] In an exemplary embodiment, after the main feature vector set and the secondary feature vector set are respectively input into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, the method further includes: when the prediction result is that the target disk is in a faulty state, collecting basic disk information corresponding to the target disk, wherein the basic disk information is used to indicate the disk storage location and the specifications of the disk; and sending the basic disk information to the target object that manages the target disk to instruct the target object to perform maintenance on the target disk.
[0013] According to another embodiment of the present application, a disk failure prediction device is also provided, including: an acquisition module, used to acquire multiple attribute parameters of a target disk, construct a variable time series data set based on the multiple attribute parameters, and transform the variable time series data set to obtain an input feature vector set; a calculation module, used to calculate the sensitivity index of each sub-feature vector in the input feature vector set; a division module, used to sort the features in the input feature vector set according to the sensitivity index, and divide the input feature vector set into a main feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the main feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset; a prediction module, used to input the main feature vector set and the secondary feature vector set into a target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty.
[0014] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.
[0015] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0016] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.
[0017] Through this application, multiple attribute parameters of the target disk are obtained, a variable time series data set is constructed based on the multiple attribute parameters, and the variable time series data set is converted to obtain an input feature vector set; the sensitivity index of each sub-feature vector in the input feature vector set is calculated; the features in the input feature vector set are sorted according to the sensitivity index, and the input feature vector set is divided into a main feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the main feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset; the main feature vector set and the secondary feature vector set are respectively input into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty. By adopting the above technical solution, it is possible to reasonably judge which attributes have the greatest "correlation" with the disk status, separate or weaken the influence of irrelevant redundant attributes on the prediction results, and then when the input features are input into the model for prediction, the influence of redundant attributes on the prediction results is reduced through advance analysis of sensitivity, solving the problem of low prediction accuracy of the disk failure prediction scheme in the existing method. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0019] Figure 1 It is a hardware structure block diagram of a computer terminal according to a disk failure prediction method of an embodiment of the present application;
[0020] Figure 2 is a flow chart of a method for predicting disk failure according to an embodiment of the present application;
[0021] Figure 3It is a flowchart of a BiLSTM disk failure prediction method based on feature sensitivity analysis according to an embodiment of the present application;
[0022] Figure 4 is a structural block diagram of a disk failure prediction device according to an embodiment of the present application;
[0023] Figure 5 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0024] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0026] The disk failure prediction method embodiment provided in the present application can be executed in a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a disk failure prediction method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the task configuration method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0028] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0029] In this embodiment, a disk failure prediction method is provided, including but not limited to being applied in a computer terminal. Figure 2 is a flowchart of a disk failure prediction method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:
[0030] Step S202: acquiring multiple attribute parameters of the target disk, constructing a variable time series data set based on the multiple attribute parameters, and converting the variable time series data set to obtain an input feature vector set;
[0031] Step S204: calculating the sensitivity index of each sub-feature vector in the input feature vector set;
[0032] Step S206: sorting the features in the input feature vector set according to the sensitivity index, and dividing the input feature vector set into a primary feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the primary feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset;
[0033] Step S208: inputting the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty.
[0034] Through the above method, multiple attribute parameters of the target disk are obtained, a variable time series data set is constructed based on the multiple attribute parameters, and the variable time series data set is converted to obtain an input feature vector set; the sensitivity index of each sub-feature vector in the input feature vector set is calculated; the features in the input feature vector set are sorted according to the sensitivity index, and the input feature vector set is divided into a main feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the main feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset; the main feature vector set and the secondary feature vector set are respectively input into the target bidirectional long short-term memory network model to obtain the prediction result of whether the target disk is faulty. By adopting the above technical solution, it is possible to reasonably judge which attributes have the greatest "correlation" with the disk status, separate or weaken the influence of irrelevant redundant attributes on the prediction results, and then when the input features are input into the model for prediction, the influence of redundant attributes on the prediction results is reduced through advance analysis of sensitivity, solving the problem of low prediction accuracy of the disk failure prediction scheme in the existing method.
[0035] In an exemplary embodiment, the main feature vector set and the secondary feature vector set are respectively input into a target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, including: initializing the target bidirectional long short-term memory network model, determining the first weight and number of layers corresponding to the initial setting of the model; using the main feature vector set as the input of the first layer sub-network in the model, and calculating the first output feature corresponding to the main feature vector set as input according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model; connecting the first output feature with the secondary feature vector set as the input of the second layer sub-network in the model, and again calculating the first output feature according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model; The forward propagation algorithm corresponding to the target bidirectional long short-term memory network model is used to calculate to obtain the second output feature; the above process of determining the output feature is executed repeatedly until the target output feature that meets the numerical value corresponding to the number of layers is obtained; wherein the number of neurons contained in each sub-network is less than or equal to the first weight; the target output feature is flattened to obtain a target feature vector; the target feature vector is synchronized to the fully connected layer of the target bidirectional long short-term memory network model for classification processing to obtain a prediction confidence value for indicating a disk failure; the prediction confidence value is compared with the prediction confidence threshold; and a prediction result of whether the target disk is faulty is generated according to the comparison result.
[0036] As an optional implementation, the input vector is divided into main features and secondary features through Sobol sensitivity analysis, and the high-sensitivity main features are input into the first layer of the BiLSTM (Bi-directional Long Short-Term Memory, BiLSTM for short) network, and the low-sensitivity supplementary features are input into the subsequent layers of the BiLSTM network, avoiding the disadvantages caused by feature redundancy. Finally, by chaining and stacking BiLSTM, each layer learns the main features and secondary features in turn, fully capturing the multi-scale information of the disk operation status, so that the model has good fitting ability and generalization performance.
[0037] It should be noted that Sobol sensitivity analysis is a method used to assess the degree of influence of model input variables on output variables. This method determines which variables have the greatest impact on changes in output variables by decomposing the total variance and attributing it to individual input variables and their interactions. Sobol sensitivity analysis can help researchers understand the interactions between the input variables of the model and identify those that are most important. This helps guide further modeling work and optimize the design. The results of Sobol sensitivity analysis are usually presented in the form of Sobol indicators, which represent the total impact of each input variable on the output variable and the impact of the interaction between the variable and other variables. By analyzing these indicators, it is possible to determine which variables have the greatest impact on the output variables of the model and make corresponding adjustments or optimizations.
[0038] In summary, through the above implementation, the main feature vector is used as the input vector of the first layer of the BiLSTM network for feature extraction, and the output vector is obtained and connected with the secondary feature to form the input vector of the next layer of BiLSTM. Finally, by chain stacking BiLSTM, each layer learns the main features and secondary features in turn, and the BiLSTM output obtained in the last layer is the final prediction result. Compared with the traditional disk failure prediction based on LSTM (Long Short-Term Memory, LSTM for short), it has higher prediction accuracy and robustness.
[0039] In an exemplary embodiment, a prediction result of whether the target disk is faulty is generated based on the comparison result, including: when it is determined that the prediction confidence value is greater than the prediction confidence threshold, determining to generate a first prediction result that the target disk is a faulty disk; when it is determined that the prediction confidence value is less than or equal to the prediction confidence threshold, determining to generate a second prediction result that the target disk is a healthy disk.
[0040] That is to say, since the above-mentioned bidirectional long short-term memory network model contains a fully connected layer, and the confidence pair output by the fully connected layer determines whether the prediction result of the current target disk is faulty or healthy, and in order to improve the accuracy of classification, by setting the prediction confidence threshold λ, if the prediction confidence is greater than λ, it is judged as a faulty disk, otherwise it is judged as a healthy disk; then the final prediction result can be quickly obtained through the confidence, so as to realize the judgment of the fault condition of the target disk.
[0041] In an exemplary embodiment, before the main feature vector set and the secondary feature vector set are respectively input into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, the method also includes: obtaining historical fault status information of the target disk, and a historical main feature subset and a historical secondary feature subset corresponding to the historical fault status information; using the historical main feature subset and the historical secondary feature subset as model input, and using the historical fault status information as model output, performing model training on a preset bidirectional long short-term memory network model to obtain a target bidirectional long short-term memory network model.
[0042] Optionally, through the above implementation, the known fault status information and corresponding feature training model are utilized, so that the subsequent model can quickly determine the fault status of the target disk by inputting the current features of the target disk, and is trained by the historical information corresponding to the target disk, thereby further improving the accuracy of the prediction.
[0043] In an exemplary embodiment, after model training is performed on a preset bidirectional long short-term memory network model, the method further includes: updating the weights of each sub-network layer and the fully connected layer deviation in the preset bidirectional long short-term memory network model through a back-propagation algorithm to obtain a second weight and a target deviation; when it is determined that the second weight is greater than the first weight and the target deviation is less than the preset deviation, error convergence is determined, and the trained target bidirectional long short-term memory network model is output.
[0044] It is understandable that in order to ensure the training effect of the model, after each prediction, the weights of the sub-network and the deviation of the fully connected layer are updated in real time through the back-propagation algorithm. The update makes the prediction accuracy for the target disk better. After using the bidirectional long short-term memory network model multiple times, the bidirectional long short-term memory network model can more comprehensively understand the disk operation status and have better prediction accuracy and robustness.
[0045] In an exemplary embodiment, multiple attribute parameters of a target disk are obtained, and a variable time series data set is constructed based on the multiple attribute parameters, including: obtaining input data transmitted by the target disk within a preset time period; identifying attribute parameters associated with the input data to obtain the multiple attribute parameters; determining a time change interval corresponding to each of the multiple attribute parameters; and performing combined data normalization processing according to a preset arrangement order of the attribute parameters and the time change interval to obtain a variable time series data set.
[0046] In an exemplary embodiment, after the main feature vector set and the secondary feature vector set are respectively input into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, the method further includes: when the prediction result is that the target disk is in a faulty state, collecting basic disk information corresponding to the target disk, wherein the basic disk information is used to indicate the disk storage location and the specifications of the disk; and sending the basic disk information to the target object that manages the target disk to instruct the target object to perform maintenance on the target disk.
[0047] In summary, through the above implementation, after determining that the target disk has a fault, the corresponding target disk administrator can be notified in time, so that the administrator can make corresponding handling methods before determining that the target disk may have a fault, which will reduce the probability of failure and improve the overall operation stability of the target disk.
[0048] Obviously, the above-described embodiments are only embodiments of a part of the present invention, rather than all embodiments. In order to better understand the above method, the above process is described below in conjunction with embodiments, but it is not intended to limit the technical solutions of the embodiments of the present invention, specifically:
[0049] In practical applications, there are many SMART attributes used to describe the disk status. When predicting disk failures, traditional prediction methods, such as those based on simple linear strategies or shallow machine learning models, cannot handle complex and changeable disk operation data. They often find it difficult to accurately capture their inherent temporal and nonlinear characteristics, thus limiting the prediction accuracy. The traditional LSTM network can alleviate the above problems to a certain extent. However, the traditional LSTM only trains input signals in one direction, which makes it easy to ignore past and future feature information, affecting the prediction performance. BiLSTM can access past and future information at the same time at each time step, thereby understanding the sequence data more comprehensively. In addition, there are many SMART attributes used to describe the disk status. How to reasonably judge which attributes have the greatest "correlation" with the disk status and separate or weaken irrelevant redundant attributes has a significant impact on the prediction results. How to reasonably judge which attributes have the greatest "correlation" with the disk status and separate or weaken irrelevant redundant attributes has a significant impact on the prediction results.
[0050] In order to avoid the above-mentioned problems, in an exemplary embodiment, a BiLSTM disk failure prediction method based on feature sensitivity analysis is proposed. The method first constructs a multivariate time series data set according to the SMART (Self Monitoring Analysis and Reporting Technology) attribute of the disk. The data set essentially belongs to a multidimensional feature vector describing the operation status of the disk. Then, Sobol sensitivity analysis is performed on the input feature vector to output the main feature vector and the secondary feature vector. The sensitivity analysis quantitatively evaluates the impact of each input parameter of the model on the output result. The main feature is a feature that is highly sensitive to the output, and the secondary feature is a feature that is not so sensitive to the output. Then, the main feature vector is used as the input vector of the first layer of the BiLSTM network for feature extraction. After obtaining the output vector, it is connected with the secondary feature to form the input vector of the next layer of BiLSTM. Finally, by chain stacking BiLSTM, each layer learns the main feature and the secondary feature in turn, and the BiLSTM output obtained in the last layer is the final prediction result. Compared with the traditional LSTM-based disk failure prediction, this method has higher prediction accuracy and robustness.
[0051] As an optional implementation, Figure 3 : is a flow chart of a BiLSTM disk failure prediction method based on feature sensitivity analysis according to an embodiment of the present application. The specific implementation steps of the above-mentioned BiLSTM disk failure prediction method based on feature sensitivity analysis are as follows:
[0052] Step 1. Construct a multivariate time series data set based on the SMART operating status of the disk. Assume that the multivariate time series is represented as Xrxm = [x1, x2, ..., xr-1, xr], where xi = [e1, e2, ..., em] (1 < i < r) is the input data of time step i, m is the number of selected disk SMART attributes, and Xr×m is normalized by Z-sCOre. Divide the data set into training set and test set in a ratio of 8:2;
[0053] Step 2: Sobol sensitivity analysis. Specifically, Sobol sensitivity calculation is performed on the feature vector Xr×m to obtain the Sobol sensitivity index SI of each input feature vector.
[0054] Step 3: Sort the feature vector Xr×m according to the SI value, and divide it into the main input feature X1 and the secondary input feature X2 according to the SI influence coefficient;
[0055] Step 4: Initialize the weights and number of layers of BiLSTM, set the number of BiLSTM layers to 3, and the number of hidden neurons to 50; use the main features as the input vector of the first layer of the BiLSTM network for feature extraction, that is, calculate its first layer output H(1) according to the BiLSTM forward propagation algorithm; connect the first layer BiLSTM output H(1) with the secondary feature X2 as the next layer BiLSTM input U(1); repeat the calculation in the previous step until the final output U(3) of the third layer H(3) is calculated;
[0056] Step 5: Flatten the final output U(3) of BiLSTM to obtain F(1)=[f1, f2, ..., fn], where n is the number of neurons in the last layer of BiLSTM;
[0057] Step 6: Input F(1) into two fully connected layers for Softmax classification processing, and the output value is the prediction confidence of disk failure. For example, set the prediction confidence threshold λ. If the prediction confidence is greater than λ, it is judged as a faulty disk, otherwise it is judged as a healthy disk.
[0058] Step 7: Determine the prediction results.
[0059] It should be noted that in the process of implementing the above process, the weights and biases of each BiLSTM layer and the fully connected layer are updated through the back-propagation algorithm until convergence;
[0060] In summary, by using the BiLSTM network, by simultaneously modeling the disk status signals in two directions, the limitation of LSTM training only on input signals in one direction is made up. This allows for a more comprehensive understanding of the disk operation status. Secondly, through Sobol sensitivity analysis, the input vector is divided into primary features and secondary features. The primary features are used as the input vector of the first layer of the BiLSTM network for feature extraction. After the output vector is obtained, it is connected with the secondary features to form the input vector of the next layer of BiLSTM. Finally, by chaining and stacking BiLSTM, each layer learns the primary features and secondary features in turn, and the BiLSTM output obtained in the last layer is the final prediction result. Compared with the traditional LSTM-based disk failure prediction, this method has higher prediction accuracy and robustness.
[0061] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0062] In this embodiment, a disk failure prediction device is also provided to implement the above-mentioned embodiments and preferred implementation modes, which have been described and will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the modules described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0063] Figure 4 is a structural block diagram of a disk failure prediction device according to an embodiment of the present application, such as Figure 4 As shown, the device comprises:
[0064] An acquisition module 42 is used to acquire multiple attribute parameters of the target disk, construct a variable time series data set based on the multiple attribute parameters, and transform the variable time series data set to obtain an input feature vector set;
[0065] A calculation module 44, used to calculate the sensitivity index of each sub-feature vector in the input feature vector set;
[0066] A partitioning module 46, configured to sort the features in the input feature vector set according to the sensitivity index, and divide the input feature vector set into a primary feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the primary feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset;
[0067] The prediction module 48 is used to input the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty.
[0068] The above device obtains multiple attribute parameters of the target disk, constructs a variable time series data set based on the multiple attribute parameters, and converts the variable time series data set to obtain an input feature vector set; calculates the sensitivity index of each sub-feature vector in the input feature vector set; sorts the features in the input feature vector set according to the sensitivity index, and divides the input feature vector set into a main feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the main feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset; and inputs the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty. By adopting the above technical solution, it is possible to reasonably determine which attributes have the greatest "correlation" with the disk status, separate or weaken the influence of irrelevant redundant attributes on the prediction results, and then when the input features are input into the model for prediction, the influence of redundant attributes on the prediction results is reduced through advance analysis of sensitivity, thereby solving the problem of low prediction accuracy of the disk failure prediction scheme in the existing method.
[0069] In an exemplary embodiment, the prediction module is further used to initialize the target bidirectional long short-term memory network model, determine the first weight and number of layers corresponding to the initial setting of the model; use the main feature vector set as the input of the first layer sub-network in the model, and calculate the first output feature corresponding to the main feature vector set as the input according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model; connect the first output feature with the secondary feature vector set as the input of the second layer sub-network in the model, and calculate again according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model to obtain the second output feature; loop the above process of determining the output feature until the target output feature that meets the numerical value corresponding to the number of layers is obtained; wherein the number of neurons contained in each sub-network is less than or equal to the first weight; flatten the target output feature to obtain the target feature vector; synchronize the target feature vector to the fully connected layer of the target bidirectional long short-term memory network model for classification processing to obtain a prediction confidence value for indicating a disk failure; compare the prediction confidence value with the prediction confidence threshold; and generate a prediction result of whether the target disk is faulty according to the comparison result.
[0070] In an exemplary embodiment, the above-mentioned prediction module is also used to determine to generate a first prediction result that the target disk is a faulty disk when it is determined that the prediction confidence value is greater than the prediction confidence threshold; and to determine to generate a second prediction result that the target disk is a healthy disk when it is determined that the prediction confidence value is less than or equal to the prediction confidence threshold.
[0071] In an exemplary embodiment, the above-mentioned device also includes: a training module, which is used to input the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain the prediction result of whether the target disk is faulty, and obtain the historical fault status information of the target disk, as well as the historical main feature subset and historical secondary feature subset corresponding to the historical fault status information; use the historical main feature subset and the historical secondary feature subset as model input, use the historical fault status information as model output, perform model training on the preset bidirectional long short-term memory network model, and obtain the target bidirectional long short-term memory network model.
[0072] In an exemplary embodiment, the above-mentioned training module also includes: an updating unit, which is used to update the weights and fully connected layer deviations of each layer of the sub-network in the preset bidirectional long short-term memory network model through a back propagation algorithm after model training, to obtain a second weight and a target deviation; when it is determined that the second weight is greater than the first weight and the target deviation is less than the preset deviation, it is determined that the error converges, and the target bidirectional long short-term memory network model that has completed the training is output.
[0073] In an exemplary embodiment, the above-mentioned acquisition module is also used to acquire input data transmitted by the target disk within a preset time period; identify attribute parameters associated with the input data to obtain the multiple attribute parameters; determine the time change interval corresponding to each attribute parameter in the multiple attribute parameters; perform combined data standardization processing according to the preset arrangement order of the attribute parameters and the time change interval to obtain a variable time series data set.
[0074] In an exemplary embodiment, the above-mentioned device also includes: a maintenance unit, which is used to input the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty. When the prediction result is that the target disk is in a faulty state, the disk basic information corresponding to the target disk is collected, wherein the disk basic information is used to indicate the disk storage location and the specifications of the disk; and the disk basic information is sent to the target object that manages the target disk to instruct the target object to perform maintenance on the target disk.
[0075] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0076] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0077] Optionally, in this embodiment, the computer program may be configured to perform the following steps by means of the computer program:
[0078] S1, obtaining multiple attribute parameters of a target disk, constructing a variable time series data set based on the multiple attribute parameters, and converting the variable time series data set to obtain an input feature vector set;
[0079] S2, calculating the sensitivity index of each sub-feature vector in the input feature vector set;
[0080] S3, sorting the features in the input feature vector set according to the sensitivity index, and dividing the input feature vector set into a primary feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the primary feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset;
[0081] S4, inputting the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty.
[0082] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0083] The embodiment of the present application also provides an electronic device, such as Figure 5 As shown, the electronic device includes a memory 502 and a processor 504. The memory 502 stores a computer program, and the processor 504 is configured to execute the steps in any of the above method embodiments through the computer program.
[0084] Optionally, in this embodiment, the processor 504 may be configured to perform the following steps through a computer program:
[0085] S1, obtaining multiple attribute parameters of a target disk, constructing a variable time series data set based on the multiple attribute parameters, and converting the variable time series data set to obtain an input feature vector set;
[0086] S2, calculating the sensitivity index of each sub-feature vector in the input feature vector set;
[0087] S3, sorting the features in the input feature vector set according to the sensitivity index, and dividing the input feature vector set into a primary feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the primary feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset;
[0088] S4, inputting the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty.
[0089] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0090] Alternatively, a person skilled in the art may understand that: Figure 5 The structure shown is for illustration only. Figure 5 The structure of the electronic device is not limited. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 5 Different configurations shown.
[0091] Among them, the memory 502 can be used to store software programs and modules, such as the program instructions / modules corresponding to the disk failure prediction method and device in the embodiment of the present application. The processor 504 executes various functional applications and data processing by running the software programs and modules stored in the memory 502, that is, realizing the above-mentioned disk failure prediction method. The memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 502 may further include a memory remotely located relative to the processor 504, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 502 can be specifically, but not limited to, used to store information such as system configuration files. As an example, such as Figure 5 As shown, the memory 502 may include but is not limited to all modules in the disk failure prediction device. In addition, it may also include but is not limited to other module units in the disk failure prediction device, which will not be described in detail in this example.
[0092] Optionally, the transmission device 506 is used to receive or send data via a network. Specific examples of the network may include wired networks and wireless networks. In one example, the transmission device 506 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 506 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0093] In addition, the electronic device mentioned above further includes: a display 508; and a connection bus 510, which is used to connect various module components in the electronic device mentioned above.
[0094] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.
[0095] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.
[0096] The embodiments of the present application also provide a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.
[0097] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0098] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for predicting disk failure, characterized in that: include: Acquire multiple attribute parameters of the target disk, construct a variable time series data set based on the multiple attribute parameters, and transform the variable time series data set to obtain an input feature vector set; Calculating a sensitivity index of each sub-feature vector in the input feature vector set; Sorting the features in the input feature vector set according to the sensitivity index, and dividing the input feature vector set into a primary feature subset and a secondary feature subset based on the sorting result, wherein a minimum sensitivity index associated with the primary feature subset is greater than or equal to a maximum sensitivity index in the secondary feature subset; The main feature vector set and the secondary feature vector set are respectively input into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty.
2. The method according to claim 1, characterized in that Inputting the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty, including: Initializing the target bidirectional long short-term memory network model, and determining the first weight and number of layers corresponding to the initial setting of the model; Using the main feature vector set as the input of the first layer sub-network in the model, and calculating the first output feature corresponding to the main feature vector set as the input according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model; Connecting the first output feature with the secondary feature vector set as the input of the second layer sub-network in the model, and calculating again according to the forward propagation algorithm corresponding to the target bidirectional long short-term memory network model to obtain the second output feature; The above process of determining the output feature is executed repeatedly until the target output feature that meets the value corresponding to the number of layers is obtained; wherein the number of neurons included in each sub-network is less than or equal to the first weight; Performing a flattening operation on the target output feature to obtain a target feature vector; Synchronizing the target feature vector to the fully connected layer of the target bidirectional long short-term memory network model for classification processing to obtain a prediction confidence value for indicating a disk failure; comparing the prediction confidence value with a prediction confidence threshold; A prediction result of whether the target disk is faulty is generated according to the comparison result.
3. The method according to claim 2, characterized in that Generating a prediction result of whether the target disk is faulty according to the comparison result, including: In the case where it is determined that the prediction confidence value is greater than the prediction confidence threshold, determining to generate a first prediction result that the target disk is a failed disk; When it is determined that the prediction confidence value is less than or equal to the prediction confidence threshold, a second prediction result that the target disk is a healthy disk is generated.
4. The method according to claim 1, characterized in that: Before respectively inputting the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, the method further includes: Acquire historical fault status information of the target disk, and a historical main feature subset and a historical secondary feature subset corresponding to the historical fault status information; The historical main feature subset and the historical secondary feature subset are used as model input, the historical fault status information is used as model output, and a preset bidirectional long short-term memory network model is trained to obtain a target bidirectional long short-term memory network model.
5. The method according to claim 4, characterized in that After performing model training on the preset bidirectional long short-term memory network model, the method further includes: Updating the weights of each sub-network and the fully connected layer deviation in the preset bidirectional long short-term memory network model through a back-propagation algorithm to obtain a second weight and a target deviation; When it is determined that the second weight is greater than the first weight and the target deviation is less than the preset deviation, it is determined that the error converges and the trained target bidirectional long short-term memory network model is to be output.
6. The method according to claim 1, characterized in that Acquire multiple attribute parameters of the target disk, and construct a variable time series data set based on the multiple attribute parameters, including: Acquire input data transmitted by the target disk within a preset time period; Identifying attribute parameters associated with the input data to obtain the plurality of attribute parameters; Determine a time variation interval corresponding to each of the plurality of attribute parameters; Combined data standardization processing is performed according to the preset arrangement order of the attribute parameters and the time change interval to obtain a variable time series data set.
7. The method according to claim 1, characterized in that After respectively inputting the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model to obtain a prediction result of whether the target disk is faulty, the method further includes: In the case where the prediction result is that the target disk is in a fault state, collecting disk basic information corresponding to the target disk, wherein the disk basic information is used to indicate the disk storage location and the specifications of the disk; The disk basic information is sent to a target object that manages the target disk to instruct the target object to perform maintenance processing on the target disk.
8. A disk failure prediction device, characterized in that: include: An acquisition module, used for acquiring multiple attribute parameters of a target disk, constructing a variable time series data set based on the multiple attribute parameters, and converting the variable time series data set to obtain an input feature vector set; A calculation module, used for calculating the sensitivity index of each sub-feature vector in the input feature vector set; a partitioning module, configured to sort the features in the input feature vector set according to the sensitivity index, and divide the input feature vector set into a primary feature subset and a secondary feature subset based on the sorting result, wherein the minimum sensitivity index associated with the primary feature subset is greater than or equal to the maximum sensitivity index in the secondary feature subset; The prediction module is used to input the main feature vector set and the secondary feature vector set into the target bidirectional long short-term memory network model respectively to obtain a prediction result of whether the target disk is faulty.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.