Hard disk failure prediction method and device, computer readable medium and electronic device

Abstract

Embodiments of the present application provide a hard disk failure prediction method, device, computer readable medium and electronic equipment, the method comprising: obtaining feature data and labels corresponding to a plurality of hard disks respectively; performing feature elimination operation on a plurality of features according to the feature data and based on at least one feature elimination method to obtain target features; clustering the hard disks indicated by each label to have a failure according to the feature values corresponding to the target features to obtain clustering results; correcting the labels of the hard disks indicated to have a failure according to the clustering results; extracting time sequence feature data corresponding to the hard disks; training a deep learning model using the time sequence feature data corresponding to the hard disks and the labels as training samples to obtain a hard disk failure prediction model, and predicting the failure of a target hard disk based on the hard disk failure prediction model. The present application can predict the failure of a hard disk in advance and improve the accuracy of hard disk failure prediction.

Description

Technical Field

[0001] This application relates to the field of hardware management technology, and more specifically, to a hard disk failure prediction method, apparatus, computer-readable medium, and electronic device. Background Technology

[0002] Currently, hard drive manufacturers generally use threshold-based methods to predict hard drive failures. This involves using SMART (Self-Monitoring Analysis and Reporting Technology) to collect various performance metrics of the hard drive during operation and comparing them with pre-defined hard drive failure warning thresholds. If the threshold is exceeded, a failure alarm is triggered. To reduce the number of hard drives returned for inspection and repair due to failure warnings, thereby lowering costs, hard drive manufacturers often choose to minimize the false alarm rate when setting thresholds, but this comes at the cost of predictive accuracy. Summary of the Invention

[0003] The embodiments of this application provide a hard disk failure prediction method, apparatus, computer-readable medium, and electronic device, which can at least to some extent predict hard disk failures in advance and improve the accuracy of hard disk failure prediction.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a hard disk failure prediction method is provided. The method includes: acquiring feature data and tags corresponding to multiple hard disks, wherein the feature data includes multiple features of the hard disk and feature values ​​corresponding to each feature, the tags indicate whether the hard disk is faulty, and the tags are assigned to the hard disks based on the detection of hard disk failure according to a specified method; performing feature removal operations on the multiple features based on the feature data and at least one feature removal method to obtain target features; performing clustering processing on hard disks whose tags indicate failure according to the feature values ​​corresponding to the target features to obtain clustering results; performing tag correction on hard disks whose tags indicate failure according to the clustering results to correct the tags of at least one hard disk whose tags indicate failure to indicate that the hard disk is normal; extracting time-series feature data corresponding to the hard disks, wherein the time-series feature data is multiple feature values ​​corresponding to the target features collected according to time series; training a deep learning model using the time-series feature data and tags corresponding to the hard disks as training samples to obtain a hard disk failure prediction model, and predicting the failure of a target hard disk based on the hard disk failure prediction model.

[0006] According to one aspect of the embodiments of this application, a hard disk failure prediction device is provided. The device includes: an acquisition unit, configured to acquire feature data and tags corresponding to multiple hard disks respectively, wherein the feature data includes multiple features of the hard disk and feature values ​​corresponding to each feature, and the tags indicate whether the hard disk is faulty, and the tags are assigned to the hard disks based on the detection of hard disk failure according to a specified method; a removal unit, configured to perform feature removal operations on the multiple features based on the feature data and at least one feature removal method to obtain target features; a clustering unit, configured to perform clustering processing on hard disks whose tags indicate failure according to the feature values ​​corresponding to the target features to obtain clustering results; a correction unit, configured to correct the tags of hard disks whose tags indicate failure according to the clustering results, so as to correct the tags of at least one hard disk whose tags indicate failure to indicate that the hard disk is normal; an extraction unit, configured to extract time-series feature data corresponding to the hard disks, wherein the time-series feature data is multiple feature values ​​corresponding to the target features collected according to time series; and a training and prediction unit, configured to train a deep learning model using the time-series feature data and tags corresponding to the hard disks as training samples to obtain a hard disk failure prediction model. And based on the hard disk failure prediction model, the target hard disk is used to predict failures.

[0007] In some embodiments of this application, based on the foregoing scheme, the feature removal method includes any one of the following: feature removal based on the variance of each feature value corresponding to the feature, feature removal based on the failure rate three times the standard deviation corresponding to the feature, and feature removal based on the comparison between the feature value corresponding to the feature and the corresponding feature threshold. The failure rate three times the standard deviation is the percentage of hard drives with labels indicating failure among hard drives whose feature values ​​reach three times the standard deviation, and the failure rate three times the standard deviation is the sum of the average value of each feature value corresponding to the feature and three times the standard deviation of each feature value corresponding to the feature.

[0008] In some embodiments of this application, based on the foregoing scheme, the elimination unit is further configured to perform feature elimination based on the variance of each feature value corresponding to the feature as follows: for each feature, determine the variance of each feature value based on each feature value corresponding to the feature in each feature data; and eliminate the feature if the variance determined for the feature is less than a preset variance threshold.

[0009] In some embodiments of this application, based on the foregoing scheme, the rejection unit is further configured to reject the feature when performing feature rejection based on the failure rate three times the standard deviation of the feature: the feature is rejected if the failure rate three times the standard deviation of the feature is less than a preset failure rate threshold.

[0010] In some embodiments of this application, based on the foregoing scheme, the elimination unit is further configured to perform feature elimination by comparing the feature value corresponding to the feature with the corresponding feature threshold as follows: determine among the plurality of hard disks the hard disks whose feature value corresponding to the feature does not exceed the first feature threshold corresponding to the feature; determine the proportion of hard disks with corresponding label indicating failure among the determined hard disks; and eliminate the feature based on the proportion of the number being less than a preset proportion threshold.

[0011] In some embodiments of this application, based on the foregoing scheme, the extraction unit is configured to: extract first time-series feature data of a faulty hard drive, wherein the faulty hard drive is a hard drive identified as having failed; extract second time-series feature data of a normal hard drive, wherein the normal hard drive is a hard drive identified as not having failed; and perform normalization and standardization operations on the first time-series feature data and the second time-series feature data to obtain the time-series feature data corresponding to the hard drive.

[0012] In some embodiments of this application, based on the foregoing scheme, the extraction unit is configured to: extract first time-series feature data of hard disks whose labels indicate faults.

[0013] In some embodiments of this application, based on the foregoing scheme, the specifying method includes: collecting feature data of the hard disk at time step intervals; performing fault monitoring on the hard disk based on the feature data; the extraction unit is configured to: take the end time of a predetermined time period after the time when the label indicating the fault of the hard disk is marked as the end point, take the time located before the end point and separated from the end point by a predetermined number of time steps as the start point, and extract the first time-series feature data between the start point and the end point.

[0014] In some embodiments of this application, based on the foregoing scheme, the extraction unit is further configured to: if, according to the feature data corresponding to the hard disk, it is determined that the feature value corresponding to the target feature in the feature data is less than the second feature threshold corresponding to the target feature, then the hard disk is determined to be a faulty hard disk; taking the moment when the feature value corresponding to the target feature in the collected feature data is less than the second feature threshold corresponding to the target feature as the abnormal moment, taking the first abnormal moment and the last abnormal moment as the end point respectively, and taking the moment located before the end point and spaced a predetermined number of time steps from the end point as the start point, extract the first time-series feature data of the faulty hard disk between the start point and the end point.

[0015] In some embodiments of this application, based on the foregoing scheme, the deep learning model includes a gated recurrent unit.

[0016] In some embodiments of this application, based on the foregoing scheme, the deep learning model is trained according to a preset loss function, which includes a penalty term; during the training process of the deep learning model, the penalty term in the preset loss function is dynamically adjusted according to whether the predicted label obtained by the deep learning model from predicting the time-series feature data is consistent with the label in the training sample to which the time-series feature data belongs.

[0017] In some embodiments of this application, based on the foregoing scheme, the deep learning model predicts the target hard drive as a faulty hard drive based on the predicted probability of the target hard drive belonging to a first probability interval, and predicts the target hard drive as a normal hard drive based on the predicted probability of the target hard drive belonging to a second probability interval, wherein the length of the first probability interval is less than the length of the second probability interval.

[0018] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the face-related data management method as described in the above embodiments.

[0019] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the hard disk failure prediction method as described in the above embodiments.

[0020] In some embodiments of this application, after acquiring feature data and labels corresponding to multiple hard drives, feature removal is performed based on the feature data and one or more feature removal methods to obtain target features. Compared with other removed features, the target features are more closely related to whether the hard drive is faulty. The hard drive fault prediction model is trained based on the feature data corresponding to the target features, and the hard drive fault prediction model is a deep learning model, thus improving the accuracy of the hard drive fault prediction model. The labels corresponding to the multiple hard drives are automatically labeled after the hard drives are monitored for faults in a specified way, thus greatly reducing the cost of obtaining samples. After determining the target features, the hard drives with labels indicating faults are clustered, and the labels are corrected based on the clustering results, making the data used for model training more accurate and further improving the accuracy of the hard drive fault prediction model. The data used to train the model is the extracted time-series feature data corresponding to the hard drives, thus enabling early prediction of hard drive faults and reducing the cost of business migration.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0023] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown; Figure 2 A flowchart of a hard disk failure prediction method according to an embodiment of this application is shown; Figure 3 A schematic diagram showing the coverage of various SMART features of a certain model of hard disk according to an embodiment of this application is illustrated; Figure 4 A flowchart illustrating feature removal based on the variance of each feature value corresponding to a feature, according to an embodiment of this application, is shown. Figure 5 A flowchart illustrating feature removal based on a comparison of the feature value corresponding to the feature with the corresponding feature threshold according to an embodiment of this application is shown. Figure 6 The illustration shows a schematic diagram of a specific implementation method for feature removal based on a comparison of the feature value corresponding to the feature and the corresponding feature threshold according to an embodiment of this application; Figure 7 A flowchart illustrating the extraction of timing feature data corresponding to a hard disk according to an embodiment of this application is shown; Figure 8 A schematic diagram illustrating a first method for extracting positive sample data according to an embodiment of this application is shown; Figure 9 A schematic diagram illustrating a second method for extracting positive sample data according to an embodiment of this application is shown; Figure 10 A flowchart illustrating the extraction of first timing feature data of a faulty hard disk according to an embodiment of this application is shown; Figure 11 A schematic diagram illustrating a third method for extracting positive sample data according to an embodiment of this application is shown; Figure 12 A schematic diagram illustrating a fourth method for extracting positive sample data according to an embodiment of this application is shown; Figure 13 A schematic diagram illustrating a method for extracting negative sample data according to an embodiment of this application is shown; Figure 14 A schematic diagram illustrating the principle of a hard disk failure prediction model according to an embodiment of this application is shown; Figure 15 A block diagram of a hard disk failure prediction apparatus according to an embodiment of this application is shown; Figure 16 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0025] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0026] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0028] As the company's business grows, the number of servers increases, which accelerates the rise in the probability of failure. This will have a significant negative impact on the business and greatly increase operating costs. Hard drive failures account for 75% of all failures.

[0029] SMART (Self-Monitoring Analysis and Reporting Technology) is an automated hard drive status detection and early warning system and standard provided by hard drive manufacturers.

[0030] Currently, hard drive manufacturers generally use threshold-based methods to predict hard drive failures. This involves using SMART technology to collect various performance metrics of the hard drive during operation and comparing them to a set failure warning threshold. Exceeding the threshold triggers a failure alarm. To reduce the number of hard drives returned for inspection and repair due to failure warnings, thereby lowering costs, manufacturers often aim to minimize false alarms when setting thresholds, but this comes at the expense of prediction accuracy. The prediction accuracy using threshold-based methods is approximately 3%-10%, with a false alarm rate of about 0.1%. Therefore, because current solutions only monitor the hard drive, replacing or returning it for repair only after a failure is detected, this incurs high costs for business migration. Furthermore, there is a high NTF (No Trouble Found) rate after hard drives are returned to the factory, meaning that after replacement or repair, the faulty component is deemed fault-free.

[0031] The applicant discovered that enterprise-level hard drive failure prediction technology is a rare class prediction problem on an extremely imbalanced binary classification dataset. In this dataset, failed drives are the rare class, while non-failed drives are the majority class.

[0032] Currently, popular data classification methods include Support Vector Machines (SVMs) and Random Forests. SVMs are supervised learning models and related learning algorithms used in classification and regression analysis. The applicant argues that SVMs have low sensitivity to data imbalance and poor model performance; while Random Forests can only make predictions using data from a single time point, resulting in similarly poor prediction performance.

[0033] The applicant discovered that SMART data exhibits a certain trend of change over time, thus allowing for the use of time series data to learn a superior model.

[0034] Therefore, this application first provides a hard disk failure prediction method. Through this method, a hard disk failure prediction model can be trained using SMART data in the time dimension. On this basis, SMART data of a certain hard disk in the time dimension can also be input into the hard disk failure prediction model to predict the failure of the hard disk in advance and ensure the accuracy of the failure prediction.

[0035] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown. For example... Figure 1 As shown, the system architecture 100 may include a computer device 101, a hard disk monitoring server 102, a network 103, and monitored servers. The monitored servers may specifically include a first server 104, a second server 105, a third server 106, etc. Each monitored server is equipped with a hard disk and can communicate with the hard disk monitoring server 102 via the network 103. The computer device 101 can also communicate directly with the hard disk monitoring server 102. The computer device 101 can be used by users, such as server maintenance engineers. Agent programs are deployed on each monitored server, while monitoring programs are deployed on the hard disk monitoring server 102. Figure 1 The process of hard drive failure prediction implemented by the system architecture shown is as follows: First, the monitoring program deployed on the hard disk monitoring server 102 sends monitoring commands to each monitored server via network 103. Each monitored server reports its hard disk's SMART data to the hard disk monitoring server 102 at specified time intervals according to the monitoring commands. The monitoring program on the hard disk monitoring server 102 analyzes the received SMART data. When the analysis results indicate that the hard disk of the monitored server corresponding to the SMART data is faulty, a fault record is created for that hard disk. Since the hard disk SMART data is reported to the hard disk monitoring server 102 by the monitored servers at specified time intervals, the SMART data obtained by the hard disk monitoring server 102 is the time-series SMART data of each hard disk. The hard disk monitoring server 102 stores this time-series SMART data.

[0036] Next, the hard disk monitoring server 102 uses the SMART data stored in its time series to train a deep learning model, thereby obtaining a hard disk failure prediction model.

[0037] Then, one or more monitored servers continue to report SMART data of the hard drives to the hard drive monitoring server 102. These monitored servers may be the same as or different from the monitored servers that report data to the hard drive monitoring server 102 for training the model. The hard drive monitoring server 102 analyzes the received SMART data of the hard drives and makes fault predictions by deploying a hard drive fault prediction model on it, thereby continuously monitoring the status of the hard drives on the monitored servers.

[0038] Computer device 101 can interact with hard disk monitoring server 102. For example, a monitoring page can be displayed on computer device 101, which shows the status data of each hard disk provided by hard disk monitoring server 102. This allows users of computer device 101 to promptly identify which hard disk on the monitored server will fail.

[0039] In some embodiments of this application, each monitored server is equipped with multiple hard drives. Each time a monitored server reports data, it reports the SMART data of multiple hard drives to the hard drive monitoring server 102.

[0040] In some embodiments of this application, the hard disk monitoring server 102 labels hard disks that have undergone fault reporting operations with fault labels and labels hard disks that have not undergone fault reporting operations with normal labels. The hard disk monitoring server 102 trains a deep learning model based on the SMART data in the time series and the labels corresponding to the SMART data.

[0041] In some embodiments of this application, the SMART data of the hard disk includes multiple features and feature values ​​corresponding to each feature. Before training the deep learning model, the hard disk monitoring server 102 also performs feature removal based on the SMART data of the hard disk and using at least one feature removal method to obtain target features. Finally, the deep learning model is trained based on the feature values ​​corresponding to the target features.

[0042] In some embodiments of this application, before training the deep learning model, the hard disk monitoring server 102 also performs clustering operations on the hard disks labeled with fault tags based on the SMART data of the hard disks, and corrects the labels of the hard disks according to the clustering results.

[0043] It should be noted that, Figure 1 The illustration shown is merely one embodiment of this application. Although in Figure 1 In the embodiment, each monitored server reports SMART data of the hard drive to the hard drive monitoring server 102 via network 103, and the computer device 101 can directly obtain the information on the hard drive monitoring server 102. However, in other embodiments of this application, each monitored server can be directly connected to the hard drive monitoring server 102 and report the SMART data of the hard drive to the hard drive monitoring server 102. A network can also be established between the computer device 101 and the hard drive monitoring server 102, and the computer device 101 needs to obtain the information on the hard drive monitoring server 102 through the network. Although in Figure 1 In the embodiment, the device for collecting SMART data from the hard drive and the device for deploying the hard drive fault prediction model trained using this SMART data are the same device. However, in other embodiments or specific applications of this application, they can be different terminal devices. Figure 1In the embodiment, the trained hard disk failure prediction model is ultimately deployed on the same device, and on a device other than the device with the hard disk to be monitored. However, in other embodiments of this application, the trained hard disk failure prediction model can also be deployed on various devices with the hard disk to be monitored. Figure 1 In the embodiments described, the device deploying the hard drive failure prediction model and the device providing monitoring information about the hard drive to the user are different devices. However, in other embodiments of this application, the same device can perform both tasks. This application does not limit the scope of protection in any way.

[0044] It is easy to understand that the hard disk failure prediction method provided in this application embodiment is generally executed by a server, and correspondingly, the hard disk failure prediction device is generally located in the server. However, in other embodiments of this application, the terminal may also have similar functions to the server, thereby executing the hard disk failure prediction scheme provided in this application embodiment.

[0045] Therefore, the solutions in this application embodiment can be applied to terminals or servers. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0046] In some embodiments of this application, the collection of SMART data from the hard drive and the deployment of the hard drive failure prediction model are both completed in the cloud.

[0047] The cloud is built upon cloud computing technology. Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go.

[0048] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0049] Based on logical function, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be deployed directly on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally speaking, SaaS and PaaS are upper layers compared to IaaS.

[0050] The implementation details of the technical solutions in the embodiments of this application are described in detail below: Figure 2 A flowchart of a hard disk failure prediction method according to an embodiment of this application is shown. This method can be executed by any device with computing, data processing, communication, or other functions, and these devices may also be equipped with untrained deep learning models. (Refer to...) Figure 2 As shown, this hard drive failure prediction method includes at least the following steps: In step 210, feature data and tags corresponding to multiple hard drives are obtained. The feature data includes multiple features of the hard drive and the feature value corresponding to each feature. The tags indicate whether the hard drive is faulty. The tags are assigned to the hard drives based on the detection of hard drive faults in a specified way.

[0051] The hard drives mentioned here can be of various types, including mechanical hard drives and solid-state drives (SSDs), and multiple hard drives can be of the same type and model. Of course, in other embodiments of this application, the types or models of the individual hard drives can be different.

[0052] Normally, multiple hard drives correspond to the same multiple characteristics, but it is possible that individual hard drives do not correspond to one or more characteristics.

[0053] In this step, the feature data obtained can be hard disk data collected at all time points of the time series, or hard disk data collected at one or more time points.

[0054] The label content can be any information that indicates whether the hard drive is malfunctioning. For example, the label content corresponding to whether the hard drive is malfunctioning can be "normal" and "malfunctioning", "non-malfunctioning" and "malfunctioning", "good disk" and "bad disk", "good product" and "bad product", "0" and "1", etc. In addition, the label content can also be represented by other characters or text.

[0055] The specified method can be any method that can determine whether a hard drive has failed. For example, the specified method can be a hard drive failure detection model based on support vector machines or random forests. Alternatively, the specified method can be manual fault detection performed on the hard drive by hard drive maintenance engineers or hard drive manufacturers.

[0056] Under a specified method, hard drive failures can be monitored based on the characteristics of the hard drive and the corresponding characteristic values. The tags reflect whether a hard drive with these characteristic values ​​is faulty under the judgment rules of the specified method.

[0057] In one embodiment of this application, the specified method is hard disk SMART technology, and multiple features of the hard disk are SMART characteristics.

[0058] When using SMART technology for hard drive fault monitoring, a fault log is created when a fault is detected, and the hard drive is tagged with a label indicating the fault. A SMART characteristic of a hard drive is a performance indicator. SMART data is data on certain performance indicators of the hard drive collected at a specific point in time; it can also be called SMART snapshot data.

[0059] As mentioned earlier, although hard drive SMART technology detects hard drive failures by comparing the collected hard drive operating metrics with the set hard drive failure warning thresholds, in reality, hard drive SMART technology may employ more complex metric judgment logic, and does not simply rely on a single metric to determine whether a hard drive is faulty.

[0060] Please refer to Table 1, which shows the various SMART characteristics of the hard drive model "ST4000NM0035". The values ​​in the "thresh" column for each SMART characteristic are the threshold values ​​for that SMART characteristic. When the value of a SMART characteristic is less than its corresponding threshold, it can often be determined that the hard drive has a potential fault or problem with that SMART characteristic. Hard drive SMART technology does not simply determine whether a hard drive is faulty by checking if a SMART characteristic value is less than its corresponding threshold; rather, it uses more complex rules for judgment.

[0061]

[0062] Table 1 In step 220, feature removal operations are performed on multiple features based on feature data and at least one feature removal method to obtain target features.

[0063] Feature elimination, also known as feature filtering, involves selecting target features from a pool of multiple features. The selected target features are more strongly correlated with hard drive malfunction than the other eliminated features.

[0064] Different features can be removed using different numbers and types of feature removal methods.

[0065] For example, all features can be divided into group 1 and group 2. Features in group 1 can be removed using feature removal method A, while features in group 2 can be removed using feature removal method C. Alternatively, features in group 1 can be removed using both feature removal methods A and B, while features in group 2 can be removed using feature removal method C.

[0066] Feature removal is achieved by processing and analyzing feature data.

[0067] In one embodiment of this application, after obtaining feature data and tags corresponding to multiple hard drives respectively, the hard drive failure prediction method further includes: removing the tags indicating hard drive failure and the corresponding sample data marked due to hard drive disconnection.

[0068] Specifically, when a hard drive goes offline, a fault record is created for the hard drive. In many cases, the fault record is created due to human error. For example, although 60% of hard drive offline cases are judged as hard drive failures, these hard drives are actually good drives. In this embodiment, the labels indicating hard drive failures and the corresponding sample data marked due to hard drive offlines are removed, thus ensuring the accuracy of the data and improving the accuracy of the final model.

[0069] In one embodiment of this application, before obtaining the target feature by performing feature removal operations on multiple features based on feature data and at least one feature removal method, the hard disk fault prediction method further includes: for multiple hard disks of the same model, for each feature threshold of each feature, determining the ratio of the number of hard disks with the feature threshold under that feature to the total number of hard disks in the multiple hard disks; determining the final feature threshold corresponding to each feature based on the ratio; removing hard disks whose feature thresholds are inconsistent with the final feature thresholds as data removal hard disks; and removing the feature data and tags corresponding to the data removal hard disks.

[0070] Hard drives of the same model may have different thresholds for a certain SMART feature due to differences in FW (Firmware version). In such cases, the threshold with the highest percentage can be selected as the final threshold for that SMART feature of the hard drive model. According to statistics, the percentage of the highest percentage threshold for almost every SMART feature is over 99%.

[0071] Figure 3 This diagram illustrates the coverage of various SMART features of a hard drive of a certain model according to an embodiment of this application. Please refer to... Figure 3 It shows the coverage of various SMART features for the hard drive with model number "ST4000NM0035". Each SMART feature is identified by its corresponding ID, English ID, and Chinese ID. For example, spin_up_time_value is the English ID of a SMART feature, and its corresponding Chinese ID is spindle start time, with ID 3. Figure 3 The SMART feature acquisition coverage was calculated using the following formula: SMART Feature Acquisition Coverage = Number of Feature Reports / Number of HDD Disks Reporting Data. Here, the number of feature reports refers to the threshold for a specific SMART feature, and the number of hard drives with the same threshold for that SMART feature. The number of HDD disks reporting data is the total number of hard drives that reported data. Figure 3 As can be seen, the SMART feature acquisition coverage rate for most SMART features is 99%, therefore, most of the hard drive's feature data can be preserved.

[0072] Therefore, this application embodiment ensures data reliability by filtering data based on hard disk feature thresholds, thereby improving the accuracy of the trained hard disk fault prediction model.

[0073] The feature removal method will be described in detail below.

[0074] In one embodiment of this application, the feature removal method includes any one of the following: feature removal based on the variance of each feature value corresponding to the feature, feature removal based on the failure rate three times the standard deviation of the feature, and feature removal based on the comparison between the feature value corresponding to the feature and the corresponding feature threshold. The failure rate three times the standard deviation of the feature is the proportion of hard drives with labels indicating failure among hard drives whose feature values ​​reach three times the standard deviation of the feature. The failure rate three times the standard deviation of the feature is the sum of the average value of each feature value corresponding to the feature and three times the standard deviation of each feature value corresponding to the feature.

[0075] Feature removal methods all rely on determining whether the feature value matches the hard drive's fault record in extreme cases, i.e., whether it matches the hard drive's label.

[0076] Figure 4 A flowchart illustrating feature removal based on the variance of each feature value corresponding to a feature, according to one embodiment of this application, is shown. Figure 4 As shown, it includes the following steps: In step 410, for each feature, the variance of each feature value is determined based on the feature values ​​corresponding to the feature in the feature data.

[0077] As mentioned earlier, the feature data of each hard drive includes feature values ​​corresponding to multiple features. Moreover, each hard drive has the same multiple features. Therefore, for each feature, the feature data of each hard drive has a feature value corresponding to that feature.

[0078] You can first calculate the average value of each eigenvalue, and then calculate the variance of each eigenvalue based on the average value of each eigenvalue.

[0079] In step 420, features are removed if the variance determined for a feature is less than a preset variance threshold.

[0080] The preset variance threshold can be a value close to 0.

[0081] If the variance of a feature value is close to 0, it means that the feature has no distinguishing effect between normal and faulty hard drives. Therefore, the feature can be removed.

[0082] In one embodiment of this application, the feature data further includes the original feature value corresponding to the feature. The feature value is obtained by mapping the original feature value corresponding to the feature. The target feature is obtained by performing feature removal operations on multiple features based on the feature data and based on at least one feature removal method. This includes: performing feature removal operations on multiple features based on the feature value and the original feature value in the feature data and based on at least one feature removal method to obtain the target feature.

[0083] Specifically, the raw value of a feature can be the raw value of a SMART characteristic, while the feature value can be the value of a SMART characteristic. Both the raw and value values ​​of a SMART characteristic can play a corresponding role depending on the different hard disk data processing methods.

[0084] Specifically, when removing features based on variance, it can be done as follows: Calculate the average and variance of the raw and value values ​​of each SMART feature over the global data. If the variance is close to 0, it indicates that there is no distinction between good and bad disks, and it can be removed.

[0085] In one embodiment of this application, feature rejection based on the failure rate three times the standard deviation of the corresponding feature includes: rejecting the feature if the failure rate three times the standard deviation of the corresponding feature is less than a preset failure rate threshold.

[0086] The preset failure rate threshold is set manually, for example, it can be set to 20%.

[0087] The three-standard-deviation out-of-failure rate is used to identify outliers. Specifically, the three-standard-deviation out-of-failure rate is the sum / difference of the average of the features corresponding to the feature and three times the standard deviation of the features corresponding to the feature.

[0088] for example, Figure 3 The SMART characteristic poweronhours_count_value (cumulative power-on time POH) shown above, if the average (avg) of the corresponding characteristic values ​​is 45.13072 and the standard deviation (std) is 6.957636, then the failure rate three times the standard deviation can be calculated using the following expression:

[0089] Here, number is used to calculate the number of hard drives, value is the characteristic value corresponding to this SMART feature, and failure=1 means that the hard drive's tag indicates a fault.

[0090] Therefore, when a feature's value is not within three standard deviations out of range, it indicates that the feature is an outlier. For that feature, the more hard drives whose corresponding feature values ​​are outside three standard deviations out of range, the more likely that feature is related to hard drive failure. Therefore, the failure rate outside three standard deviations can be used to effectively distinguish whether a feature is related to hard drive failure.

[0091] In one embodiment of this application, feature elimination based on the out-of-range failure rate corresponding to a feature includes: sorting each feature from largest to smallest according to the out-of-range failure rate corresponding to the three-standard-deviation failure rate; taking the features ranked first by a predetermined proportion as target features, and eliminating features other than target features.

[0092] As mentioned earlier, the higher the failure rate outside three standard deviations corresponding to a feature, the more hard drives have the feature value outside three standard deviations, which indicates that the feature is more likely to be related to hard drive failure.

[0093] You can start from the location of the SMART attribute with ID 242. Figure 3 The SMART features shown are divided into two groups. The first group includes SMART features with IDs between 184 and 242, while the second group includes the SMART feature with ID 1 and the SMART feature located below the SMART feature with ID 1.

[0094] For the second group of SMART features, we can first eliminate features based on the variance of each feature value, and then eliminate features based on the failure rate three times the standard deviation of the feature. Finally, we can select the following 7 SMART features as target features: raw_read_error_rate_value, spin_up_time_value, reallocated_sector_ct_raw, reallocated_sector_ct_value, seek_error_rate_raw, spin_retry_count_raw, and current_pending_sector_raw.

[0095] Figure 5 A flowchart illustrating feature removal based on a comparison of the feature value corresponding to the feature with a corresponding feature threshold, according to an embodiment of this application, is shown. Figure 5 As shown, the specific steps may include: In step 510, among the multiple hard disks, the hard disks whose feature values ​​corresponding to the feature do not exceed the first feature threshold corresponding to the feature are identified.

[0096] The first feature threshold and the final feature threshold mentioned earlier can be the same or different. The first feature threshold can be a threshold set manually based on experience.

[0097] A hard drive is identified when the feature value corresponding to a feature is less than or equal to the first feature threshold corresponding to that feature.

[0098] In step 520, the percentage of hard drives with corresponding labels indicating failure is determined among the identified hard drives.

[0099] Figure 6 This diagram illustrates a specific implementation of feature removal based on a comparison of the feature value corresponding to the feature with a corresponding feature threshold, according to an embodiment of this application. It can be determined that... Figure 6 The example implementation uses feature removal based on data from August 1, 2018 to March 10, 2019. During this period, 2830 fault reports were generated, containing 2524 SNs (Serial Numbers). Each SN identifies a hard drive. The number of SNs is less than the number of fault reports because duplicate reports were generated. Of these, 167 SNs met the condition plist+glist>6500. plist represents the number of defective sectors generated during the hard drive's manufacturing process, while glist represents the number of defective sectors generated during the hard drive's use. These 167 SNs were the hard drives for which fault reports were generated because plist+glist>6500. It can be seen that the hard drive SMART technology does not generate reports by comparing a single indicator value to a threshold; here, plist and glist are two indicator values, and the sum of these two indicator values ​​must exceed a threshold for a fault report to be generated.

[0100] Figure 6The data includes four columns: multiple SMART characteristics, threshold conditions and the number of SMART snapshots that meet the threshold conditions, the number of hard drives, the number of hard drives matched with fault tickets, and the matching rate. For example, for the SMART characteristic `end_to_end_error_value`, the corresponding threshold condition is actually `value<=99&&value!=-1`. Here, 99 can be the first feature threshold mentioned above, and `value!=-1` is used to exclude data where `value=-1`. For example, the initial value of the SMART characteristic `end_to_end_error_value` can be -1. If no other value of this SMART characteristic is collected after initialization, there may be a case where `value=-1`. For the SMART characteristic `end_to_end_error_value`, the number of SMART snapshots that meet the threshold condition `value<=99&&value!=-1` is 103982. These SMART snapshots correspond to 94 hard drives, of which 9 hard drives match fault tickets. That is, out of these 94 hard drives, 9 hard drives have fault tickets created. The matching rate is the ratio of the number of hard drives that match the fault report to the total number of hard drives, which is the percentage of quantities mentioned in step 520; for example, 9.60% is obtained by calculating the ratio of 9 to 94. Therefore, the higher the matching rate for a SMART characteristic, the more likely that SMART characteristic is associated with hard drive failure.

[0101] In step 530, features are removed based on their proportion being less than a preset proportion threshold.

[0102] For example, the preset quantity percentage threshold can be 20% or 40%. When the quantity percentage determined based on a feature reaches more than 40%, that feature will be retained.

[0103] exist Figure 6 In the data, the matching rates for the two SMART features, reported_uncorrect_value and command_timeout_value, both exceed 20%, while the matching rates for other SMART features are very low, all far below 20%. Therefore, other SMART features can be removed, and reported_uncorrect_value and command_timeout_value can be selected as the target features.

[0104] In general, if the proportion of hard drives with a feature value less than or equal to the first feature threshold corresponding to a feature is smaller among the hard drives with labels indicating failure, then the feature is less likely to be associated with hard drive failure. Therefore, the embodiments of this application can achieve feature removal more accurately.

[0105] Figure 6 The SMART features mentioned earlier are equivalent to the first set of SMART features in the two sets of SMART features mentioned earlier. The first set of SMART features eliminates features by comparing the feature value corresponding to the feature with the corresponding feature threshold. Through feature elimination, the two SMART features reported_uncorrect_value and command_timeout_value were selected as target features from the first set of SMART features. Together with the 7 SMART features selected from the second set of SMART features mentioned earlier, a total of 9 SMART features can be selected as target features.

[0106] While this application's embodiments assume that a smaller feature value corresponds to a more likely hard drive failure, other embodiments of this application may also assume that a larger feature value corresponds to a more likely hard drive failure. In this case, it is necessary to first identify hard drives whose feature values ​​exceed the corresponding feature threshold.

[0107] Please continue reading Figure 2 In step 230, based on the feature values ​​corresponding to the target features, the hard drives with faults indicated by each label corresponding to the feature values ​​are clustered to obtain the clustering results.

[0108] Clustering is a machine learning technique that involves grouping data points. Given a set of data points, we can use clustering algorithms to assign each data point to a specific group. Clustering is an unsupervised learning method and a statistical data analysis technique commonly used in many fields.

[0109] Specifically, the K-MEANS algorithm can be used for clustering.

[0110] Of course, other clustering methods can be used, such as density-based clustering (DBSCAN). This method does not require a fixed number of clusters and identifies outliers as noise. Furthermore, it can effectively find clusters of arbitrary size and shape.

[0111] In step 240, the labels of hard drives that indicate failure are corrected according to the clustering results, so as to correct the labels of at least one hard drive that indicates failure to indicate that the hard drive is normal.

[0112] As mentioned earlier, creating a fault report for a hard drive does not necessarily mean that the hard drive is actually faulty. In other words, even if a fault report is created for a hard drive and a label indicating a fault is attached, it may still be inaccurate. Moreover, there are some faults that cannot be identified by the SMART feature. Therefore, clustering can be used to further filter hard drives with fault labels, and identify hard drives that are more likely to be faulty.

[0113] In one embodiment of this application, after correcting the labels of hard drives that indicate faults based on clustering results to change the labels of at least one hard drive that indicates a fault to indicate that the hard drive is normal, the hard drive fault prediction method further includes: removing the feature data and labels corresponding to the hard drives that have undergone label correction.

[0114] As mentioned earlier, hard drive failure reporting is based on a variety of factors, and the accuracy of hard drive failure reporting is poor. Therefore, even if some hard drives have their labels corrected, these hard drives may still be faulty. Therefore, removing the data from these hard drives can improve the validity of the data.

[0115] In step 250, the time-series feature data corresponding to the hard disk is extracted. The time-series feature data consists of multiple feature values ​​corresponding to the target feature collected based on the time series.

[0116] The timing characteristic data corresponding to the hard drive can be extracted from the characteristic data corresponding to the multiple hard drives mentioned earlier, or it can be extracted from the characteristic data corresponding to other hard drives, which are different from the multiple hard drives.

[0117] Time series feature data can be SMART snapshot data collected at various time points in a time series.

[0118] To train the model, data needs to be collected for training.

[0119] Figure 7 A flowchart illustrating the extraction of timing feature data corresponding to a hard disk according to one embodiment of this application is shown. Please refer to... Figure 7 This may include the following steps: In step 710, the first timing feature data of the faulty hard disk is extracted. The faulty hard disk is the hard disk that is identified as having failed.

[0120] The first timing characteristic data of the faulty hard drive is positive sample data.

[0121] A faulty hard drive can be identified as faulty through various means. For example, a faulty hard drive can be one whose fault record is created using SMART technology, in which case a label indicating the hard drive's failure will be attached. A faulty hard drive can also be one whose failure is identified through manual inspection.

[0122] In one embodiment of this application, extracting first timing characteristic data of the faulty hard disk includes: Extract the first timing characteristic data of the hard drive whose label indicates a failure.

[0123] The hard drive with the label indicating a fault is the one whose fault was identified through hard drive SMART technology and for which a fault report was created.

[0124] In one embodiment of this application, feature data of the hard disk is collected at time step intervals; fault monitoring of the hard disk is performed based on the feature data; and first time-series feature data of the hard disk with a label indicating a fault is extracted, including: taking the end time of a predetermined time period after the time when the label indicating a fault is marked on the hard disk as the end point, taking the time before the end point and the time interval of a predetermined number of time steps from the end point as the start point, and extracting the first time-series feature data between the start point and the end point.

[0125] In other words, a time step is a fixed-length time interval. The hard disk SMART technology collects SMART snapshot data every time step. The length of a time step can be arbitrary, such as a day, 6 hours, 8 hours, etc.

[0126] Figure 8 A schematic diagram illustrating a first method for extracting positive sample data according to an embodiment of this application is shown. Please refer to... Figure 8 The initial alarm time is the moment when the hard drive is labeled with a fault indicator, while the Smart Observation Point is the end of a predetermined time period. The initial alarm time and the Smart Observation Point together constitute the predetermined time period, which is 7 days in length. Figure 8 In this example, positive sample data from 120 time steps prior to the Smart observation point are extracted. It can be seen that the duration of these 120 time steps is longer than the time between the first alarm and the Smart observation point, i.e., more than 7 days. When the time step length is 6 hours, 120 time steps would take as long as 30 days. For hard drives that have only recently been deployed, the number of time steps for data extraction can be less than 120.

[0127] pass Figure 8 The extraction method shown extracts data both before and after the first alarm, and can extract data reflecting the changing trend of the faulty hard drive.

[0128] In one embodiment of this application, extracting the first time-series feature data of the hard drive with the label indicating a fault further includes: taking the time when the label indicating a fault is marked on the hard drive as the final end point selection point, taking the time point that is before the final end point selection point and is separated from the final end point selection point by a set prediction period as the start end point selection point, determining the end point between the start end point selection point and the final end point selection point, taking the time point that is before the end point and is separated from the end point by a predetermined number of time steps as the start point, and extracting the first time-series feature data between the start point and the end point.

[0129] You can randomly select a time point between the start and end point selection points and the final end point selection point as the end point. You can also set multiple prediction periods with different time lengths, and then extract the corresponding first time series feature data for each prediction period.

[0130] Figure 9 A schematic diagram illustrating a second method for extracting positive sample data according to an embodiment of this application is shown. Please refer to... Figure 9 The time point of the first alarm is the moment when the hard drive is labeled with a fault indication, which is also the final end point selection point. The time point from the first alarm to the prediction period is the start and end point selection point. The prediction period lies between the final end point selection point and the start and end point selection points; therefore, the end point is within the prediction period. Figure 9 The example also extracts positive sample data from the 120 time steps preceding the end point. For some hard drives, since the online time may be very short, the number of time steps for extracting positive sample data can be less than 120.

[0131] Figure 10 A flowchart illustrating the extraction of first timing characteristic data of a faulty hard disk according to an embodiment of this application is shown. See also... Figure 10 This includes the following steps: In step 1010, if the feature value corresponding to the target feature in the feature data is less than the second feature threshold corresponding to the target feature, the hard drive is identified as a faulty hard drive.

[0132] The second feature threshold can be a threshold set manually based on experience. For the same feature, the second feature threshold can be the same as or different from the first feature threshold and the final feature threshold mentioned above.

[0133] If a hard drive's feature data contains a target feature whose value is less than the second feature threshold, the hard drive can be considered a faulty hard drive.

[0134] Of course, in other embodiments of this application, for some target features, when the feature values ​​of these target features of a hard disk are greater than the corresponding second feature threshold, the hard disk can also be identified as a faulty hard disk.

[0135] In step 1020, the moment when the feature value corresponding to the target feature in the collected feature data is less than the second feature threshold corresponding to the target feature is taken as the abnormal moment. The first abnormal moment and the last abnormal moment are taken as the end point, and the moment that is before the end point and is separated from the end point by a predetermined number of time steps is taken as the start point. The first time-series feature data of the faulty hard disk between the start point and the end point is extracted.

[0136] Although the embodiments of this application also extract time series feature data within a predetermined number of time steps, in reality, a different number of time series feature data within time steps than those in the aforementioned embodiments can also be extracted.

[0137] Figure 11 A schematic diagram illustrating a third method for extracting positive sample data according to an embodiment of this application is shown. Please refer to... Figure 11 The time point of the first anomaly is the first abnormal moment. Taking the first abnormal moment as the end point, the first time-series feature data of the faulty hard drive in the 120 time steps before the end point are extracted.

[0138] Figure 12 A schematic diagram illustrating a fourth method for extracting positive sample data according to an embodiment of this application is shown. Please refer to... Figure 12 The time points between the first and last anomalies are all considered anomaly points. The time point of the last anomaly is the last abnormal moment. Taking the last abnormal moment as the end point, the first time-series feature data of the faulty hard disk in the 120 time steps before the end point are extracted.

[0139] In this embodiment of the application, for the case of hard disk failure where the feature value corresponding to the target feature is less than the corresponding second feature threshold, the time-series feature data before the time point of the first anomaly and the time point of the last anomaly are extracted respectively, so that the change trend data of the hard disk from normal to failure can be completely extracted.

[0140] Please continue reading. Figure 7 In step 720, the second timing feature data of the normal hard disk is extracted. The normal hard disk is the hard disk that is identified as not having a fault.

[0141] Hard drives that are excluded from all hard drives, including faulty drives and drives whose characteristic threshold is not equal to the final characteristic threshold, are considered normal hard drives. There are various ways to extract the second time-series characteristic data from normal hard drives.

[0142] In one embodiment of this application, extracting the second time-series feature data of a normal hard disk includes: randomly selecting an end point, taking the moment that is before the end point and is spaced a predetermined number of time steps away from the end point as the starting point, and extracting the second time-series feature data of the normal hard disk between the starting point and the end point.

[0143] In one embodiment of this application, extracting the second timing feature data of a normal hard disk includes: A specified number of normal hard drives are randomly selected from all normal hard drives, and the second time-series feature data is extracted from the selected normal hard drives.

[0144] Since the number of good disks in the entire network is too large, with as many as 400,000 good disks and only 1,600 bad disk samples, 50,000 can be randomly selected from the good disks, and only one timing sample can be selected from each disk.

[0145] Figure 13 A schematic diagram illustrating a method for extracting negative sample data according to an embodiment of this application is shown. The second time-series feature data of a normal hard drive is the negative sample data. A normal hard drive is one where the threshold values ​​corresponding to the features are all final feature thresholds, no fault reports have been generated, and no situation has occurred where all feature values ​​are less than the corresponding final feature thresholds. Please refer to [link to relevant documentation]. Figure 13 The negative sample data is extracted in the following way: for each normal hard drive, the number of times SMART data is collected is counted, and it is denoted as n. Then, from [1 / 5] n, 4 / 5 Within the interval [n], a random integer is selected, denoted as r; finally, the end time of the r-th time step of the SMART data collection is taken as the end point, and the second time series feature data of 120 time steps forward is extracted.

[0146] In step 730, the first time-series feature data and the second time-series feature data are normalized and standardized to obtain the time-series feature data corresponding to the hard disk.

[0147] Specifically, we can calculate the average and variance of the feature values ​​corresponding to each target feature, and then use the average and variance to perform normalization and standardization operations respectively, so that the dimensions of each target feature are unified.

[0148] Please continue reading. Figure 2 In step 260, the time-series feature data and labels corresponding to the hard disk are used as training samples to train the deep learning model, thereby obtaining the hard disk fault prediction model, and the target hard disk is fault predicted based on the hard disk fault prediction model.

[0149] The hard drive failure prediction model can output the probability of failure on a certain day, or it can output the probability that the hard drive will fail at a certain time on a certain day.

[0150] In one embodiment of this application, a deep learning model is trained using the time-series feature data and labels corresponding to the hard disk as training samples to obtain a hard disk fault prediction model. The method includes: dividing the time-series feature data and labels corresponding to the hard disk into a training set and a test set according to a predetermined ratio; using the time-series feature data and labels corresponding to the hard disk in the training set as training samples to train the deep learning model to obtain the hard disk fault prediction model; and using the test set to test the hard disk fault prediction model.

[0151] The predetermined ratio can be any ratio set manually, such as 9:1, 8:2, etc. The time-series feature data and labels corresponding to the hard drive can also be divided into training, validation, and test sets according to a set ratio, such as 8:1:1.

[0152] Deep learning is a collection of algorithms that use various machine learning algorithms on multi-layered neural networks to learn the inherent patterns and representational layers of sample data, thereby enabling the completion of various tasks. Deep learning models can be any model capable of processing time-series data.

[0153] For example, deep learning models can use LSTM (Long Short Term Memory networks) models, which are an improved version of RNNs.

[0154] RNN: Recurrent Neural Network, is a type of neural network with a feedback structure. Its output depends not only on the current input and the network weights, but also on previous network inputs. RNNs model time by adding self-connected hidden layers that span across time points. They are primarily used to process sequential data, such as text sequences or time-series data.

[0155] LSTM models can model historical time series data, thereby improving the accuracy of hard drive failure prediction.

[0156] In one embodiment of this application, the deep learning model includes a gated recurrent unit.

[0157] GRU (gated recurrent unit) is a highly effective variant of LSTM network. It is essentially a neural network. GRU has a simpler structure than LSTM network and performs very well, making it a very popular network today.

[0158] Compared to LSTM, which has more learning parameters and greater complexity, GRU has half the number of parameters. Therefore, the simpler structure makes it easier for gradients to backpropagate and saves computational resources consumed in model training.

[0159] Figure 14 A schematic diagram illustrating the principle of a hard disk failure prediction model according to an embodiment of this application is shown. Please refer to... Figure 14 The input to the model is a feature matrix composed of the feature vectors at each time step in time sequence. The GRU outputs the hidden layer vector at the last time step, and finally outputs the predicted probability of each category through a softmax classifier.

[0160] Assuming the total number of samples is m, a certain sample Where i = (1, 2, ..., m), x is a sample The set of feature vector sequences (by) , … (composed of eigenvectors), y is The actual label.

[0161] The model's calculation process is as follows: First, the following calculation process is performed in GRU: , , , , in, It is the feature vector at time step t. , , , , , These are all parameters that need to be learned and optimized during the training process. It is the sigmoid function.

[0162] Next, the final time step in the GRU is obtained as follows: , Finally, the following calculations are performed in the softmax classifier: , in, , These are the linear parameters that need to be learned, and softmax is the activation function. It is the predicted label, and also the label with the highest probability value.

[0163] Besides GRU and LSTM, other models can also be used in the embodiments of this application. CNN can be used, specifically by extracting local features from SMART data through convolutional local information, and then using pooling techniques for dimensionality reduction and training. Alternatively, the number of hidden layers in the neural network can be increased, but overfitting should be avoided. A combination of RNN, CNN, and other classifiers can also be used to improve model accuracy.

[0164] Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that include convolutional computations and have a deep structure. They are one of the representative algorithms of deep learning and are good at handling image processing, especially image recognition and other related machine learning problems.

[0165] In one embodiment of this application, the deep learning model is trained according to a preset loss function, which includes a penalty term. During the training process of the deep learning model, the penalty term in the preset loss function is dynamically adjusted according to whether the predicted label obtained by the deep learning model from predicting the time-series feature data is consistent with the label in the training sample to which the time-series feature data belongs.

[0166] Specifically, the traditional classification loss function is as follows:

[0167] in, It is a sample The probability corresponding to the actual label in the softmax probability value vector.

[0168] However, the inventors discovered that the sample data from normal hard drives and faulty hard drives differed significantly. Therefore, the inventors improved the loss function, taking into account the unique distribution of natural data related to hard drive failures and the impact on business operations. The loss function was optimized as follows:

[0169] in, For predicted labels.

[0170] For a single training sample, if the predicted label matches the actual label, the probability corresponding to the actual label in the softmax probability vector is still used. If they do not match, the loss penalty term needs to be increased. The hyperparameter C>1 represents the penalty coefficient, and increasing the loss penalty term can be achieved by adjusting the penalty coefficient.

[0171] During model training, batch training can be performed using the training set. After each batch is trained, the accuracy is calculated on the validation set. After multiple rounds of iterative training, until the loss function no longer decreases within a very small range, the model parameters that perform best on the validation set are retained, training is stopped, and the model is tested on the test set.

[0172] In one embodiment of this application, the deep learning model predicts the target hard drive as a faulty hard drive based on the predicted probability of the target hard drive belonging to a first probability interval, and predicts the target hard drive as a normal hard drive based on the predicted probability of the target hard drive belonging to a second probability interval, wherein the length of the first probability interval is less than the length of the second probability interval.

[0173] In many cases, the model uses a probability threshold of 0.5 when predicting hard drive failures. For example, when the prediction probability is greater than 0.5, the hard drive is predicted as normal; when the prediction probability is less than 0.5, the hard drive is predicted as faulty. However, this will misclassify more normal hard drives as faulty, leading to losses due to hard drive repairs and returns to the factory. Therefore, to reduce losses, the business has a lower tolerance for misclassifying good drives as faulty than for misclassifying faulty drives as good.

[0174] The embodiments of this application actually adjust the probability threshold. For example, the probability threshold can be adjusted from 0.5 to 0.3.

[0175] This application embodiment reduces the length of the probability interval corresponding to predicting the target hard drive as a faulty hard drive, that is, makes the hard drive less likely to be predicted as a faulty hard drive, raises the threshold for predicting the hard drive as a faulty hard drive, thereby making the predicted faulty hard drives more accurate, and thus reducing business losses.

[0176] Currently, the inventors have completed modeling for older disk types (over 2.5 years old) with a high coverage rate in the existing network. The online performance of the model is evaluated using online SMART sequence data and fault order data. The model employs a GRU component, and the comparison algorithm is SVM. Specifically: Accuracy was assessed by generating datasets for three different prediction periods (week, month, and 45 days), training three different models, and then validating them online.

[0177] Accuracy (weekly): The accuracy of predicting hard drive failures within the next week.

[0178] Accuracy (monthly): The accuracy of predicting hard drive failures within the next month.

[0179] Accuracy (45 days): The accuracy of predicting hard drive failures within the next 45 days.

[0180] The evaluation results are shown in Table 2:

[0181] Table 2 The results above show that the hard drive failure prediction model constructed according to the embodiments of this application performs better in all three prediction periods on the three models of hard drives. On another model of hard drive, it performs slightly worse than the comparison algorithm, possibly because the sample size is too small to be fully learned by the deep learning algorithm. Therefore, this model can be used for prediction on models with a large hard drive coverage.

[0182] In summary, the hard disk failure prediction method provided in this application obtains target features by first removing features based on the feature data and labels corresponding to multiple hard disks using one or more feature removal methods after acquiring the feature data. The target features are more closely related to whether the hard disk is faulty compared to the removed features. The hard disk failure prediction model is trained based on the feature data corresponding to the target features and is a deep learning model, thus improving its accuracy. The labels corresponding to the multiple hard disks are automatically labeled after fault monitoring of the hard disks using a specified method, significantly reducing the cost of sample acquisition. After determining the target features, the hard disks with labels indicating faults are clustered, and the labels are corrected based on the clustering results, making the final data used for model training more accurate and further improving the accuracy of the hard disk failure prediction model. The data used to train the model is extracted time-series feature data corresponding to the hard disks, enabling early prediction of hard disk failures and reducing the cost of business migration.

[0183] The following describes an apparatus embodiment of this application, which can be used to execute the hard disk failure prediction method in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the hard disk failure prediction method described above.

[0184] Figure 15 A block diagram of a hard disk failure prediction device according to an embodiment of this application is shown.

[0185] Reference Figure 15 As shown, a hard disk fault prediction device 1500 according to an embodiment of this application includes: an acquisition unit 1510, a rejection unit 1520, a clustering unit 1530, a correction unit 1540, an extraction unit 1550, and a training and prediction unit 1560.

[0186] The acquisition unit 1510 is used to acquire feature data and labels corresponding to multiple hard drives, respectively. The feature data includes multiple features of the hard drive and feature values ​​corresponding to each feature. The labels indicate whether the hard drive is faulty. The labels are assigned to the hard drive based on a specified method to detect whether the hard drive is faulty. The elimination unit 1520 is used to perform feature elimination operations on the multiple features based on the feature data and at least one feature elimination method to obtain target features. The clustering unit 1530 is used to perform clustering processing on the hard drives whose labels indicate faults according to the feature values ​​corresponding to the target features to obtain clustering results. The correction unit 1540 is used to correct the labels of the hard drives whose labels indicate faults according to the clustering results, so as to correct the labels of at least one hard drive whose labels indicate faults to indicate that the hard drive is normal. The extraction unit 1550 is used to extract the time-series feature data corresponding to the hard drive. The time-series feature data consists of multiple feature values ​​corresponding to the target features collected according to the time series. The training and prediction unit 1560 is used to train a deep learning model using the time-series feature data and labels corresponding to the hard drive as training samples to obtain a hard drive fault prediction model. And based on the hard disk failure prediction model, the target hard disk is used to predict failures.

[0187] In some embodiments of this application, based on the foregoing scheme, the feature removal method includes any one of the following: feature removal based on the variance of each feature value corresponding to the feature, feature removal based on the failure rate three times the standard deviation corresponding to the feature, and feature removal based on the comparison between the feature value corresponding to the feature and the corresponding feature threshold. The failure rate three times the standard deviation is the percentage of hard drives with labels indicating failure among hard drives whose feature values ​​reach three times the standard deviation, and the failure rate three times the standard deviation is the sum of the average value of each feature value corresponding to the feature and three times the standard deviation of each feature value corresponding to the feature.

[0188] In some embodiments of this application, based on the foregoing scheme, the elimination unit 1520 is further configured to perform feature elimination based on the variance of each feature value corresponding to the feature when performing feature elimination: for each feature, determine the variance of each feature value based on each feature value corresponding to the feature in each feature data; and eliminate the feature if the variance determined for the feature is less than a preset variance threshold.

[0189] In some embodiments of this application, based on the foregoing scheme, the elimination unit 1520 is further configured to eliminate the feature when performing feature elimination based on the failure rate three times the standard deviation of the feature: the feature is eliminated when the failure rate three times the standard deviation of the feature is less than a preset failure rate threshold.

[0190] In some embodiments of this application, based on the foregoing scheme, the elimination unit 1520 is further configured to perform feature elimination by comparing the feature value corresponding to the feature with the corresponding feature threshold as follows: determine among the plurality of hard disks the hard disks whose feature value corresponding to the feature does not exceed the first feature threshold corresponding to the feature; determine the proportion of hard disks with corresponding label indicating failure among the determined hard disks; and eliminate the feature based on the proportion of the number being less than a preset proportion threshold.

[0191] In some embodiments of this application, based on the aforementioned scheme, the extraction unit 1550 is configured to: extract first time-series feature data of a faulty hard disk, wherein the faulty hard disk is a hard disk identified as having failed; extract second time-series feature data of a normal hard disk, wherein the normal hard disk is a hard disk identified as not having failed; and perform normalization and standardization operations on the first time-series feature data and the second time-series feature data to obtain the time-series feature data corresponding to the hard disk.

[0192] In some embodiments of this application, based on the foregoing scheme, the extraction unit 1550 is configured to extract first timing feature data of the hard disk whose label indicates a fault.

[0193] In some embodiments of this application, based on the foregoing scheme, the specified method includes: collecting feature data of the hard disk at time step intervals; performing fault monitoring on the hard disk based on the feature data; the extraction unit 1550 is configured to: take the end time of a predetermined time period after the time when the label indicating the fault is marked on the hard disk as the end point, take the time before the end point and the time interval of a predetermined number of time steps from the end point as the start point, and extract the first time-series feature data between the start point and the end point.

[0194] In some embodiments of this application, based on the foregoing scheme, the extraction unit 1550 is further configured to: if, based on the feature data corresponding to the hard disk, it is determined that the feature value corresponding to the target feature in the feature data is less than the second feature threshold corresponding to the target feature, then the hard disk is determined to be a faulty hard disk; taking the moment when the feature value corresponding to the target feature in the collected feature data is less than the second feature threshold corresponding to the target feature as the abnormal moment, taking the first abnormal moment and the last abnormal moment as the end point respectively, and taking the moment located before the end point and spaced a predetermined number of time steps from the end point as the start point, extract the first time-series feature data of the faulty hard disk between the start point and the end point.

[0195] In some embodiments of this application, based on the foregoing scheme, the deep learning model includes a gated recurrent unit.

[0196] In some embodiments of this application, based on the foregoing scheme, the deep learning model is trained according to a preset loss function, which includes a penalty term; during the training process of the deep learning model, the penalty term in the preset loss function is dynamically adjusted according to whether the predicted label obtained by the deep learning model from predicting the time-series feature data is consistent with the label in the training sample to which the time-series feature data belongs.

[0197] In some embodiments of this application, based on the foregoing scheme, the deep learning model predicts the target hard drive as a faulty hard drive based on the predicted probability of the target hard drive belonging to a first probability interval, and predicts the target hard drive as a normal hard drive based on the predicted probability of the target hard drive belonging to a second probability interval, wherein the length of the first probability interval is less than the length of the second probability interval.

[0198] Figure 16 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0199] It should be noted that, Figure 16 The computer system 1600 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0200] like Figure 16 As shown, the computer system 1600 includes a Central Processing Unit (CPU) 1601, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1602 or programs loaded from storage portion 1608 into Random Access Memory (RAM) 1603, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1603. The CPU 1601, ROM 1602, and RAM 1603 are interconnected via bus 1604. An input / output (I / O) interface 1605 is also connected to bus 1604.

[0201] The following components are connected to I / O interface 1605: an input section 1606 including a keyboard, mouse, etc.; an output section 1607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1608 including a hard disk, etc.; and a communication section 1609 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1609 performs communication processing via a network such as the Internet. A drive 1610 is also connected to I / O interface 1605 as needed. Removable media 1611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1610 as needed so that computer programs read from them can be installed into storage section 1608 as needed.

[0202] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1609, and / or installed from removable medium 1611. When the computer program is executed by central processing unit (CPU) 1601, it performs various functions defined in the system of this application.

[0203] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0204] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0205] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0206] In one aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0207] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0208] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0209] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0210] The data collection and processing plan outlined in this application must be implemented in strict accordance with the requirements of relevant national laws and regulations, obtaining the informed consent or separate consent of the data subject (or having a legal basis as stipulated by the relevant national laws and regulations), and conducting subsequent data use and processing within the scope authorized by laws and regulations and the data subject.

[0211] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A hard disk failure prediction method, characterized in that, The method includes: Acquire feature data and tags corresponding to multiple hard drives respectively. The feature data includes multiple features of the hard drive and feature values ​​corresponding to each feature. The tags indicate whether the hard drive is faulty. The tags are assigned to the hard drives based on the detection of hard drive faults in a specified manner. Based on the feature data and using at least one feature removal method, a target feature is obtained by performing feature removal operations on the multiple features; the feature removal method includes: removing the feature based on the failure rate of three times the standard deviation corresponding to the feature being less than a preset failure rate threshold; wherein, the failure rate of three times the standard deviation is the proportion of hard drives with labels indicating failure among hard drives whose feature values ​​reach three times the standard deviation value, and the failure rate of three times the standard deviation value is the sum of the average value of each feature value corresponding to the feature and three times the standard deviation of each feature value corresponding to the feature; Based on the feature values ​​corresponding to the target features, clustering is performed on the hard drives that indicate faults according to each tag corresponding to the feature values ​​to obtain the clustering results; Based on the clustering results, the labels of hard drives that indicate failure are corrected so that the labels of at least one hard drive that indicates failure are corrected to indicate that the hard drive is normal. The end point is the end of a predetermined time period after the time when the label indicating a fault is marked on the hard drive. The start point is the time before the end point and at a predetermined number of time steps away from the end point. First time-series feature data is extracted between the start point and the end point. The hard drive labeled with the fault indication is the hard drive for which a fault record is created. If, based on the feature data corresponding to the hard drive, it is determined that the feature value corresponding to the target feature in the feature data is less than the second feature threshold corresponding to the target feature, then the hard drive is identified as a faulty hard drive. The moment when the feature value corresponding to the target feature in the collected feature data is less than the second feature threshold corresponding to the target feature is taken as the abnormal moment. The first abnormal moment and the last abnormal moment are taken as the end point, and the moment that is before the end point and is separated from the end point by a predetermined number of time steps is taken as the start point. The first time-series feature data of the faulty hard disk between the start point and the end point is extracted. Extract the second time-series feature data of normal hard drives. Normal hard drives are those that are identified as not having experienced any failures. Normal hard drives are those that have not been registered for failures and have never had all feature values ​​less than the corresponding final feature threshold. Normalization and standardization operations are performed on the first time-series feature data and the second time-series feature data to obtain the time-series feature data corresponding to the hard disk. The time-series feature data consists of multiple feature values ​​corresponding to the target feature collected based on the time series. The time-series feature data and labels corresponding to the hard disk are used as training samples to train a deep learning model, thereby obtaining a hard disk fault prediction model, and fault prediction of the target hard disk is performed based on the hard disk fault prediction model. The deep learning model is trained according to a preset loss function, which includes a penalty term. During the training process, the penalty term in the preset loss function is dynamically adjusted based on whether the predicted label obtained by the deep learning model from the time-series feature data matches the label in the training sample to which the time-series feature data belongs. If they do not match, the penalty term is increased. The deep learning model predicts the target hard drive as a faulty hard drive based on the predicted probability of the target hard drive falling into a first probability interval, and predicts the target hard drive as a normal hard drive based on the predicted probability of the target hard drive falling into a second probability interval. The length of the first probability interval is less than the length of the second probability interval.

2. The hard disk failure prediction method according to claim 1, characterized in that, The feature removal method also includes any one of the following: removing features based on the variance of each feature value corresponding to the feature, or removing features based on the comparison between the feature value corresponding to the feature and the corresponding feature threshold.

3. The hard disk failure prediction method according to claim 2, characterized in that, The feature removal based on the variance of each feature value corresponding to the feature includes: For each of the aforementioned features, the variance of each of the aforementioned feature values ​​is determined based on the feature values ​​corresponding to the feature in the feature data. If the variance determined for the feature is less than a preset variance threshold, the feature is removed.

4. The hard disk failure prediction method according to claim 2, characterized in that, The feature removal process, which involves comparing the feature value corresponding to the feature with the corresponding feature threshold, includes: Among the plurality of hard drives, identify those whose feature values ​​corresponding to the feature do not exceed the first feature threshold corresponding to the feature; Determine the percentage of hard drives with corresponding labels indicating failure among the identified hard drives; If the proportion of a feature is less than a preset proportion threshold, the feature is removed.

5. The hard disk failure prediction method according to claim 1, characterized in that, The specified method includes: Collect hard drive feature data in time step intervals; The hard drive is monitored for faults based on the aforementioned feature data.

6. The hard disk failure prediction method according to any one of claims 1-5, characterized in that, The deep learning model includes a gated recurrent unit.

7. A hard disk failure prediction device, characterized in that, The device includes: An acquisition unit is used to acquire feature data and tags corresponding to multiple hard drives respectively. The feature data includes multiple features of the hard drive and feature values ​​corresponding to each feature. The tags indicate whether the hard drive is faulty. The tags are assigned to the hard drive based on whether the hard drive is faulty detected by a specified method. A rejection unit is configured to perform feature rejection operations on the plurality of features based on the feature data and at least one feature rejection method to obtain a target feature. The feature rejection method includes feature rejection based on the failure rate three times the standard deviation of the feature. When performing feature rejection based on the failure rate three times the standard deviation of the feature, the rejection unit is further configured to: reject the feature if the failure rate three times the standard deviation of the feature is less than a preset failure rate threshold. Wherein, the failure rate three times the standard deviation of the feature is the percentage of hard drives with labels indicating failure among hard drives whose feature values ​​reach three times the standard deviation of the feature, and the failure rate three times the standard deviation of the feature is the sum of the average value of each feature value corresponding to the feature and three times the standard deviation of each feature value corresponding to the feature. The clustering unit is used to cluster the hard drives that indicate faults according to the feature values ​​corresponding to the target features, and obtain the clustering results. The correction unit is used to correct the labels of hard drives that indicate failure based on the clustering results, so as to correct the labels of at least one hard drive that indicates failure to indicate that the hard drive is normal. An extraction unit is used to extract time-series feature data corresponding to the hard disk, wherein the time-series feature data consists of multiple feature values ​​corresponding to the target feature collected based on a time series. The extraction unit is configured to: extract first time-series feature data of a faulty hard drive, wherein the faulty hard drive is a hard drive identified as having failed; extract second time-series feature data of a normal hard drive, wherein the normal hard drive is a hard drive identified as not having failed, and a normal hard drive is a hard drive that has not had a fault report created and has not had all feature values ​​less than the corresponding final feature threshold; and perform normalization and standardization operations on the first time-series feature data and the second time-series feature data to obtain the time-series feature data corresponding to the hard drive. The extraction unit is configured to: take the end of a predetermined time period after the time when the label indicating the fault is marked on the hard drive as the end point, and take the time before the end point and at a predetermined number of time steps away from the end point as the start point, and extract the first time-series feature data between the start point and the end point; the hard drive labeled with the fault indication is the hard drive for which a fault record is created. The extraction unit is further configured to: if, based on the feature data corresponding to the hard disk, it is determined that the feature value corresponding to the target feature in the feature data is less than the second feature threshold corresponding to the target feature, then the hard disk is identified as a faulty hard disk; taking the moment when the feature value corresponding to the target feature in the collected feature data is less than the second feature threshold corresponding to the target feature as the abnormal moment, taking the first abnormal moment and the last abnormal moment as the end point respectively, and taking the moment located before the end point and spaced a predetermined number of time steps from the end point as the start point, extract the first time-series feature data of the faulty hard disk between the start point and the end point; The training and prediction unit is used to train a deep learning model using the time-series feature data and labels corresponding to the hard drive as training samples to obtain a hard drive failure prediction model, and to predict the failure of the target hard drive based on the hard drive failure prediction model. The deep learning model is trained according to a preset loss function, which includes a penalty term. During the training process of the deep learning model, the penalty term in the preset loss function is dynamically adjusted according to whether the predicted label obtained by the deep learning model from the time-series feature data is consistent with the label in the training sample to which the time-series feature data belongs. If they are inconsistent, the penalty term is increased. The deep learning model predicts the target hard drive as a faulty hard drive based on the prediction probability of the target hard drive belonging to a first probability interval, and predicts the target hard drive as a normal hard drive based on the prediction probability of the target hard drive belonging to a second probability interval. The length of the first probability interval is less than the length of the second probability interval.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the hard disk failure prediction method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the hard disk failure prediction method as described in any one of claims 1 to 6.

Images