Disk Failure Prediction via Null Value Preprocessing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing disk failure prediction models struggle with accuracy due to noise in raw operation data, particularly null values, which conventional methods find difficult to handle effectively, leading to unreliable predictions and potential data loss or service interruptions.
Innovation Solution
A method that involves obtaining operation data, identifying and adjusting for null values, and processing the adjusted data using a machine learning model to predict disk failure within a predetermined time period, thereby improving prediction accuracy by preprocessing the data and utilizing machine learning for enhanced analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional methods are used to process raw operation data, then the processing is simple, but the prediction accuracy deteriorates due to noise and null values in the data
Solution Approach 1:
The patent applies preliminary action by performing data preprocessing before feeding data to the machine learning model. Specifically, null values are identified and adjusted (filled with median values or marked as unknown) in advance, and data noise is filtered. This preliminary data cleaning ensures that the machine learning model receives high-quality input data, significantly improving prediction accuracy while keeping the overall system architecture relatively simple.
2Reliability
If null values are not handled in the operation data, then the data processing is faster, but the prediction reliability deteriorates due to unreliable predictions
Solution Approach 1:
The patent extracts and handles null values separately from the valid data. By identifying null values and adjusting them (either filling with median values or marking as unknown), the system ensures that these problematic data points do not negatively impact the machine learning model's training and predictions. This extraction approach maintains prediction reliability while adding minimal processing time.
3Measurement precision
If raw operation data is processed directly without adjustment, then the processing is simpler, but the prediction accuracy deteriorates due to noise in the data
Solution Approach 1:
The patent converts the harmful effect of data noise and null values into a beneficial process by systematically addressing them through predefined methods. Null values are transformed into valid data points through median imputation or unknown marking, and noise is filtered through statistical methods. This transformation turns what would be detrimental factors into controlled variables that actually improve the quality of input data for the machine learning model, thereby enhancing prediction accuracy.
Data Source
AI summary
Embodiments of the present disclosure provide method, device and computer product for predicting disk failure. The method disclosed herein comprising: obtaining operation data of a disk, each data item of the operation data indicating values for one or more attributes of the disk at an associated time point; identifying null values for the one or more attributes from the data items of the operation data; adjusting the operation data based at least on the identifying of the null values; and processing the adjusted operation data with a machine learning model, to obtain a failure prediction on whether the disk will fail within a predetermined time period after the associated time point.