Storage Failure Prediction via Conformal Feature Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage device failure detection in large-scale installations is inefficient, leading to data unavailability and potential permanent data loss, as existing methods fail to proactively detect device health issues, especially in large-scale enterprise storage systems.
Innovation Solution
Implementing a device failure prediction system using filter-based feature selection and a conformal prediction framework, which involves obtaining monitoring data, ranking features, selecting a subset of features, generating failure predictions, and applying a conformal prediction framework to provide confidence and credibility measures for initiating remedial actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional failure detection methods are used in large-scale storage systems, then the system can operate with simple monitoring, but the failure detection accuracy is insufficient leading to data loss
Solution Approach 1:
The patent segments the monitoring system into multiple independent components: feature extraction module, filter-based feature selection module (with multiple algorithms), classifier module, and conformal prediction module. Each component performs a specific function, allowing the system to achieve high detection accuracy through specialized processing while managing complexity through modular design.
Solution Approach 2:
The patent applies partial action by selecting only the most relevant features from the complete set of monitoring data using filter-based feature selection. Instead of analyzing all available features, the system identifies and processes only the subset of features that provide the most predictive value, reducing computational complexity while maintaining or improving detection accuracy.
2Reliability
If comprehensive monitoring data is collected from all devices, then more information is available for analysis, but the computational cost increases significantly
Solution Approach 1:
The patent extracts only the essential and relevant features from the comprehensive monitoring data using filter-based feature selection algorithms. By separating and selecting only the critical features that contribute to failure prediction, the system reduces the volume of data requiring computational processing while preserving the information necessary for reliable predictions.
Solution Approach 2:
The system performs partial analysis by focusing computational resources on the most informative features rather than processing all monitoring data equally. The filter-based feature selection identifies the subset of features that provide the highest predictive value, allowing the system to achieve reliable predictions with reduced computational energy consumption.
3Measurement precision
If multiple feature selection algorithms are used to rank features, then the feature selection accuracy improves, but the processing time increases
Solution Approach 1:
The patent merges the results from multiple filter-based feature selection algorithms by combining their rankings to produce a consolidated feature ranking. This approach leverages the strengths of different algorithms while distributing the computational workload, achieving accurate feature selection through ensemble methodology without requiring any single algorithm to process all data independently.
Data Source
AI summary
An apparatus comprises a processing device configured to obtain monitoring data for devices comprising a plurality of features, determining two or more rankings of the features using two or more filter-based feature selection algorithms, and selecting a subset of the features based at least in part on the two or more rankings. The processing device is also configured to generate a failure prediction for a given one of the devices using at least one classifier that takes as input the selected subset of features, and applying a conformal prediction framework to the generated failure prediction to obtain a confidence measure indicating a quality of the generated failure prediction and a credibility measure indicating a quality of the monitoring data. The processing device is further configured to initiate one or more remedial actions based at least in part on the generated failure prediction, the confidence measure, and the credibility measure.