Explainable Failure Prediction for Data Processing Infrastructure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing inference models for predicting device failures lack transparency and trustworthiness, making it difficult for downstream consumers to rely on their predictions without manual validation, which is time-consuming and inefficient.
Innovation Solution
Implementing explainable AI to extract hidden knowledge from inference models, converting it into structured knowledge attributes, and using a large language model to generate failure prediction prompts that are customizable and human-readable, thereby increasing the interpretability and trustworthiness of failure predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If inference models are used for predicting device failures, then prediction capability is improved, but transparency and trustworthiness deteriorate
Solution Approach 1:
The patent introduces an explainable AI layer as an intermediary between the inference model and end users. This layer extracts hidden knowledge from the model's internal representations and transforms it into structured, human-readable explanations. The intermediary preserves the high prediction accuracy of the original model while making the decision-making process transparent and interpretable for downstream consumers.
2Reliability
If manual validation of predictions is performed, then trustworthiness is improved, but time consumption increases
Solution Approach 1:
The system implements self-service by enabling the explainable AI layer to automatically generate structured knowledge attributes and explanations without requiring manual validation. The model autonomously extracts its own hidden knowledge and presents it in an interpretable format, allowing downstream consumers to trust and act on predictions immediately without time-consuming manual review processes.
3Loss of information
If hidden knowledge is extracted from inference models, then interpretability is improved, but system complexity increases
Solution Approach 1:
The patent applies extraction by isolating the explanation generation functionality into a separate explainable AI layer. This layer extracts hidden knowledge from the inference model's internal representations and outputs structured knowledge attributes. By separating the explanation function from the prediction function, the system maintains the original model's simplicity while adding interpretability capabilities in a modular, manageable way.
Data Source
AI summary
Methods and systems for managing data processing systems are disclosed. A data processing system may include and depend on the operation of hardware and/or software components. Inference models may be implemented to predict future system infrastructure outcomes (e.g., component failures) using information recorded in logs that reflect the operation of the components. However, the models may be complex “black boxes” and may generate critical outcome predictions for downstream consumers without explanations of how the predictions are determined, resulting in downstream consumers having low confidence in the predictions. Therefore, hidden knowledge (e.g., structured knowledge attributes) of the models may be extracted and/or used to understand the underlying processes that the models use to predict the system infrastructure outcomes. The hidden knowledge may be stored in a repository and may be provided for downstream use in order to increase the likelihood of preventing and/or mitigating future data processing system failures.