Explainable Reinforcement Learning for Aircraft Maintenance Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing aircraft maintenance scheduling methods lack explainability, leading to confusion and ineffectiveness in implementing maintenance schedules, as they do not provide clear explanations for the selected actions.
Innovation Solution
An explainable Deep Reinforcement Learning (XDRL) method using a decomposed reward Deep Q-Network (drDQN) algorithm with two DQNs to maximize mission accomplishment and minimize maintenance costs, providing aircraft maintenance decisions and explanations through a scheduling environment and an explainable module.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional optimization or machine learning methods are used for aircraft maintenance scheduling, then scheduling solutions can be generated, but the methods do not provide explanations for the selected actions
Solution Approach 1:
The patent introduces an explainable AI module as an intermediary between the reinforcement learning agent and human operators. This module translates the agent's maintenance decisions into human-understandable explanations, preserving both scheduling efficiency and decision transparency. The intermediary processes the raw decisions and generates explanations that communicate the reasoning behind each maintenance action.
2Loss of information
If explainable AI is added to provide decision explanations, then transparency is improved, but system complexity increases
Solution Approach 1:
The system is segmented into distinct functional modules: the reinforcement learning agent that generates maintenance decisions, and the explainable AI module that provides explanations. This segmentation allows each component to specialize in its function while working together through standardized interfaces, managing overall system complexity through modular architecture.
Solution Approach 2:
The explainable AI module serves as an intermediary layer that sits between the complex reinforcement learning agent and human operators. It handles the complexity of interpreting agent decisions internally while presenting simplified explanations externally, effectively shielding users from system complexity.
3Productivity
If multiple objectives (mission accomplishment and cost minimization) are optimized simultaneously, then overall performance is improved, but the decision-making process becomes more complex
Solution Approach 1:
The patent segments the decision-making process into two parallel Deep Q-Networks: one dedicated to maximizing mission accomplishment and another to minimizing maintenance costs. Each network independently optimizes its specific objective, and their decisions are integrated in the explainable AI module, which manages the complexity of coordinating multiple objectives.
Solution Approach 2:
The system uses parameter changes in the form of reward functions that guide each Deep Q-Network's optimization process. By adjusting reward parameters, the system can balance between mission accomplishment and cost minimization objectives, managing decision-making complexity through parameter control rather than structural complexity.
Data Source
AI summary
A method for aircraft maintenance scheduling includes using a scheduling environment as a reinforcement learning (RL) environment to simulate an operational concept, train an RL agent, and generate aircraft maintenance decisions and explanations; providing a decomposed reward Deep Q-Network (drDQN) algorithm, wherein the drDQN algorithm includes a first Deep Q-Network (DQN) and a second DQN; using the first DQN to maximize a mission accomplishment objective; using the second DQN to minimize a maintenance cost objective; providing a trained drDQN agent; using the trained drDQN agent to obtain the aircraft maintenance decisions and corresponding mission accomplishment and maintenance cost rewards; using a scheduling module to arrange aircraft maintenance activities; and using an explainable module to get reasons to detail why the decisions are made and present tradeoffs between the decisions and non-selected alternatives.