Certifiable Piloting Rules via Reinforcement Learning Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing aircraft piloting systems using neural networks with reinforcement learning algorithms are not certifiable due to the potential for generating inconsistent piloting commands, which can jeopardize aircraft safety.
Innovation Solution
A method for assisting aircraft piloting that involves acquiring a piloting model and a reward function with piloting constraints, applying a reinforcement learning algorithm, and generating certifiable piloting rules by associating aircraft states with piloting actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a neural network with reinforcement learning algorithm is used for aircraft piloting, then the piloting system can learn and determine commands autonomously, but the system becomes non-certifiable due to potential inconsistent commands
Solution Approach 1:
The patent segments the piloting system into two distinct parts: a reinforcement learning module that generates piloting rules autonomously, and a certification module that verifies these rules against formal specifications. This segmentation allows the automated learning component to operate independently while the certification component ensures reliability by validating that generated rules meet safety requirements before deployment.
Solution Approach 2:
The patent introduces a formal verification module as an intermediary between the reinforcement learning algorithm and the actual piloting execution. This intermediary validates the learned piloting rules against formal safety specifications and constraints, acting as a bridge that ensures the autonomous system's outputs meet certifiable standards before being applied to aircraft control.
2Productivity
If reinforcement learning is applied to generate piloting rules, then a large number of rules can be generated without human intervention, but the rules may be inconsistent and unsafe
Solution Approach 1:
The patent implements a feedback mechanism where the certification module evaluates generated piloting rules against formal specifications and provides validation feedback. If rules fail to meet safety criteria, the system can iteratively refine them through additional reinforcement learning episodes, ensuring that only consistent and safe rules are deployed while maintaining high productivity in rule generation.
Solution Approach 2:
The patent applies preliminary verification actions to piloting rules before they are deployed to the aircraft. The certification module performs formal verification checks on generated rules to ensure they meet safety specifications in advance, preventing inconsistent or unsafe rules from being executed during actual piloting operations.
3Reliability
If traditional certifiable piloting systems are used, then safety and consistency are guaranteed, but the system lacks autonomous learning capability
Solution Approach 1:
The patent merges two previously separate systems: a reinforcement learning system that provides autonomous learning capability and a formal verification system that ensures safety and consistency. By combining these systems in a unified architecture where learned rules are formally verified before deployment, the patent achieves both autonomous learning and certifiable safety simultaneously.
Data Source
AI summary
A method for assisting the piloting of an aircraft, including acquiring an aircraft piloting model and a reward function including a piloting constraint, and application, to the piloting model, of a reinforcement learning algorithm to obtain state variables and piloting commands. The method also includes formation of data group(s) from the state variables and the commands. For the or each group, the method includes assignment of at least one aircraft state to the state variables and at least one piloting action to the commands, to generate a piloting rule including the state(s) and piloting action(s). The method also includes transmission of at least one piloting rule to a display device for display to a pilot of the aircraft.