ML Control Unit With Safety Module for Permissible Actions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning-based control systems for technical systems often fail to ensure that control actions comply with safety constraints, leading to suboptimal performance in safety-critical applications.
Innovation Solution
A control device that integrates a security module with a machine learning module to validate and convert control signals into permissible actions, ensuring safety compliance while optimizing performance by using state-specific security information and performance feedback for training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If control actions from a trained machine learning policy are validated against safety constraints, then safety compliance is improved, but system performance deteriorates
Solution Approach 1:
The patent incorporates safety constraints directly into the training process of the machine learning policy through a safety module. This preliminary action ensures that the policy learns to generate safe control actions from the beginning, eliminating the need for post-training validation that would otherwise degrade performance. The safety module provides guidance during training so the policy internalizes safety requirements.
Solution Approach 2:
The patent implements a feedback mechanism where the safety module provides real-time guidance signals to the machine learning policy during training. This feedback loop allows the policy to learn from safety violations and adjust its behavior accordingly, improving both safety compliance and performance simultaneously by integrating safety learning into the optimization process.
2Reliability
If a machine learning policy is restricted to only validated control actions, then safety is improved, but optimization capability deteriorates
Solution Approach 1:
The patent makes the safety constraints dynamic by allowing the safety module to adapt its guidance based on the current state and context. Rather than applying static restrictions that limit optimization, the safety module dynamically adjusts its interventions, providing more flexibility when safe and maintaining stricter control when necessary. This dynamic approach preserves the policy's optimization capability while ensuring safety.
Solution Approach 2:
The patent changes the parameters of the control policy during training by incorporating safety-related parameters and constraints into the learning process. The safety module modifies the policy's parameter space to include safety considerations, allowing the policy to optimize performance within safe boundaries rather than being restricted to pre-defined action sets.
3Loss of information
If safety information is incorporated into the training process, then expert knowledge utilization is improved, but system complexity increases
Solution Approach 1:
The patent introduces a safety module as an intermediary between the machine learning policy and the safety constraints. This intermediary component handles the complex task of interpreting and enforcing safety requirements, allowing the main policy to focus on optimization while the safety module manages expert knowledge integration. This modular approach manages system complexity by separating concerns.
Solution Approach 2:
The safety module is designed to be universal and multi-functional, handling various types of safety constraints and expert knowledge through a unified interface. Rather than creating separate mechanisms for different safety requirements, the safety module provides a general-purpose framework that can accommodate diverse safety information, reducing overall system complexity through consolidation.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
To configure a control device (CTL) for a technical system (TS), a safety module (SM) reads in state-specific safety information (SI) regarding the permissibility of a control action signal. Furthermore, a state signal (ZS) indicating the state of the technical system (TS) is fed into a machine learning module (NN) and the safety module (SM). Additionally, an output signal (OS) from the machine learning module (NN) is fed into the safety module (SM). Based on the safety information (SI) and the state signal (ZS), the safety module (SM) converts the output signal (OS) into a permissible control action signal (AS). Finally, the performance of controlling the technical system (TS) using the permissible control action signal (AS) is determined, and the machine learning module (NN) is trained to optimize this performance.The control unit (CTL) is then configured by the trained machine learning module (NN).