Twin-Roll Casting Control With Actor-Critic Chatter Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The twin-roll casting process faces challenges in controlling rapid thermo-mechanical dynamics during start-up, leading to strip chatter and surface defects due to variations in casting machine vibrations, which are exacerbated by human operator inconsistencies and variations in adjusting process control setpoints.
Innovation Solution
A reinforcement learning (RL) agent, comprising an actor-critic model, is trained on datasets from human operators to autonomously adjust the casting roll separation force setpoint, using a reward function that emulates operator behavior and incentivizes desired performance metrics to minimize chatter and achieve stable production.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If human operators manually adjust the force setpoint frequently during start-up, then chatter can be mitigated and strip quality improved, but operator variability and inconsistencies lead to unpredictable process behavior
Solution Approach 1:
The system uses self-learning reinforcement learning agents that autonomously adjust the force setpoint without human intervention. The agents learn optimal control policies through interaction with the casting process, automatically adapting to different operating conditions and eliminating operator variability while maintaining high strip quality.
Solution Approach 2:
The patent replaces the mechanical human operator decision-making process with computational intelligence systems (reinforcement learning agents). These software-based controllers substitute human judgment and manual adjustments with algorithmic decision-making, providing consistent and repeatable control actions.
2Reliability
If human operators adjust multiple setpoints and switches during start-up, then strip production can be stabilized, but the complexity of manual control increases and productivity decreases
Solution Approach 1:
The reinforcement learning agents autonomously manage all control decisions during start-up without requiring human operators to manually adjust multiple setpoints and switches. The system self-regulates the casting process, rapidly stabilizing production while reducing start-up time and increasing overall productivity.
Solution Approach 2:
The RL agents are pre-trained on extensive casting data to learn optimal control strategies before actual operation. This preliminary learning phase enables the agents to execute rapid and accurate control actions during start-up, stabilizing the process quickly without requiring operators to methodically adjust numerous parameters.
3Extent of automation
If model-based control approaches are used to automate casting processes, then automation can be improved, but the difficulty of modeling rapid thermo-mechanical dynamics during transient periods increases
Solution Approach 1:
The patent replaces model-based control approaches with model-free reinforcement learning. Instead of requiring complex mathematical models of the rapid thermo-mechanical dynamics, the system uses data-driven learning where agents directly learn control policies from interaction with the process, eliminating the need for explicit system modeling while achieving high automation.
Solution Approach 2:
The approach transitions from fixed model-based control parameters to adaptive learning parameters. The reinforcement learning agents continuously adapt their control strategies based on real-time process conditions, changing their behavior dynamically without requiring a predetermined mathematical model of the system dynamics.
4Manufacturing precision
If the force setpoint is decreased to reduce chatter, then strip quality improves, but other quality issues may arise and productivity decreases
Solution Approach 1:
The reinforcement learning agents provide dynamic, real-time adjustment of the force setpoint based on actual process conditions. Rather than using fixed conservative force levels, the agents continuously optimize the force parameter, reducing chatter when necessary while maintaining higher forces when conditions permit, thus preserving productivity and avoiding unnecessary quality issues.
Solution Approach 2:
The system implements closed-loop feedback control where the RL agents continuously monitor chatter levels and other process parameters, then adjust the force setpoint accordingly. This feedback mechanism enables the system to reduce chatter only when and where needed, maintaining optimal casting speed and avoiding the productivity losses associated with consistently low force settings.
Data Source
AI summary
A twin roll casting system comprises counter-rotating casting rolls having a nip between the casting rolls and capable of delivering cast strip downwardly from the nip, a casting roll controller configured to adjust at least one process control setpoint between the casting rolls in response to control signals, a cast strip sensor capable of measuring at least one parameter of the cast strip, and a controller coupled to the cast strip sensor to receive cast strip measurement signals from the cast strip sensor and coupled to the casting roll controller to provide control signals to the casting roll controller, the controller comprising a reinforcement learning (RL) Agent. The RL Agent further comprises a model-free actor-critic agent having a value function and a policy function, the RL Agent having been trained on a plurality of casting system operation datasets composed of casting runs executed by a plurality of different human operators.