Conveyor Control Using Reinforcement Learning for Piece-Good Spacing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing conveyor systems face challenges in efficiently adjusting control processes for singulating and orienting piece goods due to varying properties, which are not observable in camera images and require manual adjustments, leading to inefficiencies and reduced control performance when goods deviate from standard properties.
Innovation Solution
Implementing Reinforcement Learning methods to control conveyor elements, where a control device determines actions based on state vectors and action vectors to achieve alignment and spacing, allowing for adaptive control strategies that learn and optimize in real-time based on piece good properties.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual adjustment of control processes is used for singulation and alignment, then control quality can be optimized for standard goods, but the system becomes time-consuming to adjust and cannot adapt to varying good properties
Solution Approach 1:
The control device automatically adapts to different piece good properties by using sensors to detect actual properties and autonomously adjusting control parameters without manual intervention. The system serves itself by learning from observed goods characteristics and modifying control processes accordingly.
Solution Approach 2:
The system dynamically changes control parameters such as conveyor element velocities based on detected piece good properties. By adjusting these parameters in real-time according to actual goods characteristics, the system maintains optimal control quality across varying conditions without requiring manual reconfiguration.
2Productivity
If the control system is designed for standard goods with fixed properties, then control processes work efficiently for those goods, but the system cannot reliably handle goods with different properties
Solution Approach 1:
The control system transitions from a static configuration optimized for standard goods to a dynamic system that continuously adapts its control parameters based on real-time detection of piece good properties. This enables the system to maintain high throughput efficiency while handling diverse goods types by adjusting velocities and control strategies according to actual conditions.
Solution Approach 2:
The control device is designed to handle multiple types of piece goods with varying properties through a universal detection and adaptation mechanism. By incorporating sensors that detect general properties and an automatic control adjustment system, the same control device can efficiently process different goods types without requiring separate specialized configurations.
3Manufacturing precision
If high cycle rate control adjustments are implemented to achieve efficient singulation, then control performance improves, but the complexity of controlling multiple conveyor elements increases significantly
Solution Approach 1:
The control problem is segmented by dividing the conveyor system into multiple independently controllable conveyor elements, each capable of being adjusted individually. This segmentation allows the system to achieve precise singulation and alignment by controlling each element separately rather than managing the entire conveyor as a single complex unit.
Solution Approach 2:
The system implements feedback mechanisms where sensors detect the actual positions and properties of piece goods, and this information is fed back to the control device. The control device uses this feedback to automatically adjust the velocities of individual conveyor elements in real-time, reducing the complexity of high-rate control adjustments by relying on automated feedback-driven modifications rather than pre-programmed complex control sequences.
Data Source
AI summary
A computer-implemented method, a device for data processing and a computer system for controlling a control device of a conveyor system to achieve an alignment and/or a defined spacing of piece goods, wherein the control of the control device is determined by an agent acting according to Reinforcement Learning methods. An individual, local state vector of predefined dimension that is the same for all the piece goods is created for each of the piece goods and an action vector is selected from an action space according to a strategy that is the same for all piece goods for the current state vector of this piece good. These action vectors are projected onto the conveying elements, wherein conflicts are resolved. After a cycle time has elapsed, state vectors are created again for each piece good and evaluated with rewards and the strategy is adjusted.