Factory Workflow Scheduling Using Reinforcement Learning Simulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional scheduling systems in manufacturing factories rely heavily on rule-based methods and historical data, limiting their ability to optimize processes in real-time and adapt to changing conditions, especially in complex environments where multiple processes with precedence relationships are involved.
Innovation Solution
A factory simulator-based scheduling system using reinforcement learning, which trains a neural network agent to determine optimal next actions by simulating various scenarios and collecting states, actions, and rewards, allowing for real-time decision-making without relying on past history data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If reinforcement learning is applied to train neural network agents for factory scheduling, then scheduling efficiency and adaptability are improved, but training time and computational resources increase
Solution Approach 1:
The patent pre-generates training data through factory simulator simulations before actual deployment. By simulating various factory scenarios and pre-computing state-action-reward tuples, the system prepares training datasets in advance, reducing the time needed for real-time scheduling decisions while maintaining high scheduling efficiency
Solution Approach 2:
The patent creates virtual copies of factory environments through simulation to generate training data. Instead of training directly on real factory operations, the system uses simulated factory models that replicate real-world conditions, enabling efficient pre-training without disrupting actual production
2Measurement precision
If deep learning with multi-layer neural networks is used for scheduling, then decision-making accuracy is improved, but device complexity and data requirements increase
Solution Approach 1:
The patent divides the complex scheduling problem into smaller sub-problems by processing different factory processes separately. Each neural network agent handles specific scheduling decisions for individual processes or process types, making the overall system more manageable while maintaining high decision-making accuracy through specialized agents
Solution Approach 2:
The patent transforms the scheduling problem into a reinforcement learning framework with clearly defined states, actions, and rewards. By changing the problem representation to use discrete state spaces and action spaces suitable for neural networks, the system achieves accurate decision-making without requiring excessively complex network architectures
3Measurement precision
If history data from past factory operations is used for training, then model accuracy is improved, but adaptability to new conditions deteriorates
Solution Approach 1:
The patent implements a dynamic training approach where the neural network agents are trained on pre-generated simulation data that can be regenerated to reflect new factory conditions. This allows the system to adapt to changing conditions by updating the simulation parameters and re-training, rather than relying on static historical data that may become obsolete
Solution Approach 2:
The system uses self-generated training data through the factory simulator instead of relying on external historical data. The simulator automatically generates realistic training scenarios and computes corresponding rewards, enabling the system to adapt to new conditions by simply updating the simulation environment without requiring manual data collection or labeling
4Reliability
If all process variables and past characteristics are included in training, then comprehensive understanding is improved, but training load and complexity increase
Solution Approach 1:
The patent extracts only the essential features and state variables needed for scheduling decisions from the complex factory environment. By identifying and extracting key state parameters, actions, and reward signals that are most relevant to scheduling, the system reduces training load while maintaining comprehensive understanding of the scheduling problem
Data Source
AI summary
The present invention relates to a factory simulator-based scheduling system using reinforcement learning, which schedules a process by training a neural network agent that determines a next action when a current state of a workflow is given in a factory environment in which a plurality of processes having a precedence relationship with each other constitutes a workflow and products are produced when the processes in the workflow are performed, and there is provided a factory simulator-based scheduling system using reinforcement learning, the system comprising a neural network agent having at least one neural network that outputs, when a state of a factory workflow (hereinafter, referred to as a workflow state) is input, a next work to be processed in the workflow state, wherein the neural network is trained by a reinforcement learning method; a factory simulator for simulating the factory workflow; and a reinforcement learning module for simulating the factory workflow using the factory simulator, extracting reinforcement learning data from a simulation result, and training the neural network of the neural network agent using the extracted reinforcement learning data. According to the system as described above, as learning data is configured by extracting a next state and a performance when an action of a specific process is performed in various process conditions through a simulator, there is an effect of stably training a neural network agent within a shorter time, and as a result, directing a more optimized work in the field.