Deep Reinforcement Learning Scheduling for Capacity-Limited Production
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the context of smart manufacturing, existing methods fail to efficiently schedule production processes to minimize overall processing time while ensuring that production device capacity is not exceeded, leading to inefficiencies and potential bottlenecks.
Innovation Solution
A dynamic production scheduling method based on deep reinforcement learning, which acquires static and dynamic characteristics of jobs and system dynamics to optimize job execution sequences through a trained actor and critic network, determining whether to process jobs in batches or sequentially, thereby adjusting network parameters to minimize completion time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional scheduling methods are used to allocate jobs to production devices, then the scheduling process is simple to implement, but the overall processing time of jobs increases and production efficiency decreases
Solution Approach 1:
The patent replaces traditional mechanical scheduling methods with deep reinforcement learning algorithms. The scheduling system uses neural networks (actor-critic architecture) to learn optimal scheduling policies from historical data and dynamically adjust job allocation, substituting rule-based mechanical scheduling with intelligent adaptive scheduling that minimizes processing time while handling complexity through automated learning.
Solution Approach 2:
The system dynamically changes scheduling parameters based on real-time production state. The reinforcement learning model adjusts scheduling decisions by changing parameters such as job sequence, batch size, and device allocation based on learned patterns from training data, allowing the system to adapt to varying production conditions and minimize overall processing time.
2Productivity
If deep reinforcement learning is used to optimize job scheduling, then the overall processing time is reduced and production efficiency is improved, but the system complexity and computational requirements increase
Solution Approach 1:
The scheduling system is segmented into multiple independent neural network components: an actor network that generates scheduling decisions, a critic network that evaluates those decisions, and separate modules for different scheduling objectives. This segmentation allows each component to be trained and optimized independently, reducing the computational complexity of the overall system while maintaining high production efficiency through coordinated operation of specialized modules.
3Adaptability or versatility
If dynamic scheduling adjustments are made based on real-time production state, then the adaptability to changing conditions is improved, but the computational overhead and processing delay increase
Solution Approach 1:
The system performs preliminary action by pre-training the reinforcement learning models offline using historical production data. The actor and critic networks are trained in advance to learn optimal scheduling policies for various production scenarios. During real-time operation, the pre-trained models can quickly infer scheduling decisions without requiring extensive computational resources, thus maintaining high adaptability while minimizing computational delay.
Solution Approach 2:
The critic network provides continuous feedback to the actor network by evaluating the quality of scheduling decisions. This feedback mechanism allows the system to learn from past decisions and adjust future scheduling actions. The feedback loop operates efficiently by using the pre-trained critic network to quickly assess scheduling states, enabling real-time adaptability without significant computational overhead.
Data Source
AI summary
The embodiments of the present invention provide a dynamic production scheduling method, apparatus and electronic device based on deep reinforcement learning, which relate to the technical field of Industrial Internet of Things, and can reduce the overall processing time of jobs on the basis of not exceeding the processing capacity of production device. The embodiments of the present invention includes: acquiring static characteristics, dynamic characteristics of each of jobs and system dynamic characteristics, inputting the static characteristics, dynamic characteristics of each of jobs to be scheduled and system dynamic characteristics into a scheduling model to obtain a job execution sequence or batch execution sequence of the jobs in each production stage, wherein, the static characteristics of the job include an amount of tasks and time required for completion, the dynamic characteristics of the job include reception moment, and the system dynamic characteristics include a remaining amount of tasks that can be performed by the device in each production stage. The scheduling model is a model obtained after training a first actor network based on static characteristics and dynamic characteristics of a sample job, system dynamic characteristics, and a first critic network.