Discrete Manufacturing Scheduling With Memory-Guided Deep RL
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing intelligent decision-making methods for discrete manufacturing are limited by high computational complexity and are unsuitable for large-scale dynamic scheduling, failing to adapt to dynamic factors such as operator flow, order insertion, and device failure, leading to inefficiencies and reduced production efficiency.
Innovation Solution
An intelligent interactive decision-making method utilizing a deep reinforcement learning algorithm with a memory mechanism to store high-reward states, iteratively updating the model with prior knowledge to improve training speed and adapt to dynamic environments, enabling optimal production scheduling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a genetic algorithm is used for production scheduling, then high-quality solutions can be obtained, but the computation amount increases exponentially with problem scale
Solution Approach 1:
The patent segments the production scheduling problem into discrete state representations that can be stored and retrieved from memory. By dividing the problem space into manageable state-action-reward tuples, the system avoids exhaustive computation while maintaining solution quality through selective reuse of previously learned experiences.
Solution Approach 2:
The patent performs preliminary action by pre-training the deep reinforcement learning model on historical production data and storing high-reward states in memory. This preliminary learning phase enables the system to quickly retrieve and apply proven scheduling strategies without re-computing from scratch, significantly reducing real-time computational burden.
2Measurement precision
If a genetic algorithm is used for production scheduling, then optimal solutions can be found, but the training speed is too low
Solution Approach 1:
The patent implements feedback mechanisms through the deep reinforcement learning framework, where the system continuously receives reward signals from the production environment and updates its policy accordingly. The memory component provides feedback by retrieving past high-reward experiences, enabling faster convergence to optimal scheduling solutions compared to traditional genetic algorithms.
Solution Approach 2:
The patent replaces the mechanical evolutionary process of genetic algorithms with a neural network-based deep reinforcement learning system. This substitution enables parallel processing and gradient-based optimization, dramatically accelerating training speed while maintaining the ability to find optimal scheduling solutions.
3Device complexity
If traditional scheduling methods are used, then computation is manageable, but the system cannot adapt to dynamic factors such as operator flow, order insertion, and device failure
Solution Approach 1:
The patent implements dynamics by designing a reinforcement learning system that continuously learns from and adapts to changing production conditions. The model processes real-time state information including dynamic factors like operator availability, order changes, and device status, enabling flexible adaptation without requiring complex re-computation for each scenario.
Solution Approach 2:
The patent utilizes parameter changes by representing dynamic production environments as varying state parameters in the reinforcement learning framework. When dynamic events occur (e.g., device failure, order insertion), the system updates relevant state parameters and retrieves or learns appropriate scheduling actions, maintaining adaptability with manageable computation through efficient parameter updating rather than complete re-optimization.
Data Source
AI summary
Disclosed is an intelligent interactive decision-making method for a discrete manufacturing system. The method includes the following steps: step 1, establishing a production scheduling optimization model and strategy for discrete manufacturing for an actual application scene; step 2, training the scheduling strategy with existing production data on the basis of a deep reinforcement learning algorithm, and storing a state having a high reward in a training process in a memory; step 3, updating the state according to prior knowledge in the memory; step 4, inputting the updated state into a deep reinforcement learning network, obtaining a corresponding reward, and updating the memory according to the reward; and step 5, repeating step 4 until model parameters converge, and saving and putting the model into an actual production scene.