Flexible Job-Shop Scheduling Using MDP and Actor-Critic Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for flexible job-shop scheduling, such as meta-heuristic algorithms and heuristic methods, face challenges in real-time performance, computational efficiency, and generalization ability, particularly in feature extraction and exploration of production units within flexible job-shop environments.
Innovation Solution
A flexible job-shop scheduling method that generates multiple environments based on a preset production target, constructs data sets, and uses a Markov decision process to optimize feature extraction networks, actor networks, and critic networks, determining an optimal scheduling plan for maximum completion time through iterative training and validation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If accurate methods such as branch and bound method and mathematical programming method are used, then optimal solution can be obtained, but computational complexity increases exponentially and real-time scheduling requirements cannot be met
Solution Approach 1:
The patent segments the scheduling problem into multiple sub-problems by dividing the job-shop environment into different states and using hierarchical decomposition. The complex scheduling problem is broken down into smaller decision units that can be processed independently and then integrated, reducing the exponential computational complexity while maintaining solution quality.
Solution Approach 2:
The patent transforms the scheduling problem from a deterministic optimization problem into a probabilistic decision-making problem using Markov decision processes. By changing the problem parameters from seeking exact optimal solutions to seeking near-optimal solutions with probabilistic guarantees, the computational complexity is reduced from exponential to polynomial time while maintaining real-time performance.
2Productivity
If meta-heuristic algorithms such as genetic algorithm and particle swarm optimization algorithm are used, then real-time performance is improved, but performance is sensitive to parameters and generalization ability is poor
Solution Approach 1:
The patent implements feedback mechanisms through the Markov decision process framework, where the system continuously observes the current state, evaluates the quality of scheduling decisions, and adjusts future decisions based on accumulated experience. This feedback loop enables the system to adapt to different job-shop environments and improve generalization ability while maintaining real-time performance.
Solution Approach 2:
The patent performs preliminary action by pre-training the scheduling model on diverse job-shop environments using offline reinforcement learning. The system learns optimal scheduling policies in advance through extensive simulation and training, so that when deployed in actual production, it can quickly adapt to new scenarios without requiring complex parameter tuning, thus improving generalization ability.
3Ease of manufacture
If heuristic methods with preset rules are used, then program realization is simple and real-time performance is good, but solution quality is insufficient and adaptability to specific scenarios is limited
Solution Approach 1:
The patent replaces traditional mechanical heuristic rules with an intelligent decision-making system based on Markov decision processes and reinforcement learning. Instead of using fixed preset rules that require manual design and adjustment, the system automatically learns optimal scheduling strategies from data, achieving both program simplicity and high solution quality simultaneously.
Data Source
AI summary
The present disclosure provides a flexible job-shop scheduling method and system and an electronic device. The method according to the present disclosure includes: randomly generating a plurality of flexible job-shop environments in a production job-shop according to a preset production target; constructing a scheduling strategy model of the production job-shop based on a Markov decision process; optimizing a feature extraction network, an actor network and a critic network simultaneously by using the scheduling strategy model and a plurality of data sets, and determining a scheduling plan corresponding to a maximum completion time as an optimal scheduling plan after the completion of the optimization; and completing the preset production target based on the optimal scheduling plan. According to the present disclosure, feature extraction is performed on the plurality of job-shop environments to generate a scheduling scheme, so as to improve the efficiency and rationality of flexible job-shop scheduling.