Reinforcement Learning for Manufacturing Due Date and Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current manufacturing systems lack an effective method to simultaneously determine optimal due date quotations and production scheduling policies, which are mutually dependent, leading to inefficiencies and increased risks of past due deliveries.
Innovation Solution
A virtual environment is created to simulate factory production lines and order arrivals, where policies for due date quotations and scheduling are learned through reinforcement learning, updating both policies simultaneously based on feedback to improve target metrics such as tardiness, and then applied to the actual operation system for real-time decision-making.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If factory emulation is used to simulate production lines for due date quotation, then the accuracy of due date estimation is improved, but the computation time increases and it becomes difficult to generate a due date reply within a desired period
Solution Approach 1:
The system performs preliminary simulation runs during an offline learning phase to train a machine learning model. The simulation environment pre-computes various production scenarios and stores the results in a trained model, so that during online operation, due date quotations can be generated instantly without performing full simulations in real-time.
Solution Approach 2:
The system creates a virtual copy of the factory production lines as a simulation environment. This digital twin allows the system to learn from simulated scenarios without affecting actual production, and the learned patterns are then applied to real due date quotations, avoiding the need to run full simulations for each new order.
2Loss of time
If production scheduling rules are fixed to estimate lead time using machine learning, then the computation time is reduced, but the ability to adapt to changing production conditions is limited
Solution Approach 1:
The system transitions from fixed scheduling rules to dynamic, learned policies. The machine learning model is trained offline to capture complex production patterns and relationships, enabling it to adapt to changing conditions without requiring real-time computation or reconfiguration of fixed rules. The model dynamically adjusts predictions based on the current state inputs.
3Productivity
If optimal production scheduling is searched given fixed due dates, then the scheduling efficiency is improved, but the due date quotation and scheduling become mutually dependent requiring simultaneous consideration
Solution Approach 1:
The system performs preliminary joint optimization offline by training the machine learning model to simultaneously learn due date quotation and scheduling policies. The simulation environment pre-explores the joint search space of due dates and schedules, capturing their mutual dependencies in the trained model. During online operation, both due date quotations and scheduling decisions are generated together based on the pre-learned policies, avoiding the need for iterative coordination.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Example implementations described herein involve methods and systems with one or more machines on a factory floor. Example implementations involve, in response to received orders, determining an initial scheduling policy for internal processes to meet the order and a due date policy for the order; a) executing a simulation involving scheduling decisions and due date quotations based on the initial scheduling policy and the due date policy; b) executing a machine learning process on the simulation results to update the scheduling policy 261 and the due date policy by evaluating the scheduling decisions and the due date quotations according to a scoring function which is common for evaluating the scheduling decisions and evaluating the due date quotations; iteratively executing a) and b) until a finalized scheduling policy and the due date policy is determined; and output the finalized scheduling policy and the due date policy in response to the order.