Evolved SARSA Flow Shop Scheduling to Escape Local Optima
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for solving Flow Shop Scheduling Problems (FSSP) face limitations when integrating reinforcement learning (RL) and genetic algorithms (GA), as they often get stuck at local optima and fail to achieve the minimum makespan efficiently.
Innovation Solution
A hybrid framework combining Reinforced State-Action-Reward-State-Action (R-SARSA) reinforcement learning with evolutionary crossover and heuristics, which iteratively refines job sequences through genetic operators and heuristic perturbations to prevent stagnation at local optima and achieve global optima.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If reinforcement learning (RL) and genetic algorithms (GA) are integrated to solve Flow Shop Scheduling Problems, then the solution quality improves, but the system complexity increases and the algorithms may still get stuck at local optima
Solution Approach 1:
The patent merges reinforcement learning (R-SARSA) with genetic algorithms (evolutionary crossover and mutation operators) into a hybrid framework. The RL component generates initial job sequences while the GA component refines them through evolutionary operations, combining the exploration strength of RL with the exploitation capability of GA to achieve better solution quality while managing system complexity through modular integration
Solution Approach 2:
The patent introduces a heuristic module as an intermediary component that bridges the RL and GA components. This heuristic module processes the output from RL, applies domain-specific knowledge and rules to guide the evolutionary process, and facilitates the transition between different algorithmic phases, thereby managing system complexity through structured intermediate processing
2Reliability
If reinforcement learning and genetic algorithms are integrated, then convergence to optimal solution improves, but the computational time and iterations required increase
Solution Approach 1:
The patent applies preliminary action by using the reinforcement learning component to generate initial promising job sequences before initiating the genetic algorithm. This preliminary exploration by RL provides a better starting population for the GA, reducing the number of iterations needed for convergence and thereby decreasing overall computational time while maintaining reliability of finding optimal solutions
Solution Approach 2:
The patent implements periodic action through iterative cycles where the RL component periodically updates the policy based on accumulated experience, and the GA component periodically applies evolutionary operations. This periodic alternation between exploration (RL) and exploitation (GA) phases enables the system to converge reliably while managing computational time through structured periodic updates rather than continuous exhaustive search
3Reliability
If the algorithm explores diverse job sequences to avoid local optima, then the chance of finding global optimum increases, but the computational overhead and complexity of sequence management increases
Solution Approach 1:
The patent applies dynamics by making the algorithm adaptive through experience. The R-SARSA learning module dynamically updates its policy based on accumulated rewards and observations, adjusting its exploration-exploitation balance over time. The genetic algorithm parameters such as crossover and mutation rates can be dynamically adjusted based on population diversity and convergence status, enabling the system to explore diverse sequences effectively while managing complexity through adaptive parameter control
Data Source
AI summary
Flow Shop Scheduling Problems (FSSP) solved using combination of Reinforcement Learning (RL), Genetic Algorithm (GA) and Heuristics is effective if can provide makespan as minimum as possible. Embodiments herein provide a method and system for evolved State-Action-Reward-State-Action (evolved SARSA) RL for flow shop scheduling, which is a hybrid framework of hierarchical RL with evolutionary techniques and heuristics method to solve FSSP. An optimum job sequence is estimated that minimizes the makespan thereby achieving maximum utilization of the resources. The evolutionary and heuristics strategy is applied in a reinforced way of learning for estimating the optimal schedule. The framework refines FSSP solution provided by Reinforced-SARSA (R-SARSA) using the evolutionary Genetic Algorithms (GAs), which is further guided by heuristic in moving towards the optimal solutions and prevents from being stuck at a local optimum.