A flexible job shop scheduling method based on improved Seagull optimization algorithm

Abstract

The present invention discloses a flexible job shop scheduling method based on an improved Seagull optimization algorithm, which belongs to the field of shop scheduling. The method includes: 1) solving the problem of small search space by improving the initialization population; 2) solving the problem that the Seagull optimization algorithm is prone to fall into the local optimum by redesigning the local search model; 3) enabling the offline trained model to quickly adapt to the new problem environment by integrating reinforcement learning. Through the present invention, the problem that the population diversity of the Seagull optimization algorithm decreases in the later iteration period and the algorithm is prone to fall into the local optimum can be solved; the convergence speed of the algorithm is accelerated and the accuracy is improved, so that the processing efficiency of the flexible job shop can be effectively improved, and the economic benefits of the factory can be improved. Finally, the method provided by the present invention is verified by examples and compared with other algorithms, which reflects the superiority of the method proposed by the present invention.

Description

Technical Field

[0001] The invention relates to the technical field of flexible job shop scheduling, and in particular to a flexible job shop scheduling method based on an improved Seagull optimization algorithm. Background Art

[0002] The Job Shop Scheduling Problem (JSP) has a long history and is defined as an NP-hard problem. Its main feature is that as the number of machines and jobs in the workshop increases, the number of scheduling solutions will explode, so JSP is a very complex engineering mathematical problem. The Flexible Job Shop Scheduling Problem (FJSP) is an extension of JSP, which increases the flexibility and complexity of scheduling. FJSP involves flexible manufacturing. Each production process has multiple machines corresponding to it, so it is necessary to select machines and equipment while sorting the processes. FJSP is a selection and optimization process, that is, it is necessary to optimize production goals such as processing time, cost, and energy consumption under the premise of meeting processing production conditions and constraints, so as to meet the needs of factory production. FJSP is one of the most difficult scheduling problems to solve in actual processing and production of enterprises. It has the characteristics of complex difficulty in solving and wide classification of problems. It is not only a key problem in scientific research, but also directly related to the production quality of workshops in many industries such as construction, aerospace, and steel. Therefore, it has been valued by the research community and enterprises as soon as it was proposed.

[0003] Seagull Optimization Algorithm (SOA) is a swarm intelligence optimization algorithm inspired by the biological behavior of seagulls. The algorithm mainly simulates the migration of seagulls in nature and the attack behavior (foraging behavior) during the migration process. Migration behavior means that seagulls fly from a place that is not suitable for survival at this stage to another place that is suitable for survival. During migration, each seagull is in a different position to avoid collisions with each other. In a group, seagulls can move in the direction of the best position and change their own position. Migration behavior affects the global exploration ability of the SOA algorithm. Attack behavior means that seagulls attack and forage for food on the ground and in the water during flight. When attacking, the seagull group makes a spiral movement pattern. The attack behavior affects the local development ability of the SOA algorithm.

[0004] Reinforcement learning is a type of machine learning. Its general process is to obtain the current state through the interaction between the agent and the environment, select a decision based on the current state and knowledge base and act on the environment, receive the evaluation reward of the decision from the environment and update the knowledge base about the decision. In this way, reinforcement learning can gradually learn how to obtain the maximum reward value through continuous "trial and error". Summary of the invention

[0005] In order to overcome the shortcomings of the flexible job shop scheduling problem in the above-mentioned prior art, the present invention provides a flexible job shop scheduling method of the Seagull optimization algorithm integrated with reinforcement learning, which solves the problem of small search space by improving the initialization population; solves the problem that the Seagull optimization algorithm is prone to fall into local optimality by redesigning the local search model; and enables the offline trained model to quickly adapt to the new problem environment by integrating reinforcement learning.

[0006] To achieve the above objectives, the following technical solutions are adopted:

[0007] A flexible job shop scheduling method integrating Q-learning improved Seagull optimization algorithm applies Logistic chaotic mapping population initialization scheme, improved local search model, and reinforcement learning algorithm in parameter selection to optimize the flexible job shop scheduling problem, so as to obtain a better production scheduling scheme and improve production efficiency. The specific process includes the following steps:

[0008] It is known that there are n workpieces (J1, J2, ..., J n ) to be performed on m machines (M1,M2,...,M m ) processing; the i-th workpiece J i Contains one or more processes, where the jth process is described as O ij ; The process sequence is predetermined; Each process can be processed on multiple different processing machines; Workpiece O ij The processing time on machine k is described as t ijk , t ijk The scheduling goal is to select the most suitable machine for each process, determine the best processing sequence and start time for each process on each machine, and optimize the maximum completion time of the entire system.

[0009] Step 1: Encode the production sequence using integer encoding. i =(x i1 ,x i2 ,...,x in ) for each dimension xij are all real numbers between 0 and 1, n is the maximum dimension of each individual, and also the number of processes; the n real numbers of the seagull individuals are recorded in the corresponding order according to the minimum arrangement criterion and this order is recorded as the production order matching the processing process and the machine;

[0010] Step 2: Initialize algorithm parameters. Mainly including population size numAgent, maximum number of iterations maxIteration, upper bound ub, lower bound lb, proportion of improved local search P_Ipatical, dimension dim; where numAgent = 50, maxIteration = 1000, ub = 1, lb = 0, P_Ipatical = 0.5. In the present invention, the optimization goal is to minimize the maximum completion time of the flexible job shop, and the algorithm terminates when the maximum number of iterations is reached.

[0011] Step 3: Use the improved Logistic chaotic mapping to initialize the position of each seagull, so that the initial population has a certain quality and diversity;

[0012] Step 4: Calculate the fitness of each seagull in all initialized populations, that is, the objective function value of the model, and take the position of the seagull with the smallest objective function value as the best seagull position of this generation;

[0013] Step 5: Use the reinforcement learning algorithm to determine the calculation formula of the decision variable according to the optimal fitness value of this generation and the best seagull position of this generation;

[0014] Step 6: Drive each seagull individual in the population to approach the optimal seagull position according to the seagull migration model formula;

[0015] Step 7: Use the roulette wheel method to select the seagull attack model formula or the improved local search model to update the new position of each seagull in the population;

[0016] Step 8: Record the position of each individual and the global optimal solution after the t-th iteration according to the fitness value; if the fitness value of the global optimal solution xi after the t-th iteration is less than the fitness value of the t-1th iteration, update the position of xi, otherwise keep the individual position of the previous generation of seagulls;

[0017] Step 9: Determine whether the number of iterations t reaches the maximum number of iterations maxIteration. If so, output the optimal solution for the individual seagull, i.e. the optimal scheduling plan; otherwise, t = t + 1, and go to step 5, and return the reward value of this iteration to evaluate the pros and cons of this decision.

[0018] The scheme of initializing the seagull population using Logistic chaotic mapping is:

[0019] A d-dimensional vector with each dimension ranging from 0 to 1 is randomly generated as the first individual. Then, each dimension of the first individual is iterated using the chaotic mapping expression to obtain the remaining (M-1) individuals. Finally, the variable values ​​generated by the Logistic mapping are mapped to the seagull individuals using the seagull initialization formula.

[0020] The expression of Logistic chaos map is:

[0021] X n+1 =X n ×μ×(1-X n )

[0022] Where μ is the mapping parameter, μ∈[0,4], and X is the mapping variable, X∈[0,1].

[0023] The Seagull initialization formula is:

[0024]

[0025] Where LB is the lower bound of the Seagull search space, UB is the upper bound of the Seagull search space, and X n is the random amount of the nth chaotic mapping.

[0026] The reinforcement learning algorithm is specifically:

[0027] In the Seagull optimization algorithm, a parameter A is used to balance global search and local search. However, during the optimization process, the Seagull algorithm cannot obtain prior experience about the value of A and can only change A through a fixed function, which will result in the inability to fully develop the algorithm's capabilities. To this end, the Q-learning algorithm in reinforcement learning is integrated into the Seagull algorithm, and the position of the seagull population is input into the Q-learning algorithm as a state; according to the current state, the Q-learning algorithm selects an action a suitable for the current state according to the Q table, that is, the function adjusts the A value; observes the result of the execution of this action, that is, obtains the next state s', and obtains a reward value r; finally, the state transfer matrix [s, a, r, s'] composed of the current state, action, reward, and next state is learned, and the Q table is updated. When reinforcement learning obtains the goal, the maximum cumulative reward is obtained. In the present invention, the reward value r is set to the negative maximum completion time. The implementation steps of the Seagull optimization algorithm integrating Q-learning are as follows:

[0028] Step 1: Initialize the Q table or read the existing Q table;

[0029] Step 2: Input the seagull population as state s into the Q-learning algorithm;

[0030] Step 3: According to the current Q table and state s, use the ε-greedy strategy to select an action a from the action set;

[0031] Step 4: Execute action a, the position of the seagull population is updated to s', and the reward r is obtained;

[0032] Step 5: The agent transfers the matrix [s,a,r,s'] according to the state and updates the Q table;

[0033] Q(s,a)←Q(s,a)+α*[r+γ*max a′ Q(s',a')-Q(s,a)]

[0034] Step 6: Update the position of the seagull population, s←s′;

[0035] Step 7: Determine whether state s is the terminal state or the number of iterations reaches the maximum value. If not, go to Step 3.

[0036] in:

[0037] ε-greedy strategy - means randomly selecting an action with a probability of ε, and selecting the action with the highest reward value in the Q table with a probability of (1-ε);

[0038] α——represents the learning rate of reinforcement learning;

[0039] γ——represents the attenuation value of future rewards.

[0040] Specifically, for the Seagull optimization algorithm integrated with Q-learning described in the present invention, the action set selected is a calculation function of the balance parameter A for global exploration and local development. At the tth iteration:

[0041] (1) Adopting a linear descent strategy

[0042]

[0043] (2) Using the sigmoid descent strategy

[0044]

[0045] (3) Adopting nonlinear differential change strategy

[0046]

[0047] (4) Cubic function change strategy

[0048]

[0049] The improved local search model is specifically:

[0050] In view of the poor performance of the Seagull algorithm and the slow convergence in the early stages of iteration, an improved local search model was designed. The local search model formula is:

[0051]

[0052] in is the position of the nth seagull, The best solutions are saved and further updated with the positions of other search agents.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention adopts Logistic chaotic mapping to initialize the population in the population initialization stage. Compared with the original random initialization population strategy, it can obtain an initial population with certain quality and diversity, which is conducive to accelerating the convergence speed of the algorithm.

[0055] The present invention introduces the Q-learning algorithm in reinforcement learning when the Seagull algorithm searches and updates the positions of individuals in the population, so that the algorithm can learn adaptively and improve its own knowledge base through offline learning, which is conducive to speeding up the solution of scheduling problems and enabling individuals in the population to quickly reach the optimal state.

[0056] The present invention designs four parameter iteration functions to balance the performance of the algorithm, enriches the exploration means of the Seagull algorithm, and balances the global exploration and local development capabilities of the Seagull optimization algorithm through reinforcement learning, overcomes the problem that the algorithm is prone to fall into local optimality, and thus can obtain a better scheduling solution.

[0057] The present invention improves the migration process of the seagull optimization algorithm and adds a new local search model, which can improve the ability to jump out of the local optimum, avoid the population from falling into the local optimal solution too early, and improve the convergence accuracy.

[0058] The present invention can solve the problem that the population diversity of the Seagull optimization algorithm decreases in the later stage of iteration and the algorithm is prone to falling into the local optimum; the convergence speed of the algorithm is accelerated and the accuracy is improved, which can effectively improve the processing efficiency of the flexible job shop and improve the economic benefits of the factory. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is the algorithm flow chart of the present invention;

[0060] Figure 2 This is a schematic diagram of Q-learning training of the present invention;

[0061] Figure 3 This is a 35×6 example diagram of the present invention;

[0062] Figure 4 It is a comparison diagram of the maximum completion time convergence curve in the present invention;

[0063] Figure 5 This is the Gantt chart of the optimal scheduling solution obtained by the method of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described below in conjunction with the accompanying drawings.

[0065] Aiming at the characteristics of the flexible job shop problem, the present invention proposes a Q-learning fused seagull optimization method for the flexible job shop scheduling problem to solve the flexible job shop scheduling problem with the goal of minimizing the completion time. This method integrates the Logistic initialization population, the reinforcement learning Q-learning algorithm, and the local reinforcement strategy into the seagull optimization algorithm to optimize the flexible job shop scheduling problem.

[0066] 1. Flexible job shop scheduling problem

[0067] 1.1. Problem Description

[0068] The Flexible Job-shop Scheduling Problem (FJSP) is an extension of the classic JSP, in which each process is allowed to be processed on any one of a set of available machines.

[0069] n workpieces (J1, J2, ..., J n ) to be performed on m machines (M1,M2,...,M m ) processing; the i-th workpiece J i Contains one or more processes, where the jth process is described as O ij ; The process sequence is predetermined; Each process can be processed on multiple different processing machines; Workpiece O ij The processing time on machine k is described as t ijk , t ijk It is fixed; the processing time of the process varies with the processing machine; each process cannot be interrupted during processing; a machine can only process one workpiece at any time.

[0070] The scheduling goal is to select the most suitable machine for each process, determine the best processing sequence and start time for each process on each machine, and optimize the maximum completion time of the entire system.

[0071] 1.2. Mathematical model

[0072] Assume there are n workpieces (J1, J2, ..., J n), the number of processes for each workpiece is l i , we need to have m machines (M1,M2,...,M m ) processing, the main constraints are as follows:

[0073] C max ≥c i i∈{1,2,…,n} (1)

[0074]

[0075]

[0076]

[0077]

[0078] Where:

[0079] C max — maximum completion time;

[0080] c i ——Completion time of workpiece i;

[0081] c ij ——Process O ij completion time;

[0082] c ijk ——Process O ij Makespan on machine k;

[0083] s ijk ——Process O ij start time on machine k;

[0084] p ijk ——Process O ij Processing time on machine k;

[0085] v ijk ——The value is 1, indicating process O ij Processed on machine k,

[0086] A value of 0 indicates other situations;

[0087] L – a very large positive number.

[0088] In the recursive relationship of equations (1)-(5), equations (1) and (2) respectively specify the value constraints of the maximum completion time and the workpiece completion time; equation (3) specifies the time constraint for the process to be processed on a certain machine; equation (4) specifies the order constraint for the processing of the same workpiece process; and equation (5) specifies that each process can only be processed on one machine. Through the above constraints, the goal of the scheduling problem is finally obtained - the minimum maximum completion time:

[0089] minC max

[0090] 2. Integrate Q-learning and Seagull optimization algorithm to solve the problem of flexible job shop scheduling

[0091] The population in this algorithm shows the location of individual seagulls and is considered by the following matrix:

[0092]

[0093] The population matrix X is n rows and d columns, where n is the total number of seagulls, d is the decision dimension, i.e., the number of processes p × the number of machines m. Each row represents the location information of an individual in the seagull population, x ij Represents the j-th dimension position information of the i-th seagull in the population.

[0094] 2.1. Encoding

[0095] Each dimension x of the seagull population i =(x i1 ,x i2 ,...,x id ) represents a seagull individual, where each decision value x ij They are all real numbers between 0 and 1, representing the process-machine processing mark v ijk The d real numbers in the seagull population are converted into a p×m matrix, and the processing steps and processing machines are matched in ascending order until all the processes are arranged. For example, in a flexible scheduling scenario with 3 processes and 2 machines, an individual x in the seagull population i =(0.567, 0.455, 0.424, 0.357, 0.575, 0.661), first convert it into a 3×2 matrix:

[0096]

[0097] Then find the matching process and machine in the order from small to large. In this embodiment, the matching situation of the processing process and the machine is: O2 is processed in M2 → O1 is processed in M2 → O3 is processed in M1.

[0098] 2.2. Parameter initialization

[0099] Initialize the parameters used in the algorithm. Mainly including the population size numAgent, the maximum number of iterations maxIteration, the upper bound ub, the lower bound lb, the proportion of improved local search P_Ipatical, and the dimension dim related to the Seagull Optimization Algorithm; where numAgent=50, maxIteration=1000, ub=1, lb=0, P_Ipatical=0.5. And the learning rate lr, the decay coefficient γ, the ε-greedy parameter ε, the target value target, and the initial reward value reward related to the Q-learning algorithm; where lr=0.01, γ=0.9, ε=0.9, target=60, reward=-999.

[0100] 2.3. Population initialization

[0101] The improved Logistic chaotic map is used to initialize the position of each seagull, so that the initial population has a certain quality and diversity. The scheme of initializing the seagull population using Logistic chaotic map is as follows:

[0102] First, a d-dimensional vector with each dimension ranging from 0 to 1 is randomly generated as the first individual; then, the chaotic mapping expression is used to iterate each dimension of the first individual to obtain the second individual, and the remaining individuals are iterated in this way until the number of individuals reaches the population size numAgent; finally, the seagull initialization formula is used to map the variable values ​​generated by the logistic mapping to the seagull individuals.

[0103] The expression of Logistic chaos map is:

[0104] X n+1 =X n ×W×(1-X n )

[0105] Where μ is the mapping parameter, μ∈[0,4], and X is the mapping variable, X∈[0,1].

[0106] The Seagull initialization formula is:

[0107]

[0108] Where LB is the lower bound of the Seagull search space, UB is the upper bound of the Seagull search space, and X n is the random amount of the nth chaotic mapping.

[0109] 2.4. Calculate the optimal position and fitness value of the seagull population

[0110] According to the position of the seagull population and the coding method, each individual seagull will obtain a set of matching results between processing procedures and machines. The individual with the shortest maximum completion time is the best position for the first generation of seagulls, and its fitness value is the minimum maximum completion time of this generation of seagulls.

[0111] 2.5. Selecting the Seagull Population Convergence Strategy Based on the Q-learning Algorithm

[0112] The Q-learning algorithm is a type of reinforcement learning. Reinforcement learning is a learning method in which the agent learns by trial and error, and the reward obtained by interacting with the environment guides the behavior. The goal is to make the agent obtain the maximum reward. The specific usage is as follows:

[0113] In the Seagull optimization algorithm, a parameter A is used to balance global search and local search. However, during the optimization process, the Seagull algorithm cannot obtain prior experience about the value of A and can only change A through a fixed function, which will result in the inability to fully develop the algorithm's capabilities. To this end, the Q-learning algorithm in reinforcement learning is integrated into the Seagull algorithm, and the position of the seagull population is input into the Q-learning algorithm as a state; according to the current state, the Q-learning algorithm selects an action a suitable for the current state according to the Q table, that is, the function adjusts the A value; observes the result of the execution of this action, that is, obtains the next state s', and obtains a reward value r; finally, the state transfer matrix [s, a, r, s'] composed of the current state, action, reward, and next state is learned, and the Q table is updated. When reinforcement learning obtains the goal, the maximum cumulative reward is obtained. In the present invention, the reward value r is set to the negative maximum completion time.

[0114] The implementation steps of the Seagull optimization algorithm integrated with Q-learning are as follows:

[0115] Step 1: Initialize the Q table or read an existing Q table;

[0116] Step 2: Input the seagull population as state s into the Q-learning algorithm;

[0117] Step 3: According to the current Q table and state s, use the ε-greedy strategy to select an action a from the action set;

[0118] Step 4: Execute action a, the position of the seagull population is updated to s', and the reward r is obtained;

[0119] Step 5: The agent transfers the matrix [s,a,r,s'] according to the state and updates the Q table;

[0120] Q(s,a)←Q(s,a)+α*[r+γ*max a′Q(s',a')-Q(s,a)]

[0121] Step 6: Update the position of the seagull population, s←s′;

[0122] Step 7: Determine whether state s is the terminal state or the number of iterations reaches the maximum value. If not, go to Step 3.

[0123] in:

[0124] ε-greedy strategy - means randomly selecting an action with a probability of ε, and selecting the action with the highest reward value in the Q table with a probability of (1-ε);

[0125] α——represents the learning rate of reinforcement learning;

[0126] γ——represents the attenuation value of future rewards.

[0127] Specifically, for the Seagull optimization algorithm integrated with Q-learning described in the present invention, the action set selected is a calculation function of the balance parameter A for global exploration and local development. At the tth iteration:

[0128] (1) Adopting a linear descent strategy

[0129]

[0130] (2) Using the sigmoid descent strategy

[0131]

[0132] (3) Adopting nonlinear differential change strategy

[0133]

[0134] (4) Cubic function change strategy

[0135]

[0136] 2.6. Update individual seagull positions

[0137] The mathematical model obtained by the seagull optimization algorithm mainly includes the migration process and the attack process, among which the migration process dominates the global development and exploration ability of the seagull population, while the attack model dominates the local search ability of the seagull population.

[0138] 2.6.1. Migration model

[0139] The migration model is designed based on the biological behavior of seagulls. In this process, seagull individuals need to meet three conditions:

[0140] (1) Prevent individual collisions

[0141]

[0142] A=f c -(x×(f c / MAX iteration ))

[0143] x=0,1,2,…,MAX iteration

[0144] in, - the location of the search agent, i.e. the individual seagull;

[0145] - Search the current location of the agent;

[0146] x – the current iteration number;

[0147] A – the movement parameters that the search agent searches for in the target space;

[0148] f c ——Control variable A from f c Down to 0, f c Generally take 2;

[0149] MAX iteration ——The maximum number of iterations.

[0150] (2) Move toward the best neighbor

[0151]

[0152] B = 2 × A 2 ×rd

[0153] in, ——Indicates the direction in which the current seagull individual is flying to where the most seagull individual is;

[0154] B——Random numbers, enhancing the balance between global optimization and local optimization of the algorithm;

[0155] rd——a random number in the range of [0, 1];

[0156] ——The most individual position of seagulls.

[0157] (3) Approaching the optimal individual

[0158]

[0159] in, ——The distance between the current seagull and the seagull with the smallest fitness value.

[0160] 2.6.2. Attack Model

[0161] The spiral upward attack model of the seagull population in the three-dimensional plane is described as follows:

[0162] x′ = r × cos(k)

[0163] y′ = r × sin(k)

[0164] Z' = r × k

[0165] r = u × e kv

[0166]

[0167] 2.6.3. Improved Local Search Model

[0168] In view of the disadvantages of the seagull algorithm, such as poor development performance and slow convergence in the early stage of iteration, the present invention additionally designs an improved local search model. The formula of its local search model is:

[0169]

[0170] where is the position of the nth seagull, saves the best solution and further updates the positions of other search agents. In order not to affect the characteristics of the seagull model in global exploration ability in the early stage, the present invention sets to randomly select between the attack model and the improved local search model by roulette, and designs a probability threshold Ps. Each seagull individual has two parallel update methods in each iteration; if rand < Ps, the position is updated according to the following seagull attack formula:

[0171]

[0172] Otherwise, the seagull individual updates its position according to the following formula:

[0173]

[0174] 2.7. Execution Process of the Q-learning Seagull Optimization Algorithm

[0175] Step 1: Parameter initialization;

[0176] Step 2: Population initialization and Logistic chaotic mapping;

[0177] Step 3: Calculate the fitness value of the population and the best position;

[0178] Step 4: The reinforcement learning agent selects an action;

[0179] Step 5: Update the seagull position;

[0180] Step 6: The reinforcement learning agent learns this strategy;

[0181] Step 7: Update population position, fitness value and optimal position;

[0182] Step 8: Check whether the stop condition is met. If yes, go to step 9; otherwise, increase the number of iterations by one and go to step 4.

[0183] Step 9: Match the processing procedures and machines according to the optimal position of the population and output them.

[0184] 3. Experiment and result analysis

[0185] The present invention discloses a Q-learning-integrated seagull optimization method for a flexible job shop scheduling problem. The effect of the invention is further illustrated by the following experiments.

[0186] This experiment was simulated using PyChram 2020.1 (Professional Edition) and Python 3.8 in a Windows 10 system with an i5-7300U CPU @ 2.60GHz and 16.0GB of RAM. The simulation experiment scheduling used a flexible job shop scheduling example with 7 workpieces, 6 machines, and 35 processes as the research object. The specific processing data is as follows Figure 3 shown.

[0187] The parameter settings of the simulation experiment design are as follows: population size numAgent = 50, maximum number of iterations maxIteration = 300, upper bound ub = 1, lower bound lb = 0, proportion of improved local search P_Ipatical = 0.5, dimension dim = 210; and Q-learning algorithm-related learning rate lr = 0.05, decay coefficient γ = 0.9, ε-greedy parameter ε = 0.9, target value target = 60, initial reward value reward = -999.

[0188] The untrained fusion Q-learning Seagull Optimization Algorithm (QSOA_0), the fusion Q-learning Seagull Optimization Algorithm (QSOA_1000) after one thousand trainings, the Seagull Optimization Algorithm (SOA) and the Genetic Algorithm (GA) were compared and tested. They were run ten times independently on the simulation instance respectively. The results are shown in Table 1.

[0189] Table 1 Calculation results of QSOA_0, QSOA_1000, SOA and GA

[0190] 1 2 3 4 5 6 7 8 9 10 Ave QSOA_0 73.8 69.2 81.5 76.6 71.0 65.0 67.6 73.5 66.3 65.9 71.04 QSOA_1000 68.7 64.9 74.1 75.2 67.0 66.6 67.8 64.6 67.0 70.8 68.67 SOA 68.8 79.4 79.1 67.9 87.7 103.9 78.4 74.5 92.0 71.0 80.27 GA 79.6 78.1 80.5 75.6 81.7 79.8 77.4 76.5 81.6 80.7 79.15

[0191] A better scheduling scheme can achieve the goal of saving production time and improving production efficiency. According to the experimental data obtained in Table 1, the untrained QSOA_0 algorithm and the QSOA_1000 algorithm trained a thousand times both achieved better results than the SOA and GA algorithms for the flexible job shop instance with 7 workpieces, 6 machines and 35 processes. Among them, the trained QSOA_1000 algorithm has the best effect, and the optimal value and average scheduling results of ten scheduling results are better than other algorithms.

[0192] In summary, the QSOA algorithm proposed in the present invention can effectively solve the problem of flexible job shop scheduling and achieve the purpose of improving production line production efficiency. The untrained QSOA algorithm can achieve performance superior to other algorithms. After offline training and learning, the algorithm effect will gradually improve, and as the learning time increases, the algorithm's convergence range, convergence depth and convergence speed will gradually improve. .

Claims

1. A flexible job shop scheduling method based on an improved Seagull optimization algorithm, characterized in that: The Logistic chaotic mapping population initialization scheme, the improved local search model, and the reinforcement learning algorithm are integrated in parameter selection to optimize the flexible job shop scheduling problem. The specific steps include: Step 1: Encode the production sequence using integer encoding; Step 2: Initialize algorithm parameters; Step 3: Use the improved Logistic chaotic mapping to initialize the position of each seagull, so that the initial population has a certain quality and diversity; Step 4: Calculate the fitness of each seagull in all initialized populations, that is, the objective function value of the model being sought, and take the position of the seagull with the smallest objective function value as the best position of this generation; according to the position of the seagull population and the encoding method, each seagull individual will obtain a set of matching results between the processing procedures and the machine, among which the individual with the smallest maximum completion time is the best position of the first generation of seagulls, and its fitness value is the minimum maximum completion time of this generation of seagulls; Step 5: Use the reinforcement learning algorithm to determine the calculation formula of the decision variable according to the optimal fitness value of this generation and the best seagull position of this generation; The reinforcement learning algorithm is specifically: According to the current state of the seagull population, the Q-learning algorithm selects an action a suitable for the current state according to the Q table, that is, the function adjusts the A value; observes the result of the execution of this action, that is, obtains the next state s', and obtains a reward value r; finally, the state transfer matrix [s, a, r, s'] composed of the current state, action, reward, and next state is learned, and the Q table is updated. The goal of reinforcement learning is to obtain the maximum cumulative reward, and the reward value r is set to the negative maximum completion time; The implementation steps of the Seagull optimization algorithm integrated with Q-learning are as follows: Step 1: Initialize the Q table or read the existing Q table; Step 2: Input the seagull population as state s into the Q-learning algorithm; Step 3: According to the current Q table and state s, use the ε-greedy strategy to select an action a from the action set; Step 4: Execute action a, the position of the seagull population is updated to s', and the reward r is obtained; Step 5: The agent transfers the matrix [s,a,r,s'] according to the state and updates the Q table; Q(s,a)←Q(s,a)+α*[r+γ*max a′ Q(s′,a′)-Q(s,a)] Step 6: Update the position of the seagull population, s←s′; Step 7: Determine whether state s is the terminal state or the number of iterations reaches the maximum value. If not, go to Step 3; in: ε-greedy strategy - means randomly selecting an action with a probability of ε, and selecting the action with the highest reward value in the Q table with a probability of (1-ε); α——represents the learning rate of reinforcement learning; γ——represents the attenuation value of future rewards; Step 6: Drive each seagull individual in the population to approach the optimal seagull position according to the seagull migration model formula; Step 7: Use the roulette wheel method to select the seagull attack model formula or the improved local search model to update the new position of each seagull in the population; Step 8: Record the position of each individual and the global optimal solution after the t-th iteration according to the fitness value; if the fitness value of the global optimal solution xi after the t-th iteration is less than the fitness value of the t-1th iteration, update the position of xi, otherwise keep the individual position of the previous generation of seagulls; Step 9: Determine whether the number of iterations t reaches the maximum number of iterations maxIteration. If so, output the optimal solution for the individual seagull, i.e. the optimal scheduling plan; otherwise, t = t + 1, and go to step 5, and return the reward value of this iteration to evaluate the pros and cons of this decision.

2. The flexible job shop scheduling method based on the improved Seagull optimization algorithm according to claim 1 is characterized in that: Individual x in the seagull population in step 1 i =(x i1 ,x i2 ,...,x in ) for each dimension x ij are all real numbers between 0 and 1, n is the maximum dimension of each individual, and is also the possible number of matches between processes and machines; the n real numbers obtained by seagull individuals are recorded in corresponding order according to the minimum arrangement criterion and this order is recorded as the production order of matching the processing steps and machines.

3. The flexible job shop scheduling method based on the improved Seagull optimization algorithm according to claim 1 is characterized in that: Step 2 specifically includes the population size numAgent, the maximum number of iterations maxIteration, the upper bound ub, the lower bound lb, the proportion of improved local search P_Ipatical, and the dimension dim; where numAgent = 50, maxIteration = 1000, ub = 1, lb = 0, P_Ipatical = 0.5, and the minimum maximum completion time of the flexible job shop is the optimization goal, and the algorithm terminates when the maximum number of iterations is reached.

4. The flexible job shop scheduling method based on the improved Seagull optimization algorithm according to claim 1 is characterized in that: The scheme for initializing the seagull population using the Logistic chaotic map is: A d-dimensional vector with each dimension ranging from 0 to 1 is randomly generated as the first individual. Then, each dimension of the first individual is iterated using the chaotic mapping expression to obtain the remaining (M-1) individuals. Finally, the variable value generated by the logistic mapping is mapped to the seagull individual using the seagull initialization formula. The expression of Logistic chaos map is: X n+1 =X n ×μ×(1-X n ) Where μ is the mapping parameter, μ∈[0,4], X is the mapping variable, X∈[0,1]; The Seagull initialization formula is: Where LB is the lower bound of the Seagull search space, UB is the upper bound of the Seagull search space, and X n is the random amount of the nth chaotic mapping.

5. The flexible job shop scheduling method based on the improved Seagull optimization algorithm according to claim 1 is characterized in that: The action set selected by the Seagull optimization algorithm integrated with Q-learning is a calculation function of the balance parameter A for global exploration and local development. At the tth iteration: (1) Adopting a linear descent strategy (2) Using the sigmoid descent strategy (3) Adopting nonlinear differential change strategy (4) Cubic function change strategy 6. The flexible job shop scheduling method based on the improved Seagull optimization algorithm according to claim 1 is characterized in that: The improved local search model is specifically: in is the position of the nth seagull, The best solutions are saved and further updated with the positions of other search agents.

Images