Hybrid flow shop scheduling method considering preparation time and equivalent parallel machine

Abstract

The invention discloses a hybrid flow shop scheduling method considering preparation time and an equivalent parallel machine. The method comprises the following steps: step 1, establishing a hybrid flow shop scheduling model considering preparation time and an equivalent parallel machine according to hybrid flow shop scheduling characteristics; 2, designing a strengthened mayfly naiad algorithm, and fusing a Q learning algorithm and the mayfly naiad algorithm; the outer layer adopts a QL algorithm, and the inner layer adopts a mayfly naiad algorithm; and 3, solving the problem model by adopting a strengthened mayfly naiad algorithm, and optimizing the machine distribution and processing sequence of the workpiece to obtain the minimum value of the maximum completion time. According to the method, the defects of a single algorithm can be overcome, and higher adaptability and flexibility can be shown in practical application.

Description

Technical Field

[0001] The present invention relates to the technical field of hybrid flow shop scheduling, and in particular to a hybrid flow shop scheduling method considering preparation time and equivalent parallel machines. Background Art

[0002] In the modern manufacturing system, shop scheduling plays a vital role. It coordinates production activities, coordinates various resources, ensures that workpieces are processed in a predetermined order and time, and provides support for the efficiency, economy and flexibility of the manufacturing process. The Hybrid Flow Shop Scheduling Problem (HFSP) is an extension of the Job Shop Scheduling Problem (JSP). In the context of intelligent manufacturing, the significance of HFSP is further highlighted. Shop scheduling improves production efficiency and reduces cost consumption through automation and intelligent technology. But it also adds new dimensions and challenges to HFSP. Problems such as collaborative scheduling of multiple equipment in the workshop and unified management of information technology need to be improved urgently, which also restricts the interconnection between equipment and overall production efficiency. Traditional algorithms show fatigue when facing such large-scale problems, and lack targeted design for problems such as collaborative scheduling of multiple equipment. Combined with the actual production process of the job shop, the following problems exist:

[0003] 1. During the workshop production process, the production preparation process requires a certain amount of production preparation time to prepare and replace tools and adjust machine tools. 2. When batch processing a certain process, there is a lack of machine selectivity. 3. The scheduling of each process stage is not only related to the workpiece processing sequence of the current stage, but also affected by the scheduling of the previous stage.

[0004] Currently, research on hybrid flow shop scheduling problems primarily relies on the use of swarm intelligence algorithms (such as genetic algorithms) to solve them. For example, Zhang Yang used genetic algorithms in his paper "Research on Green Hybrid Flow Shop Scheduling with Uncorrelated Parallel Machines" to solve hybrid flow shop scheduling problems. However, this method has certain limitations: high computational complexity and slow convergence; parameter settings are sensitive, with different parameter settings significantly impacting algorithm performance; and the algorithm's initial optimization capabilities are weak, with the risk of being trapped in a local optimal solution. In some cases, the genetic algorithm may be unable to escape the local optimal solution, resulting in an inability to find a global optimal solution. Furthermore, it exhibits fatigue in the face of complex hybrid flow shop problems. Summary of the Invention

[0005] To overcome the above technical problems, the present invention aims to provide a hybrid flowshop scheduling method that considers setup time and equivalent parallel machines. By fusing the QL algorithm with the Mayfly algorithm, this method combines the advantages of both, achieving a complementary balance between global exploration and local exploitation. This approach achieves higher efficiency and more stable optimization results when solving hybrid flowshop scheduling problems. This fusion method not only overcomes the shortcomings of a single algorithm but also demonstrates greater adaptability and flexibility in practical applications.

[0006] The technical solution adopted in the present invention is:

[0007] A hybrid flow shop scheduling method considering setup time and equivalent parallel machines includes the following steps:

[0008] Step 1: Based on the scheduling characteristics of hybrid flow shop, a hybrid flow shop scheduling model is established that takes into account setup time and equivalent parallel machines;

[0009] Step 2: Design an enhanced Mayfly algorithm that integrates the Q-learning algorithm with the Mayfly algorithm; the outer layer uses the QL algorithm and the inner layer uses the Mayfly algorithm;

[0010] Step 3: Use the enhanced Mayfly algorithm to solve the problem model, optimize the machine allocation and processing sequence of the workpiece, and obtain the minimum value of the maximum completion time. Compare with the original algorithm and two classic algorithms to verify the feasibility and effectiveness of the enhanced Mayfly algorithm.

[0011] The symbolic definition in the hybrid flow shop scheduling model considering preparation time and equivalent parallel machines in step 1; establishing constraints and establishing an objective function based on the characteristics of the hybrid flow shop workpiece considering the processing preparation time and parallel machines.

[0012] The step 1 is specifically as follows:

[0013] Step 1: The hybrid flow shop scheduling problem can be described as follows: There are several workpieces to be processed, which need to be processed in the order of process 1, process 2, ... process m. At least one process has multiple equivalent machines running in parallel. The same workpiece requires the same amount of time to be processed on each equivalent machine tool. The machine tool does not need setup time to process the first workpiece, but when processing the second workpiece, if it is different from the previous workpiece, the machine tool needs a certain amount of setup time. Taking machine tool M as an example, the workpieces are arranged as N1-N2-N2. After processing N1, processing N2 requires setup time, but processing the second process of N2 does not require setup time.

[0014] Step 2, symbolic definition in the hybrid flow shop scheduling model considering setup time and equivalent parallel machines:

[0015] N: total number of workpieces processed;

[0016] J: number of processes for a single workpiece;

[0017] M: the number of machine tools;

[0018] i: the index number of the workpiece (i∈{1,2,…,N});

[0019] j: the process index number of the workpiece (j∈{1,2,…,J};

[0020] M(j): the set of parallel machines available for process j;

[0021] I_J(k): the set of processing steps on machine tool k;

[0022] k: the index number of the machine tool (k∈{1,2,…,K});

[0023] b i,j : The starting time of the jth process of workpiece i;

[0024] e i,j : Completion time of process j of workpiece i;

[0025] t i,j,k : The processing time of workpiece i on parallel machine k at stage j;

[0026] O i,j : j-th process of workpiece i;

[0027] X i,l : Decision variable, when workpiece i is placed in the lth position waiting for processing, X i,l =1;

[0028] Y i,j,k : Decision variables, workpiece i's process j is processed on machine tool k, then X i,j,k =1, otherwise

[0029] X i,j,k =0;

[0030] C i : Completion time of workpiece i;

[0031] C max =max{C1,C2,...,C n}:maximum completion time;

[0032] st i,j : The processing preparation time required for workpiece i at stage j;

[0033] Step 3: Establish constraints based on the characteristics of workpiece processing in a hybrid flow shop:

[0034] Constraint 1: Each process of each workpiece can only be performed on one machine.

[0035]

[0036] Constraint 2: There must be a one-to-one correspondence between each workpiece and its processing sequence

[0037]

[0038] Constraint 3: The completion time e of the jth process of workpiece i on parallel machine k i,j Equal to the sum of the processing start time, processing preparation time and actual processing time

[0039]

[0040] Constraint 4: In the same processing stage, the workpiece with the highest order will be processed first

[0041]

[0042] Constraint 5: The workpiece must complete the processing task of the previous stage before entering the next stage of processing

[0043] b i,j+1 -e i,j ≥0,i=1,...,n,j=1,...,s-1(5)

[0044] Constraint 6: The maximum completion time of job i is the maximum completion time of the last process of all jobs

[0045] C i =max(E ijtm ),j=1,2,...,P i (6)

[0046] Step 4, establish the objective function of the model, the expression is as follows:

[0047] Min C max =max{C i |i=1,2,...,n} (7).

[0048] This method establishes an objective function using the maximum completion time of a workpiece as the optimization objective to minimize the maximum completion time. Through step 1, the hybrid flow shop scheduling problem, which considers setup time and equivalent parallel machines, is transformed into a concrete mathematical model. The corresponding constraints and objective function are established, paving the way for subsequent problem solving.

[0049] The step 2 is specifically as follows:

[0050] Step 1, Intelligent Agent Design

[0051] The Outer QL algorithm dynamically updates the mutation rate by designing an agent that meets the characteristics of the hybrid flow shop scheduling problem, further enhancing the performance of the Mayfly algorithm and improving the efficiency of the problem-solving model. The agent design is divided into state space, action space, reward and penalty functions, and greedy strategies. The design process requires defining the state space, action space, and reward and penalty functions based on the Mayfly algorithm and the hybrid flow shop scheduling problem to ensure that they can effectively capture the requirements of dynamic adjustment of the mutation rate while maintaining the algorithm's adaptability and flexibility at different iteration stages. The detailed division of the state space must balance the breadth and depth of solution exploration, while the construction of the reward and penalty function requires comprehensive consideration of the magnitude of fitness optimization and the characteristics of the convergence phase to achieve continuous incentives for the objective function.

[0052] (2.1) State Space: The values ​​of the mutation rate parameters in the Mayfly algorithm constitute the state space of the agent. The mutation rate parameters in state space S directly affect the processing order of workpieces on each machine tool and the selection of machine tools. Based on the characteristics of the hybrid flow shop scheduling problem, to improve the algorithm's precision and flexibility and ensure that individuals retain some of their parent's genes after mutation, the mutation rate parameters in the state space are discretized into 11 state values ​​within the interval [0.2, 0.7] with a step size of 0.05. That is, there are a total of 11 states in state space S. State space S = {0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7}. These 11 discrete states cover a reasonable range from low to high mutation rates, with an upper limit of 0.7. A low mutation rate (0.2-0.35) helps to retain the current better processing sequence, while a high mutation rate (0.5-0.7) enhances the ability to explore new processing sequences to adapt to the complex workpiece processing requirements in the hybrid flow shop scheduling problem.

[0053] (2.2) Action Space: The actions in action space A (increasing, maintaining, or decreasing the mutation rate) directly affect the algorithm's ability to explore and exploit the hybrid flow shop scheduling problem. Increasing the mutation rate facilitates a more extensive search within the solution space, seeking new optimal solutions; decreasing the mutation rate facilitates a more refined search near the current solution, optimizing the existing solution.

[0054] (2.3) Reward and Penalty Function: The design of the reward and penalty function takes into account the optimization objective of the hybrid flow shop scheduling problem, namely, minimizing the maximum completion time. The reward function is not only related to the degree of optimization of the fitness value, but also takes into account the characteristic that the fitness value gradually converges in the later stages of the algorithm, encouraging the algorithm to adopt appropriate mutation rates in different iteration stages to optimize the maximum completion time. The reward function is shown in Equation (8). This reward function is not only related to the degree of optimization of the fitness value, but also takes into account the characteristic that the fitness value gradually converges in the later stages of the algorithm. Even if the degree of optimization of a fitness value in a certain generation is the same as that of the previous generation, as the number of iterations increases, a higher reward should be obtained in the later stages. This design aims to encourage the algorithm to adopt a more appropriate mutation rate, thereby improving its ability to solve the hybrid flow shop scheduling problem.

[0055]

[0056] Where Δf is the difference in fitness between the previous generation and the current generation of mayflies, f current is the fitness value of the current generation of mayflies, t is the current iteration number, T is the maximum iteration number, and k is the reward coefficient.

[0057] Q-value update: The Q-value update formula of the QL algorithm is shown in Equation (9), where Q(s, a) is the Q-value of the action taken under the current state and action. First, maxQ(s', a') is attenuated by the discount factor, then subtracted from Q(s, a), and then the reward r is added. The result is weighted again by the learning rate and added to the current Q-value, replacing the original Q(s, a).

[0058] Q(s,a)=Q(s,a)+α(r+γ*maxQ(s',a')-Q(s,a)) (9)

[0059] The pseudocode for the QL agent is as follows: it takes as input an action and a fitness value parameter, makes decisions based on the agent, and outputs an adjusted mutation rate parameter. The QL agent dynamically adjusts the mutation rate throughout the iterations of the Mayfly algorithm, improving the algorithm's adaptability at different stages and significantly enhancing its global search and local optimization capabilities when solving hybrid flow shop scheduling models.

[0060] Step 2: Fusion of QL algorithm and Mayfly algorithm

[0061] The enhanced mayfly algorithm (RMA) is obtained by fusing the QL algorithm with the mayfly algorithm. The specific steps of the algorithm process are as follows:

[0062] Step (1): Initialization phase combined with scheduling problem

[0063] When initializing the Mayfly algorithm parameters, a population is designed for the hybrid flow shop scheduling problem, including mutation rate initialization and population initialization. N individuals are randomly generated from both male and female populations, and each individual (Mayfly) represents a possible scheduling solution (the first-level gene represents the workpiece processing sequence (e.g., [3,1,2]), and the second-level gene represents the equivalent parallel machine number assigned to each process (e.g., [1,2,1]). ) To ensure that the initial solution covers different machine tool load balancing states. When the Q learning algorithm is initialized, the "state" of the Q table is defined as the performance indicator of the current scheduling solution (such as the maximum completion time).

[0064] Step (2): Dynamic adjustment and scheduling optimization

[0065] In the early stages of iteration, since it is not possible to judge the quality of individuals by fitness values, actions are randomly selected from the action space to explore diverse scheduling schemes (randomly adjust the workpiece processing order or machine tool allocation). In subsequent iterations, actions are selected based on the Q table:

[0066] If the current scheduling plan has a bottleneck (such as a machine tool with too high a load), select "Increase mutation rate" to escape the local optimum.

[0067] If the scheduling plan is already good (such as a short completion time), select "Reduce mutation rate" for fine-tuning.

[0068] Step (3): Mayfly algorithm execution and schedule generation

[0069] The updated mutation rate is applied to the Mayfly algorithm, and the inner Mayfly algorithm begins executing. This process includes multiple steps: first, individual updates, then an overall population update, and finally, crossover operations between individuals, as well as the mutation strategy introduced in the Mayfly algorithm.

[0070] Individual update: simulates the flow of workpieces between machine tools and adjusts the processing order according to the speed vector (giving priority to long and time-consuming processes).

[0071] Crossover operation: Exchange the workpiece sequence segments of the parent scheduling plan to generate a child scheduling plan (for example: adjust workpiece 3 from machine tool 1 to machine tool 2).

[0072] Mutation operation: Randomly perturb the scheduling scheme (exchanging the processing order of adjacent workpieces) to cope with dynamic changes in the workshop.

[0073] Step (4): Fitness calculation and scheduling evaluation

[0074] The fitness of the mutated individuals is calculated and compared with the individuals before mutation. The fitness function is directly linked to the scheduling objective: calculating the maximum completion time for each schedule in the current population. Relatively good schedules (individuals with shorter completion times) are retained, while inferior ones are eliminated to ensure improvements in the next generation of schedules. If the preset maximum number of iterations is reached during the process, the process ends. If not, the subsequent steps are continued. This looping mechanism helps ensure that the algorithm fully utilizes the evolutionary results of each generation during the optimization process, continuously approaching the optimal solution (minimum completion time).

[0075] Step (5): Q-learning feedback and scheduling strategy optimization

[0076] Assuming the process doesn't terminate at this point, the outer QL algorithm is executed after the mutation. Based on the individual fitness (the reduction in completion time) obtained in step 4, a reward is calculated and the Q table is updated. If the mutated schedule is better, the reward is positive, reinforcing the current action (increasing the mutation rate). If the schedule deteriorates, the penalty is negative to avoid repeated inefficient adjustments. Finally, the loop returns to step 2 and continues until the preset termination condition (number of iterations) is met, ultimately achieving the optimal solution (minimum completion time).

[0077] Through step 2, the two algorithms are combined to form an enhanced Mayfly algorithm with an inner-outer nested optimization structure. This algorithm leverages the global exploration capabilities of the Mayfly algorithm and the dynamic decision-making advantages of the QL algorithm. During the algorithmic operation, the algorithm improves its efficiency in solving hybrid flowshop scheduling problems.

[0078] The enhanced mayfly algorithm solution model includes encoding, decoding, crossover and mutation;

[0079] The coding in step 3 refers to a two-layer coding method for solving the problem of machine tool allocation for the processing sequence and processing stage of the workpiece on each machine tool;

[0080] The decoding in step 3 means first decoding the process code in the processing stage code, then decoding the machine tool code, and then constructing the machine matrix J by extracting the machine code and process code information in the chromosome. M , workpiece processing time matrix T1 and machine preparation time matrix T2. According to the processing schedule, the end time is calculated. During the decoding process, the workpiece processing end time E ijkm Compare and analyze the idle time period of the machine, and then select the machine with the earliest completion time among all machines that can process the process;

[0081] The crossover in step 3 refers to performing a crossover operation on the coded part in the double-layer coding;

[0082] The mutation in step 3 is to randomly select a part of genes for mutation according to the mutation rate to generate new individuals, increase the diversity of the population, and improve the ability of the enhanced mayfly algorithm to solve the hybrid flow shop scheduling problem.

[0083] Beneficial effects of the present invention:

[0084] Aiming at the hybrid flow shop scheduling problem, a hybrid flow shop scheduling model considering preparation time and equivalent parallel machines is established through step 1. The hybrid flow shop scheduling problem is expressed by a mathematical model, making the hybrid flow shop scheduling problem clearer and more specific.

[0085] The fusion of the Q-learning algorithm (QL) and the Mayfly algorithm (MA) does not simply superimpose the two, but overcomes the disadvantages of a single algorithm through deep integration, giving full play to the synergistic advantages of the two, and significantly improving the solution performance in the hybrid flow shop scheduling problem. This fusion method shows excellent performance in both global exploration and local development. Especially when the solution space is significantly increased, its optimization ability and convergence performance are significantly better than the simple Mayfly algorithm. By combining the advantages of both, the fusion method achieves more efficient and stable optimization effects when solving the hybrid flow shop scheduling problem, showing greater adaptability and flexibility. Specifically, this fusion method brings the following innovative benefits:

[0086] (1) Dynamic sample update and improved optimization accuracy: During the fusion process, the iterative process of MA provides high-quality samples for the QL algorithm, which can be used as the basis for the QL algorithm to update learning in real time. This mechanism significantly improves the optimization accuracy of the QL algorithm in local development, enabling it to adjust the strategy more accurately and gradually approach the optimal solution. Through this dynamic sample update mechanism, the QL algorithm can better adapt to the complexity of the hybrid flow shop scheduling problem, thereby achieving higher accuracy and efficiency in the local optimization process.

[0087] (2) Dynamic parameter adjustment: In traditional applications, the parameters of the Mayfly algorithm remain unchanged once set, making the algorithm performance more sensitive to the parameter settings. However, in the fusion method, by introducing the QL algorithm, the dynamic update of the mutation rate parameter of the MA is achieved. According to the progress of the algorithm, the size of the mutation rate is adjusted in a timely manner, thereby achieving adaptive optimization of the algorithm performance. This dynamic parameter adjustment mechanism not only improves the flexibility of the algorithm, but also enhances its adaptability at different stages, further improving the overall optimization performance.

[0088] (3) Enhanced stability and avoidance of local optimality: The QL algorithm itself has excellent convergence performance. By embedding the QL algorithm in step 2, the convergence stability of MA is further improved and the possibility of falling into local optimality is significantly reduced. This combination achieves a balance between expanding the exploration scope and deepening the development level, making the fusion method perform well in both global exploration and local development. Through this balance mechanism, the fusion method can not only effectively avoid the local optimal trap, but also achieve more stable and reliable optimization results in complex hybrid flow shop scheduling problems.

[0089] The enhanced Mayfly algorithm can be applied to shop scheduling problems and can effectively solve mixed flow shop scheduling problems. It improves the search and optimization capabilities under problems with a larger solution space and can efficiently optimize the processing schedule and total processing time of workpieces.

[0090] The processing preparation time of the workpiece and the equivalent parallel machine are taken into consideration, which is more in line with the actual workshop production requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 This is the pseudo code diagram of the QL agent algorithm of the present invention.

[0092] Figure 2 This is the flow chart of the enhanced mayfly algorithm of the present invention.

[0093] Figure 3 This is a schematic diagram of the chromosome encoding of the present invention.

[0094] Figure 4 It is a cross schematic diagram of the present invention.

[0095] Figure 5 Schematic diagram of the variation iteration of the present invention.

[0096] Figure 6 This is an analysis diagram of the orthogonal experiment results.

[0097] Figure 7 Iteration curves of the four algorithms.

[0098] Figure 8 Scheduling Gantt chart for the present invention. DETAILED DESCRIPTION

[0099] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0100] like Figures 1-8 As shown:

[0101] Step 1: Based on the scheduling characteristics of hybrid flow shop, a hybrid flow shop scheduling model is established that takes into account setup time and equivalent parallel machines;

[0102] Step 1: The hybrid flow shop scheduling problem can be described as follows: There are several workpieces to be processed, which need to be processed sequentially in the order of process 1, process 2, ... process m. At least one process involves multiple equivalent parallel machines. The same workpiece requires the same processing time on each equivalent machine tool. The first workpiece requires no setup time for the machine tool. However, the second workpiece, if different from the previous one, requires setup time. Taking machine tool M as an example, the workpieces are arranged as N1-N2-N2. After processing N1, processing N2 requires setup time, while the second process of processing N2 requires no setup time.

[0103] Step 2, symbolic definition in the hybrid flow shop scheduling model considering setup time and equivalent parallel machines:

[0104] N: total number of workpieces processed;

[0105] J: number of processes for a single workpiece;

[0106] M: the number of machine tools;

[0107] i: the index number of the workpiece (i∈{1,2,…,N});

[0108] j: the process index number of the workpiece (j∈{1,2,…,J};

[0109] M(j): the set of parallel machines available for process j;

[0110] I_J(k): the set of processing steps on machine tool k;

[0111] k: the index number of the machine tool (k∈{1,2,…,K});

[0112] b i,j : The starting time of the jth process of workpiece i;

[0113] e i,j : Completion time of process j of workpiece i;

[0114] t i,j,k : The processing time of workpiece i on parallel machine k at stage j;

[0115] O i,j : j-th process of workpiece i;

[0116] X i,l : Decision variable, when workpiece i is placed in the lth position waiting for processing, X i,l =1;

[0117] Y i,j,k : Decision variables, workpiece i's process j is processed on machine tool k, then X i,j,k =1, otherwise

[0118] X i,j,k =0;

[0119] C i : Completion time of workpiece i;

[0120] C max =max{C1,C2,...,C n}:maximum completion time;

[0121] st i,j : The processing preparation time required for workpiece i at stage j;

[0122] Step 3: Establish constraints based on the characteristics of workpiece processing in a hybrid flow shop:

[0123] Constraint 1: Each process of each workpiece can only be performed on one machine.

[0124]

[0125] Constraint 2: There must be a one-to-one correspondence between each workpiece and its processing sequence

[0126]

[0127] Constraint 3: The completion time e of the jth process of workpiece i on parallel machine k i,j Equal to the sum of the processing start time, processing preparation time and actual processing time

[0128]

[0129] Constraint 4: In the same processing stage, the workpiece with the highest order will be processed first

[0130]

[0131] Constraint 5: The workpiece must complete the processing task of the previous stage before entering the next stage of processing

[0132] b i,j+1 -e i,j ≥0,i=1,...,n,j=1,...,s-1(5)

[0133] Constraint 6: The maximum completion time of job i is the maximum completion time of the last process of all jobs

[0134] C i =max(Eijtm ),j=1,2,...,P i (6)

[0135] Step 4, establish the objective function of the model, the expression is as follows:

[0136] Min C max =max{C i |i=1,2,...,n} (7)

[0137] This method establishes an objective function using the maximum completion time of a workpiece as the optimization objective to minimize the maximum completion time. Through step 1, the hybrid flow shop scheduling problem, which considers setup time and equivalent parallel machines, is transformed into a concrete mathematical model. The corresponding constraints and objective function are established, paving the way for subsequent problem solving.

[0138] Step 2: For the hybrid flow shop scheduling model, an enhanced mayfly algorithm is designed to prepare for solving the problem.

[0139] Enhanced Mayfly Algorithm

[0140] The fusion of the Q-learning algorithm and the Mayfly algorithm overcomes the disadvantages of the two algorithms and combines their advantages, combining the advantages of global exploration and local development. The specific fusion steps are as follows:

[0141] Step 1, Intelligent Agent Design

[0142] The Outer QL algorithm dynamically updates the mutation rate by designing an agent that meets the characteristics of the hybrid flow shop scheduling problem, further enhancing the performance of the Mayfly algorithm and improving the efficiency of the problem-solving model. The agent design is divided into state space, action space, reward and penalty functions, and greedy strategies. The design process requires defining the state space, action space, and reward and penalty functions based on the Mayfly algorithm and the hybrid flow shop scheduling problem to ensure that they can effectively capture the requirements of dynamic adjustment of the mutation rate while maintaining the algorithm's adaptability and flexibility at different iteration stages. The detailed division of the state space must balance the breadth and depth of solution exploration, while the construction of the reward and penalty function requires comprehensive consideration of the magnitude of fitness optimization and the characteristics of the convergence phase to achieve continuous incentives for the objective function.

[0143] (2.1) State Space: The values ​​of the mutation rate parameters in the Mayfly algorithm constitute the state space of the agent. The mutation rate parameters in state space S directly affect the processing order of workpieces on each machine tool and the selection of machine tools. Based on the characteristics of the hybrid flow shop scheduling problem, to improve the algorithm's precision and flexibility and ensure that individuals retain some of their parent's genes after mutation, the mutation rate parameters in the state space are discretized into 11 state values ​​within the interval [0.2, 0.7] with a step size of 0.05. That is, there are a total of 11 states in state space S. State space S = {0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7}. These 11 discrete states cover a reasonable range from low to high mutation rates, with an upper limit of 0.7. A low mutation rate (0.2-0.35) helps to retain the current better processing sequence, while a high mutation rate (0.5-0.7) enhances the ability to explore new processing sequences to adapt to the complex workpiece processing requirements in the hybrid flow shop scheduling problem.

[0144] (2.2) Action Space: The actions in action space A (increasing, maintaining, or decreasing the mutation rate) directly affect the algorithm's ability to explore and exploit the hybrid flow shop scheduling problem. Increasing the mutation rate facilitates a more extensive search within the solution space, seeking new optimal solutions; decreasing the mutation rate facilitates a more refined search near the current solution, optimizing the existing solution.

[0145] (2.3) Reward and Penalty Function: The design of the reward and penalty function takes into account the optimization objective of the hybrid flow shop scheduling problem, that is, minimizing the maximum completion time. The reward function is not only related to the degree of optimization of the fitness value, but also takes into account the characteristic that the fitness value gradually converges in the later stages of the algorithm, and encourages the algorithm to adopt appropriate mutation rates in different iteration stages to optimize the maximum completion time. The reward function is shown in formula (8). This reward function is not only related to the degree of optimization of the fitness value, but also takes into account the characteristic that the fitness value gradually converges in the later stages of the algorithm. Even if the degree of optimization of a certain fitness value is the same as that of the previous generation, as the number of iterations increases, a higher reward should be obtained in the later stages. This design aims to encourage the algorithm to adopt a more appropriate mutation rate, thereby improving its ability to solve the hybrid flow shop scheduling problem.

[0146]

[0147] Where Δf is the difference in fitness between the previous generation and the current generation of mayflies, f current is the fitness value of the current generation of mayflies, t is the current iteration number, T is the maximum iteration number, and k is the reward coefficient.

[0148] Q-value update: The Q-value update formula of the QL algorithm is shown in Equation (9), where Q(s, a) is the Q-value of the action taken under the current state and action. First, maxQ(s', a') is attenuated by the discount factor, then subtracted from Q(s, a), and then the reward r is added. The result is weighted again by the learning rate and added to the current Q-value, replacing the original Q(s, a).

[0149] Q(s,a)=Q(s,a)+α(r+γ*maxQ(s',a')-Q(s,a)) (9)

[0150] The pseudo code of QL agent is as follows Figure 1 As shown in the figure, by inputting parameters such as actions and fitness values, and after the agent's decision-making, the adjusted mutation rate parameters are output. The QL algorithm's agent dynamically adjusts the mutation rate throughout the entire iteration of the Mayfly algorithm, which not only improves the Mayfly algorithm's adaptability at different stages but also significantly enhances its global search and local optimization capabilities when solving the hybrid flow shop scheduling problem model.

[0151] Step 2: Fusion of QL algorithm and Mayfly algorithm

[0152] The enhanced mayfly algorithm (RMA) is obtained by fusing the QL algorithm with the mayfly algorithm. The enhanced mayfly algorithm process is as follows: Figure 2 The specific steps of the algorithm are as follows:

[0153] Step 1: Initialize the Mayfly algorithm parameters, including mutation rate and population initialization. Randomly generate N individuals from each male and female population to ensure the algorithm fully utilizes population diversity. Also, initialize the QL algorithm parameters, setting the Q table to 0.

[0154] Step 2: In the first iteration, since fitness cannot yet be used to determine the quality of individuals, actions are randomly selected from the action space. In subsequent iterations, the algorithm selects the action that maximizes reward based on the current Q-table value and the greedy strategy, or selects random actions with a certain probability to balance exploration and exploitation. The mutation rate is dynamically updated based on the results of the selected actions.

[0155] Step 3: Apply the updated mutation rate to the Mayfly algorithm, beginning the inner Mayfly algorithm process. This process involves multiple steps: first, updating the individual population, then updating the entire population, then performing crossover operations between individuals, and then implementing the mutation strategy introduced in the Mayfly algorithm. These steps aim to improve the diversity and adaptability of the population, thereby enhancing the algorithm's global search capabilities and local development capabilities.

[0156] Step 4: Calculate the fitness of the mutated individuals and compare them with those before the mutation. This step is crucial because it determines whether the individuals will be retained in the next generation. If the maximum number of iterations is reached, the process ends. If not, the next step continues. This iterative mechanism helps ensure that the algorithm fully utilizes the evolutionary results of each generation during the optimization process, continuously approaching the optimal solution.

[0157] Step 5: Assuming the process doesn't terminate at this point, the outer QL algorithm is executed after the mutation. Rewards are calculated based on the individual fitness obtained in Step 4, and the Q table is updated so that it gradually reflects the value of different actions under different states. This process not only helps the algorithm optimize the current solution space but also provides valuable information for action selection in the next iteration. Finally, the algorithm returns to Step 2 and continues the loop until the preset termination condition is met.

[0158] In step 2, appropriate parameters were selected based on the characteristics of the hybrid flowshop scheduling problem. The outer QL algorithm and the inner Mayfly algorithm were then combined to form an enhanced Mayfly algorithm with a nested optimization structure. This algorithm leverages the global exploration capabilities of the Mayfly algorithm and the dynamic decision-making advantages of the QL algorithm. During the algorithmic operation, the algorithm's efficiency in solving the hybrid flowshop scheduling problem was improved.

[0159] Step 3 is as follows:

[0160] Step 1, coding

[0161] For the hybrid flow shop scheduling problem, this method uses a two-layered chromosome encoding to solve the problem of determining the processing order of workpieces on each machine tool and assigning them to each processing stage. The first-layer encoding represents the process code, which is generated by arranging all workpiece numbers from left to right without duplication. Process codes closer to the left indicate earlier processing. The second-layer machine tool code consists of all available parallel machines in that stage, with each gene bit corresponding to a workpiece number in the first-layer process code.

[0162] exist Figure 3 In the chromosome encoding shown, assuming there are 5 workpieces, the number of available parallel machines in the first stage is 2. During initialization, the first-level process codes are randomly sorted according to the number of workpieces N and are not repeated. The second-level machine tool codes randomly select the available parallel machine in the first stage: parallel machine 1 or 2.

[0163] Step 2, decoding

[0164] The chromosome decoding process converts genetically encoded information into a scheduling solution for the workpiece process. This is mainly to solve the problem of the processing order of the workpiece on each machine tool and the machine tool allocation problem in the hybrid flow shop scheduling problem. This process adopts a hierarchical parsing strategy: first decode the process code part, then decode the machine tool code. In the process of decoding the machine selection part, the machine code information in the chromosome needs to be extracted in order from left to right, and then the machine matrix J is constructed. M , workpiece processing time matrix T1 and machine preparation time matrix T2. In the decoding link of the process sorting part, the sub-batch process scheduling code in the chromosome is read one by one in the same order from left to right. During the decoding process, the workpiece processing end time E ijkm Compare and analyze the idle time period of the machine, and then select the machine with the earliest completion time among all the machines that can process the process. Figure 3 For example, the first-level process code indicates that the processing sequence for the first-stage workpiece is 3-2-4-5-1. The second-level machine tool code identifies the machine tool number used for the workpiece. Combining these two, the first-stage processing schedule is determined as follows: the first digit, workpiece 3, uses machine tool 1; the second digit, workpiece 2, uses machine tool 2; the third digit, workpiece 4, uses machine tool 2; the fourth digit, workpiece 5, uses machine tool 2; and the fifth digit, workpiece 1, uses machine tool 1. This decoding solves the hybrid flow shop scheduling problem by determining the processing sequence for each workpiece on each machine tool and assigning machines to each processing stage, and by calculating the end time based on the processing schedule.

[0165] Step 3, cross

[0166] The crossover operation involves swapping genes from parent chromosomes to create new individuals. This operation impacts algorithm performance. In a two-layer encoding scheme, changes to the upper-layer process code will also affect the lower-layer machine tool code. Therefore, when performing a crossover, only the upper-layer process code needs to be changed. By directly swapping gene segments from the parent chromosome within the crossover interval and performing mapping and filling outside of the interval, this method effectively preserves some structural information from the parent chromosome, maintaining the desirable characteristics of the parent while increasing population diversity. Simultaneously, the lower-layer machine code should also be adjusted to align with the sub-batch process scheduling code.

[0167] according to Figure 4 The crossover operation process is as follows:

[0168] Step 1: Given two parent chromosomes, p1 and p2, for each gene position in offspring 1, with p1 as the base, each position selects a gene from p1 with a 50% probability and passes it on to offspring 1, and the remaining positions select from p2. The genes at each position in offspring 2 will inherit the two parent genes that were not selected by offspring 1. Figure 4 For example, the genes {7, 2, 3, 9, 4} of parent p1 are inherited to offspring 1, and {5, 1, 8, 6, 10} are not inherited. Therefore, offspring 2 will select the genes {5, 1, 8, 6, 10} of parent p1, and the other positions will be selected from parent p2.

[0169] Step 2: For the first row of the daughter chromosome, record the used genes and repair the gene positions outside the crossover interval according to the mapping relationship to ensure that each gene is unique.

[0170] Step 3: Adjust the lower machine tool coding

[0171] Mutations

[0172] In evolutionary algorithms, mutation is the main way for population evolution. A mutation rate that is too high or too low may lead to premature convergence of the algorithm. Therefore, this method introduces a non-uniform mutation strategy. The QL algorithm enables the agent to adaptively adjust the mutation rate and applies it to the mutation operator of the Mayfly algorithm. This non-uniform mutation operator better explores the solution space by increasing diversity and randomness. Its mutation amplitude gradually decreases with the increase of evolutionary generations, allowing the algorithm to perform a large-scale search in the early stage and a fine search in the later stage. The mutation operation randomly selects a part of genes for mutation based on the mutation rate to generate new individuals, thereby increasing the diversity of the population. The number of mutations is determined by formula (9):

[0173] n=μ·N (9)

[0174] Where μ is the mutation rate and N is the gene length. For the selected gene, the variation δ is calculated as shown in (10).

[0175]

[0176] Where UpperBound is the upper bound, LowerBound is the lower bound, is a uniformly distributed random number, is the current number of iterations, and is the maximum number of iterations. Figure 5 This is a graph showing the amount of change in non-uniform mutation versus the number of iterations. The upper bound is 10 and the lower bound is -10. It can be seen that the amount of change slowly decreases with the increase in the number of iterations.

[0177] When implementing mutation operations on the process scheduling coding part of the hybrid flow shop scheduling problem, the following two methods are followed:

[0178] (1) When mutating the process scheduling code in the upper layer code, a gene in the process scheduling code is randomly selected and assigned a new value within the specified range. If this operation results in an illegal chromosome, the chromosome is repaired using the repair method used in the crossover operation.

[0179] (2) When performing mutation operations on process scheduling codes, the double-point permutation method is used, that is, two process scheduling codes and their corresponding machine selection codes are randomly selected and exchanged with each other.

[0180] The encoding and decoding in step 3 determine the machine tool allocation for each processing stage of the hybrid flow shop scheduling problem and the processing order of the workpieces on each machine tool. Crossover mutation is used to improve the genetic iteration behavior of the mayflies, increase the diversity of the population, and enhance the ability of the enhanced mayfly algorithm to solve the hybrid flow shop scheduling problem.

[0181] A series of examples that conform to HFSP-ST-EPM are used to illustrate the practicality and effectiveness of the present invention. The example information is as follows: the workpiece N scale is 10, 30, and 60, the number of process stages S is 5 and 10, the processing time of each process stage follows a uniform integer distribution of [3, 20], and the preparation time follows a uniform integer distribution of [1, 9]. The time is dimensionless. In order to be closer to the actual production layout environment of the workshop, five actual workshop layout schemes are referenced. The specific workshop layout types are shown in Table 1. Based on the above parameter combinations, there are a total of 30 (3×2×5, workpiece×process×layout scheme) scale examples. In order to eliminate the influence of random factors on the experimental results, 3 groups of examples of each scale are randomly generated, and finally 90 (30×3) test examples are obtained.

[0182] Table 1 Workshop layout

[0183]

[0184] The Q-learning algorithm gradually converges and reaches a stable state during long-term training. This algorithm typically uses a low learning rate. A higher learning rate may cause the agent to overemphasize low-value information, leading to Q-value fluctuations and, in turn, affecting the algorithm's convergence efficiency and ultimate performance. The discount factor, a key parameter for evaluating the weighting of future rewards, will gradually become effective and stabilize with sufficient sample size and multiple iterations. The exploration rate determines the probability of exploring more possible options, balancing the frequency of new actions with the frequency of known actions. Therefore, based on the characteristics of the Q-learning algorithm, the learning rate α is set to 0.1, the discount factor γ is set to 0.9, and the exploration rate ε is set to 0.2.

[0185] The Mayfly algorithm has numerous parameters. In addition to the mutation rate, key parameters include population size, individual positive attraction coefficient, population positive attraction coefficient, and random walk coefficient. The size and combination of these parameters can affect the algorithm's progress and performance. To prevent the impact of improper algorithm parameter settings on the experiment, an orthogonal experiment was used to identify an optimal parameter setting. Three different levels of each parameter were proposed for the four parameters shown in Table 2. An L9 orthogonal table was used to cover all cases, resulting in nine experimental groups. Each level of each parameter appeared an equal number of times in the experiment to ensure uniformity and representativeness. Each parameter combination in the orthogonal experiment was run ten times, and the algorithm was terminated after 50 iterations. The experimental case was N30S10d1, where N30 represents 30 workpieces, S10 represents 10 stages, d represents the type of shop floor layout, and 1 represents the first set of randomly generated production data. The parameter grouping and experimental results are shown in Table 3.

[0186] Table 2 Parameters at different levels

[0187]

[0188] Table 3 Parameter orthogonal experiment results

[0189]

[0190] Figure 6 The following is an analysis chart of the orthogonal experiment results. The bold line represents the trend of the orthogonal experiment results. It can be seen that the experimental results of group 7 are the best, with a maximum completion time of 348.7. The experiment uses the parameter combination of group 7: n = 30, α1 = 0.8, α2 = 1.7, and λ = 1.5.

[0191] The parameter settings of the comparison algorithm selected by the present invention are shown in Table 4.

[0192] Table 4 Parameter settings of three comparison algorithms

[0193]

[0194]

[0195] We conducted simulation experiments using the RMA, the original MA, the GA, and the PSO in this case study. Each case was run ten times independently. The specific experimental data is shown below. Tables 5 and 6 show the experimental data results for the 5-stage and 10-stage cases, respectively. (The bold data in the table represents the optimal scheduling results.)

[0196] Table 5 Results of the five-stage example run

[0197]

[0198] Table 5 Results of the five-stage example run (continued)

[0199]

[0200]

[0201] Table 5 Results of the five-stage example run (continued)

[0202]

[0203] Analysis of the experimental results in the table above shows that, among the algorithms tested, RMA successfully found the optimal solution in 41 of 45 cases, with only 4 failing to find the optimal solution. MA found the optimal solution in 28 of 45 cases, with 17 failing to find the optimal solution. GA found the optimal solution in 34 of 45 cases, with 11 failing to find the optimal solution. PSO found the optimal solution in 26 of 45 cases, with 19 failing to find the optimal solution. The optimality rates of the algorithms were 91.1% for RMA, 62.2% for MA, 75.6% for GA, and 57.8% for PSO. Further analysis revealed that for shop floor layout types a, b, and e, the solution space is relatively small, as these layouts have bottlenecks with only one machine tool at a certain stage. In these cases, nearly all four algorithms found the optimal solution. When the workshop layout types are c and d, the solution space for the problem increases, and the RMA algorithm's optimization advantage becomes more prominent, with its optimization rate far exceeding that of the other three algorithms. This fully demonstrates RMA's superior search and optimization capabilities when faced with complex problems with larger solution spaces.

[0204] In terms of mean performance, RMA only had three suboptimal results, MA had 24 suboptimal results, GA had 22 suboptimal results, and PSO had 30 suboptimal results. In terms of RPD performance, based on its mean, RMA had four suboptimal results on the Min metric, with an average RPD of 0.72%. The average RPDs for MA, GA, and PSO were 1.6%, 4.9%, and 2.4%, respectively. This shows that even when RMA misses the optimal solution, its results are close to the optimal value, and it can still find a high-quality solution. RMA's convergence and robustness are superior to the other three algorithms. Figure 7 The comparison chart of the iteration curves of four algorithms used in experiments on a certain example.

[0205] Table 610 Stage Calculation Results

[0206]

[0207] Table 610 Phase Calculation Results (Continued)

[0208]

[0209] Table 610 Stage Calculation Results (Continued)

[0210]

[0211] In the 10-stage case scale, RMA found the optimal solution in 41 cases and did not find the optimal solution in 4 cases; MA found the optimal solution in 30 cases and did not find the optimal solution in 15 cases; GA found the optimal solution in 32 cases and did not find the optimal solution in 13 cases; PSO found the optimal solution in 21 cases and did not find the optimal solution in 24 cases. The optimization rates of each algorithm are: RMA 91.1%, MA 66.7%, GA 71.1%, PSO 46.7%.

[0212] In terms of mean performance, RMA failed to reach optimality in six cases, MA failed to reach optimality in 31 cases, GA failed to reach optimality in 30 cases, and PSO failed to reach optimality in 40 cases. As the number of stages increased and the solution space expanded, the convergence ability of the compared algorithms decreased. However, while RMA failed to reach optimality in six cases compared to the five-stage algorithm, it achieved optimal mean performance in five of these cases, even though mean performance did not reach optimality. The mean RPD values ​​of the four algorithms were 0.59%, 0.86%, 0.66%, and 1.26%, respectively. RMA performed well in multi-stage cases, especially when the solution space expanded significantly, where its optimization and convergence performance outperformed the other compared algorithms. RMA was able to find optimal solutions in most cases and demonstrated high stability and reliability in terms of mean performance. In contrast, the optimization and convergence performance of the MA, GA, and PSO algorithms decreased with an increase in the number of stages, resulting in poor optimization rates and mean RPD values.

[0213] Taking the N10S5C (10-workpiece 5-stage C-type workshop layout) mentioned above as an example, Table 7 shows an example of the hybrid flow shop scheduling problem. In this scheduling problem, there are 10 types of workpieces, each workpiece has 5 processes, and each process can be processed by multiple parallel machines.

[0214] Table 7 Example of hybrid flow shop scheduling problem

[0215]

[0216]

[0217]

[0218] In accordance with the actual production requirements of the workshop, when the workpiece is processed on different machines, the installation, positioning and tool replacement of the workpiece require preparation time. Based on the data in Table 7, the preparation time of the workpiece is added. Then, the enhanced ephemera algorithm is used to solve the problem. Compared with other algorithms in Table 5, the maximum completion time is further reduced to 125. The resulting scheduling diagram is shown in the figure below. Figure 8 As shown, each colored block represents the processing process of the workpiece, different colors represent different workpieces, and the data in the color block has its corresponding meaning. Taking p(10)=7 as an example, it means that the first process of workpiece No. 10 is processed on machine 1, and the processing time is 7. The white blocks between different color blocks represent the preparation process, and the numbers on the left side of the white blocks represent the preparation time.

Claims

1. A hybrid flow shop scheduling method considering setup time and equivalent parallel machines, characterized in that: The following steps are included: Step 1: Based on the scheduling characteristics of hybrid flow shop, a hybrid flow shop scheduling model is established that takes into account setup time and equivalent parallel machines; Step 2: Design an enhanced Mayfly algorithm that integrates the Q-learning algorithm with the Mayfly algorithm; the outer layer uses the QL algorithm and the inner layer uses the Mayfly algorithm; Step 3: Use the enhanced mayfly algorithm to solve the problem model, optimize the machine allocation and processing sequence of the workpiece, and obtain the minimum value of the maximum completion time.

2. A hybrid flow shop scheduling method considering setup time and equivalent parallel machines according to claim 1, characterized in that: The symbolic definition in the hybrid flow shop scheduling model considering preparation time and equivalent parallel machines in step 1; establishing constraints and establishing an objective function based on the characteristics of the hybrid flow shop workpiece considering the processing preparation time and parallel machines.

3. A hybrid flow shop scheduling method considering setup time and equivalent parallel machines according to claim 2, characterized in that: The step 1 is specifically as follows: Step 1: The hybrid flow shop scheduling problem can be described as follows: There are several workpieces to be processed, which need to be processed in the order of process 1, process 2, ... process m. At least one process has multiple equivalent machines running in parallel. The same workpiece requires the same amount of time to be processed on each equivalent machine tool. The machine tool does not need setup time to process the first workpiece, but when processing the second workpiece, if it is different from the previous workpiece, the machine tool needs a certain amount of setup time. Taking machine tool M as an example, the workpieces are arranged as N1-N2-N2. After processing N1, processing N2 requires setup time, but processing the second process of N2 does not require setup time. Step 2, symbolic definition in the hybrid flow shop scheduling model considering setup time and equivalent parallel machines: N: total number of workpieces processed; J: number of processes for a single workpiece; M: the number of machine tools; i: the index number of the workpiece (i∈{1,2,…,N}); j: the process index number of the workpiece (j∈{1,2,…,J}; M(j): the set of parallel machines available for process j; I_J(k): the set of processing steps on machine tool k; k: the index number of the machine tool (k∈{1,2,…,K}); b i,j : The starting time of the jth process of workpiece i; e i,j : Completion time of process j of workpiece i; t i,j,k : The processing time of workpiece i on parallel machine k at stage j; O i,j : j-th process of workpiece i; X i,l : Decision variable, when workpiece i is placed in the lth position waiting for processing, X i,l =1; Y i,j,k : Decision variables, workpiece i's process j is processed on machine tool k, then X i,j,k =1, otherwise X i,j,k ＝0； C i : Completion time of workpiece i; C max =max{C1,C2,...,C n }:maximum completion time; st i,j : The processing preparation time required for workpiece i at stage j; Step 3: Establish constraints based on the characteristics of workpiece processing in a hybrid flow shop: Step 4, establish the objective function of the model, the expression is as follows: My C max =max{C i |i=1,2,...,n} (7).

4. A hybrid flow shop scheduling method considering setup time and equivalent parallel machines according to claim 3, characterized in that: Constraint 1: Each process of each workpiece can only be processed on one machine Constraint 2: There must be a one-to-one correspondence between each workpiece and its processing sequence Constraint 3: The completion time e of the jth process of workpiece i on parallel machine k i,j Equal to the sum of the processing start time, processing preparation time and actual processing time Constraint 4: In the same processing stage, the workpiece with the highest order will be processed first Constraint 5: The workpiece must complete the processing task of the previous stage before entering the next stage of processing b i,j+1 -e i,j ≥0,i=1,...,n,j=1,...,s-1(5) Constraint 6: The maximum completion time of job i is the maximum completion time of the last process of all jobs C i =max(E ijtm ),j=1,2,...,P i (6)。 5. The hybrid flow shop scheduling method considering setup time and equivalent parallel machines according to claim 4, characterized in that: The step 2 is specifically as follows: Step 1: (2.1) State space: The mutation rate parameter of the state space is discretized into 11 state values ​​in the interval [0.2, 0.7] with a step size of 0.

05. That is, there are 11 states in the state space S; state space S = {0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7}. The 11 discrete states cover a reasonable range from low mutation rate to high mutation rate, with an upper limit of 0.

7. (2.2) Action Space: The actions in the action space A directly affect the algorithm's ability to explore and exploit the hybrid flow shop scheduling problem. Increasing the mutation rate allows for a wider search in the solution space to find new optimal solutions; decreasing the mutation rate allows for a more refined search near the current solution to optimize the existing solution. (2.3) Reward and Penalty Function: The design of the reward and penalty function takes into account the optimization goal of the hybrid flow shop scheduling problem, which is to minimize the maximum completion time; The reward function is shown in formula (8): Where Δf is the difference in fitness between the previous generation and the current generation of mayflies, f current is the fitness value of the current generation of mayflies, t is the current iteration number, T is the maximum iteration number, and k is the reward coefficient; Q-value update: The Q-value update formula of the QL algorithm is as shown in Equation (9), where Q(s, a) is the Q-value of the action taken under the current state and action; First, maxQ(s',a') is attenuated by the discount factor, then subtracted from Q(s,a), and then added with the reward r. After being weighted again with the learning rate, it is added to the current Q value and the new result is used to update and replace the original Q(s,a). Q(s,a)=Q(s,a)+α(r+γ*maxQ(s',a')-Q(s,a)) (9) The specific pseudo code of the QL agent is to input the action and fitness value parameters, make the decision of the agent, and output the adjusted mutation rate parameters; Step 2: The QL algorithm is integrated with the Mayfly algorithm to obtain the enhanced Mayfly algorithm.

6. A hybrid flow shop scheduling method considering setup time and equivalent parallel machines according to claim 5, characterized in that: The specific steps of the algorithm process are as follows: Step (1): When initializing the parameters of the Mayfly algorithm, a population is designed for the hybrid flow shop scheduling problem, including mutation rate initialization and population initialization. N individuals are randomly generated from the male and female populations, each representing a possible scheduling solution. The second-layer gene represents the equivalent parallel machine number assigned to each process to ensure that the initial solution covers different machine tool load balancing states. When the Q learning algorithm is initialized, the "state" of the Q table is defined as the performance indicator of the current scheduling solution. Step (2): At the beginning of the iteration, randomly select actions from the action space to explore diverse scheduling schemes; in subsequent iterations, select actions based on the Q table: If the current scheduling solution has a bottleneck, select "Increase mutation rate" to escape the local optimum. If the scheduling plan is already optimal, select "Reduce mutation rate" for fine-tuning. Step (3): First, the individual is updated, then the whole population is updated, then the crossover operation between individuals, and the mutation strategy introduced in the Mayfly algorithm; Individual update: simulates the flow of workpieces between machine tools and adjusts the processing sequence according to the velocity vector; Crossover operation: swapping the artifact sequence segments of the parent scheduling solution to generate the child scheduling solution; Mutation operation: Randomly perturb the scheduling plan to cope with dynamic changes in the workshop; Step (4): Calculate the fitness of the individuals after mutation and compare them with those before mutation. The fitness function is directly related to the scheduling goal: calculate the maximum completion time of each scheduling scheme in the current population; retain the better scheduling schemes and eliminate the inferior ones to ensure the improvement of the next generation scheduling scheme; if the preset maximum number of iterations is reached during the process, the process will end. If not, continue to the next step; Step (5): Assuming that the process does not end at this time, the outer QL algorithm is executed after the mutation; the reward is calculated based on the individual fitness obtained in Step (4) and the Q table is updated. If the scheduling plan is better after the mutation, the reward value is positive and the current action is strengthened; if the plan becomes worse, the penalty value is negative to avoid repeated inefficient adjustments; finally, return to the second step and continue to execute the loop until the preset termination condition is met, and finally the optimal solution is obtained.

7. A hybrid flow shop scheduling method considering setup time and equivalent parallel machines according to claim 6, characterized in that: In step 3, the enhanced mayfly algorithm solves the model including encoding, decoding, crossover and mutation; The coding refers to a two-layer coding method for solving the problem of machine tool allocation for the processing sequence and processing stage of workpieces on each machine tool; The decoding means first decoding the process code in the processing stage code, then decoding the machine tool code, and then constructing the machine matrix J by extracting the machine code and process code information in the chromosome. M , workpiece processing time matrix T1 and machine preparation time matrix T2. According to the processing schedule, the end time is calculated. During the decoding process, the workpiece processing end time E ijkm Compare and analyze the idle time period of the machine, and then select the machine with the earliest completion time among all machines that can process the process; The crossover refers to performing a crossover operation on the coding part in the double-layer coding; The mutation is to randomly select a part of genes for mutation according to the mutation rate to generate new individuals, increase the diversity of the population, and improve the ability of the enhanced mayfly algorithm to solve the hybrid flow shop scheduling problem.

Images