A Multi-Objective Distributed Hybrid Flow Shop Scheduling Method

Abstract

This invention relates to a multi-objective distributed hybrid flow shop scheduling method, belonging to the technical field of hybrid flow shop scheduling. The invention establishes a corresponding objective function with the simultaneous minimization of maximum completion time and maximum processing time in distributed hybrid flow shop scheduling. First, a vector-based evaluation genetic algorithm and a fitness function based on Pareto dominance and non-dominance relationships are used to divide the population into three meme groups. Then, PSO is used to perform a global search on each meme group, exploring solutions in multiple directions of the Pareto front to accelerate the convergence speed of the upper and lower edges and the central region of the Pareto front. Second, critical-factory insert and critical-factory swap multi-neighborhood search operators are used to perform local searches on individuals, enhancing the quality of solutions in the meme groups. Third, a Q-Learning variable neighborhood search strategy is used to further enhance the algorithm's search capability on the three meme groups, thereby improving the quality and diversity of solutions and preventing the algorithm from converging prematurely and failing to find better solutions.

Description

Technical Field

[0001] This invention relates to a multi-objective distributed hybrid flow shop scheduling method, belonging to the field of hybrid flow shop scheduling technology. Background Technology

[0002] Production scheduling plays a crucial role in many manufacturing systems, and effective production scheduling is a key factor in improving industrial production efficiency and resource utilization. Therefore, the modeling and optimization of production scheduling is an important research topic in systems engineering, and studying production scheduling problems, especially developing effective solution algorithms, is of great significance. The Hybrid Flow Shop Scheduling Problem (HFSP) is a complex optimization decision problem in manufacturing and is currently widely used in industries such as electronics, glass, textiles, papermaking, and semiconductors. HFSP is an extension of the classic permutation flow shop scheduling problem, suitable for flexible production scheduling with multiple parallel machines at each stage. To effectively solve HFSP, many metaheuristic algorithms have been proposed, including Genetic Algorithm (GA), Artificial Bee Colony (ABC), Iterative Greedy Algorithm (IG), and Particle Swarm Optimization (PSO). With increasing market competition and personalized customer demands, centralized manufacturing seems unable to adapt to market needs. Therefore, many enterprises are expanding production into distributed environments, establishing factories or production centers in different locations. Distributed manufacturing facilitates resource sharing, improves economic efficiency, and enables rapid response to market changes. In this context, the Distributed Hybrid Flow Shop Scheduling Problem (DHFSP) is studied. DHFSP is more difficult than both HFSP and the Distributed Permutation Flow Shop Scheduling Problem. It requires simultaneously considering three tightly coupled subproblems: allocating jobs to different factories, selecting appropriate machines for jobs, and determining the processing order on each machine. With the increase in cooperating factories and the large number of jobs and machines in practice, the time scale for solving the problem can grow exponentially. Clearly, developing efficient optimization algorithms using emerging technologies to solve large-scale DHFSP has significant practical implications.

[0003] Early research on the distributed workshop scheduling problem relied on traditional mathematical methods that required extensive searches for every permutation and combination to find the optimal solution. For medium and large-scale problems, the solution time for exact algorithms was extremely long. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-objective distributed hybrid flow shop scheduling method to solve the problem of long calculation time in the current distributed hybrid flow shop scheduling solution process.

[0005] To solve the above-mentioned technical problems, this invention provides a multi-objective distributed hybrid flow shop scheduling method, which includes the following steps:

[0006] 1) Construct an objective function that simultaneously minimizes the maximum completion time and maximum processing time of the distributed hybrid flow shop scheduling;

[0007] 2) The workpiece and factory are encoded using an integer-based discrete sorting encoding method, and the population is initialized according to the encoding results;

[0008] 3) Divide the population into three meme groups and perform a global PSO search on each meme group;

[0009] 4) Perform interactive updates on the three meme groups and perform local reinforcement on key factories, where the key factories refer to the factories with the worst and best target values.

[0010] 5) Perform localized reinforcement on the internal structures of the selected factories within the factory set;

[0011] 6) Select the better individuals from the meme group after PSO search and the meme group after local reinforcement, and carry out the next population evolution for the selected individuals until the termination condition is met. The best individual obtained at the end is taken as the solution of the objective function.

[0012] This invention combines the global search capability of Particle Swarm Optimization (PSO) with the local search capability of Variable Neighborhood Search (VNSS) to design a multi-meme group interactive update meme algorithm. First, the population is divided into three meme groups, and PSO is used to perform a global search on each meme group. Second, a local search is performed on individuals targeting key factories to enhance the quality of solutions within the meme groups. Third, local reinforcement is applied within the selected factory set to further enhance the algorithm's search capability across the three meme groups, thereby improving the quality and diversity of solutions and preventing premature convergence that would prevent the algorithm from finding better solutions. This invention, through the synergy of global and local search strategies, can accelerate the processing speed of the meme algorithm for distributed hybrid flow shop scheduling problems and improve the quality of solutions.

[0013] Further, in step 3), the population is divided into three meme groups using a vector-based evaluation genetic algorithm and a fitness function method based on Pareto dominance and non-dominance relationships. The three meme groups are respectively the solution that satisfies the objective of minimizing the maximum completion time, the solution that minimizes the maximum processing time, and the solution that minimizes both the maximum completion time and the maximum processing time, i.e., the solution with the minimum fitness function value.

[0014] This invention uses a Vector Evaluated Genetic Algorithm (VEGA) and a Pareto Dominating and Dominated Relationship-based Fitness Function (PDDR-FF) to divide the population into three meme groups, which can accurately achieve the division of meme groups.

[0015] Furthermore, the local enhancement in step 4) is implemented using two local search operators. The first local search operator is critical-factory insert: randomly select a workpiece from the factory with the worst target value and insert it into a random position in the factory with the best target value; the second local search operator is critical-factory swap: randomly select two workpieces from the factory with the worst target value and the factory with the best target value, and swap these two workpieces.

[0016] Furthermore, the local strengthening process in step 4) is as follows:

[0017] a. Evaluate the population, interactively update the three meme groups, and obtain individuals from the meme groups;

[0018] b. Perform a local search on the individual within the critical-factory insert neighborhood. If the updated individual has a better target value than the individual before the update, continue searching within this neighborhood; otherwise, proceed to step c.

[0019] c. Perform a local search on the individual in the critical-factory swap neighborhood. If the updated individual has a better target value than the individual before the update, continue the search in this neighborhood; otherwise, proceed to step d.

[0020] d. Determine if the termination condition has been met. If it has, stop the search; otherwise, return to step b.

[0021] This invention utilizes critical-factory insert and critical-factoryswap multi-neighborhood search operators for critical factories to perform local searches on individuals, thereby preventing the phenomenon of wasted time due to excessive differences in the number of workpieces between critical factories, and avoiding the situation where, although the workpieces are evenly distributed, the processing time of most workpieces distributed in the same factory is too long, leading to an increase in the total processing time, thus enhancing the quality of solutions in the meme set.

[0022] Furthermore, step 5) employs Q-Learning and VNS for local search.

[0023] Furthermore, the local reinforcement process in step 5) is as follows:

[0024] A. Set the initial Q value and state of each individual in each meme group, and select the factory set to be locally enhanced from the action set;

[0025] B. For the selected factory set, use the insertion neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue the search within this neighborhood; otherwise, proceed to step C.

[0026] C. Use the inverted neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue the search within this neighborhood; otherwise, proceed to step D.

[0027] D. Use the exchange neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue the search within this neighborhood; otherwise, proceed to step E.

[0028] E. Measure the reward of each individual after the local search according to the reward function and determine the state of the individual, and update the Q table according to the Q-Learning formula;

[0029] F. Determine if Q-Learning has met the termination condition. If it has, stop VNS; otherwise, skip to step B until Q-Learning meets the termination condition.

[0030] This invention uses Q-Learning to guide VNS, enabling VNS to adaptively perform local reinforcement within certain factories. This maintains a balance between the diversity and convergence of the updated meme set and improves the quality of the solution, thus making the algorithm more efficient in solving DHFSP.

[0031] Furthermore, the reward function in step E is set according to the different meme groups, and the reward function for each meme group is as follows:

[0032] R = f1(x) - f1(VNS(x))

[0033] R = f2(x) - f2(VNS(x))

[0034] R=f1(x)-f1(VNS(x))+f2(x)–f2(VNS(x))

[0035] Where f1(x) represents the first objective of individual x, i.e., minimizing the maximum completion time; f1(VNS(x)) represents the first objective of the individual after the VNS operation; f2(x) represents the second objective of individual x, i.e., minimizing the maximum processing time; and f2(VNS(x)) represents the second objective of the individual after the VNS operation.

[0036] This invention designs different reward functions for different meme groups. By designing the corresponding reward functions, more accurate feedback can be given to the agent's behavior, making the local search for solutions more reasonable and efficient.

[0037] Furthermore, the formula for Q-Learning is:

[0038] Q(s,a)=Q(s,a)+α(R+γmax Q(s',a')-Q(s,a))

[0039] The parameter α is the learning rate; γ is the discount factor; s and s' represent the current state and the next state, respectively; a represents the action; and R represents the reward. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the solution of the multi-objective distributed hybrid flow shop scheduling method of the present invention;

[0041] Figure 2 This is an example of chromosome encoding and decoding used in this invention;

[0042] Figure 3 This is an example of the sequence swapping operation used in this invention;

[0043] Figure 4 This is an example of the first step of the PSO search process in this invention: the sequence swapping operation.

[0044] Figure 5 This is an example of the second step of the PSO search process in this invention: the sequence swapping operation.

[0045] Figure 6 This is an example of the third step of the PSO search process in this invention: the sequence swapping operation.

[0046] Figure 7 This is an example of the operation of inserting a neighboring region in the local reinforcement process between key factories according to the present invention;

[0047] Figure 8 This is an example of the operation of flipping the neighborhood in the local reinforcement process between key factories according to the present invention;

[0048] Figure 9 This is an example of the operation of exchanging neighborhoods in the local reinforcement process between key factories according to the present invention;

[0049] Figure 10 This is a schematic diagram of the Q-learning VNS model used in this invention;

[0050] Figure 11 This is a schematic diagram of the MAPSO-QLS process framework for solving DHFSP used in this invention.

[0051] Figure 12a This is a schematic diagram of the HV index in 100 operations, 5 stages, and 4 factory examples of the present invention;

[0052] Figure 12b This is a schematic diagram of the IGD index in 100 operations, 5 stages, and 4 factory examples of the present invention. Detailed Implementation

[0053] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0054] This invention establishes an objective function with the simultaneous minimization of maximum completion time and maximum processing time in distributed hybrid flow shop scheduling. It uses a multi-objective memetic algorithm based on particle swarm optimization and local search with Q-learning (MAPSO-QLS) for solution. First, the population is divided into three memetic groups using a Vector Evaluated Genetic Algorithm (VEGA) and a Pareto Dominating and Dominated Relationship-based Fitness Function (PDDR-FF). Then, PSO is used to perform a global search on each memetic group, exploring solutions in multiple directions of the Pareto front to accelerate convergence at the upper and lower edges and the central region of the Pareto front. Second, critical-factory inserts and critical-factory... The swap multi-neighborhood search operator performs local searches on individuals, enhancing the quality of solutions within the meme set. Furthermore, the Q-Learning variable neighborhood search (VNS) strategy further enhances the algorithm's search capability across the three meme sets, thereby improving the quality and diversity of solutions and preventing premature convergence that would prevent the algorithm from finding better solutions. The specific process and implementation framework of the MAPSO-QLS algorithm are as follows: Figure 1 and Figure 11 As shown, the following is a detailed explanation.

[0055] 1. Construct the objective function.

[0056] This invention establishes a corresponding objective function with the goal of simultaneously minimizing the maximum completion time and the maximum processing time of the distributed hybrid flow workshop scheduling, that is, establishing a dual objective function that minimizes the maximum completion time and the maximum processing time.

[0057] 2. Set the encoding / decoding method.

[0058] This invention employs an integer-based discrete sorting encoding method. The encoding consists of two parts: a chromosome vector representing the workpiece sequence, where each gene position on the chromosome represents a unique workpiece number, allowing the processing order of the workpieces in the solution to be obtained from the gene sequence; and a factory sequence, the same length as the workpiece sequence, where each gene position corresponds to a factory number, but can be repeated. During decoding, workpieces are assigned to different factories for processing according to the correspondence between workpiece and factory numbers. Two initialization encoding methods are used: a completely random encoding method, where both the workpiece and factory sequences use randomly generated integers (note that each factory must be assigned at least one workpiece); and a uniformly random encoding method, where the workpiece sequence uses randomly generated integers, while the factory sequence uses uniformly distributed integers (the number of workpieces within a factory is uniform).

[0059] For DHFSP, all workpieces need to be assigned to factories, and then the processing sequence of workpieces and the machine allocation for each stage are determined within each factory. In DHFSP, the encoded arrangement generally only represents the workpiece sequence of the first stage; the workpiece sequence of subsequent stages is affected by the completion time of workpieces in the previous stage. Therefore, this invention utilizes scheduling rules, namely the First-In-First-Out (FIFO) rule, the Earliest Finish Time (ETC) rule, and the First Available Machine (FAM) rule, to decode the solution into a complete schedule. Using the above encoding scheme, three vectors are used to represent the decoded solution. During decoding, both parts of the chromosome are traversed from left to right. First, all workpieces corresponding to factory 1 are selected without changing their positions; the remaining workpiece sequence is the operation sequence of workpieces within factory 1. Similarly, the operation sequence of workpieces within each factory is also determined. After determining the workpiece processing sequence within each factory, the processing sequence of the first stage is also determined. In other stages, the completion times of all workpieces in the previous stage are sorted in ascending order, and then the processing sequence of workpieces in the current stage is determined according to the ETC rule. When processing workpieces, the FAM rule is used to select machines for each stage. If some workpieces have the same completion time at the end of a stage, they are placed in the same relative order as in the previous stage (FIFO rule) when entering the next stage. This encoding and decoding method can solve the problems of workpiece allocation to the factory, machine allocation at each stage, and the processing order of workpieces on machines in DHFSP. Two-vector encoding and three-vector decoding schemes are as follows... Figure 2 As shown.

[0060] 3. Construct a population and divide it into three meme groups, and use PSO to perform a global search on each of the three meme groups.

[0061] The population was divided into three meme groups using a Vector Evaluated Genetic Algorithm (VEGA) and a Pareto Dominating and Dominated Relationship-based Fitness Function (PDDR-FF). Two selection strategies were required for dividing the population into three meme groups: PDDR-FF and VEGA. VEGA can find solutions that satisfy a single objective and form meme groups 1 and 2, while PDDR-FF can find solutions that satisfy two objectives simultaneously and form meme group 3.

[0062] PSO search uses crossover orders between individuals to update individuals in the meme group. The specific operation of crossover orders is as follows: Figure 3As shown, then according to the PSO formula, the individuals in the meme group are updated with exchange order, as follows: Figure 4 , Figure 5 and Figure 6 As shown. The use of the permutation order makes the PSO search more rational and efficient. Finally, PSO can search for solutions to meme sets in three specific directions, thereby accelerating the convergence speed of the algorithm and enhancing its overall convergence and distribution performance.

[0063] 4. Perform interactive updates on the three meme groups and implement localized reinforcements for key factories.

[0064] Because this invention uses two initialization methods, it is difficult to maintain a balance between the diversity and convergence of new solutions generated during individual updates. Furthermore, to prevent excessive time wastage due to large differences in the number of workpieces between critical factories, or to avoid situations where, although workpieces are evenly distributed, the processing time for most workpieces in the same factory is too long, leading to an increase in total processing time, this invention first uses two local operation operators targeting critical factories. The first local search operator is critical-factory insert: randomly selects a workpiece from factory fmax (the factory with the worst objective value) and inserts it into a random position in factory fmin (the factory with the best objective value). If the objective value of the extracted solution is improved, the above steps are executed until the termination condition is met. It should be noted that if two factories have the same objective value, the factory with more workpieces is selected. The second is critical-factory swap: randomly selects two workpieces from factories fmax (the factory with the worst objective value) and fmin (the factory with the best objective value), and then swaps these two workpieces. If the objective value of the swapped solution is improved, the above steps are executed until the termination condition is met. It is important to note that if two factories have the same objective value, one of them is randomly selected. The main purpose of applying the two operators is to balance the processing capacity of the factories. Therefore, these two operators are used as two neighborhood structures in the variable neighborhood descent method to perform a local search for the solution. The specific operations are as follows:

[0065] Step 1: Evaluate the population, interactively update the three meme groups, and then obtain the individuals in the meme groups.

[0066] Step 2: Perform a local search on the individual within the critical-factory insert neighborhood. If the updated individual is better than the target value of the individual before the update, continue searching within this neighborhood; otherwise, proceed to step 3. It is worth noting that individuals in meme group 1 are compared against the first target, individuals in meme group 2 are compared against the second target, and individuals in meme group 3 are compared against the PDDR-FF value.

[0067] Step 3: Perform a local search on the individual within the critical-factory swap neighborhood. If the updated individual has a better target value than the individual before the update, continue the search within this neighborhood; otherwise, proceed to step 4.

[0068] Step 4: Determine if the termination condition has been met. If it has, stop the search; otherwise, skip to step 2. The termination condition here refers to the number of searches for individuals in each meme group, i.e., the number of loops.

[0069] (The above steps apply to each individual in the meme group)

[0070] 5. Perform localized reinforcement on the internal facilities of the selected factory set.

[0071] Since the three meme groups will interact and update after performing a global PSO search and a local search between critical plants, efficient job scheduling within the plants cannot be guaranteed. Therefore, to maintain a balance between the diversity and convergence of the updated meme groups and improve the quality of the solution, this invention introduces a VNS with Q-Learning to locally adjust the processing order of jobs within the plants, thereby achieving better scheduling. Q-Learning plays a crucial role in the local search strategy of MAPSO-QLS. Although both objectives of the solution are optimized after a local search between critical plants, scheduling within the plants is not necessarily good. To better balance the algorithm's exploration and development capabilities, Q-Learning can be used to guide the VNS, enabling it to adaptively perform local reinforcement within certain plants, thus making the algorithm more efficient in solving DHFSP.

[0072] VNS is an improved local search algorithm. For corresponding workpieces within a factory, VNS uses the neighborhood structure formed by different workpieces to perform alternating searches, thereby adjusting the processing order of workpieces in different factories, which is more conducive to optimizing the multi-objective value of the solution. This invention designs three neighborhood operation operators—"insertion," "reversal," and "exchange"—for the neighborhood structure of VNS, and designs three components of VNS: "perturbation," "local search," and "neighborhood transformation." The specific search operations of the three neighborhood operators—"insertion," "reversal," and "exchange"—are as follows: Figure 7 , 8 As shown in Figures 9 and 10, the local search requires repeating these three parts until the termination condition is met, and finally, the optimal solution is output.

[0073] Q-Learning is an algorithm with strong self-learning capabilities. It learns the action-value function to find the optimal value of an action, takes a specific action in a specific state, follows the best policy, and ultimately obtains the desired reward. Its goal is to find the policy that maximizes cumulative reward. When constructing Q-Learning, the instantaneous reward needs to be defined first to guide the agent in choosing an action, thereby obtaining the Q-matrix. The Q-value is updated as follows:

[0074] Q(s,a)=Q(s,a)+α(R+γmax Q(s',a')-Q(s,a))

[0075] Here, parameter α is the learning rate, with a value of 1, which determines the degree of Q-value iterative updates; γ is the discount factor, which determines the importance of future rewards, ranging from 0 to 1. s and s' represent the current state and the next state, a represents the action, and R represents the reward function. This invention stores Q-values ​​in a Q-table and updates the values ​​in the Q-table iteratively. The agent makes decisions by selecting a strategy based on ε-greedy behavior. The symbol ε represents the probability used to control the degree of greed. In this invention, the agent's decisions are fixed; the Q-Learning model will be described in detail below.

[0076] Since the goal of DHFSP is to find the minimum completion time and the minimum total plant processing time, Q-Learning requires constructing the environment state based on the population's fitness function values.

[0077] This invention sets two states based on the feedback results of VNS: (1) the original solution is better than the solution after VNS; (2) the solution after VNS is better than the original solution; and judges the quality of the solution based on the classification criteria of the meme group to which the individual belongs. For example: meme group 1 is the first target of the individual being compared, meme group 2 is the second target of the individual being compared, and meme group 3 is the PDDR-FF value of the individual being compared.

[0078] Since the scheduling of workpieces within each factory may not be optimal after a local search among key factories, a better target value can be obtained by reordering the workpieces within the factories. To ensure that the updated meme set retains or improves excellent solutions, a more efficient local search of the population is needed without reducing the current scheduling efficiency. Therefore, this local search only requires strengthening the order of workpieces within the factories to achieve better scheduling. Benming designed a unique action set. Assuming there are 3 factories, and the factory number set is represented by {1,2,3}, then these 3 factories have 7 combined subsets: {1,2,3}, {1,2}, {1,3}, {2,3}, {1}, {2}, and {3}. For each solution, the agent uses these factory sets as action sets to select a factory set and then performs a VNS local search on the workpieces within the selected factory. This action set not only allows for more efficient local search of solutions but also avoids losing currently excellent solutions.

[0079] In Q-Learning, the reward function is designed based on the target value of the solution. This invention designs different reward functions for different meme groups. By designing appropriate reward functions, more accurate feedback can be provided to the agent's actions, making the local search for solutions more reasonable and efficient. The reward functions are as follows:

[0080] R = f1(x) - f1(VNS(x))

[0081] R = f2(x) - f2(VNS(x))

[0082] R=f1(x)-f1(VNS(x))+f2(x)–f2(VNS(x))

[0083] Based on the above work, the Q-Learning model is as follows: Figure 10 As shown, the specific steps for solving the DHFSP using Q-Learning VNS are as follows:

[0084] Step 1: Define three neighborhoods: insertion, inversion, and swapping. For each individual in each meme group, initialize the Q-table and state decision for the individual, and then select the factory set to be locally reinforced from the action set.

[0085] Step 2: For the selected factory set, use the insertion neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue searching within this neighborhood; otherwise, proceed to Step 3.

[0086] Step 3: Use the inverted neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual is better than the target value of the individual before the update, continue the search within this neighborhood; otherwise, proceed to step 4.

[0087] Step 4: Use the exchange neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue searching within this neighborhood; otherwise, proceed to Step 5.

[0088] Step 5: Measure the reward and determine the individual's status, then update the Q table according to the Q-Learning formula.

[0089] Step 6: Determine if Q-Learning has reached the termination condition. If it has, stop VNS; otherwise, skip to step 2. The termination condition here refers to the number of iterations of Q-Learning.

[0090] The VNS using Q-Learning can not only locally enhance the scheduling within the plant, but also improve the quality of the solution and increase the diversity of the population.

[0091] 6. Select individuals with better PDDR-FF values ​​from the meme groups after PSO search and the locally enhanced meme groups, and perform the next evolution on the selected individuals until the termination condition is met. The termination condition here means that the original population number of new individuals is selected, and the termination condition is reached.

[0092] Therefore, this invention divides the population into three meme groups. First, it uses PSO to perform a global search on each meme group, exploring solutions in multiple directions of the Pareto front, thus accelerating the convergence speed of the upper and lower edges and the central region of the Pareto front. Second, it uses critical-factory insert and critical-factoryswap multi-neighborhood search operators for critical factories to perform local searches on individuals, enhancing the quality of solutions in the meme groups. Finally, it uses the Q-Learning variable neighborhood search (VNS) strategy to further enhance the algorithm's search capability on the three meme groups, thereby improving the quality and diversity of solutions, preventing the algorithm from converging too early and failing to find better solutions, and enabling the algorithm to perform update operations on individuals more conveniently and reasonably, thus improving the overall performance of the algorithm.

[0093] To verify the aforementioned effects of this invention and to test the effectiveness of inter-factory local search and intra-factory local search, as well as the effectiveness of Q-Learning in the local search strategy, these three parts of the experiment were tested under 36 different distributed hybrid flow shop scheduling problems. The convergence and distribution performance of the algorithm were verified using the comprehensive evaluation metrics HV and IGD. Table 1 compares the experimental results of the inter-factory local search algorithm (MAPSO) with those of the inter-factory local search algorithm (MAPSO-LS1). The results in Table 1 show that the inter-factory local search strategy achieves better results on all problems, and the significance analysis results indicate that the inter-factory local search strategy has significant convergence and distribution performance. Table 2 compares the experimental results of MAPSO-LS1 with the inter-factory local search algorithm (MAPSO-LS2). The results in Table 2 show that the intra-factory local search algorithm has significantly better performance on all problems, fully demonstrating the effectiveness of the intra-factory local search. Table 3 shows the comparative experimental results between MAPSO-LS2 and MAPSO-QLS. As can be seen from the results in Table 3, except for a few specific problems, the results of using Q-Learning-based local search are significantly better on other problems. This indicates that using Q-Learning allows the solution to be updated more effectively, improving the algorithm's local search capability and convergence performance.

[0094] Table 1

[0095]

[0096] Table 2

[0097]

[0098] Table 3

[0099]

[0100] The comparative experiments of this invention with other optimization algorithms use a standard distributed hybrid flow shop scheduling problem dataset to demonstrate the performance of the MAPSO-QLS algorithm of this invention. The evaluation metrics used in the experiments are HV, IGD, and Spread to evaluate the algorithm's performance. Algorithms used for comparative experiments include MOPSODS-DE, MOHEA-DE, NSGA-II, SPEA2, and MOEA-LS. Each algorithm is executed 30 times, and the solution is recorded as a Pareto approximation each time.

[0101] This validation experiment (DOE) focused on the parameters of the PSO velocity and position update formulas and the Q-learning formula, as well as the number of iterations for local search in MAPSO-QLS. The parameters of MAPSO-QLS were designed as follows: PSO parameters r1 and r2 at four levels (0.2, 0.4, 0.6, and 0.8); the number of iterations for local search at three levels (10, 20, and 30); the number of iterations for Q-Learning at three levels (30, 40, and 50); the discount factor γ at three levels (0.7, 0.8, and 0.9); and the greed rate ε at three levels (0.7, 0.8, and 0.9). These five parameters have 324 (4x3x3x3x3) different combinations. The DOE was run 30 times on a 100_5_4 instance (100 jobs, 5 stages, and 4 factories). Therefore, a full factorial experimental design with a 9720 (324x30) treatment was used to determine the parameter settings. Specifically, the size of the three meme groups was set to one-third of the total population size. The main effects of each parameter are as follows: Figure 12a and Figure 12b As shown, the optimal parameters are set to r1, r2 = 0.4, the number of local search iterations is 30, the number of Q-Learning iterations is 50, the discount factor γ = 0.8, and the greed rate ε = 0.9. To ensure a fair comparison with other algorithms, this invention also performs a DOE (Design of Experiments) on the parameters of other comparison algorithms. The optimal parameter combinations for each algorithm are shown in Table 4.

[0102] Table 4

[0103]

[0104] Experimental results show that MAPSO-QLS performs exceptionally well on the HV index in Table 5. Among the 36 problems, MAPSO-QLS has the highest HV value, with a significant difference. Therefore, it can be seen that MAPSO-QLS has the most outstanding convergence performance. The IGD index results in Table 6 show that MAPSO-QLS performs best on all 36 problems. Therefore, it can be concluded that among all compared algorithms, the MAPSO-QLS of this invention has the best convergence and distribution performance, and the algorithm's performance is highly significant. Furthermore, the Spread index results in Table 7 show that MAPSO-QLS performs best on 32 of the 36 problems. While there is no significant difference between MAPSO-QLS and other algorithms on a few individual problems, MAPSO-QLS's distribution performance remains outstanding on the remaining problems. In summary, the MAPSO-QLS of this invention has good convergence performance, and MAPSO-QLS can also make the distribution of solutions obtained by the algorithm more diverse and extensive.

[0105] Table 5

[0106]

[0107] Table 6

[0108]

[0109] Table 7

[0110]

[0111] In summary, the results of the three metrics—HV, IGD, and Spread—show significant differences between the MAPSO-QLS proposed in this invention and other algorithms, with MAPSO-QLS achieving the best metric values. It can be said that using MA as a framework and employing two initialization methods—PSO based on exchange sequences as a global search strategy—can improve the convergence speed of the population and the convergence ability of the algorithm; while using Q-learning local search as a local reinforcement strategy and interactively updating the meme set can further improve the quality of the solutions and enhance the convergence and distribution of the algorithm.

Claims

1. A multi-objective distributed hybrid flow shop scheduling method, characterized in that, The scheduling method includes the following steps: 1) Construct an objective function that simultaneously minimizes the maximum completion time and maximum processing time of the distributed hybrid flow shop scheduling; 2) The workpiece and factory are encoded using an integer-based discrete sorting encoding method, and the population is initialized according to the encoding results; 3) Divide the population into three meme groups and perform a global PSO search on each meme group; 4) Perform interactive updates on the three meme groups and perform local reinforcement on key factories, where the key factories refer to the factories with the worst and best target values. 5) Local reinforcement within the selected factory set: Q-Learning is used to guide the VNS so that the VNS can adaptively perform local reinforcement within some specific factories. 6) Select the better individuals from the meme group after PSO search and the meme group after local reinforcement, and carry out the next population evolution for the selected individuals until the termination condition is met. The best individual obtained at the end is taken as the solution of the objective function.

2. The multi-objective distributed hybrid flow shop scheduling method according to claim 1, characterized in that, Step 3) uses a vector-based genetic algorithm and a fitness function method based on Pareto dominance and non-dominance relationships to divide the population into three meme groups. The three meme groups are the solution that satisfies the objective of minimum maximum completion time, the solution that minimizes maximum processing time, and the solution that minimizes both maximum completion time and maximum processing time, i.e., the solution with the minimum fitness function value. The two meme groups corresponding to the solution that satisfies the objective of minimum maximum completion time and the solution that minimizes maximum processing time are found using VEGA, and the meme group corresponding to the solution that minimizes both maximum completion time and maximum processing time is found using PDDR-FF.

3. The multi-objective distributed hybrid flow shop scheduling method according to claim 1, characterized in that, The local reinforcement in step 4) is implemented using two local search operators. The first local search operator is critical-factoryinsert: randomly select a workpiece from the factory with the worst target value and insert it into a random position in the factory with the best target value. The second local search operator is critical-factory swap: randomly select two workpieces from the factory with the worst target value and the factory with the best target value, and swap these two workpieces.

4. The multi-objective distributed hybrid flow shop scheduling method according to claim 3, characterized in that, The local strengthening process in step 4) is as follows: a. Evaluate the population, interactively update the three meme groups, and obtain individuals from the meme groups; b. Perform a local search on the individual within the critical-factory insert neighborhood. If the updated individual has a better target value than the individual before the update, continue searching within this neighborhood; otherwise, proceed to step c. c. Perform a local search on the individual in the critical-factory swap neighborhood. If the updated individual has a better target value than the individual before the update, continue the search in this neighborhood; otherwise, proceed to step d. d. Determine if the termination condition has been met. If it has, stop the search; otherwise, return to step b.

5. The multi-objective distributed hybrid flow shop scheduling method according to claim 1, characterized in that, Step 5) uses Q-Learning and VNS for local search.

6. The multi-objective distributed hybrid flow shop scheduling method according to claim 5, characterized in that, The local reinforcement process in step 5) is as follows: A. Set the initial Q value and state of each individual in each meme group, and select the factory set to be locally enhanced from the action set; B. For the selected factory set, use the insertion neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue the search within this neighborhood; otherwise, proceed to step C. C. Use the inverted neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue the search within this neighborhood; otherwise, proceed to step D. D. Use the exchange neighborhood to perform a local search on the sequence of workpieces within a specific factory. If the updated individual has a better target value than the individual before the update, continue the search within this neighborhood; otherwise, proceed to step E. E. Measure the reward of each individual after the local search according to the reward function and determine the state of the individual, and update the Q table according to the Q-Learning formula; F. Determine if Q-Learning has met the termination condition. If it has, stop VNS; otherwise, jump to step B until Q-Learning meets the termination condition.

7. The multi-objective distributed hybrid flow shop scheduling method according to claim 6, characterized in that, The reward function in step E is set according to the different meme groups, and the reward function for each meme group is as follows: R = f1(x) - f1(VNS(x)) R = f2(x) - f2(VNS(x)) R=f1(x)-f1(VNS(x))+f2(x)-f2(VNS(x)) Where f1(x) represents the first objective of individual x, i.e., minimizing the maximum completion time; f1(VNS(x)) represents the first objective of the individual after the VNS operation; f2(x) represents the second objective of individual x, i.e., minimizing the maximum processing time; and f2(VNS(x)) represents the second objective of the individual after the VNS operation.

8. The multi-objective distributed hybrid flow shop scheduling method according to claim 6, characterized in that, The formula for Q-Learning is as follows: Q(s,a)=Q(s,a)+α(R+γmax Q(s',a')-Q(s,a)) The parameter α is the learning rate; γ is the discount factor; s and s' represent the current state and the next state, respectively; a represents the action; and R represents the reward.

Images