Crossing variation rate adaptive NSGA-II multi-target flexible job shop scheduling method

Through the cross-variability adaptive NSGA-II method combined with Q-learning and entropy value method, multi-objective flexible operation workshop scheduling is optimized, the problem of fixed cross-variability and variability is solved, and the maximum in flexible operation workshop scheduling is achieved Effective optimization of completion time and total machine energy consumption improves the quality and efficiency of the scheduling plan.

CN118963276APending Publication Date: 2024-11-15ZHENGZHOU UNIVERSITY OF AERONAUTICS
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411030016.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing multi-objective flexible operation workshop scheduling method is difficult to effectively optimize the maximum completion time and total machine energy consumption, and the problem of fixed cross-rate and variability rates in the intelligent optimization algorithm has not been completely solved.

Method used

A cross-variability adaptive NSGA-Ⅱ multi-objective flexible operation workshop scheduling method is proposed. Combined with the Q-learning algorithm, the cross-variability and variability adaptive strategies are constructed, and the cross-variability and mutation operations are optimized, and the target is weighted by entropy value method is used to process the target , introduce taboo search strategies to improve the diversity and uniformity of solution sets.

Benefits of technology

The adaptability of cross-mutation parameters in multi-objective flexible operation workshop scheduling is achieved, the convergence speed of the algorithm and the diversity of solution sets are improved, local optimal solutions are avoided, and the quality and efficiency of the scheduling scheme are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118963276A_ABST
    Figure CN118963276A_ABST
Patent Text Reader

Abstract

The invention discloses a crossover variation rate adaptive NSGA-II multi-target flexible job shop scheduling method, and the method comprises the steps: building a multi-target flexible job shop scheduling model with the minimum maximum completion time and the minimum total energy consumption of a machine as optimization targets; the cross-over rate and variation rate self-adaptive strategy based on Q-learning is combined with NSGA-II population cross-over and variation links, the defect that the cross-over rate and variation rate in NSGA-II are fixed is overcome, finally, an optimal solution is gradually obtained by solving an objective function of a multi-objective flexible job shop scheduling model, an initial solution is generated by using three initialization strategies, and the optimal solution is finally obtained. Sorting and selecting the initial population based on an entropy method; secondly, initializing a learning process of Q-learning, and randomly selecting a crossover rate and a mutation rate within a certain range; next, a new population is obtained after crossover mutation operation, the state and reward of the new population are calculated, and a Q table is updated by using a Q table updating formula in Q-learning; and finally, obtaining a non-dominated solution by using a local search algorithm with a tabu search strategy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of flexible job shop scheduling, and in particular to a crossover mutation rate adaptive NSGA-II multi-objective flexible job shop scheduling method. Background Art

[0002] In recent years, new manufacturing systems need to develop towards flexibility and intelligence, but there is still a lot of room for improvement in how to quickly and accurately find the optimal scheduling solution while ensuring flexibility. The flexible job shop scheduling studied in this application refers to the process of balancing the work to be processed on each machine in the workshop production process, by reasonably commanding, controlling and adjusting effective resources with production as the center, so as to minimize the completion time and thus achieve the purpose of improving economic benefits.

[0003] The methods for solving FJSP can be divided into three categories: exact algorithms, heuristic algorithms, and intelligent optimization algorithms. The exact algorithm has a large time complexity and is not suitable for solving large-scale problems. The heuristic algorithm has a fast solution speed, but the quality of the solution is often not good. For intelligent optimization algorithms, although scholars have applied intelligent algorithms such as genetic algorithms, ant colony algorithms, particle swarm algorithms, simulated annealing algorithms, etc. to solve this problem in recent years and have achieved certain results, a completely good solution has not yet been obtained, so this problem still has a lot of room for research.

[0004] Therefore, this application proposes a crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling method for the multi-objective flexible job shop scheduling problem (MOFJSP), and simultaneously optimizes the two key objectives of maximum completion time and total machine energy consumption. Summary of the invention

[0005] The present application provides a cross-mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling method, which can reasonably command, control and adjust effective resources so that the jobs to be processed can be evenly arranged on each machine to minimize the completion time, thereby achieving the process of improving economic benefits.

[0006] The crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling method is characterized by comprising the following steps:

[0007] S1: Aiming at the multi-objective flexible job shop scheduling problem, a multi-objective flexible job shop scheduling model is established with the minimization of the maximum completion time and the minimum total energy consumption of the machine as the optimization objectives;

[0008] S2: Construct a crossover rate and mutation rate adaptive strategy based on Q-learning, including: creating two Q tables for crossover and mutation, defining the Q table update formula; calculating the fitness value with the objective function value, and then using the size of the fitness value as the state space; for each generation, the agent can take different actions to obtain the appropriate crossover rate (Pc) and mutation rate (Pm), and Pc and Pm respectively select probability values ​​of different intervals as the action space; design the reward and punishment function based on the overall average fitness and population diversity; adopt the improved ε-greedy strategy as the action selection strategy; realize the adaptive strategy of crossover rate and mutation rate;

[0009] S3: The Q-learning-based crossover rate and mutation rate adaptive strategies in S2 are combined with the NSGA-Ⅱ population crossover and mutation links to make up for the defects of the fixed crossover rate (Pc) and mutation rate (Pm) in NSGA-Ⅱ, and finally the optimal solution is gradually obtained by solving the objective function of the multi-objective flexible job shop scheduling model in S1, including: first, using three initialization strategies to generate the initial solution, and then sorting and selecting the initial population based on the entropy method; then, initializing the Q-learning learning process, Pc and Pm are randomly selected within a certain range; next, after the crossover and mutation operations, a new population is obtained, their states and rewards are calculated, and the Q-table update formula in Q-learning is used to update the Q table; finally, a local search algorithm with a taboo search strategy is used to obtain a non-dominated solution.

[0010] In summary, this application has the following beneficial effects:

[0011] This application studies the crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling problem, and proposes a QNSGA-Ⅱ algorithm with the optimization objectives of minimizing the maximum completion time and the total energy consumption of the machine. For the problem model, three population initialization rules are designed, and multiple objectives are weighted using the entropy method so as to improve the diversity of the population by using the roulette method. In order to make the crossover mutation parameters adaptive to the evolutionary state of the current population, the Q-learning algorithm in reinforcement learning is integrated with NSGA-Ⅱ, which makes up for the defects of the fixed crossover rate (Pc) and mutation rate (Pm) in NSGA-Ⅱ, so that the population is evenly distributed and diverse.

[0012] By adding a local search strategy to the improved taboo search algorithm, a taboo table is used during the search process to avoid falling into the local optimal solution, so that it can jump out of the local optimal solution and continue to explore the solution space, thereby achieving local and global search capabilities while retaining the diversity and uniformity of its solution set.

[0013] In the action selection strategy, the classic ε-greedy strategy is extended and an elite preservation strategy is introduced, so that the adaptive parameters can better select more unknown actions in the early stage of iteration, converge better in the later stage of iteration, and preserve the better individuals in each generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the algorithm flow chart of this application;

[0015] Figure 2 This is the 3×4 scheduling Gantt chart of this application;

[0016] Figure 3 It is a schematic diagram of the production flow chart of the workshop in this application;

[0017] Figure 4 It is a schematic diagram of the two-stage coding of this application;

[0018] Figure 5 It is a schematic diagram of plug-in decoding of this application;

[0019] Figure 6 Schematic diagram of crossover and mutation operators for this application;

[0020] Figure 7 This is a schematic diagram of the reinforcement learning model framework for this application;

[0021] Figure 8 This is a schematic diagram of the Q table update for this application;

[0022] Fig. 9 This is a schematic diagram of the main effects of the mean for this application;

[0023] Fig.10 This is a box-line diagram of IGD values ​​for different algorithms in this application;

[0024] Fig.11 This is a box line diagram of HV values ​​for different algorithms in this application;

[0025] Fig.12 This is a box diagram of SC values ​​for different algorithms in this application;

[0026] Fig.13 Schematic diagram of the non-dominated frontier of the three algorithms in this application;

[0027] Fig.14 This is a schematic diagram of the convergence curve of the MK02 example in this application;

[0028] Fig.15 This is the Gantt chart for the MK02 example in this application. DETAILED DESCRIPTION

[0029] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0030] 1 Problem description and mathematical model establishment

[0031] 1.1 Description based on MOFJSP

[0032] Multi-objective flexible job shop scheduling refers to the problem of scheduling jobs in a production workshop by considering multiple objectives or multiple constraints at the same time. These objectives involve aspects such as completion time, cost, and delivery time. An n×m flexible job shop scheduling can be represented as n independent workpieces J = {J1; J2; ..., J n} and m independently processable machines M = {M1,M2,…,M m}, each workpiece J i Contains several processes O ij , O ij is the jth process of the i-th workpiece (j = 1, 2, 3, ..., h). The scheduling of this problem includes two sub-problems: machine selection and process sequencing. Machine selection is to select the appropriate machine for each process, and process sequencing is the processing order of the processes on each machine.

[0033] Figure 2 It is a workshop production flow chart, which includes different workpieces, different machines, and the workpieces that can be processed by each machine are not exactly the same. It can intuitively show how the flexible job shop scheduling works.

[0034] The assumptions for the MOFJSP problem are as follows: a) At time zero, all machines and workpieces are in a ready state; b) A machine can only process one workpiece at a time; c) Each process can only be processed by one of its candidate machines, and the next process cannot be started before all previous processes are processed; d) The processing of a process cannot be stopped or paused before completion; e) The movement time of all workpieces is negligible; f) The processes of the same workpiece have priority constraints, and the workpieces are independent of each other; g) The transportation time and clamping time of the workpiece between the buffer and the machine are ignored; h) The processing cannot be interrupted and there is no rework.

[0035] 1.2 Mathematical model

[0036] The symbols and definitions of the variables related to this application are shown in Table 1.

[0037] Table 1 Variable description

[0038]

[0039]

[0040] The decision variable is X ijk , used to indicate whether the jth process of workpiece i is processed on machine k

[0041]

[0042] in:

[0043] The optimization goal is to minimize the maximum completion time and the total energy consumption of the machine. The objective function and constraints are as follows:

[0044] (1) Minimize the maximum completion time:

[0045] Completion time is the time point when the production workshop completes the production task and delivers the product to the next process within a certain time range according to the production plan and resource conditions. The completion time of the last process needs to be minimized to improve production efficiency. Its objective function is shown in (3):

[0046] f1=min{max{C i}}, i = 1, 2, ..., n (3)

[0047] (2) Minimize the total energy consumption of the machine:

[0048] The machine consumes energy during the processing, and the scheduling arrangement with the lowest total energy consumption is required. Its objective function is shown in (4):

[0049]

[0050] (3) Constraints:

[0051]

[0052] C ij -C i(j-1) ≥P ijk X ijk (6)

[0053] S ijk ≥S epk +P epk (7)

[0054] Expression (5) limits the process to only one processing; Expression (6) indicates that the processing sequence of the same workpiece must be carried out according to the process sequence; Expression (7) requires that the machine can only process one workpiece at a time;

[0055] In order to facilitate the understanding of the multi-objective flexible job shop scheduling problem (MOFJSP), Table 2 provides an example of 3 workpieces and 4 machines. In this example, workpieces J1, J2, and J3 have 2, 3, and 2 processes respectively. Each process has optional processing machines. When selecting these processing machines, the processing time and machine energy consumption are different. Figure 3 It is a Gantt chart based on the 3×4 scheduling example in Table 2.

[0056] Table 2 3×4 scheduling example

[0057]

[0058] 2 Solving MOFJSP based on QNSGA-Ⅱ

[0059] 2.1 QNSGA-Ⅱ Algorithm Framework

[0060] This section introduces the algorithm framework of QNSGA-Ⅱ. The adaptive ability of the intelligent agent can effectively solve the scheduling problems in complex and changeable production environments.

[0061] This algorithm integrates the Q-learning algorithm in reinforcement learning with NSGA-Ⅱ to make up for the defects of the fixed crossover rate (Pc) and mutation rate (Pm) in NSGA-Ⅱ.

[0062] The specific description is as follows:

[0063] First, three initialization strategies are used to generate the initial population and initialize the learning process of Q-learning, namely the state space, action space and Q table.

[0064] Secondly, the target fitness value is calculated based on the initial population, and each target fitness value is weighted. The roulette wheel selection method is used to select the initial population to perform the crossover and mutation operation. After a local search of the population after crossover and mutation, the population is selected using NSGA-Ⅱ.

[0065] Then, based on the initialized Q-learning process, the fitness after the crossover and mutation operation, and the current state calculated by the population selected by NSGA-Ⅱ, the reward and punishment function is used to calculate the reward and punishment values ​​and update the Q table.

[0066] Finally, the ε-descent strategy is used to select the next action to be taken and adjust the crossover mutation rate.

[0067] The algorithm flow chart is shown in Figure 1, where gen represents the current iteration number.

[0068] 2.2 Encoding and Decoding

[0069] (1) Two-stage coding

[0070] This application adopts a two-stage coding method, namely the machine selection part and the process sorting part.

[0071] For the machine selection part, first find out all the machines that can process each process of each workpiece and the number of processes corresponding to each workpiece, and calculate the number of machines. Each number in the machine represents how many machines can process the corresponding process; for the process sorting part, the workpiece number corresponding to each workpiece depends on the sorting of the workpiece, and the number of processes corresponding to the workpiece is the size of the workpiece gene segment.

[0072] like Figure 4 As shown in the figure, the first half of this figure is the machine selection part, and the second half is the process sorting part. There are 7 processes in total. The first row in the left dotted box indicates that there are 3 processing machines for the first process of workpiece 1, and the third machine is selected for processing; the fourth row indicates that there are 3 processing machines for the second process of workpiece 2, and the second machine is selected for processing, and so on. The first 1 in the right dotted box is the first process of workpiece 1, the second 1 is the second process of workpiece 1, and so on.

[0073] (2) Plug-in decoding

[0074] The decoding process is to convert the scheduling plan formed in the encoding process into a time list that can represent each process on each machine. Each chromosome corresponds to a scheduling plan. This application uses plug-in decoding to decode the scheduling plan. The specific operation is as follows:

[0075] Step 1: Take out the first half of the gene, that is, the machine selection part, and establish the machine usage matrix (MU), processing time matrix (PJT), and machine energy consumption matrix (PC); take out the second half of the gene, that is, the process sorting part, in order to better distinguish the process corresponding to the workpiece, use the workpiece number * 100 + process number method to generate the scheduling process.

[0076] Step 2: For the decoding of the first process, after finding the machine and processing time of the corresponding process from MU and PJT, take out the machine start time (TMval) and workpiece start time (TPval), then find out the time when the workpiece can start processing and the processing end time, and save them in Pval, and finally record the machine's gap start time (TSval) and gap end time (TEval) for subsequent insertion.

[0077] Step 3: If it is not the first process, after finding the time when the workpiece can start processing in step 2, first identify the size relationship between TSval, TPval and TEval to determine whether the insertion can be successful. If the gap time cannot be inserted, perform normal sorting. Figure 5 As shown, O a1Indicates the operation to be inserted. The left figure shows the case where TEval' - TSval' ≥ PJT' and insertion can be performed. The right figure shows the case where TEval' - TSval' < PJT'. The gap interval time is less than the processing time and insertion cannot be performed.

[0078] Step 4: If the insertion condition is satisfied, insert the current operation and re - sort the operation part.

[0079] Step 5: Repeat Step 3 and Step 4 until all operations are traversed.

[0080] Step 6: Find the maximum value in Pval, which is the makespan of this scheduling scheme.

[0081] The pseudo - code of the algorithm is as follows.

[0082]

[0083]

[0084] 2.3 Population Initialization

[0085] Population initialization is very important for the algorithm. It directly affects the diversity and quality of the solutions. The diversity of the solutions means that there is a larger search space for the solutions, and the quality of the solutions indicates that the initial solutions are closer to the optimal solutions.

[0086] Therefore, this application designs three initialization rules to ensure the convergence speed of the fitness value and the quality of the initial population. These initialization strategies are: (1) Completely randomly generate the initial solutions; (2) Generate the initial solutions according to the minimum time; (3) Generate the initial solutions according to processing the largest remaining time first. In the initial population, to ensure greater randomness of the initial solutions, the proportions of the three initialization strategies are 50%, 25%, and 25% respectively.

[0087] (1) Completely randomly generate the initial solutions. The purpose of this initialization strategy is to expand the diversity of the population. For both the gene segments of machine selection and operation sequencing, they are randomly generated within a certain range.

[0088] (2) Generate the initial solutions according to the minimum time. The purpose of this strategy is to effectively shorten the makespan. For the machine selection part, select the machine with the minimum time, and for the operation sequencing part, it is randomly generated within a certain range.

[0089] (3) Generate the initial solutions according to processing the largest remaining time first. The purpose of this initialization strategy is to improve the quality of the solutions by balancing the processing times of the operations in the workpiece. For the gene segment of machine selection, it is randomly generated within a certain range. For the operation sequencing part, select the operation corresponding to the workpiece with the longest largest remaining time to be processed first.

[0090] 2.4 Sorting and selection based on entropy method

[0091] Sorting is to perform a fast non-dominated sorting of the initial solution, and the selection operation is to generate a new population with a higher fitness value from the current population. First, the individual target value is converted into a fitness value using a ranking-based fitness allocation function. The selection operation uses the roulette selection method. Since the roulette selection method is usually applied to single-objective optimization methods, the entropy method is used to weight two different target values, namely, minimizing the maximum completion time and machine energy consumption, and combine them into a comprehensive target value to support the subsequent roulette selection method. The specific process is as follows:

[0092] (1) Assuming that n samples and m indicators are analyzed, then x ij is the value of the jth indicator of the ith sample

[0093] (2) Normalization of indicators: Homogenization of heterogeneous indicators

[0094] Since the measurement units of various indicators are not uniform, they must be standardized before they are used to calculate the comprehensive indicators, that is, the absolute values ​​of the indicators are converted into relative values, so as to solve the homogeneity problem of the values ​​of various indicators of different qualities. The values ​​of positive indicators and negative indicators have different meanings (the higher the value of positive indicators, the better, and the lower the value of negative indicators, the better), so different methods need to be used for data standardization for positive and negative indicators:

[0095] Positive indicators:

[0096]

[0097] Negative indicators:

[0098]

[0099] For convenience, the normalized data is still x ij .

[0100] (3) Calculate the proportion of the i-th sample value under the j-th indicator to the indicator:

[0101]

[0102] (4) Calculate the entropy value of the jth indicator:

[0103]

[0104] Among them, k=1 / ln(n)>0, satisfying e j ≥0

[0105] (5) Calculate information entropy redundancy (difference)

[0106] d j =1-e j j=1,…,m (12)

[0107] (6) Calculate the weight of each indicator

[0108]

[0109] (7) Calculate the comprehensive score of each sample

[0110]

[0111] This application adopts the elite retention strategy, which is the process of retaining the best individuals and excluding bad individuals from the population in each iteration. It can make individuals evolve continuously in the right direction and accelerate the convergence speed.

[0112] 2.5 Crossover and Mutation

[0113] In traditional genetic algorithms, crossover and mutation are two core links. Crossover is used to increase the diversity of the population and discover new solutions, while mutation is used to increase the randomness of the optimization process. In order to better increase the diversity of the population, this application directly uses the population after crossover as the initial population for mutation, rather than the traditional direct merging of the population, as follows:

[0114] For the crossover method, it is divided into a machine selection part and a process sorting part. The machine selection part adopts a multi-point crossover method, and the process sorting part adopts a workpiece exchange method, such as Figure 6 shown.

[0115] For the variation method, it is also divided into two parts: machine selection and process sorting. The machine selection part randomly selects available machines, and the process sorting part randomly exchanges the positions of two processes, such as Figure 6 Shown

[0116] 2.6 Local Search Strategy Based on Improved Taboo Search Algorithm

[0117] The taboo search algorithm is a meta-heuristic search algorithm used to solve combinatorial optimization problems. It avoids falling into the local optimal solution by using the taboo table in the search process, so that it can jump out of the local optimal solution and continue to explore the solution space.

[0118] In FJSP, the critical path is the scheduling path that has the greatest impact on the completion time. It is crucial to identify and optimize the critical path in scheduling.

[0119] Therefore, this application designs a neighborhood structure based on critical path moving operations to effectively reduce the minimized maximum completion time while combining tabu search to avoid falling into local optimality.

[0120] The critical path has the longest total duration in the scheduling plan and has the greatest impact on the completion time of the entire scheduling plan. There may be multiple critical paths. The first step is to find the critical process. The specific process is shown in Algorithm 2.

[0121]

[0122]

[0123] The second step is to transfer the key process to other machines while ensuring the constraints of the previous and next processes. The specific process is shown in Algorithm 3.

[0124]

[0125] The third step is to establish a taboo table, use Algorithm 3 to perform local search on individuals of a single target and record relevant information into the taboo table. The specific process is shown in Algorithm 4.

[0126]

[0127]

[0128] 3 Crossover and mutation adaptive strategies based on Q-learning

[0129] 3.1 Introduction to Q-learning

[0130] In recent years, the research of reinforcement learning has received more and more attention. Given a state (s) of an environment, the agent selects a corresponding action (a) according to a certain strategy (p), and after executing the action (a), the environment will change, that is, the state (s) will be converted to a new state (s') and after each action (a) is executed, the agent will receive an incentive value (positive or negative), and the agent will adjust its strategy according to the size of the incentive value, so that after all steps are executed, the sum of the rewards (r) is maximized. The model framework of reinforcement learning is as follows: Figure 7 shown.

[0131] The Q-learning algorithm is a type of reinforcement learning and a strategy selection method. It uses the action value function Q(s,a) to estimate the expected return of taking action a in a given state s. The core and training goal is to select a suitable strategy so that the sum of rewards obtained at the end of each cycle is maximized. Q-learning continuously updates the Q table in iterations under limited combinations of states and actions so that it can eventually converge, that is, the agent can determine the action it wants to take by looking up the table based on the current state. The Q table update formula is shown in formula (15).

[0132]

[0133] 3.2 State Space

[0134] Before constructing the state space, MOFJSP, which aims to minimize the maximum completion time and the total energy consumption of the machine, needs to calculate the fitness value based on the objective function value and use the size of the fitness value as the state space. The following aspects are mainly considered for the fitness value function:

[0135] (1) Average fitness, as shown in formula (16)

[0136] (2) Population diversity, as shown in formula (17)

[0137] (3) The best individual fitness, as shown in formula (18)

[0138] Formula (19) is the weighted sum of formulas (16)(17)(18)

[0139]

[0140]

[0141]

[0142]

[0143] represents the target value of the ith individual in the tth generation, represents the optimal target value of the i-th individual in the t-th generation, A(t) is the average fitness of the t-th generation, A(1) is the average fitness of the 1st generation, and D(t), D(1), B(t), and B(1) have similar meanings to A(t) and A(1). w1, w2, and w3 are weight values, where w1+w2+w3=1. In this application, the average fitness value and diversity of the population can reflect the state of the entire population and make it easier to obtain excellent individuals, so the weights of w1, w2, and w3 are set to 0.35, 0.35, and 0.3, respectively.

[0144] According to the overall fitness, this application divides the state space into 21 states, where s = (s1, s2, ..., s 21 ), when the weighted fitness value is between [0,1], 0.05 is the interval value, and every 0.05 is a state. When the fitness value is greater than 1, it is the 21st state, as shown in formula (20)

[0145]

[0146] 3.3 Action Space

[0147] For each generation, the agent will take different actions to obtain the appropriate crossover rate (Pc) and mutation rate (Pm). Pc and Pm select 10 actions as the action space. For Pc, the commonly used range is 0.4-0.9, and each action interval is 0.05; for Pm, the commonly used range is 0.01-0.2, and each action interval is 0.02, as shown in Table 3.

[0148] Table 3 Action space

[0149]

[0150] 3.4 Reward and Punishment Mechanism

[0151] In QNSGA-Ⅱ, the average fitness value and diversity of the population have a greater impact on obtaining excellent individuals. Therefore, the average fitness and population diversity before and after crossover mutation are compared. If the weighted increase of the average fitness value and population diversity of the population is increased, a positive reward is given, otherwise a negative reward is given. Because the average fitness value and diversity have little impact on excellent individuals, a1 and a2 are both 0.5. The specific formulas are shown in formulas (21) and (22):

[0152]

[0153]

[0154] In the formula, s t is the current state, a t The reward obtained after taking action a in state s after taking action r, γ is the decay value of future rewards, Q(s t ,a) is the Q value corresponding to taking action a in state s.

[0155] 3.5 Action Selection Strategy

[0156] The action selection strategy of Q-learning is called the search strategy, which is the rule for the agent to decide what action to take in a given state. These strategies are crucial for the agent to learn the optimal behavior because they directly affect the interaction effect and learning efficiency of the agent and the environment.

[0157] This application adopts an improved ε-Greedy Strategy. As shown in Algorithm 5, the agent generates a random number before making each decision. If this rand is smaller than ε, then a random action is selected. Otherwise, the action that can maximize the Q value under the current known conditions is selected. This is shown in formula (23). In order to make the ε-greedy strategy converge better in the later stages of iteration, this application designs an ε iterative descent strategy. The ε value will gradually decay with the increase of iterations, and stop decaying after decaying to the set minimum value. This makes the algorithm tend to explore new solutions in the early stages and tend to retain and utilize learning results in the later stages. This is shown in formula (24).

[0158]

[0159] ε=ε1-t((ε1-0.05) / max t) (24)

[0160] Where rand is a random number between [0,1], ε1 is the initial value of ε, and max t is the maximum number of iterations.

[0161]

[0162]

[0163] 3.6 Parameter Adaptation Strategy Based on Q-learning

[0164] After determining the state space, action space, reward function and action selection strategy, we can perform Q-learning-based crossover rate and mutation rate adaptation. First, the agent selects action a according to the action selection strategy in state S (the size of the current fitness value). c (crossover rate) and a m (Mutation rate), and obtain the reward r and the next state s' (the size of the next generation fitness value); among them, which action the agent chooses is determined by the Q table, where the rows of the Q table represent the states and the columns represent the actions.

[0165] During the iteration of the algorithm, the Q table is updated based on learning experience and predictions of future rewards. The Q update formula is formula (15). The Q table update process is as follows: Figure 8 shown.

[0166] 4 Simulation Experiment Analysis

[0167] 4.1 Example Test

[0168] The algorithm proposed in this application is implemented on Intel(R)Core(TM)i5-1035G7 CPU@1.20GHz 1.50GHz and 8.00GB RAM, and MATLAB2023a is used to perform an example simulation experiment on the proposed QNSGA-Ⅱ algorithm. First, Minitab software is used to conduct a multi-factor Taguchi orthogonal experiment on the algorithm and evaluate the problem in combination with the evaluation index to determine the optimal parameter combination of multiple factors, among which 10 FJSP benchmark test instances MK01~MK10 are used for solution. Then, the QNSGA-Ⅱ algorithm is experimentally compared with NSGA-Ⅱ and IGA to further verify the effectiveness of the algorithm.

[0169] The MRO1~MRO10 examples used in this application are randomly generated based on 10 FJSP benchmark test examples MK01~MK10 using the rules given by MATLAB.

[0170] 4.2 Performance indicators

[0171] This application aims at solving MOFJSP with QNSGA-Ⅱ and evaluates the quality of Pareto Front (PF) from three aspects, namely Inverted Generational Distance (IGD), Set Coverage (SC) and Hypervolume (HV).

[0172] For each solution y in the true optimal Pareto front (PF*), IGD finds the solution x in the nearest PF, calculates its Euclidean distance, and takes the average without square root. If the number of PF* is greater than the number of PF, then IGD can most completely express the performance of PF. The better the IGD value, the better the algorithm diversity and convergence. The specific formula is shown in formula (25).

[0173]

[0174] SC is used to evaluate the dominance relationship between two Pareto solution sets. Assuming that A and B are two PFs, the C value can be expressed as follows: |B| represents the number of solutions in B, and C(A,B) represents the percentage of solutions in B that are dominated by a solution in A. The larger the C(A,B) value, the better the performance of A. The specific formula is shown in formula (26).

[0175]

[0176] HV is used to evaluate the degree to which the target space is covered by an approximate set and is the most common evaluation indicator. A reference point is required, and the HV value is the volume of the hypercube formed between the PF and the reference point. The comparison of HV does not require prior knowledge and does not require finding the true PF. If an approximate set A completely dominates another approximate set B, then the hypercapacity HV of A will be greater than that of B. Therefore, HV can be fully used for Pareto comparison. The specific formula is shown in formula (27).

[0177]

[0178] 4.3 Parameter settings

[0179] Because the learning rate The three parameters of discount factor γ and initial greed factor ε directly determine the performance of the Q-learning part. Therefore, this application uses Minitab2019 to set a three-factor four-level Taguchi orthogonal experimental design (design of experiments) for the three parameters, and uses IGD as the evaluation index to select the optimal parameter combination;

[0180] For each parameter set to γ∈{0.5, 0.6, 0.7, 0.8}, ε1∈{0.6, 0.7, 0.8, 0.9}, forming L16 (4) The DOE orthogonal array is shown in Table 4. There are 16 groups of experiments, each group of experiments is performed 5 times and the average value is calculated. The smaller the IGD value, the better the matching of the group of measured parameters. Table 5 shows other parameters in QNSGA-Ⅱ. Using the data in Table 4, Minitab software is used to obtain Fig. 9 The results shown in the figure show that the optimal parameter combination is γ=0.6,ε1=0.7.

[0181] Table 4 Parameter combination experiment table

[0182]

[0183]

[0184] Table 5 Other parameters

[0185] parameter value Population size (N) 120 Maximum number of iterations (MAXGEN) 200 Optimization target number 2 Maximum neighborhood search range 50 Jumping out of the neighborhood value unchanged algebra 20

[0186] 4.4 Strategy effectiveness verification

[0187] In order to verify the effectiveness of the two strategies of Q-learning and local search in QNSGA-Ⅱ, two variants of QNSGA-Ⅱ were constructed, namely QNSGA-Ⅱ(A) and QNSGA-Ⅱ(B). QNSGA-Ⅱ(A) represents the case where the local search strategy is not adopted, and QNSGA-Ⅱ(B) represents the case where the Q-learning algorithm is not used for parameter adaptation. The above three indicators are used to evaluate the performance of the three algorithms. In order to avoid the randomness of the algorithm, each algorithm instance is run 10 times and the average value is taken. Table 6 shows the results of 10 runs and their average values ​​of the three algorithms QNSGA-Ⅱ, QNSGA-Ⅱ(A) and QNSGA-Ⅱ(B) evaluated using the IGD evaluation index under the MK01 case condition, which can provide a clearer understanding of how to obtain the final data of each instance; Table 7 shows the comparison of the results of the three algorithms QNSGA-Ⅱ, QNSGA-Ⅱ(A) and QNSGA-Ⅱ(B) evaluated using IGD and HV under the 10 case conditions; Table 8 shows the comparison of the results of the three algorithms QNSGA-Ⅱ, QNSGA-Ⅱ(A) and QNSGA-Ⅱ(B) evaluated using SC under the 10 case conditions, and the optimal value is indicated in bold.

[0188] Table 6 Comparison of IGD values ​​of various algorithms of MK01

[0189]

[0190]

[0191] Table 7 Comparison of IGD value and HV value

[0192]

[0193] Table 8 SC value comparison table

[0194]

[0195] By comparing QNSGA-Ⅱ and QNSGA-Ⅱ(A), it can be seen that the IGD of QNSGA-Ⅱ is smaller than that of QNSGA-Ⅱ(A) in most cases, and the HV and SC are greater than those of QNSGA-Ⅱ(A) in most cases. The improvement effects of each part have been verified, which proves the effectiveness of using Q-learning to adjust the values ​​of Pc and Pm and accelerate the convergence speed of the algorithm.

[0196] By comparing QNSGA-Ⅱ and QNSGA-Ⅱ(B), it can be seen that the IGD of QNSGA-Ⅱ is smaller than that of QNSGA-Ⅱ(B) in all cases, and the HV and SC in all cases are greater than those of QNSGA-Ⅱ(B), which verifies the effectiveness of the local search strategy based on the critical path moving operation, which is conducive to finding better individuals in the search process.

[0197] 4.5 Comparative test

[0198] In order to further evaluate the effectiveness of the proposed QNSGA-Ⅱ, it is compared with the other two cutting-edge algorithms studied in the existing literature, namely IGA and NSGA-Ⅱ. The above three evaluation indicators are used to evaluate the performance of the three algorithms. Table 9 is a comparison of the results of using IGD and HV to evaluate the three algorithms QNSGA-Ⅱ, IGA and NSGA-Ⅱ under 10 case conditions. The optimal value is in bold. In order to make the display effect better, the data in the table are used to draw Fig.10 and Fig.11 Box plot; Table 10 is the comparison of the results of using SC to evaluate the three algorithms QNSGA-Ⅱ, IGA and NSGA-Ⅱ under 10 examples. The optimal value is in bold. Fig.12 The box plot can more clearly show the distribution of the corresponding values ​​of SC(Q,I) and SC(Q,N).

[0199] Table 9 Comparison of IGD value and HV value

[0200]

[0201] Table 10 SC value comparison table

[0202]

[0203] Fig.13 The non-dominated frontiers of the three algorithms QNSGA-Ⅱ, IGA, and NSGA-Ⅱ are shown in Figure 1. To ensure randomness, four MK series examples are randomly selected for demonstration, namely MK02, MK03, MK06, and MK08. It can be clearly seen from the figure that the non-dominated frontiers in QNSGA-Ⅱ are more evenly distributed than those in the other two algorithms, and the quality of the solution is better.

[0204] From Table 9, we can see that under the IGD and HV evaluation indicators, the IGD of QNSGA-Ⅱ is better than that of IGA and NSGA-Ⅱ in each case condition, which shows that QNSGA-Ⅱ is superior to IGA and NSGA-Ⅱ in terms of algorithm convergence and diversity. Fig.10 and Fig.11 It can be seen that the QNSGA-Ⅱ algorithm is obviously better than the IGA algorithm overall and slightly better than the NSGA-Ⅱ algorithm.

[0205] Table 10 shows the comparison of QNSGA-Ⅱ, IGA, and NSGA-Ⅱ under the SC evaluation index. It can be seen that the Pareto solution set of QNSGA-Ⅱ under each case condition completely dominates the Pareto solution set of IGA, and the Pareto solution set of QNSGA-Ⅱ under most of the case conditions completely dominates the Pareto solution set of NSGA-Ⅱ. Fig.12 It can be seen from the box plot that the corresponding values ​​of SC(Q,I) and SC(Q,N) are significantly better than those of SC(I,Q) and SC(N,Q).

[0206] In order to compare the effectiveness of each algorithm more intuitively and make the results more convincing, two different types of convergence curves and Gantt charts are added, both of which are derived from a set of data obtained by the corresponding algorithms. This application selects example MK02 as a representative. Fig.14 The convergence curves of the three algorithms under the two objectives are shown. It can be clearly seen from the figure that the accuracy of QNSGA-Ⅱ is better than the other two algorithms in both objectives.

[0207] Fig.15 This is the Gantt chart obtained for MK02 using the QNSGA-Ⅱ algorithm. In the figure, the purple box represents workpiece 3. It can be seen from the figure that the first process of workpiece 3 is processed on machine 3, the second process of workpiece 3 is processed on machine 2, the third process of workpiece 3 is processed on machine 5, and so on.

[0208] In summary, the QNSGA-Ⅱ algorithm proposed in this application is superior to IGA and NSGA-Ⅱ in solving the MOFJSP problem.

[0209] This application studies the crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling problem, and proposes a QNSGA-Ⅱ algorithm with minimization of maximum completion time and total machine energy consumption as optimization objectives.

[0210] For the problem model, three population initialization rules are designed, and multiple objectives are weighted using the entropy method so as to improve the diversity of the population using a roulette wheel approach.

[0211] In order to make the crossover mutation parameters adaptive to the evolutionary state of the current population, Q-learning is added to the algorithm.

[0212] In the Q-learning framework, the fitness value is first selected as the current state space, the crossover rate and mutation rate are selected as the action space, and a reward and punishment mechanism is designed based on the average fitness value and diversity of the population.

[0213] In the action selection strategy, the classic ε-greedy strategy is extended and an elite preservation strategy is introduced, so that the adaptive parameters can better select more unknown actions in the early stage of iteration, converge better in the later stage of iteration, and preserve the better individuals in each generation.

[0214] Finally, a simulation experiment was designed, including three experiments: Taguchi orthogonal experiment, strategy effectiveness verification, and comparative experiment. The Taguchi orthogonal experiment determined the optimal parameter combination of learning rate, discount factor, and initial greedy factor. The strategy effectiveness verification ensured the effectiveness of the two key strategies proposed (Q-learning and local search). The comparative experiment compared the proposed QNSGA-Ⅱ algorithm with other cutting-edge algorithms to ensure the overall effectiveness of the algorithm.

[0215] Although the present invention has been illustrated and described with respect to the preferred embodiments, it will be appreciated by those skilled in the art that various changes and modifications may be made to the present invention without departing from the scope of the present invention as defined by the claims.

Claims

1. Crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling method, It is characterized in that The following steps are involved: S1: Aiming at the multi-objective flexible job shop scheduling problem, a multi-objective flexible job shop scheduling model is established with the minimization of the maximum completion time and the minimum total energy consumption of the machine as the optimization objectives; S2: Construct a crossover rate and mutation rate adaptive strategy based on Q-learning, including: creating two Q tables for crossover and mutation, defining the Q table update formula; calculating the fitness value with the objective function value, and then using the size of the fitness value as the state space; for each generation, the agent can take different actions to obtain the appropriate crossover rate (Pc) and mutation rate (Pm), and Pc and Pm respectively select probability values ​​of different intervals as the action space; design the reward and punishment function based on the overall average fitness and population diversity; adopt the improved ε-greedy strategy as the action selection strategy; realize the adaptive strategy of crossover rate and mutation rate; S3: The Q-learning-based crossover rate and mutation rate adaptive strategies in S2 are combined with the NSGA-Ⅱ population crossover and mutation links to make up for the defects of the fixed crossover rate (Pc) and mutation rate (Pm) in NSGA-Ⅱ, and finally the optimal solution is gradually obtained by solving the objective function of the multi-objective flexible job shop scheduling model in S1, including: first, using three initialization strategies to generate the initial solution, and then sorting and selecting the initial population based on the entropy method; then, initializing the Q-learning learning process, Pc and Pm are randomly selected within a certain range; next, after the crossover and mutation operations, a new population is obtained, their states and rewards are calculated, and the Q-table update formula in Q-learning is used to update the Q table; finally, a local search algorithm with a taboo search strategy is used to obtain a non-dominated solution.

2. The crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The establishment process of the multi-objective flexible job shop scheduling model in S1 is as follows: S1-1: Define the decision variable as X ijk , used to indicate whether the jth process of workpiece i is processed on machine k in: In the formula, C i represents the completion time of job i, C ij represents the completion time of the jth process of workpiece i, P ijk represents the processing time of the jth process of workpiece i on machine k, and the machine set M = {M k |1≤k≤m}, the workpiece set J={J i |1≤i,e≤n},J i The process set O = {O ij |1≤j,p≤n i },S ijk is the starting processing time of the jth process of workpiece i on machine k, C in represents the completion time of the last process of workpiece i; S1-2: The optimization goal is to minimize the maximum completion time. The objective function is defined as shown in (3): f 1 =min{max{C i }},i=1,2,…,n (3) Taking minimizing the total energy consumption of the machine as the optimization goal, the objective function is defined as shown in (4): In the formula, H i j k It represents the loss caused by the j-th process of workpiece i to machine k, and defines the constraints: C ij -C i(j-1) ≥P ijk X ijk (6) S ijk ≥S epk +P epk (7) Expression (5) limits the process to only one processing; Expression (6) indicates that the processing sequence of the same workpiece must be carried out according to the process sequence; Expression (7) requires that the machine can only process one workpiece at a time, where P epk represents the processing time of the pth process (the process before jth process) of workpiece e on machine k, S epk It indicates the start time of the p-th process (the process before j-th process) of workpiece e on machine k.

3. The crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The Q table update formula in S2 is: In the formula, s t is the current state, a t The reward obtained after taking action a in state s after taking action r, γ is the decay value of future rewards, Q(s t ,a) is the Q value corresponding to taking action a in state s.

4. The crossover mutation rate adaptive NSGA-II multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The fitness value function in S2 mainly considers the following aspects: The average fitness is shown in formula (9), the population diversity is shown in formula (10), and the best individual fitness is shown in formula (11). Formula (12) is the weighted combination of formulas (9), (10), and (11): In the formula, represents the target value of the ith individual in the tth generation, represents the optimal target value of the ith individual in the tth generation, A(t) is the average fitness of the tth generation, A(1) is the average fitness of the 1st generation, and D(t), D(1), B(t), and B(1) have similar meanings to A(t) and A(1). 1 、w 2 、w 3 is the weight value, and the state space is divided into different state spaces according to the overall fitness.

5. The crossover mutation rate adaptive NSGA-Ⅱ multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The formula for the reward and punishment mechanism in S2 is: In the formula, a 1 、a 2 All are 0.5, A(t) is the average fitness of the t-th generation population, D(t) is the population diversity of the t-th generation, R is the weighted value of the average fitness and diversity of the population, and r is the reward and punishment value.

6. The crossover mutation rate adaptive NSGA-II multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The improvement of the ε-greedy strategy in S2 is that the ε value will gradually decay with the increase of iterations, and stop decaying after decaying to the set minimum value. The formula is as follows: e=e 1 -t((e 1 -0.05) / max t) In the formula, rand is a random number between [0,1], ε 1 is the initial value of ε, max t is the maximum number of iterations, and A is the set of optional actions.

7. The crossover mutation rate adaptive NSGA-II multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The three initialization strategies for the population in S3 include: generating an initial solution completely randomly; generating an initial solution according to the minimum time; and generating an initial solution according to the maximum remaining time of the first processing.

8. The crossover mutation rate adaptive NSGA-II multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The sorting and selection of the entropy value method in S3 includes: weighting two different target values, namely minimizing the maximum completion time and the machine energy consumption, to combine them into a comprehensive target value for sorting, and selecting the selection operation using the roulette wheel selection method.

9. The crossover mutation rate adaptive NSGA-II multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The population after crossover in S3 is directly used as the initial population for mutation.

10. The crossover mutation rate adaptive NSGA-II multi-objective flexible job shop scheduling method according to claim 1, It is characterized in that The local search algorithm with taboo search strategy in S3 includes: the first step is to find the key process, that is, the process with the longest total duration in the scheduling plan, which has the greatest impact on the completion time of the entire scheduling plan, and there may be multiple paths; the second step is to transfer the key process to other machines under the premise of ensuring the constraints of the previous and subsequent processes; the third step is to establish a taboo table, conduct local search on individuals of a single target and record relevant information in the taboo table.

Citation Information

Cited By

  • Flexible job shop machine and AGV combined scheduling method, system and device

    CN119644959A

  • Multi-workshop job task cooperative scheduling method based on tabu search and NSGA-II fusion algorithm

    CN119647681A

  • Energy-saving flexible workshop scheduling method and device considering human factors

    CN120146541A

  • Efficient and energy-saving flexible job shop scheduling method considering adjustment time

    CN121073114A