Flexible job shop scheduling method considering fuzzy factors

Abstract

The invention discloses a flexible job shop scheduling method considering fuzzy factors. The method comprises the following steps: constructing a flexible production process model with fuzzy features; initializing a differential evolution algorithm; defining a differential evolution algorithm as a Markov decision process; and strategy optimization is carried out by using deep reinforcement learning, and flexible job shop scheduling is realized. According to the method, the fuzzy set theory and the flexible job shop scheduling problem are combined, uncertain factors in the production process are represented by fuzzy numbers, scheduling imbalance caused by changes of production and processing parameters can be prevented, the anti-interference performance of a scheduling scheme is improved, an optimization algorithm can better cope with a complex and changeable flexible production environment, and the production efficiency is improved. And a more reasonable and effective production scheduling scheme is obtained. And deep reinforcement learning is introduced to learn the variation strategy and the variation rate, and the algorithm is guided to be optimized in a more promising direction, so that the optimization process is more efficient, the solution performance is stronger, the solution result distribution is more concentrated, and the solution quality is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of job shop scheduling, and particularly to a flexible job shop scheduling method considering fuzzy factors. Background Art

[0002] Through the transformation and upgrading of the manufacturing industry, especially by promoting the application of key technologies and products such as informatization, automation, and intelligence, the supply quality and market competitiveness of the manufacturing industry can be improved. Among them, as the core link in production manufacturing, the optimization scheme of shop scheduling has a significant impact on improving production efficiency, reducing costs, and increasing enterprise profits. However, traditional shop scheduling research (JSP) often based on fixed parameter assumptions, ignoring the uncertainty and complexity of parameters in a flexible production environment, resulting in limited practical application effects.

[0003] With the in-depth development of the fuzzy set theory, researchers have recognized the advantages of fuzzy numbers in describing the actual situation of flexible job shop scheduling (FJSP), and then proposed the fuzzy flexible job shop scheduling problem (FFJSP); however, there are three major limitations in the current FFJSP (fuzzy flexible job shop scheduling problem) scheduling research: First, the solution strategies lack diversity, and most research is limited to improving within the framework of genetic algorithms by adding local search strategies or combining with other optimization algorithms, and there are few explorations of new solution paths; Second, the optimization objectives are relatively limited, mainly focusing on minimizing the processing time, and there is still insufficient research on FFJSP scheduling involving multi-objective optimization (such as cost, energy consumption, resource utilization, etc.); Third, the model construction is too idealized, and most research focuses on the standard FFJSP model, ignoring the deep integration with the actual production environment, resulting in limited practicality and generalization ability of the model. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a flexible job shop scheduling method considering fuzzy factors.

[0005] To achieve the above purpose, the technical solution provided by the present invention is as follows:

[0006] A flexible job shop scheduling method considering fuzzy factors, including:

[0007] Construct a flexible production process model with fuzzy characteristics;

[0008] Initialize the differential evolution algorithm;

[0009] Define the differential evolution algorithm as a Markov decision process;

[0010] Use deep reinforcement learning for policy optimization to achieve flexible job shop scheduling.

[0011] Furthermore, a flexible production process model with fuzzy characteristics is constructed, including:

[0012] According to different restriction conditions for machine selection in the flexible production process, the fuzzy flexible job shop scheduling problem FFJSP is divided into two categories:

[0013] The scheduling problem where any machine in the machine concentration can be selected for processing all the operations in the process route is called the total fuzzy flexible job shop scheduling problem t-FFJSP, that is

[0014] The scheduling problem where only some machines in the machine concentration can be selected for processing some of the operations in the process route is called the partial fuzzy flexible job shop scheduling problem p-FFJSP, that is

[0015] where i represents the workpiece number, j represents the operation number of the workpiece, and M ij represents the set of machines that can process O ij ; O ij represents the j-th operation of workpiece i, and M is the set of all machines;

[0016] For the processing time, triangular fuzzy numbers are used to characterize the uncertainty of the processing time of the two types of fuzzy flexible job shop scheduling problems, specifically:

[0017] For the operation O 11 of workpiece J1, its fuzzy processing time on machine M1 is (u, v, q), where u, v, and q represent the minimum value, the most likely value, and the maximum value of the processing time respectively;

[0018] Fuzzy characteristics are introduced to describe the uncertainty parameters, and common evaluation indexes in solving t-FFJSP and p-FFJSP are given:

[0019] (1) Fuzzy makespan:

[0020]

[0021] In the above formula, n is the number of workpieces, represents the fuzzy completion time of the i-th workpiece, represents obtaining the maximum value in;

[0022] (2) Fuzzy average satisfaction degree:

[0023]

[0024] where,

[0025]

[0026] In the above formula, represents the fuzzy satisfaction degree of the i-th workpiece, represents the fuzzy satisfaction degrees of all workpieces, represents the fuzzy due date of the i-th workpiece; the fuzzy completion time is represented by a triangular fuzzy number, and the fuzzy delivery time is represented by a trapezoidal fuzzy number.

[0027] Furthermore, the comparison method of triangular fuzzy numbers is as follows:

[0028] For triangular fuzzy numbers and

[0029] 1) Calculate the following formula. If then then

[0030]

[0031] 2) If then compare a2 and b2. If a2 > b2, then If a2 < b2, then

[0032] 3) If a2 = b2, then compare a3 - a1 and b3 - b1. If a3 - a1 > b3 - b1, then If a3 - a1 < b3 - b1, then

[0033] Furthermore, the initialization of the differential evolution algorithm includes:

[0034] Initialize the population;

[0035] Initialize the mutation strategy:

[0036] Mutation strategy 1:

[0037] Mutation strategy 2:

[0038] Mutation strategy 3:

[0039] Mutation strategy 4:

[0040] Among them, is the newly generated individual after the mutation operation, are respectively the individuals with fitness rankings d, r1, r2, r3, r4, r5, where the value ranges of r1, r2, r3, r4, r5 are all [1, N], and N is the population size; F is the mutation rate.

[0041] Furthermore, the state features in the state space include the fitness of the current individual, and the calculation formula for the normalized difference P1 between the worst and the best solutions is as follows:

[0042]

[0043] Among them, is the fitness value of the current parent, f bsf and f wsf are the best fitness value and the worst fitness value found currently, respectively;

[0044] The calculation formula for the normalized difference P2 between the overall average fitness of the entire population and the worst and the best solutions is as follows:

[0045]

[0046] Among them, N is the population size, is the average fitness of the current population;

[0047] The calculation formula for the standard deviation P3 of the overall fitness value of the entire population is as follows:

[0048]

[0049] Among them, std max is the standard deviation of the population with the best fitness value; is the standard deviation of the current population;

[0050] The calculation formula for the remaining budget P4 in the iterative process of the differential evolution algorithm is as follows:

[0051]

[0052] Among them, FE max is the maximum number of iterations for the differential evolution algorithm to run, and t is the number of iterations for the current run of the differential evolution algorithm.

[0053] Furthermore, the set rewards include two types, namely the reward R given according to the fitness difference between the offspring and the parent during improvement d and the reward R given when the current solution is improved to become the current best b ;

[0054] The calculation formula for the reward R d is as follows:

[0055]

[0056] If the fitness difference is greater than zero, then take the improved fitness difference value as the reward R d; If the fitness difference is less than zero, then set the reward R d to 0, indicating that there is no improvement currently;

[0057] The reward R b has the following calculation formula:

[0058]

[0059] Furthermore, an ε-greedy strategy is set when selecting actions:

[0060]

[0061] where act represents the selected action; A is the set of all possible actions; ∈ is a decimal between 0 and 1, representing the probability of selecting a random action; random action from A represents randomly selecting an action from the action set A; argmax a∈A Q(s,a) represents selecting the action a that maximizes Q(s,a) in state s; Q(s,a) is the action-value function, which is used to estimate the expected return that can be obtained after performing action a in state s.

[0062] Compared with the prior art, the principles and advantages of this technical solution are as follows:

[0063] 1. Combining the fuzzy set theory with the flexible job shop scheduling problem, representing the uncertain factors in the production process with fuzzy numbers can prevent scheduling disorders caused by changes in production and processing parameters, increase the anti-interference performance of the scheduling scheme, enable the optimization algorithm to better cope with the complex and changeable flexible production environment, and obtain a more reasonable and effective production scheduling scheme.

[0064] 2. The improved differential evolution algorithm is simple to apply, has strong robustness, and has the characteristics of faster convergence speed and higher accuracy. Applying it to the fuzzy flexible job shop scheduling problem, using multiple mutation strategies to generate new populations, and selecting the optimal new population as the sub-population to enter the next iteration can obtain better optimization effects than the particle swarm algorithm and genetic algorithm in the single-objective fuzzy flexible job shop scheduling problem.

[0065] 3. Aiming at the problems of low solution quality and weak generalization ability when the differential evolution algorithm solves the multi-objective fuzzy flexible job shop scheduling problem, deep reinforcement learning is introduced to learn its mutation strategy and mutation rate, guiding the algorithm to optimize in a more promising direction, making its optimization process more efficient, its solving performance stronger, the distribution of solving results more concentrated, and the quality of solutions higher. Description of the Drawings

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the services required in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0067] Figure 1 It is a principle flowchart of a flexible job shop scheduling method considering fuzzy factors according to the present invention;

[0068] Figure 2 It is a schematic diagram of fuzzy satisfaction degree;

[0069] Figure 3 It is a general Gantt chart;

[0070] Figure 4 It is a Gantt chart of FFJSP. Specific embodiments

[0071] The present invention will be further described below in conjunction with specific embodiments:

[0072] As Figure 1 shown, a flexible job shop scheduling method considering fuzzy factors described in this embodiment includes the following steps:

[0073] S1. Construct a flexible production process model with fuzzy characteristics, and the process includes:

[0074] According to different restrictions on machine selection in the flexible production process, the fuzzy flexible job shop scheduling problem FFJSP is divided into two categories:

[0075] The scheduling problem where any machine in the machine concentration can be selected for processing all processes in the process route is called the total fuzzy flexible job shop scheduling problem t-FFJSP, that is

[0076] The scheduling problem where only some processes in the process route can only select some machines in the machine concentration for processing is called the partial fuzzy flexible job shop scheduling problem p-FFJSP, that is

[0077] Among them, i represents the workpiece number, j represents the process number of the workpiece, M ij represents the set of machinable machines of O ij O represents the j-th process of workpiece i, and M is the set of all machines; ij For the processing time, triangular fuzzy numbers are used to characterize the uncertainty of the processing time of the two types of fuzzy flexible job shop scheduling problems, specifically:

[0078] For the processing time, triangular fuzzy numbers are used to characterize the uncertainty of the processing time of the two types of fuzzy flexible job shop scheduling problems, specifically:

[0079] For the operation O of workpiece J1 11 , its fuzzy processing time on machine M1 is (u, v, q), where u, v, and q represent the minimum value, the most likely value, and the maximum value of the processing time, respectively;

[0080] The following respectively give the fuzzy processing time tables corresponding to t-FFJSP and p-FFJSP containing two workpieces and three processing machines. For example, the operation O of workpiece J1 in the t-FFJSP fuzzy processing time table 11 has a fuzzy processing time of (5, 6, 7) on machine M1, where 5, 6, and 7 represent the minimum value, the most likely value, and the maximum value of the processing time respectively. The dash "-" in the p-FFJSP fuzzy processing time table indicates that a certain machine cannot process the corresponding operation of a certain workpiece.

[0081]

[0082] t-FFJSP fuzzy processing time table

[0083]

[0084]

[0085] p-FFJSP fuzzy processing time table

[0086] Fuzzy features are introduced to describe the uncertain parameters, and the common evaluation indexes in solving t-FFJSP and p-FFJSP are given:

[0087] (1) Fuzzy makespan:

[0088]

[0089] In the above formula, n is the number of workpieces, represents the fuzzy completion time of the i-th workpiece, represents obtaining the maximum value in;

[0090] (2) Fuzzy average satisfaction degree:

[0091]

[0092] Among them,

[0093]

[0094] In the above formula, represents the fuzzy satisfaction degree of the i-th workpiece, represents the fuzzy satisfaction degrees of all workpieces, Denote the fuzzy due date and fuzzy completion time of the $i$-th workpiece; The fuzzy due time is represented by a triangular fuzzy number The fuzzy due time is represented by a trapezoidal fuzzy number. Draw an example diagram as Figure 2 shown.

[0095] The Gantt chart of FFJSP is also different from the general Gantt chart:

[0096] The general Gantt chart is as Figure 3 shown, and the Gantt chart of FFJSP is as Figure 4 shown.

[0097] The comparison method of triangular fuzzy numbers is as follows:

[0098] For triangular fuzzy numbers and

[0099] 1) Calculate the following formula. If then then

[0100]

[0101] 2) If then compare $a_2$ and $b_2$. If $a_2 > b_2$, then If $a_2 < b_2$, then

[0102] 3) If $a_2 = b_2$, then compare $a_3 - a_1$ and $b_3 - b_1$. If $a_3 - a_1 > b_3 - b_1$, then If $a_3 - a_1 < b_3 - b_1$, then

[0103] S2. Initialize the differential evolution algorithm. The process includes:

[0104] Initialize the population;

[0105] Initialize the mutation strategy:

[0106] Mutation strategy 1:

[0107] Mutation strategy 2:

[0108] Mutation strategy 3:

[0109] Mutation strategy 4:

[0110] where is the newly generated individual after the mutation operation, Individuals with fitness rankings d, r1, r2, r3, r4, r5 respectively, where the value ranges of r1, r2, r3, r4, r5 are all [1, N], and N is the population size; F is the mutation rate.

[0111] S3. Define the differential evolution algorithm as a Markov decision process, and the process includes:

[0112] S3-1. First is the setting of the state space. The quality of the state space setting plays a crucial role in the learning of DQN (Deep Q-network). It needs to provide sufficient information for DQN to decide which action is more appropriate at the current step. However, too many state feature representations will greatly increase the algorithm complexity. Considering the balance between the two, the following four state features are used in this embodiment:

[0113] The state features in the state space include the following formula for calculating the normalized difference P1 between the fitness of the current individual and the worst and best solutions:

[0114]

[0115] Among them, is the fitness value of the current parent, f bsf and f wsf are the best fitness value and the worst fitness value found currently, respectively;

[0116] The following formula is used to calculate the normalized difference P2 between the total average fitness of the entire population and the worst and best solutions:

[0117]

[0118] Among them, N is the population size, is the average fitness of the current population;

[0119] The following formula is used to calculate the standard deviation P3 of the overall fitness value of the entire population:

[0120]

[0121] Among them, std max is the standard deviation of the population with the best fitness value; is the standard deviation of the current population;

[0122] The following formula is used to calculate the remaining budget P4 during the iterative process of the differential evolution algorithm:

[0123]

[0124] Among them, PE max$T$ is the maximum number of iterations for the differential evolution algorithm to run, and $t$ is the number of iterations for the current run of the differential evolution algorithm.

[0125] S3-2. Set to give rewards in each iteration of the differential evolution algorithm;

[0126] The rewards set include two types, namely the reward $R_1$ given according to the fitness difference between the offspring and the parent during improvement d and the reward $R_2$ given when the current solution is improved to be the current optimal, b which respectively contain different information on fitness improvement and enhance the comprehensiveness of evaluation.

[0127] The formula for the reward $R_1$ d is as follows:

[0128]

[0129] If the fitness difference is greater than zero, then take the improved fitness difference value as the reward $R_1$ d ; if the fitness difference is less than zero, then let the reward $R_1$ d be 0, indicating that there is no current improvement;

[0130] The formula for the reward $R_2$ b is as follows:

[0131]

[0132] S3-3. Set the selection action;

[0133] When setting the selection action, use the ε-greedy strategy:

[0134]

[0135] where $act$ represents the selected action; $A$ is the set of all possible actions; $\epsilon$ is a decimal between 0 and 1, representing the probability of choosing a random action; $random\ action\ from\ A$ represents randomly selecting an action from the action set $A$; $argmax$ a∈A $Q(s,a)$ represents choosing the action $a$ that maximizes $Q(s,a)$ in state $s$; $Q(s,a)$ is the action-value function, which is used to estimate the expected return that can be obtained after executing action $a$ in state $s$.

[0136] S4. Use deep reinforcement learning for policy optimization to achieve flexible job shop scheduling.

[0137] After setting the above state, reward, and action, the differential evolution algorithm is successfully constructed into a Markov decision process. Whenever an action $a$ is selected in state $S$ t , a reward $r$ will be obtained t ​t and the next state S t+1 , the obtained tuple <s t , a t , r t , s t+1 > is called an observation value, that is, a Markov decision process, used for the learning of DQN, and then to optimize the policy in the process of population iteration, so as to achieve flexible job shop scheduling.

[0138] The above-described embodiments are only preferred embodiments of the present invention, and do not limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A flexible job shop scheduling method considering fuzzy factors, characterized in that, Including: Construct a flexible production process model with fuzzy characteristics; Initialize the differential evolution algorithm; Define the differential evolution algorithm as a Markov decision process; Use deep reinforcement learning for policy optimization to achieve flexible job shop scheduling.

2. The flexible job shop scheduling method considering fuzzy factors according to claim 1, characterized in that Construct a flexible production process model with fuzzy characteristics, including: According to different machine selection constraints in the flexible production process, the fuzzy flexible job shop scheduling problem FFJSP is divided into two categories: The scheduling problem in which any machine in the machine cluster can be selected for processing in all processes of the process route is called the total fuzzy flexible job shop scheduling problem t-FFJSP, that is The scheduling problem in which some processes in the process route can only be selected to be processed on some machines in the machine set is called the partial fuzzy flexible job shop scheduling problem p-FFJSP, that is Among them, i represents the workpiece number, j represents the process number of the workpiece, and M ij represents the set of machinable machines for O ij , O ij represents the process j of workpiece i, and M is the set of all machines; For the processing time, triangular fuzzy numbers are used to characterize the uncertainty of the processing time of the two categories of fuzzy flexible job shop scheduling problems, specifically: For the process O of workpiece J1 11 , its fuzzy processing time on machine M1 is (u, v, q), u, v, and q respectively represent the minimum value, the most likely value, and the maximum value of the processing time; Introduce fuzzy characteristics to describe the uncertainty parameters, and give the common evaluation indicators in solving t-FFJSP and p-FFJSP: (1) Fuzzy makespan: In the above formula, n is the number of workpieces, represents the fuzzy completion time of the i-th workpiece, represents obtaining the maximum value in; (2) Fuzzy average satisfaction: Where, In the above formula, represents the fuzzy satisfaction degree of the i-th workpiece, represents the fuzzy satisfaction degrees of all workpieces, represents the fuzzy due date of the i-th workpiece; the fuzzy completion time is represented by a triangular fuzzy number, and the fuzzy delivery time is represented by a trapezoidal fuzzy number.

3. A flexible job shop scheduling method considering fuzzy factors according to claim 2, characterized in that, The comparison method of triangular fuzzy numbers is as follows: For triangular fuzzy numbers and 1) Calculate the following formula. If then then 2) If then compare a2 and b2. If a2 > b2, then If a2 < b2, then 3) If a2 = b2, then compare a3 - a1 and b3 - b1. If a3 - a1 > b3 - b1, then If a3 - a1 < b3 - b1, then 4. A flexible job shop scheduling method considering fuzzy factors according to claim 1, characterized in that Initializing the differential evolution algorithm includes: Initializing the population; Initializing the mutation strategy: Mutation Strategy 1: Mutation Strategy 2: Mutation Strategy 3: Mutation Strategy 4: Among them, is the newly generated individual after mutation operation, are individuals with fitness rankings d, r1, r2, r3, r4, r5 respectively, where the value ranges of r1, r2, r3, r4, r5 are all [1, N], N is the population size; F is the mutation rate.

5. A flexible job shop scheduling method considering fuzzy factors according to claim 4, characterized in that, Defining the differential evolution algorithm as a Markov decision process specifically includes: Setting the state space: The state characteristics in the state space include the normalized difference P1 between the fitness of the current individual and the worst and best solutions, the normalized difference P2 between the total average fitness of the entire population and the worst and best solutions, the standard deviation P3 of the overall fitness value of the entire population, and the remaining budget P4 during the iteration process of the differential evolution algorithm; Set to give rewards in each iteration of the differential evolution algorithm; Set the selection action.

6. A flexible job shop scheduling method considering fuzzy factors according to claim 5, characterized in that, The calculation formula for the normalized difference P1 between the fitness of the current individual and the worst and best solutions in the state characteristics in the state space is as follows: Among them, is the fitness value of the current parent, f bsf and f wsf are the best fitness value and the worst fitness value found currently, respectively; The calculation formula for the normalized difference P2 between the total average fitness of the entire population and the worst and best solutions is as follows: where N is the population size, is the average fitness of the current population; The calculation formula for the standard deviation P3 of the overall fitness value of the entire population is as follows: Among them, std max is the standard deviation of the population with the optimal fitness value; is the standard deviation of the current population; The calculation formula for the remaining budget P4 during the iteration process of the differential evolution algorithm is as follows: Among them, FE max is the maximum number of iterations for the differential evolution algorithm to run, and t is the number of iterations for the current differential evolution algorithm to run.

7. A flexible job shop scheduling method considering fuzzy factors according to claim 6, characterized in that, The set rewards include two types, namely, the reward R given according to the fitness difference between the offspring and the parent during improvement d and the reward R given when the current solution is improved to become the current optimum b ; Reward R d The calculation formula is as follows: If the fitness difference is greater than zero, then use the improved fitness difference as the reward R d ; If the fitness difference is less than zero, set the reward R d to 0, indicating that there is no current improvement; Reward R b The calculation formula is as follows:

8. A flexible job shop scheduling method considering fuzzy factors according to claim 5, characterized in that, Use the ε-greedy strategy when setting the selection action: Among them, act represents the selected action; A is the set of all possible actions; ∈ is a decimal between 0 and 1, representing the probability of choosing a random action; random action from A represents randomly selecting an action from the action set A; argmax a∈A Q(s,a) represents, in state s, choosing the action a that maximizes Q(s,a); Q(s,a) is the action-value function, used to estimate the expected return that can be obtained after executing action a in state s.

Images