Flexible job shop scheduling optimization method with robot constraint

Abstract

The invention relates to the technical field of flexible job shop scheduling in intelligent manufacturing, in particular to a flexible job shop scheduling optimization method with robot constraint. Comprising the steps of initializing parameters; performing population initialization, and generating an evolution guiding population and a knowledge driving population; population evolution: using an individual competition strategy, self-evolution and co-evolution to evolve an evolution guide population, and using an individual pairing strategy and DQN evolution to evolve a knowledge driven population; population updating: updating the evolution guiding population and the knowledge driving population by using the combined population; cP auxiliary optimization is carried out, an optimization condition is met, a CP model is constructed, an individual with the maximum completion time and the minimum completion time after population updating is used as an initial solution in the CP model, and further optimization is carried out by utilizing the global search capability of the CP model; and stopping condition check, and outputting a final solution if yes. The influence of robot loading and unloading time on production efficiency can be effectively reduced, and the equipment utilization rate is increased.

Description

Technical Field

[0001] The present invention relates to the technical field of flexible job shop scheduling in intelligent manufacturing, and in particular to an optimization method for a flexible job shop scheduling problem with robot constraints. Background Art

[0002] With the rapid development of automation technology, fully automated workshops equipped with robots have become an important development trend in the manufacturing industry. The Flexible Job Shop Scheduling Problem (FJSP) is the core scheduling problem of intelligent production systems. The extended problem of Flexible Job Shop Scheduling Problem with Robot Constraints (FJSP-RC) not only needs to solve the process sequencing and machine selection problems, but also needs to consider the loading and unloading task scheduling of robots. Among the existing methods, the Mixed Integer Linear Programming (MILP) model and the Constraint Programming (CP) model can solve small-scale instances, but are inefficient for large-scale problems. Although the traditional evolutionary algorithm (EA) has high solution efficiency, it lacks effective use of historical search information. In the field of intelligent scheduling, reinforcement learning based on Q learning is a specific implementation of reinforcement learning (RL). Q-learning is a model-free reinforcement learning algorithm that uses Q-value (action value function) to evaluate the long-term cumulative reward of taking a specific action in a certain state. The agent selects the optimal action based on the Q-value. The agent is the subject of the decision-making task to achieve the goal of maximizing the cumulative reward. However, Q-learning-based reinforcement learning (Q-learning) has limited state extraction capabilities in complex workshop environments. Compared with Q-learning-based reinforcement learning, deep reinforcement learning algorithms have obvious advantages. Deep Q-Network (DQN) is an algorithm for deep reinforcement learning. DQN combines the powerful feature extraction capabilities of deep learning and the decision optimization mechanism of reinforcement learning, effectively solving the challenges brought by high-dimensional state space, enabling the agent to understand the environmental state more accurately and make better decisions. This advantage has made DQN, a deep reinforcement learning algorithm, gradually applied in the field of workshop scheduling. In addition, the existing methods do not fully combine the global search capability of CP and the group optimization advantages of EA, resulting in insufficient solution quality and efficiency. Therefore, there is an urgent need for a hybrid algorithm that can integrate CP, EA and DQN to efficiently solve the flexible job shop scheduling problem with robot constraints. Summary of the invention

[0003] The purpose of the present invention is to propose a flexible job shop scheduling optimization method with robot constraints, which solves the problems of the inability to effectively handle robot constraints, limited solution space and low search efficiency in existing shop scheduling, so as to achieve the purpose of effectively reducing the impact of robot loading and unloading time on production efficiency and improving equipment utilization by optimizing the scheduling scheme.

[0004] The present invention provides a flexible job shop scheduling optimization method with robot constraints, which is characterized by comprising the following steps: Step 1: Initialize parameters and set the size of the evolutionary guide population N p1 , knowledge-driven population size N p2 and total running time t ; Step 2: Population initialization: randomly generate the initial population, including the evolutionary guided population and the knowledge driven population; Step 3: Population evolution: using individual competition strategy, self-evolution and co-evolution to evolve the evolution-guided population, and using individual pairing strategy and DQN evolution to evolve the knowledge-driven population; Step 4, population update, combining the evolved evolutionary guided population and knowledge driven population into one population, and using the combined population to update the evolutionary guided population and knowledge driven population; Step 5, CP-assisted optimization. If the optimization conditions are met, a CP model is constructed. The individual with the smallest maximum completion time after the population update is used as the initial solution in the CP model. The global search capability of the CP model is used for further optimization. If the optimization conditions are not met, return to step 3, where the optimization condition is that the running time after the population update is equal to the total running time. t half of; Step 6: Check the termination condition. If the termination condition is met, output the final solution. The termination condition is that the running time after CP-assisted optimization is equal to the total running time. t .

[0005] Furthermore, in step 2, the population initialization process is to loop-initialize individuals, including evolving the population through loops from 0 to N p1 Iterate, and the knowledge-driven population goes through a cycle from 0 to N p2 Iterate, and create an individual in each iteration; the individual is composed of two vectors, namely the process sorting vector and the machine selection vector; the process sorting vector is initialized, and the length of the process sorting vector is the total number of processes. The processes in the process sorting vector are in [0, n-1] is randomly generated within the range, n is the total number of workpieces. The number of times each workpiece appears in the process sorting vector is consistent with the number of processes contained in the workpiece. The machine selection vector is initialized. The length of the machine selection vector is the total number of processes. The machines in the machine selection vector are in [0, M -1] is randomly generated within the range, M is the total number of machines. If the processing time of the process on the selected machine is 0, the machines in the machine selection vector are in the range [0, M -1] until the processing time of the process on the selected machine is not 0; after completing the individual initialization, the generated N p1 The number of individuals is added to the evolutionary guide population, and the generated N p2 number of individuals are added to the knowledge-driven population.

[0006] Furthermore, in step 3, the individual competition strategy is to classify the individuals in the evolutionary guided population according to the maximum completion time, and divide the evolutionary guided population into a winner population and a loser population. The sizes of the winner population and the loser population are N p / 2, sort the individuals in the evolutionary guided population according to their maximum completion time from small to large, N p1 / 2 individuals are assigned to the winner population, and then N p1 / 2 individuals are assigned to the loser population; each individual in the winner population randomly uses one of the exchange operator, reversal operator and reallocation operator to self-evolve, where the exchange operator and reversal operator are applied to the process sorting vector, and the reallocation operator is applied to the machine selection vector; the winner population and the loser population co-evolve by using the crossover operator, which includes the priority process crossover operator and the uniform crossover operator. An individual is randomly selected from the winner population and the loser population respectively, and the process sorting vectors of the two selected individuals use the priority process crossover operator, and the machine selection vector uses the uniform crossover operator. The process sorting vector and machine selection vector of each individual are crossovered once until all individuals have completed the crossover process.

[0007] Further, the operation steps of the exchange operator, the inversion operator, the reallocation operator, the priority process crossover operator and the uniform crossover operator are as follows: The operation process of the exchange operator is to randomly select two different positions from the individual process sorting vector rand 1 and location rand 2 , swap the selected position rand 1and location rand 2 The above process; The operation process of the inversion operator is to randomly select two different positions from the individual process order vector rand 1 and location rand 2 ,Location rand 1 In Location rand 2 The position in the process sorting vector is at least 3 process intervals away from the previous position. rand 1 and location rand 2 Reverse the process sequence between and find the midpoint of the reversal interval ( rand 1 + rand 2 ) / 2, successively exchange the processes symmetrical with the midpoint of the reversal interval; The operation process of the reallocation operator is to randomly select a position in the machine selection vector, and randomly select a machine from the set of selectable machines corresponding to the process at the selected position to replace the original machine; The operation process of the priority process crossover operator is to obtain individual pop 1 and individuals pop 2 The process ordering vector is represented as a process sequence, which is randomly generated [1, n -1] num, n Indicates the total number of artifacts generated num Artifact Sets of Size I 0 , and randomly select from all artifacts num different artifacts to populate the artifact set I 0 In, traverse pop 1 and pop 2 At each position in the process sequence, if we find a workpiece set I 0 The process is then pop 1 and pop 2 Exchange the corresponding processes in; The operation process of the uniform crossover operator is to obtain individual pop 1 and individuals pop 2The machine selection vector of , where the machine selection vector is represented as a machine sequence, generates a size of N Digital Set B , N Indicates the total number of processes, a digital set B Each position in the random filling is 0 or 1, 0 and 1 are used to determine pop 1 and pop 2 The flag of whether the machine at the corresponding position in the machine sequence is exchanged, where 0 represents exchange and 1 represents no exchange. N Iterate, traverse in order from the first position to the last position pop 1 and pop 2 Each position of the machine sequence and synchronously traverse the number set B The numbers in the corresponding positions in the B If the number at the current position is 0, pop 1 and pop 2 Exchange the machine at the current traversal position in the machine sequence.

[0008] Furthermore, in step 3, the individual pairing strategy is to generate all paired individual pairs from the knowledge-driven population, the paired individual pairs are composed of two different individuals, and all individual pairs are combined together to form a paired population; DQN evolution is applied to the paired population, and DQN selects the search operator that minimizes the maximum completion time based on the process sorting vector and machine selection vector combination information of the individual pairs in the current paired population. The search operators include exchange operators, inversion operators, reallocation operators, job-based crossover operators, two-point crossover operators, and multi-point crossover operators, among which the exchange operator, inversion operator, and job-based crossover operator are applied to the process sorting vector, and the reallocation operator, two-point crossover operator, and multi-point crossover operator are applied to the machine selection vector.

[0009] Further, the operation steps of the job-based crossover operator, two-point crossover operator and multi-point crossover operator are as follows: The operation process of the job-based crossover operator is to obtain individual pop 1 and individuals pop 2 The process sorting vector is represented as a process sequence, and the initialization list C 1 and list C 2 , list C 1 To store the crosspop 1 The process sequence, list C 2 To store the cross pop 2 The process sequence randomly divides all workpieces into two different sets, which are respectively denoted as workpiece sets I 1 and artifact sets I 2 , traverse pop 1 and pop 2 At each position in the process sequence, if pop 1 The current process belongs to I 1 , copy the process to the list C 1 The corresponding position of pop 2 The current process belongs to I 2 , copy the process to the list C 2 The corresponding position of pop 2 Does not belong to I 1 The process is as follows pop 2 The order in which they are copied to the list C 1 For the unoccupied positions in pop 1 Does not belong to I 2 The process is as follows pop 1 The order in which they are copied to the list C 2 The unoccupied positions in the list C 1 The process sequence obtained after the crossover is assigned to pop 1 , the list C 2 The process sequence obtained after the crossover is assigned to pop 2 ; The operation process of the two-point crossover operator is to obtain individual pop 1 and individuals pop 2 The machine selection vector is represented as a machine sequence, initialization list C1 and list C 2 , C 1 To store the cross pop 1 The machine sequence, C 2 To store the cross pop 2 Machine sequence, randomly generate the position on the machine sequence rand 1 and location rand 2 ,Location rand 1 In Location rand 2 The position that was previously at least 1 machine interval away will be pop 1 Medium machine sequence interval [ rand 1 , rand 2 ] on the machine to the list C 1 The corresponding position in pop 2 Medium machine sequence interval [ rand 1 , rand 2 ] on the machine to the list C 2 The corresponding position in pop 1 The machine sequence interval [0, rand 1 ]and[ rand 2 , N -1] on the machine to be copied to the list C 2 The corresponding position in pop 2 The machine sequence interval [0, rand 1 ]and[ rand 2 , N -1] on the machine to be copied to the list C 1 The corresponding position in N Indicates the total number of processes, and lists C 1 The machine sequence obtained after crossover is assigned to pop 1 , the list C 2The machine sequence obtained after crossover is assigned to pop 2 ; The operation process of the multi-point crossover operator is to obtain individual pop 1 and individuals pop 2 The machine selection vector is represented as a machine sequence, which is randomly generated [0, N -1] num , N Indicates the total number of processes, pop 1 and pop 2 conduct num The machine is exchanged for each time, and the position of a machine is randomly selected. pop 1 and pop 2 The machine at the selected position in the machine sequence is exchanged.

[0010] Furthermore, in step 4, the specific process of population update is to combine the evolved evolutionary guided population and knowledge driven population into one population, sort the individuals in the combined population from small to large according to the maximum completion time; clear all individuals in the evolutionary guided population and knowledge driven population; N p1 The number of individuals are added to the evolutionary guide population in sequence, and the combined population N p2 A number of individuals are added to the knowledge-driven population in sequence.

[0011] Furthermore, in step 5, the CP model includes the following constraint set, (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) The variables of the CP model include:i Represents the artifact index; I Represents the set of all artifacts; Representation of workpiece i The number of processes; j Indicates the process index; Representation of workpiece i The process set; l An index representing a loading or unloading task; L Represents a collection of loading and unloading tasks; k Represents the machine index; K represents the set of all machines; r Indicates the robot index; R represents the set of all robots; n Indicates the total number of workpieces; N Indicates the total number of processes; O i,j Representation of workpiece i No. j process; Indicates processable operations O i,j A collection of machines; is the continuous decision variable for the maximum completion time, For workpiece i No. The interval variable of the unloading task of each process; For process O i,j The interval variable, For process O i,j In the machine k Optional interval variables for upper processing; For process O i,j The interval variable of the loading or unloading task, when l =1 indicates loading task. l =2 means uninstalling the task. For process O i,j In the machine k Optional interval variable for loading or unloading tasks of upper processing; For process O i,j The interval variable of the offloading task, For process O i,j+1 The interval variable of the loading task; To integrate the process O i,j The interval variables of the loading or unloading tasks and the kOptional interval variables for upper processing; Decision variables for machine sequences, including optional interval variables for assignment ; Decision variables for the robot sequence, including optional interval variables for assignment ; Decision variables for machine sequences, including optional interval variables for assignment and ; For process O i,j The interval variable of the loading task; For process O i,j The interval variable of the unloading task; the constraint set (1) indicates that the goal is to minimize the maximum completion time ,function Returns an interval variable End time of Constraint set (2) represents the process O i,j Only on one eligible machine Processing, function Indicates that in each interval variable Only one optional interval variable can be selected ; Constraint set (3) represents the process O i,j Loading or unloading tasks l Only on one eligible machine Processing, function Indicates that in each interval variable Only one optional interval variable can be selected ; Constraint set (4) represents the workpiece i Each process of O i,j+1 The previous process O i,j After the unloading task is completed, the process O i,j+1 The loading task can start, function Indicates the variable in the loading task interval Can only be used in uninstall task interval variables Start after completion; Constraint set (5) means and Two optional interval variables form an optional interval variable in time order ,function Function represents optional interval variable Involving collections { , } in all current intervals, The start time is the set { , }, and The end time is the set { , }The maximum end time of the optional interval variable; Constraint set (6) indicates that a process can only be processed on one machine. Function Indicates the process O i, j Optional interval variables exist the number of Constraint set (7) represents the same time machine k Only one process can be processed, function Represents all optional interval variables that exist No overlap; Constraint set (8) represents the robot at the same time r Only one loading or unloading task can be performed. Represents all optional interval variables that exist No overlap; Constraint set (9) represents the machine k The processing steps must be carried out in sequence. All optional interval variables exist and do not overlap with each other; Constraint set (10) represents the i Every process, process O i,j The processing start time cannot be in the process O i,j Before the loading task is completed, the function Represents interval variables The start time is no earlier than the interval variable End time of Constraint set (11) represents the workpiece i Every process, process O i,j The start time of the unloading task cannot be in the process O i,j Before the processing end time, function Represents interval variables The start time is not less than the interval variable The end time of .

[0012] The present invention provides a flexible job shop scheduling optimization method with robot constraints, which aims to minimize the maximum completion time. Compared with the prior art, the present invention has the following positive effects: (1) The present invention proposes a new hybrid algorithm, namely, an evolutionary algorithm considering deep Q network assisted by a constraint programming model (EA-DQN-CP), which combines the evolutionary algorithm assisted by a deep Q network (EA-DQN) and the constraint programming (CP) model, firstly uses EA-DQN to efficiently obtain high-quality solutions, and then further optimizes with the help of the CP model to expand the solution space and improve the quality of the solution, thereby obtaining a solution with a smaller maximum completion time; wherein, the population is updated as the result of EA-DQN, and the CP-assisted optimization is used as the result of EA-DQN-CP; (2) In the present invention, an evolution-guided population and a knowledge-driven population are constructed respectively. The evolution-guided population achieves overall optimization of the population through individual competition, self-evolution and co-evolution; the knowledge-driven population reduces the risk of falling into local optimality through individual pairing and DQN evolution. At the same time, six search operators are designed to comprehensively utilize their advantages to minimize the maximum completion time of the obtained solution. (3) The present invention adopts the CP model applicable to the flexible job shop scheduling problem with robot constraints. Compared with the MILP model, the CP model can more comprehensively explore the solution space, more comprehensively consider various constraints in the problem, and more effectively handle the complex relationship of job shop scheduling; In summary, the present invention effectively solves the problem of flexible job shop scheduling with robot constraints. By designing the EA-DQN-CP algorithm, i.e., an evolutionary algorithm considering deep Q network assisted by a constraint programming model, the process sequence of workpieces, machine selection, and the loading and unloading task sequence of robots are reasonably arranged, which has the positive effect of improving the utilization rate of machines and thus improving the resource utilization efficiency of the entire workshop production. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart for realizing the present invention; Figure 2 An example diagram of an individual process ordering vector and a machine selection vector of the present invention; Figure 3 A diagram of the evolutionary process of the evolutionary guide population of the present invention; Figure 4 A diagram of the evolutionary process of the knowledge-driven population of the present invention; Figure 5 This is a diagram of the connection mechanism between the CP model of the present invention and EA-DQN. DETAILED DESCRIPTION

[0014] like Figure 1As shown, the optimization method for flexible job shop scheduling with robot constraints provided by the present invention is mainly implemented through the following process.

[0015] Step 1: Initialize parameters and set the size of the evolutionary guide population N p1 , knowledge-driven population size N p2 and total running time t .

[0016] Step 2: Population initialization: randomly generate the initial population, including the evolutionary guided population and the knowledge driven population. Specifically, the population initialization process is to loop initialize individuals, including the evolutionary guided population looping from 0 to N p1 Iterate, and the knowledge-driven population goes through a cycle from 0 to N p2 Iterate, and create an individual in each iteration; the individual is composed of two vectors, namely the process sorting vector and the machine selection vector. The example of the process sorting vector and the machine selection vector of the individual is shown in the figure Figure 2 As shown, O i,j Representation of workpiece i No. j The number sequence on the process sorting vector represents the processing order of the processes. For example, the first number 1 represents the first process of workpiece 1. O 1,1 The second number 2 indicates the first process of workpiece 2 O 2,1 , the third number 3 represents the first process of workpiece 3 O 3,1 , the fourth number 1 represents the second process of workpiece 1 O 1,2 The numbers on the machine selection vector represent the machine selected for processing in each process. For example, the first number 2 represents the first process of workpiece 1. O 1,1 Processing is performed on machine 2. The second number 4 indicates the second process of workpiece 1. O 1,2 Processing is performed on machine 4. The third number 3 indicates the first process of workpiece 2. O 2,1 Processing is performed on machine 3. The fourth number 3 indicates the second process of workpiece 2. O 2,2 Processing is performed on machine 3. The process sorting vector is initialized. The length of the process sorting vector is the total number of processes. The processes in the process sorting vector are in the range [0, n -1] is randomly generated within the range, nis the total number of workpieces. The number of times each workpiece appears in the process sorting vector is consistent with the number of processes contained in the workpiece. The machine selection vector is initialized. The length of the machine selection vector is the total number of processes. The machines in the machine selection vector are in [0, M -1] is randomly generated within the range, M is the total number of machines. If the processing time of the process on the selected machine is 0, the machines in the machine selection vector are in the range [0, M -1] until the processing time of the process on the selected machine is not 0; after completing the individual initialization, the generated N p1 The number of individuals is added to the evolutionary guide population, and the generated N p2 number of individuals are added to the knowledge-driven population.

[0017] Step 3, population evolution, using individual competition strategy, self-evolution and co-evolution to evolve the evolution-guided population, and using individual pairing strategy and DQN evolution to evolve the knowledge-driven population. Specifically, the individual competition strategy is to classify the individuals in the evolution-guided population according to the maximum completion time, and divide the evolution-guided population into winners and losers. The size of the winner and loser populations is N p / 2, sort the individuals in the evolutionary guided population according to their maximum completion time from small to large, N p1 / 2 individuals are assigned to the winner population, and then N p1 / 2 individuals are assigned to the loser population; each individual in the winner population randomly uses one of the exchange operator, reversal operator and reallocation operator to evolve itself, where the exchange operator and reversal operator are applied to the process sorting vector, and the reallocation operator is applied to the machine selection vector; the winner population and the loser population co-evolve by using the crossover operator. The crossover operator is the operation mode for the winner population and the loser population to co-evolve. The crossover operator includes the priority process crossover operator and the uniform crossover operator. An individual is randomly selected from the winner population and the loser population respectively. The process sorting vectors of the two selected individuals use the priority process crossover operator, and the machine selection vector uses the uniform crossover operator. The process sorting vector and the machine selection vector of each individual undergo a crossover operation each until all individuals complete the crossover process. The evolution process of the evolution-guided population is shown in the figure below. Figure 3 Specifically, the operation steps of the exchange operator, the inversion operator, the reallocation operator, the priority process crossover operator and the uniform crossover operator are as follows: The operation process of the exchange operator is to randomly select two different positions from the individual process sorting vector rand 1and location rand 2 , swap the selected position rand 1 and location rand 2 The above process; The operation process of the inversion operator is to randomly select two different positions from the individual process order vector rand 1 and location rand 2 ,Location rand 1 In Location rand 2 The position in the process sorting vector is at least 3 process intervals away from the previous position. rand 1 and location rand 2 Reverse the process sequence between and find the midpoint of the reversal interval ( rand 1 + rand 2 ) / 2, successively exchange the processes symmetrical with the midpoint of the reversal interval; The operation process of the reallocation operator is to randomly select a position in the machine selection vector, and randomly select a machine from the set of selectable machines corresponding to the process at the selected position to replace the original machine; The operation process of the priority process crossover operator is to obtain individual pop 1 and individuals pop 2 The process ordering vector is represented as a process sequence, which is randomly generated [1, n -1] num, n Indicates the total number of artifacts generated num Artifact Sets of Size I 0 , and randomly select from all artifacts num different artifacts to populate the artifact set I 0 In, traverse pop 1 and pop 2 At each position in the process sequence, if we find a workpiece set I 0 The process is then pop 1 and pop 2 Exchange the corresponding processes in; The operation process of the uniform crossover operator is to obtain individualpop 1 and individuals pop 2 The machine selection vector of , where the machine selection vector is represented as a machine sequence, generates a size of N Digital Set B , N Indicates the total number of processes, a digital set B Each position in the random filling is 0 or 1, 0 and 1 are used to determine pop 1 and pop 2 The flag of whether the machine at the corresponding position in the machine sequence is exchanged, where 0 represents exchange and 1 represents no exchange. N Iterate, traverse in order from the first position to the last position pop 1 and pop 2 Each position of the machine sequence and synchronously traverse the number set B The numbers in the corresponding positions in the B If the number at the current position is 0, pop 1 and pop 2 The machine in the machine sequence in the current traversal position is exchanged; The individual pairing strategy is to generate all paired individual pairs from the knowledge-driven population. The paired individual pairs are composed of two different individuals. All individual pairs are combined to form a paired population. DQN evolution is applied to the paired population. DQN selects the search operator that minimizes the maximum completion time based on the process sorting vector and machine selection vector combination information of the individual pairs in the current paired population. The search operator is the operation mode of DQN evolution. The search operators include exchange operator, reversal operator, reallocation operator, job-based crossover operator, two-point crossover operator and multi-point crossover operator. Among them, the exchange operator, reversal operator and job-based crossover operator are applied to the process sorting vector, and the reallocation operator, two-point crossover operator and multi-point crossover operator are applied to the machine selection vector. The evolution process of the knowledge-driven population is shown in the figure below. Figure 4 Specifically, the operation steps of the job-based crossover operator, two-point crossover operator, and multi-point crossover operator are as follows: The operation process of the job-based crossover operator is to obtain individual pop 1 and individuals pop 2 The process sorting vector is represented as a process sequence, and the initialization list C 1 and list C2 , list C 1 To store the cross pop 1 The process sequence, list C 2 To store the cross pop 2 The process sequence randomly divides all workpieces into two different sets, which are respectively denoted as workpiece sets I 1 and artifact sets I 2 , traverse pop 1 and pop 2 At each position in the process sequence, if pop 1 The current process belongs to I 1 , copy the process to the list C 1 The corresponding position of pop 2 The current process belongs to I 2 , copy the process to the list C 2 The corresponding position of pop 2 Does not belong to I 1 The process is as follows pop 2 The order in which they are copied to the list C 1 For the unoccupied positions in pop 1 Does not belong to I 2 The process is as follows pop 1 The order in which they are copied to the list C 2 The unoccupied positions in the list C 1 The process sequence obtained after the crossover is assigned to pop 1 , the list C 2 The process sequence obtained after the crossover is assigned to pop 2 ; The operation process of the two-point crossover operator is to obtain individual pop 1 and individuals pop 2The machine selection vector is represented as a machine sequence, initialization list C 1 and list C 2 , C 1 To store the cross pop 1 The machine sequence, C 2 To store the cross pop 2 Machine sequence, randomly generate the position on the machine sequence rand 1 and location rand 2 ,Location rand 1 In Location rand 2 The position that was previously at least 1 machine interval away will be pop 1 Medium machine sequence interval [ rand 1 , rand 2 ] on the machine to the list C 1 The corresponding position in pop 2 Medium machine sequence interval [ rand 1 , rand 2 ] on the machine to the list C 2 The corresponding position in pop 1 The machine sequence interval [0, rand 1 ]and[ rand 2 , N -1] on the machine to be copied to the list C 2 The corresponding position in pop 2 The machine sequence interval [0, rand 1 ]and[ rand 2 , N -1] on the machine to be copied to the list C 1 The corresponding position in N Indicates the total number of processes, and lists C 1 The machine sequence obtained after crossover is assigned to pop1 , the list C 2 The machine sequence obtained after crossover is assigned to pop 2 ; The operation process of the multi-point crossover operator is to obtain individual pop 1 and individuals pop 2 The machine selection vector is represented as a machine sequence, which is randomly generated [0, N -1] num , N Indicates the total number of processes, pop 1 and pop 2 conduct num The machine is exchanged for each time, and the position of a machine is randomly selected. pop 1 and pop 2 The machine at the selected position in the machine sequence is exchanged.

[0018] Step 4: Population update: combine the evolved evolutionary guided population and knowledge-driven population into one population, use the combined population to update the evolutionary guided population and knowledge-driven population, and use the updated population as the result after EA-DQN. Specifically, combine the evolved evolutionary guided population and knowledge-driven population into one population, sort the individuals in the combined population from small to large according to the maximum completion time; clear all individuals in the evolutionary guided population and knowledge-driven population; and replace the former in the combined population with the latter. N p1 The number of individuals are added to the evolutionary guide population in sequence, and the combined population N p2 A number of individuals are added to the knowledge-driven population in sequence.

[0019] Step 5, CP-assisted optimization. If the optimization conditions are met, a CP model is constructed. The individual with the smallest maximum completion time after the population update is used as the initial solution in the CP model. The global search capability of the CP model is used for further optimization. The CP-assisted optimization is used as the result of EA-DQN-CP. If the optimization conditions are not met, return to step 3. The connection mechanism diagram of the CP model and EA-DQN is shown in the figure. Figure 5 As shown. Among them, the optimization condition is that the running time after the population update is equal to the total running time t The CP model contains the following set of constraints: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) The variables of the CP model include: i Represents the artifact index; I Represents the set of all artifacts; Representation of workpiece i The number of processes; j Indicates the process index; Representation of workpiece i The process set; l An index representing a loading or unloading task; L Represents a collection of loading and unloading tasks; k Represents the machine index; K represents the set of all machines; r Indicates the robot index; R represents the set of all robots; n Indicates the total number of workpieces; N Indicates the total number of processes; O i,j Representation of workpiece i No. j process; Indicates processable operations O i,j A collection of machines; is the continuous decision variable for the maximum completion time, For workpiece i No. The interval variable of the unloading task of each process; For process O i,j The interval variable, For process O i,j In the machine k Optional interval variables for upper processing; For process O i,j The interval variable of the loading or unloading task, when l =1 indicates loading task.l =2 means uninstalling the task. For process O i,j In the machine k Optional interval variable for loading or unloading tasks of upper processing; For process O i,j The interval variable of the offloading task, For process O i,j+1 The interval variable of the loading task; To integrate the process O i,j The interval variables of the loading or unloading tasks and the k Optional interval variables for upper processing; Decision variables for machine sequences, including optional interval variables for assignment ; Decision variables for the robot sequence, including optional interval variables for assignment ; Decision variables for machine sequences, including optional interval variables for assignment and ; For process O i,j The interval variable of the loading task; For process O i,j The interval variable of the offloading task; Constraint set (1) indicates that the goal is to minimize the maximum completion time ,function Returns an interval variable End time of Constraint set (2) represents the process O i,j Only on one eligible machine Processing, function Indicates that in each interval variable Only one optional interval variable can be selected ; Constraint set (3) represents the process O i,j Loading or unloading tasks l Only on one eligible machine Processing, function Indicates that in each interval variable Only one optional interval variable can be selected ; Constraint set (4) represents the workpiece i Each process ofO i,j+1 The previous process O i,j After the unloading task is completed, the process O i,j+1 The loading task can start, function Indicates the variable in the loading task interval Can only be used in uninstall task interval variables Start after completion; Constraint set (5) means and Two optional interval variables form an optional interval variable in time order ,function Function represents optional interval variable Involving collections { , } in all current intervals, The start time is the set { , }, and The end time is the set { , }The maximum end time of the optional interval variable; Constraint set (6) indicates that a process can only be processed on one machine. Function Indicates the process O i, j Optional interval variables exist the number of Constraint set (7) represents the same time machine k Only one process can be processed, function Represents all optional interval variables that exist No overlap; Constraint set (8) represents the robot at the same time r Only one loading or unloading task can be performed. Represents all optional interval variables that exist No Overlap: Constraint set (9) represents the machine k The processing steps must be carried out in sequence. All optional interval variables exist and do not overlap with each other; Constraint set (10) represents the i Every process, process O i,j The processing start time cannot be in the process O i,jBefore the loading task is completed, the function Represents interval variables The start time is no earlier than the interval variable End time of Constraint set (11) represents the workpiece i Every process, process O i,j The start time of the unloading task cannot be in the process O i,j Before the processing end time, function Represents interval variables The start time is not less than the interval variable The end time of .

[0020] Step 6: Check the termination condition. If the termination condition is met, output the final solution. The termination condition is that the running time after CP-assisted optimization is equal to the total running time. t .

[0021] The present invention is further described below by using a specific example.

[0022] The present invention was run on a computer equipped with an Intel® Core™ i7-12700 processor and an NVIDIA T600 GPU. Python was used to code in the PyCharm 2024.1.1 environment, and the CP and CPLEX solvers were provided by IBM CPLEX StudioIDE 12.7.1. Total running time t Set to 2 N Second( N is the total number of processes in the corresponding instance), and all compared algorithms are executed 10 times on each instance.

[0023] The present invention is based on the commonly used benchmark instances MFJS01-MFJS10 and MK01-MK10 in FJSP, and the instances with robot constraints are named RMFJS01-RMFJS10 and RMK01-RMK10. Each instance contains two robots, each robot is responsible for half of the machine, and the loading and unloading task time is set to 5s. In order to verify the effectiveness of the present invention in a large-scale production environment, the number of workpieces in the instances RMFJS01-RMFJS10 and RMK01-RMK10 is doubled, and new instances RLMFJS01-RLMFJS10 and RLMK01-RLMK10 are created.

[0024] First, the effectiveness of the CP model is verified. The comparison results of the existing mixed integer linear programming model (MILP model) and the constraint programming model (CP model) are shown in Table 1: Table 1 Comparison results between mixed integer linear programming model and constraint programming model ; In Table 1, the number of binary decision variables, continuous decision variables, and constraints are represented by "NB", "NC", and "NCT", respectively. In addition, "NV" represents the number of interval decision variables in the CP model. "Gap" represents the optimal gap, and a value of 0 means that the solution with the minimum maximum completion time is found. "Cmax" represents the solution found within the time limit, and "Time" represents the CPU time. Specifically, if the solution with the minimum maximum completion time is obtained, "Time" does not exceed the time limit; otherwise, it is equal to the time limit.

[0025] According to Table 1, as the instance size increases, the values ​​of NB, NC and NTC in the MILP model, as well as the values ​​of NV and NCT in the CP model, all increase significantly. The MILP model obtains the solution with the minimum maximum completion time for RMFJS01-RMFJS06, and the feasible solutions for RMFJS07-RMFJS10 and RLMFJS01-RLMFJS05. Since the scale of other instances is relatively large, the MILP model cannot find a feasible solution within the time limit. The CP model also obtains the solution with the minimum maximum completion time for RMFJS01-06, which is faster than the MILP model. For the remaining instances, the CP model obtains a solution with a smaller maximum completion time than the MILP model. In summary, the CP model is better than the MILP model.

[0026] In order to prove the effectiveness of the CP-assisted optimization method of the present invention, the evolutionary algorithm considering the deep Q network (EA-DQN-CP) assisted by the constraint programming model proposed in the present invention and the evolutionary algorithm assisted by the deep Q network (EA-DQN) are compared. The comparison results are shown in Table 2: Table 2 Comparison results between EA-DQN and EA-DQN-CP ; For the EA-DQN-CP of the present invention, the optimal configuration of eight important parameters is determined through experiments. These parameters include the evolutionary guided population size N p1 , knowledge-driven population size N p2 , Discount Factor γ , learning rate lr , Exploration Rate ε , target Q network update frequency q , Experience replay buffer size D and batch size BS The present invention executes each parameter configuration 10 times on the RMK10 instance, and determines the optimal parameter configuration based on the experimental results as follows:N p1 =500, N p2 =10, γ =0.9, lr =0.1, ε =0.95, q =15, D =512 and BS =32.

[0027] According to Table 2, "Best" represents the minimum value of the maximum completion time of each instance repeated 10 times, "AVG" represents the average value of the maximum completion time of each instance repeated 10 times, and "Mean" represents the average value of each column. In 40 instances, EA-DQN-CP is better than EA-DQN for both Best and AVG indicators, indicating that there is a significant difference between EA-DQN and EA-DQN-CP. In short, the CP-assisted optimization method enhances the search ability of EA-DQN. By combining the EA-DQN and CP models, EA-DQN-CP takes advantage of the advantages of both EA-DQN and CP models and achieves superior performance.

[0028] The CP model, EA-DQN and EA-DQN-CP methods proposed in this invention are compared with the existing literature algorithms IGA (Improved Genetic Algorithm) and QABC (Artificial Bee Colony Algorithm based on Q Learning). The comparison results are shown in Table 3: Table 3 Comparison results of IGA, QABC, CP, EA-DQN, and EA-DQN-CP ; For QABC, the population size, iteration limit, learning rate and discount factor are set to 80, 20, 0.8 and 0.1 respectively. For IGA, the population size is 300, the crossover probability is 0.8, the mutation probability is 0.1, and the diversity check generation is 300. In Table 3, "LB" represents the result with the smallest maximum completion time in the algorithm, and "ALB" represents the result with the smallest average maximum completion time in the algorithm. As shown in Table 3, in 40 instances, EA-DQN obtained 1 minimum value in Best. CP obtained 21 minimum values ​​in Best and 22 minimum values ​​in AVG. EA-DQN-CP obtained 36 minimum values ​​in Best and 35 minimum values ​​in AVG. It can be seen that for the Best and AVG indicators, the CP model, EA-DQN and EA-DQN-CP methods proposed in the present invention are superior to the existing literature algorithms QABC and IGA.

[0029] In summary, the CP model, EA-DQN, and EA-DQN-CP are effective methods for solving FJSP-RC. The CP model explores the complete solution space through its advanced search technology, EA-DQN uses the advantages of EA and DQN algorithms to achieve learning and evolution, and EA-DQN-CP uses the advantages of CP model, EA, and DQN algorithms, resulting in excellent performance.

Claims

1. A flexible job shop scheduling optimization method with robot constraints, characterized in that: The following steps are included: Step 1: Initialize parameters and set the size of the evolutionary guide population N p1 , knowledge-driven population size N p2 and total running time t ; Step 2: Population initialization: randomly generate the initial population, including the evolutionary guided population and the knowledge driven population; Step 3: Population evolution: using individual competition strategy, self-evolution and co-evolution to evolve the evolution-guided population, and using individual pairing strategy and DQN evolution to evolve the knowledge-driven population; Step 4, population update, combining the evolved evolutionary guided population and knowledge driven population into one population, and using the combined population to update the evolutionary guided population and knowledge driven population; Step 5, CP-assisted optimization. If the optimization conditions are met, a CP model is constructed. The individual with the smallest maximum completion time after the population update is used as the initial solution in the CP model. The global search capability of the CP model is used for further optimization. If the optimization conditions are not met, return to step 3, where the optimization condition is that the running time after the population update is equal to the total running time. t half of; Step 6: Check the termination condition. If the termination condition is met, output the final solution. The termination condition is that the running time after CP-assisted optimization is equal to the total running time. t .

2. The method for optimizing flexible job shop scheduling with robot constraints according to claim 1, characterized in that: In step 2, the population initialization process is to loop initialization of individuals, including evolutionary guidance of the population through loops from 0 to N p1 Iterate, and the knowledge-driven population goes through a cycle from 0 to N p2 Iterate, and create an individual in each iteration; the individual is composed of two vectors, namely the process sorting vector and the machine selection vector; the process sorting vector is initialized, and the length of the process sorting vector is the total number of processes. The processes in the process sorting vector are in [0, n -1] is randomly generated within the range, n is the total number of workpieces. The number of times each workpiece appears in the process sorting vector is consistent with the number of processes contained in the workpiece. The machine selection vector is initialized. The length of the machine selection vector is the total number of processes. The machines in the machine selection vector are in the range [0, M -1] is randomly generated within the range, M is the total number of machines. If the processing time of the process on the selected machine is 0, the machines in the machine selection vector are in the range [0, M -1] until the processing time of the process on the selected machine is not 0; after completing the individual initialization, the generated N p1 The number of individuals is added to the evolutionary guide population, and the generated N p2 number of individuals are added to the knowledge-driven population.

3. The method for optimizing flexible job shop scheduling with robot constraints according to claim 2, characterized in that: The individual competition strategy is to classify the individuals in the evolutionary guided population according to the maximum completion time, and divide the evolutionary guided population into a winner population and a loser population. The sizes of the winner population and the loser population are N p / 2, sort the individuals in the evolutionary guided population according to their maximum completion time from small to large, N p1 / 2 number of individuals are assigned to the winner population, and then N p1 / 2 individuals are assigned to the loser population; each individual in the winner population randomly uses one of the exchange operator, reversal operator and reallocation operator to self-evolve, where the exchange operator and reversal operator are applied to the process sorting vector, and the reallocation operator is applied to the machine selection vector; the winner population and the loser population co-evolve by using the crossover operator, which includes the priority process crossover operator and the uniform crossover operator. An individual is randomly selected from the winner population and the loser population respectively, and the process sorting vectors of the two selected individuals use the priority process crossover operator, and the machine selection vector uses the uniform crossover operator. The process sorting vector and machine selection vector of each individual are crossovered once until all individuals have completed the crossover process.

4. The method for optimizing flexible job shop scheduling with robot constraints according to claim 3 is characterized in that: The operation steps of the exchange operator, inversion operator, reallocation operator, priority process crossover operator and uniform crossover operator are as follows: The operation process of the exchange operator is to randomly select two different positions from the individual process sorting vector rand 1 and location rand 2 , swap the selected position rand 1 and location rand 2 The above process; The operation process of the inversion operator is to randomly select two different positions from the individual process order vector rand 1 and location rand 2 ,Location rand 1 In Location rand 2 The position in the process sorting vector is at least 3 process intervals away from the previous position. rand 1 and location rand 2 Reverse the process sequence between and find the midpoint of the reversal interval ( rand 1 + rand 2 ) / 2, successively exchange the processes symmetrical with the midpoint of the reversal interval; The operation process of the reallocation operator is to randomly select a position in the machine selection vector, and randomly select a machine from the set of selectable machines corresponding to the process at the selected position to replace the original machine; The operation process of the priority process crossover operator is to obtain individual pop 1 and individuals pop 2 The process ordering vector is represented as a process sequence, which is randomly generated [1, n -1] num,n Indicates the total number of workpieces generated num Artifact Sets of Size I 0 , and randomly select from all artifacts num different artifacts to populate the artifact set I 0 In, traverse pop 1 and pop 2 At each position in the process sequence, if we find a workpiece set I 0 The process is then pop 1 and pop 2 Exchange the corresponding processes in; The operation process of the uniform crossover operator is to obtain individual pop 1 and individuals pop 2 The machine selection vector of , where the machine selection vector is represented as a machine sequence, generates a size of N Digital Set B , N Indicates the total number of processes, a digital set B Each position in the random filling is 0 or 1, 0 and 1 are used to determine pop 1 and pop 2 The flag of whether the machine at the corresponding position in the machine sequence is exchanged, where 0 represents exchange and 1 represents no exchange. N Iterate, traverse in order from the first position to the last position pop 1 and pop 2 Each position of the machine sequence and synchronously traverse the number set B The corresponding position of the number in the B If the current position is 0, pop 1 and pop 2 Exchange the machine at the current traversal position in the machine sequence.

5. The method for optimizing flexible job shop scheduling with robot constraints according to claim 4, characterized in that: The individual pairing strategy is to generate all paired individual pairs from the knowledge-driven population. The paired individual pairs consist of two different individuals. All individual pairs are combined together to form a paired population. DQN evolution is applied to the paired population. DQN selects the search operator that minimizes the maximum completion time according to the combined information of the process ranking vector and the machine selection vector of the individual pairs in the current paired population. The search operators include exchange operator, reversal operator, reallocation operator, job-based crossover operator, two-point crossover operator and multi-point crossover operator. Among them, the exchange operator, reversal operator and job-based crossover operator are applied to the process ranking vector, and the reallocation operator, two-point crossover operator and multi-point crossover operator are applied to the machine selection vector.

6. The method for optimizing flexible job shop scheduling with robot constraints according to claim 5, characterized in that: The operation steps of the job-based crossover operator, two-point crossover operator and multi-point crossover operator are as follows: The operation process of the job-based crossover operator is to obtain individual pop 1 and individuals pop 2 The process sorting vector is represented as a process sequence, and the initialization list C 1 and list C 2 , list C 1 To store the cross pop 1 The process sequence, list C 2 To store the cross pop 2 The process sequence randomly divides all workpieces into two different sets, which are respectively denoted as workpiece sets I 1 and artifact sets I 2 , traverse pop 1 and pop 2 At each position in the process sequence, if pop 1 The current process belongs to I 1 , copy the process to the list C 1 The corresponding position of pop 2 The current process belongs to I 2 , copy the process to the list C 2 The corresponding position of pop 2 Does not belong to I 1 The process is as follows pop 2 The order in which they are copied to the list C 1 For the unoccupied positions in pop 1 Does not belong to I 2 The process is as follows pop 1 The order in which they are copied to the list C 2 The unoccupied positions in the list C 1 The process sequence obtained after the crossover is assigned to pop 1 , the list C 2 The process sequence obtained after the crossover is assigned to pop 2 ; The operation process of the two-point crossover operator is to obtain individual pop 1 and individuals pop 2 The machine selection vector is represented as a machine sequence, initialization list C 1 and list C 2 , C 1 To store the cross pop 1 The machine sequence, C 2 To store the cross pop 2 Machine sequence, randomly generate the position on the machine sequence rand 1 and location rand 2 ,Location rand 1 In Location rand 2 The position that was previously at least 1 machine interval away will be pop 1 Medium machine sequence interval [ rand 1 , rand 2 ] on the machine to the list C 1 The corresponding position in pop 2 Medium machine sequence interval [ rand 1 , rand 2 ] on the machine to the list C 2 The corresponding position in pop 1 The machine sequence interval [0, rand 1 ]and[ rand 2 , N -1] on the machine to be copied to the list C 2 The corresponding position in pop 2 The machine sequence interval [0, rand 1 ]and[ rand 2 , N -1] on the machine to be copied to the list C 1 The corresponding position in N Indicates the total number of processes, and lists C 1 The machine sequence obtained after crossover is assigned to pop 1 , the list C 2 The machine sequence obtained after crossover is assigned to pop 2 ; The operation process of the multi-point crossover operator is to obtain individual pop 1 and individuals pop 2 The machine selection vector is represented as a machine sequence, which is randomly generated [0, N -1] num , N Indicates the total number of processes, pop 1 and pop 2 conduct num The machine is exchanged for each time, and the position of a machine is randomly selected. pop 1 pop 2 The machine at the selected position in the machine sequence is exchanged.

7. The method for optimizing flexible job shop scheduling with robot constraints according to claim 6, characterized in that: The process of population updating is to combine the evolved evolution-guided population and the knowledge-driven population into one population, and sort the individuals in the combined population from small to large according to the maximum completion time; Clear all individuals in the evolutionary guided population and the knowledge driven population; N p1 The number of individuals are added to the evolutionary guide population in sequence, and the combined population N p2 A number of individuals are added to the knowledge-driven population in sequence.

8. The method for optimizing flexible job shop scheduling with robot constraints according to claim 7, characterized in that: The CP model contains the following set of constraints, （1） （2） （3） （4） （5） （6） （7） （8） （9） （10） （11） The variables of the CP model include: i Represents the artifact index; I Represents the set of all artifacts; Representation of workpiece i The number of processes; j Indicates the process index; Representation of workpiece i The process set; l An index representing a loading or unloading task; L Represents a collection of loading and unloading tasks; k Represents the machine index; K represents the set of all machines; r Indicates the robot index; R represents the set of all robots; n Indicates the total number of workpieces; N Indicates the total number of processes; O i,j Representation of workpiece i No. j process; Indicates processable operations O i,j A collection of machines; is the continuous decision variable for the maximum completion time, For workpiece i No. The interval variable of the unloading task of each process; For process O i,j The interval variable, For process O i,j In the machine k Optional interval variables for upper processing; For process O i,j The interval variable of the loading or unloading task, when l =1 indicates loading task. l =2 means uninstalling the task. For process O i,j In the machine k Optional interval variable for loading or unloading tasks of upper processing; For process O i,j The interval variable of the offloading task, For process O i,j+1 The interval variable of the loading task; To integrate the process O i,j The interval variables of the loading or unloading tasks and the k Optional interval variables for upper processing; Decision variables for machine sequences, including optional interval variables for assignment ; Decision variables for the robot sequence, including optional interval variables for assignment ; Decision variables for machine sequences, including optional interval variables for assignment and ; For process O i,j The interval variable of the loading task; For process O i,j The interval variable of the offloading task; Constraint set (1) indicates that the goal is to minimize the maximum completion time ,function Returns an interval variable End time of Constraint set (2) represents the process O i,j Only on one eligible machine Processing, function Indicates that in each interval variable Only one optional interval variable can be selected ; Constraint set (3) represents the process O i,j Loading or unloading tasks l Only on one eligible machine Processing, function Indicates that in each interval variable Only one optional interval variable can be selected ; Constraint set (4) represents the workpiece i Each process of O i,j+1 The previous process O i,j After the unloading task is completed, the process O i,j+1 The loading task can start, function Indicates the variable in the loading task interval Can only be used in uninstall task interval variables Start after completion; Constraint set (5) means and Two optional interval variables form an optional interval variable in time order ,function Function represents optional interval variable Involving collections { , } in all current intervals, The start time is the set { , }, and The end time is the set The maximum end time of the optional interval variable in; Constraint set (6) indicates that a process can only be processed on one machine. Function Indicates the process O i, j Optional interval variables exist the number of Constraint set (7) represents the same time machine k Only one process can be processed, function Represents all optional interval variables that exist No overlap; Constraint set (8) represents the robot at the same time r Only one loading or unloading task can be performed. Represents all optional interval variables that exist No overlap; Constraint set (9) represents the machine k The processing steps must be carried out in sequence. All optional interval variables exist and They do not overlap with each other; Constraint set (10) represents the i Every process, process O i,j The processing start time cannot be in the process O i,j Before the loading task is completed, the function Represents interval variables The start time is no earlier than the interval variable End time of Constraint set (11) represents the workpiece i Every process, process O i,j The start time of the unloading task cannot be in the process O i,j Before the processing end time, function Represents interval variables The start time is not less than the interval variable The end time of .

Images