A multi-agent flexible job shop autonomous scheduling method

Abstract

The present invention discloses a multi-agent flexible job shop autonomous scheduling method, which belongs to the field of production scheduling and overcomes the shortcomings of a single scheduling algorithm. The implementation of the present invention includes three stages. In the preparation stage, flexible job shop scheduling algorithms are collected, and scheduling programs are written to implement the scheduling functions described by the algorithms; scheduling agent templates are constructed for each scheduling program, and these templates are saved in a template warehouse. In the application stage, a management agent is created to receive scheduling parameters, determine the required scheduling algorithms and the number of scheduling agents; the management agent calls the corresponding template from the template warehouse to create a scheduling agent. After the scheduling agent completes the calculation, it submits the obtained scheduling plan to the management agent, which selects the optimal plan and submits it to the user. In the cleanup stage, the management agent destroys all scheduling agents. The present invention can effectively utilize the concurrent computing characteristics of multiple intelligences to improve the speed and quality of solving scheduling problems.

Description

Technical Field

[0001] The invention belongs to the field of production scheduling, and in particular relates to an autonomous scheduling method for a multi-agent flexible job shop. Background Art

[0002] Usually, a production task includes multiple processes, and the multiple processes belonging to the same production task must be processed in a specified order, but there is no processing order requirement between the processes belonging to different production tasks. In addition, a device must process the multiple processes assigned to it one by one, but the device can process these processes in different orders. Under the premise of satisfying these constraints, how to optimize the production goal (for example, minimize the maximum completion time) is the problem that production scheduling needs to solve.

[0003] The production scheduling problem is an NP-hard combinatorial optimization problem, that is, all processes belonging to a production task set can have different combinations under the premise of satisfying the constraints; and usually different combinations have different performances, that is, the values ​​of their production targets are different. For real production scenarios, it is difficult to determine the optimal solution because the scale of the combinations is large and difficult to enumerate. In order to achieve a balance between performance and efficiency, as a compromise solution, the use of search algorithms to find a suboptimal solution (also called a non-inferior solution) within a limited time has become a commonly used method in industry.

[0004] At present, the academic community has proposed a variety of scheduling algorithms, but these algorithms have different characteristics, and usually an algorithm only performs well on a part of the problem, so there is no obvious difference between them. In addition, it is difficult to accurately and meticulously classify scheduling problems, so it is difficult to establish an optimal matching relationship between algorithms and problems. These problems cause great trouble in the selection of algorithms and are a key problem that needs to be solved in the field of production scheduling. Zheng Xudong (Zheng Xudong. Research on workshop scheduling system based on multi-agent [D]. Shanghai Jiaotong University, 2007.) defines four types of agents, namely workshop scheduling agents, task allocation agents, workshop resource agents and auction agents. Agents collaborate based on auctions to complete task allocation. Dai Tao defines six types of agents, namely management agents, material feeding agents, workpiece agents, equipment agents, transportation agents and personnel agents. The bidding and reward and punishment system is used to coordinate the behavior between agents, and the ant colony algorithm is combined with reinforcement learning to achieve production scheduling. Zhao Wei (Zhao Wei. Production scheduling method based on multi-agent and its application [D]. Zhejiang University of Technology, 2004.) defined three types of agents, namely manager agents, task agents and resource agents. The agents collaborate based on the contract network protocol and realize production scheduling by combining genetic algorithms and neural network algorithms. The scheduling algorithms used in the above three comparative documents are limited, and the coordination mechanism between agents is relatively complex, and the production scenarios that can be adapted are limited. The present invention can comprehensively use all scheduling algorithms, and can flexibly increase or decrease scheduling algorithms, support users to flexibly select algorithms, and can adapt to most production scenarios. Moreover, the coordination mechanism of the present invention is simple, and only simple information transmission is required between the management agent and the scheduling agent, while multiple scheduling agents are independent and do not need to collaborate. Summary of the invention

[0005] In order to overcome the defects and shortcomings of the prior art, the present invention provides a multi-agent flexible job shop autonomous scheduling method to effectively utilize the concurrent computing characteristics of multiple intelligences and simultaneously execute multiple scheduling algorithms or multiple versions of the same scheduling algorithm to overcome the shortcomings of a single scheduling algorithm and improve the speed and quality of solving scheduling problems.

[0006] The present invention is achieved by at least one of the following technical solutions.

[0007] A multi-agent flexible job shop autonomous scheduling method comprises the following steps:

[0008] In the preparation stage, the flexible job shop scheduling algorithms are collected, combined with the scheduling program, a corresponding scheduling agent template is constructed for each scheduling algorithm, and the scheduling agent template is saved in the template warehouse;

[0009] In the application phase, a management agent is created to receive scheduling parameters, determine the required scheduling algorithm and the number of scheduling agents;

[0010] The management agent calls the scheduling agent template corresponding to the scheduling algorithm from the template warehouse to create a scheduling agent;

[0011] After the scheduling agent completes the calculation, it submits the obtained scheduling plan to the management agent, which selects the best plan and submits it to the user;

[0012] In the cleanup phase, the management agent destroys all scheduling agents.

[0013] Furthermore, the scheduling algorithm includes an algorithm based on heuristic rules, an integer programming algorithm, a genetic algorithm, a neighborhood search algorithm, a taboo search algorithm, a particle swarm algorithm, an ant colony algorithm, a deep learning algorithm, a reinforcement learning algorithm, a deep reinforcement learning algorithm, and a hybrid algorithm formed by combining two or more of the above algorithms.

[0014] Furthermore, the scheduling agent template is a Docker container image, the template repository is a Docker container image repository, and the scheduling agent is a Docker container process that executes a scheduling algorithm.

[0015] Furthermore, the management agent is used to receive the user's configuration information, create a scheduling agent, receive the scheduling plan submitted by the scheduling agent, select the optimal scheduling plan, and return the optimal scheduling plan to the user.

[0016] Furthermore, the creation of the scheduling agent refers to the management agent calling the creation instruction of the Docker daemon process, and the Docker daemon process creates the Docker container process corresponding to the scheduling agent according to the scheduling agent template specified by the management agent.

[0017] Further, multiple scheduling agents created according to different scheduling agent templates execute different scheduling algorithms;

[0018] Multiple scheduling agents created based on the same scheduling agent template use different parameter values ​​to execute the same scheduling algorithm. The parameter values ​​are set by the user using the human-computer interaction interface provided by the management agent.

[0019] Further, the determination of the required scheduling algorithm and the number of scheduling agents is specified by the user on a graphical human-computer interaction interface provided by the management agent;

[0020] The graphical human-computer interaction interface is generated according to the configuration file;

[0021] The configuration file uses a TXT file to sequentially list the names of the scheduling algorithms, the scheduling targets supported by each algorithm and the input parameters required;

[0022] The graphical human-computer interaction interface is changed by modifying the configuration file.

[0023] Furthermore, the scheduling parameters include scheduling targets and production task information;

[0024] The scheduling objectives include minimizing the maximum completion time, minimizing the maximum machine load, minimizing the total machine load, minimizing the lead time / delay, minimizing the production cost, minimizing the energy consumption, minimizing the carbon emission, or simultaneously including two or more of the above objectives;

[0025] The production task information includes the number of production tasks, the required completion time of each production task, the processes included in each production task, the processing order of the processes, the set of equipment that can process each process, and the time required for the equipment to process each process.

[0026] Furthermore, the scheduling scheme determines the processing equipment, processing start time and processing end time required for each process of each task;

[0027] The optimal solution refers to the scheduling solution that can best meet the scheduling target requirements;

[0028] Furthermore, the destroying of all scheduling agents refers to the management agent calling the destroy instruction of the Docker daemon process, and the Docker daemon process destroys the Docker container process corresponding to the scheduling agent.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] (1) The present invention can adapt to a variety of scheduling problems, make full use of computing resources, and quickly generate optimized scheduling solutions;

[0031] (2) The scheduling system developed according to the present invention is easy to use, has a graphical human-computer interaction interface, and does not require manual matching of algorithms and problems;

[0032] (3) The technical architecture of the present invention is flexible and can be flexibly expanded. It supports flexible selection of algorithm types and specified number of agents. It supports adding new scheduling algorithms and optimization goals, and also supports deleting or modifying existing scheduling algorithms and optimization goals. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flowchart of an implementation of a multi-agent flexible job shop autonomous scheduling method of the present invention, used to illustrate the main steps of the method;

[0034] Figure 2A schematic diagram of the appearance of the graphical human-computer interaction interface provided for the management agent, used to illustrate the main content of human-computer interaction;

[0035] Figure 3 It is a Gantt chart of the scheduling results of the production task instance, which is used to illustrate the main contents of the scheduling results. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] The present invention provides a multi-agent flexible job shop autonomous scheduling method, which includes the following three stages:

[0038] Phase 1, the preparation phase, includes the following steps:

[0039] Step 1: Collect flexible job shop scheduling algorithms, including algorithms based on heuristic rules, integer programming algorithms, genetic algorithms, neighborhood search algorithms, taboo search algorithms, particle swarm algorithms, ant colony algorithms, deep learning algorithms, reinforcement learning algorithms, deep reinforcement learning algorithms, and hybrid algorithms formed by combining two or more of the above algorithms;

[0040] Step 2: Write a scheduling program to implement the scheduling function of the algorithm;

[0041] Step 3: Combined with the scheduling program, a scheduling agent template is constructed for each scheduling algorithm, wherein the scheduling agent template is a Docker container image;

[0042] Step 4: Save these scheduling agent templates to the template repository, which is a Docker container image repository.

[0043] Phase 2, the application phase, includes the following steps:

[0044] Step 5: Create a management agent to receive scheduling parameters, determine the required scheduling algorithm and the number of scheduling agents. The management agent is located between the user and the scheduling agent at the functional level and is used to receive the user's configuration information, create a scheduling agent, receive the scheduling plan submitted by the scheduling agent, select the optimal scheduling plan, and return the optimized scheduling plan to the user;

[0045] The user refers to a person who directly uses the method of the present invention to obtain a scheduling solution;

[0046] The scheduling parameters include scheduling targets and production task information;

[0047] The scheduling objectives include minimizing the maximum completion time, minimizing the maximum machine load, minimizing the total machine load, minimizing the lead time / delay, minimizing the production cost, minimizing the energy consumption, minimizing the carbon emission, or simultaneously including two or more of the above objectives;

[0048] The production task information includes the number of production tasks, the required completion time (i.e., delivery date) of each production task, the processes included in each production task, the processing order of the processes, the set of equipment that can process each process, and the time required for the equipment to process each process;

[0049] The production task information is provided in a TXT file format, and is saved in the format of "task number, delivery date, process number, machine number, processing time" and can be viewed and edited using a TXT file editor;

[0050] The task number is the unique identification of the task, the process number is the unique identification of the process, and the machine number is the unique identification of the machine;

[0051] Step 6: The management agent calls the scheduling agent template corresponding to the scheduling algorithm from the template warehouse to create a scheduling agent, where the scheduling agent is a Docker container process that executes the scheduling algorithm.

[0052] The creation of the scheduling agent refers to the management agent calling the creation instruction of the Docker daemon process, and the Docker daemon process creates the Docker container process corresponding to the scheduling agent according to the scheduling agent template specified by the management agent;

[0053] Multiple scheduling agents are created based on different scheduling agent templates to execute different scheduling algorithms;

[0054] Multiple scheduling agents created based on the same scheduling agent template use different parameter values ​​to execute the same scheduling algorithm. The parameter values ​​are set by the user using the human-computer interaction interface provided by the management agent.

[0055] The required scheduling algorithm and number of scheduling agents are determined by the user specifying them on a graphical human-computer interaction interface provided by the management agent;

[0056] The designation is to tick the required algorithm and fill in the quantity and parameters;

[0057] The graphical human-computer interaction interface is generated according to the configuration file;

[0058] The configuration file uses a TXT file to sequentially list the names of the scheduling algorithms, the scheduling targets supported by each algorithm and the input parameters required;

[0059] By modifying the configuration file, the graphical human-computer interaction interface can be changed;

[0060] Step 7: After the scheduling agent completes the calculation, it submits the obtained scheduling plan to the management agent. The scheduling plan determines the processing equipment, processing start time and processing end time required for each process of each task;

[0061] Step 8: The management agent selects the optimal solution, where the optimal solution refers to the scheduling solution that best meets the scheduling target requirements;

[0062] Step 9: The management agent submits the optimal solution to the user;

[0063] The optimal solution can be submitted in two ways: JPG file and TXT file, both of which are saved in the specified directory;

[0064] The JPG file stores the scheduling result Gantt chart, which can be displayed on the graphical human-computer interaction interface provided by the management agent;

[0065] The TXT file stores the scheduling result string, which is saved in the format of "task number, completion time, process number, machine number, start time, end time" and can be viewed and edited using a TXT file editor.

[0066] Phase 3, the cleanup phase, includes the following steps:

[0067] Step 10: The management agent destroys all scheduling agents. Destroying all scheduling agents means that the management agent calls the destruction instruction of the Docker daemon process, and the Docker daemon process destroys the Docker container process corresponding to the scheduling agent.

[0068] Step 11: The management agent enters a waiting state, waiting for the user to initiate a new scheduling solution request.

[0069] Example 1

[0070] This embodiment provides a multi-agent flexible job shop autonomous scheduling method, such as Figure 1 As shown, this embodiment compiles corresponding scheduling programs for the ant colony algorithm, genetic algorithm and reinforcement learning algorithm, namely, the ant colony algorithm scheduling program, the genetic algorithm scheduling program and the reinforcement learning algorithm scheduling program. Then, based on these programs, scheduling agent templates are constructed, namely, the ant colony algorithm scheduling agent template, the genetic algorithm scheduling agent template and the reinforcement learning algorithm scheduling agent template, and are saved in the scheduling agent template warehouse.

[0071] With these templates, in actual production scheduling applications, users select the scheduling algorithm to be executed through the management agent, specify the optimization goal, set the number of agents corresponding to each algorithm, and set the parameters of each agent. The management agent then creates the required scheduling agent according to the scheduling agent template and executes the corresponding scheduling algorithm.

[0072] Example 2

[0073] This embodiment provides a graphical human-computer interaction interface for managing intelligent agents and their configuration files. Figure 2 As shown in the figure, users can choose to use ant colony algorithm, genetic algorithm and reinforcement learning algorithm through the graphical human-computer interaction interface of the management agent. If you want to use a certain algorithm, you can check the box behind the algorithm and specify the required scheduling optimization goal and the number of agents. In this example, the user needs an ant colony algorithm agent and two reinforcement learning algorithm agents. The optional scheduling optimization goals are minimum maximum completion time, maximum machine load and minimum advance / delay. The actual choice is minimum maximum completion time.

[0074] When the user clicks the corresponding algorithm name, the parameter setting page of the algorithm will pop up. Different parameters can be set for each agent. The parameters required for each algorithm are given in the configuration file, as shown in the following table:

[0075]

[0076] The configuration file lists three algorithms, namely, ant colony algorithm, genetic algorithm and reinforcement learning algorithm, as well as the objectives that each algorithm can optimize and its required parameters. The details are as follows:

[0077] Algorithm 1 is an ant colony algorithm, which can optimize only one goal, that is, the minimum maximum completion time. The algorithm requires setting five parameters, namely, the importance of pheromone, the importance of heuristic factor, the pheromone evaporation coefficient, the number of ants and the number of iterations;

[0078] Algorithm 2 is a genetic algorithm, which can optimize two goals, namely, the minimum maximum completion time and the minimum total machine load. The algorithm requires the setting of four parameters, namely, the population size, the termination evolution number of the genetic algorithm, the crossover probability and the mutation probability;

[0079] Algorithm 3 is a reinforcement learning algorithm. It can optimize two objectives, namely, minimizing the maximum completion time and minimizing the advance / delay. The algorithm requires setting a parameter, namely the discount coefficient.

[0080] When the user clicks the "Load Task" button, a file selection window will pop up, through which the user can select the production task TXT file. When the user clicks the "Start Running" button, the agent will be created and run according to the process shown in Example 1, and will automatically stop running and report the results after the set time arrives. If the user needs to stop the agent running before the time arrives, click the "Stop Running" button.

[0081] After the agent submits the result, the management agent selects an optimal scheduling solution and displays it in the "Scheduling Result Display (Gantt Chart)" area. The Gantt chart is as follows: Figure 3 shown.

[0082] Example 3

[0083] This embodiment provides two production task instance expression methods, one of which is a table format, as shown in the following table:

[0084]

[0085] The production task set includes three tasks, namely J1, J2, and J3, with delivery dates of 20, 15, and 25 time units respectively. Each of these three tasks contains three processes, that is, J1 includes process O 1,1 , O 1,2 With O 1,3 , J2 includes process O 2,1 , O 2,2 With O 2,3 , J3 includes process O 3,1 , O 3,2 With O 3,3 The number of devices that can handle each process varies from one to more. In this example, each process has two devices that can handle it. Due to different equipment performance, different devices that handle the same process often require different times. The details are as follows:

[0086] Process O 1,1 The devices are M1 and M2, and the processing times are 2 and 7 time units respectively;

[0087] Process O 1,2 The devices are M2 and M3, and the processing times are 3 and 6 time units respectively;

[0088] Process O 1,3 The devices are M1 and M3, and the processing times are 7 and 5 time units respectively;

[0089] Process O 2,1 The devices are M1 and M2, and the processing times are 3 and 8 time units respectively;

[0090] Process O 2,2The devices are M2 and M3, and the processing times are 9 and 3 time units respectively;

[0091] Process O 2,3 The devices are M1 and M2, and the processing times are 7 and 3 time units respectively;

[0092] Process O 3,1 The devices are M2 and M3, and the processing times are 3 and 8 time units respectively;

[0093] Process O 3,2 The devices are M1 and M2, and the processing times are 4 and 8 time units respectively;

[0094] Process O 3,3 The devices are M2 and M3, and the processing times are 8 and 3 time units respectively.

[0095] The other is in TXT file format, as shown in the following table:

[0096]

[0097] The information described in the two formats is exactly the same. The difference is that the table format is easier for humans to understand, while the TXT file format is easier for computers to interpret and transmit.

[0098] Example 4

[0099] This embodiment provides two methods for expressing scheduling results. Figure 3 The Gantt chart of the scheduling result is shown, with the horizontal axis being time and the vertical axis being the machine number. Through the process number listed after the machine number, we know that the scheduling result determines that the equipment M1 is used to process process O. 2,1 , O 1,1 and O 3,2 ; Processing step O with equipment M2 3,1 , O 1,2 and O 2,3 ; Processing step O with equipment M3 2,2 , O 1,3 and O 3,3 Through the position of each process on the timeline, the processing start time and processing end time of each process can be determined. The details are as follows:

[0100] Process O 1,1 The processing start time is 3 and the processing end time is 5;

[0101] Process O 1,2 The processing start time is 5 and the processing end time is 8;

[0102] Process O 1,3 The processing start time is 8 and the processing end time is 13;

[0103] Process O 2,1 The processing start time is 0 and the processing end time is 3;

[0104] Process O 2,2 The processing start time is 3 and the processing end time is 6;

[0105] Process O 2,3 The processing start time is 8 and the processing end time is 11;

[0106] Process O 3,1 The processing start time is 0 and the processing end time is 3;

[0107] Process O 3,2 The processing start time is 5 and the processing end time is 9;

[0108] Process O 3,3 The processing start time is 13, and the processing end time is 16;

[0109] The effect of saving the above information in TXT file is shown in the following table:

[0110]

[0111] The information described in the TXT file is exactly the same as that in the Gantt chart. The difference is that the Gantt chart is easier to understand by humans, while the TXT file is easier to interpret by computers and for communication transmission.

[0112] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A multi-agent flexible job shop autonomous scheduling method, characterized in that: The following steps are involved: In the preparation stage, the flexible job shop scheduling algorithms are collected, and the corresponding scheduling agent templates are constructed for each scheduling algorithm in combination with the scheduling program, and the scheduling agent templates are saved in the template warehouse; The scheduling algorithm includes an algorithm based on heuristic rules, an integer programming algorithm, a genetic algorithm, a neighborhood search algorithm, a taboo search algorithm, a particle swarm algorithm, an ant colony algorithm, a deep learning algorithm, a reinforcement learning algorithm, a deep reinforcement learning algorithm, and a hybrid algorithm formed by combining two or more of the above algorithms; In the application stage, a management agent is created to receive scheduling parameters, determine the required scheduling algorithm and the number of scheduling agents; the required scheduling algorithm and the number of scheduling agents are specified by the user on the graphical human-computer interaction interface provided by the management agent; the graphical human-computer interaction interface is generated according to the configuration file; the configuration file uses a TXT file to sequentially list the name of the scheduling algorithm, the scheduling target supported by each algorithm and the required input parameters; the graphical human-computer interaction interface is changed by modifying the configuration file; The management agent calls the scheduling agent template corresponding to the scheduling algorithm from the template warehouse to create a scheduling agent; the creation of the scheduling agent means that the management agent calls the creation instruction of the Docker daemon process, and the Docker daemon process creates the Docker container process corresponding to the scheduling agent according to the scheduling agent template specified by the management agent; After the scheduling agent completes the calculation, it submits the obtained scheduling plan to the management agent, which selects the best plan and submits it to the user; In the cleanup phase, the management agent destroys all scheduling agents.

2. A multi-agent flexible job shop autonomous scheduling method according to claim 1, characterized in that: The scheduling agent template is a Docker container image, the template repository is a Docker container image repository, and the scheduling agent is a Docker container process that executes the scheduling algorithm.

3. The multi-agent flexible job shop autonomous scheduling method according to claim 1, characterized in that: The management agent is used to receive the user's configuration information, create a scheduling agent, receive the scheduling plan submitted by the scheduling agent, select the optimal scheduling plan, and return the optimal scheduling plan to the user.

4. The multi-agent flexible job shop autonomous scheduling method according to claim 1, characterized in that: The scheduling parameters include scheduling targets and production task information; The scheduling objectives include minimizing the maximum completion time, minimizing the maximum machine load, minimizing the total machine load, minimizing the lead time / delay, minimizing the production cost, minimizing the energy consumption, minimizing the carbon emission, or simultaneously including two or more of the above objectives; The production task information includes the number of production tasks, the required completion time of each production task, the processes included in each production task, the processing order of the processes, the set of equipment that can process each process, and the time required for the equipment to process each process.

5. The multi-agent flexible job shop autonomous scheduling method according to claim 1, characterized in that: The scheduling scheme determines the processing equipment, processing start time and processing end time required for each process of each task; The optimal solution refers to the scheduling solution that best meets the scheduling target requirements.

6. The multi-agent flexible job shop autonomous scheduling method according to claim 1, characterized in that: The destroying of all scheduling agents refers to the management agent calling the destruction instruction of the Docker daemon process, and the Docker daemon process destroys the Docker container process corresponding to the scheduling agent.

Images