Flexible job shop scheduling method based on deep reinforcement learning and multi-agent graph

Abstract

The invention discloses a flexible job shop scheduling method based on deep reinforcement learning and a multi-agent graph. The method comprises the following steps: associating each machine or each job with a reinforcement learning agent, According to the process relationship between the machines and the jobs, including the process sequence between the machines and the current process of the jobs, determining the process sequence of the jobs; constructing a multi-agent graph; on the basis of the multi-agent graph, each agent adopts respective optimal action according to the surrounding environment state, that is, the machine agent arranges the next operation to process when being idle, and the operation agent is routed to a certain machine to process the next process after the current process is completed; and all the intelligent agents work cooperatively so as to take the optimal combined action. According to the method, deep reinforcement learning is utilized, operation sorting is carried out according to the environment state and each machine, operation routing is carried out on each operation, global sub-optimal scheduling is obtained through approximate calculation, heuristic search is not depended on, scheduling rules do not need to be defined in advance, and local optimum is avoided.

Description

technical field [0001] The invention relates to the technical field of flexible job shop scheduling, in particular to a flexible job shop scheduling method based on deep reinforcement learning and multi-agent graphs. Background technique [0002] Scheduling refers to arranging resources to perform tasks over a period of time. Developing optimal schedules is increasingly important in tough economic conditions or in a competitive business environment. Scheduling problems are common in fields ranging from computer engineering to industrial manufacturing. Job shop scheduling (JSS) is one of the most popular scheduling problems, which has been extensively studied in the last three decades due to its wide application in real factories. The decision-making step of JSS is job ordering, i.e., execution of jobs at specific times and on specific machines, where (1) the steps of each job need to be processed in a given order (called priority constraints), and (2) each machine is Only...

AI Technical Summary

Problems solved by technology

[0006] (1) Local Optimum: The existing technology uses heuristic search [CN110009235A, 2019] or pre-defined scheduling rules [CN109270904A, 2019; CN111199272A, 2020], it is easy to fall into a local optimal solution, so it is difficult to generate a satisfactory scheduling scheme

[0007] (2) The state representation is simple: Although the patent [CN109270904A, 2019; CN111199272A, 2020] utilizes deep reinforcement learning, that is, a deep Q network, the state representation of the factory production environment is too simple

These characteristics represent the overall condition of the factory production environment, lacking the specific details of individual machines and operations

[0008] (3) The action is indirect and single: the action of the patent [CN109270904A, 2019; CN111199272A, 2020] is to select the scheduling rule according to the state; then, based on the current scheduling rule, indirectly assign the process of the job to the appropriate time slot of the specific machine

Therefore, existing patents must define the set of scheduling rules in advance

Images