Apparatus and method for scheduling a set of jobs for a plurality of machines

Abstract

An apparatus and method for scheduling a set of jobs for a plurality of machines is provided. A method for scheduling a set of jobs for a plurality of machines, wherein each job is defined by at least one feature characterizing a processing time of the job, wherein if any machine is idle, a job is selected from the set of jobs to be executed by the machine and scheduled for the machine, wherein the job is selected as follows: a graph neural network receives as input the set of jobs and a current state of at least the idle machine, the graph neural network outputs a reward for each job if started on the machine, the state is input into the graph neural network, and the job for the idle machine is selected depending on the graph neural network output.

Description

Technical Field

[0001] This invention relates to a job scheduler for machines, particularly production machines, for continuously selecting jobs via a graph neural network. The invention further relates to a computer program, a machine-readable storage medium, and a system configured to perform the job scheduler. Background Technology

[0002] Prior Technology

[0003] Optimizing production volume or resource utilization is challenging, but it has the potential for significant cost savings and increased productivity in manufacturing plants.

[0004] One challenge lies in the combinatorial nature of the problem. Many possible schedules exist, and finding the optimal schedule is typically NP-hard. Another challenge is its online (or dynamic) nature. Additional tasks arrive during the execution of the current schedule. The characteristics of arriving jobs are not known prior to their arrival.

[0005] Mao et al. used the reinforcement learning algorithm REINFORCE, which utilizes graph convolutional embeddings, for job scheduling on a cluster.

[0006] Advantages of the present invention

[0007] The present invention discloses a job scheduler based on reinforcement learning, and preferably further based on global search, more precisely by combining a DQN (Deep Q-Network) algorithm with a graph convolutional network trained with MCTS (Monte Carlo Tree Search).

[0008] The advantages are given as follows: (1) The job scheduler performs well when a fast scheduling decision is required. In this case, there is no time for searching, and it is recommended to create a schedule based solely on the DQN value network. Experiments have shown that the schedule created by the DQN value network is superior to standard heuristics used for scheduling, such as the weighted shortest processing time heuristic, which is used for rule-based scheduling. (2) The job scheduler can make full use of any available time for deliberation. When some time is available for optimizing the schedule via MCTS. Since MCTS is an anytime algorithm, it can stop at any time and produce an output schedule. Because the output of the DQN value network is used as a search heuristic, the output schedule is at least as good as the schedule produced solely by the value network. The longer MCTS runs, the more the output improves. Given enough time, MCTS can achieve the optimal schedule. (3) The job scheduler can take into account information about how additional tasks arrive over time. Summary of the Invention

[0009] In a first aspect of the invention, a method for scheduling multiple jobs is disclosed. Preferably, it is a computer-implemented method.

[0010] In particular, the allocation and / or sequencing of work to industrial actuators as machines within a time slot can be called scheduling. A job set contains all jobs that must be performed and are currently waiting to be performed. A job can be understood as a task that must be performed by a machine. Multiple machines can be identical or similar machines and can operate in parallel. Machines can be all in one factory or distributed across several factories.

[0011] Each job is defined by at least a first feature characterizing the job's processing time. Processing time can be the duration required for the machine to perform the corresponding job. Optionally, a second feature characterizes the importance of the job.

[0012] If any machine is idle, a job is selected from the job set for that machine to execute and the job is scheduled for that machine. Ideally, the job selection minimizes the total completion time of the job set. An idle machine can be understood as either the machine having completed its current job / the current job being cancelled and the machine being able to start a new job, or the machine not starting any jobs. If a machine is executing a job, the corresponding state of that machine can be called occupied.

[0013] The following steps are taken to select jobs. A Graph Neural Network (GNN) takes a set of jobs and the current state of at least the idle machines as input. The GNN is trained to estimate the reward that will be obtained when performing a job, such that the total completion time is minimized. More precisely, the GNN is trained to output a high reward for a small completion time for a set of jobs.

[0014] The current state of a machine can include its current settings and remaining processing time. The machine state can further include, for example, its current class; whether it is currently occupied by a job, and if so, the remaining processing time. Optionally, the duration until the next maintenance job can be defined. It is also possible that the GNN receives the state of all machines as input. Then, preferably, irrelevant machines and jobs are masked, for example, by zero masking.

[0015] If we assume that the GNN starts on a machine, the reward of the set of jobs—specifically the expected cumulative discount—is fed into the GNN. It can be said that the higher the reward, the lower the expected total completion time. The jobs used for the idle machines are selected based on the output of the graph neural network that achieves the highest reward. Then, one of the idle machines, or the idle machine itself, can receive the command to execute the selected job.

[0016] It is proposed that GNNs accept—specifically, a bipartite and undirected—graph as input, comprising two disjoint, independent sets of nodes (U, V), where the first set of nodes (U) comprises the current state of each machine, and the second set of nodes (V) comprises the features of each job within the job set. Furthermore, the graph includes a set of edges (E), where edges represent connections between nodes in the first set and nodes in the second set. Additionally, the graph includes a global information vector relevant to the entire graph. This vector can be initialized to zero.

[0017] The output of the graph neural network is then a graph with the same structure but updated nodes, edges, and global feature vectors, but with node sets U and V. The updated edges are interpreted as rewards, also known as Q-values, for scheduling one of the jobs on one of the machines connected by the corresponding edge. The updated global features can characterize the idle actions of the machines. It is then possible to select the idle state of a machine based on the output of the GNN, rather than selecting a job based on the output of the GNN.

[0018] The advantage of using GNNs is that they take into account the problem of applying the same network to different scales, i.e., different numbers of jobs or machines.

[0019] Note that after a job is selected, it is assigned to a machine, which then begins executing the job. This process can be repeated until all jobs are completed. The selected job can then be removed from the job set.

[0020] It is suggested that a job be defined by a third feature representing one of several classes, where each class represents the corresponding machine settings required to perform the corresponding job. Furthermore, the current state of the machine can represent its current settings. The graph neural network receives an additional input, which is a time penalty, if the machine must be reconfigured by changing the settings of any machine due to two immediately consecutive jobs for different classes of the same machine. The time penalty can be a value that can be proportional to a measured or predetermined time, which can be the measured time or estimated based on experience regarding machine reconfiguration.

[0021] More precisely, the set of edges in the input graph of the GNN edge features encodes the setup time required to process the jobs corresponding to the nodes of the second set of nodes on the corresponding machines of the first set of nodes.

[0022] Further suggestions depend on the GNN's output, which sorts all jobs in the job set into a queue. The queue defines the order in which multiple machines continue processing jobs. Preferably, jobs continue processing according to the machines' queues. Particularly preferably, the queue is re-evaluated after one of the jobs in the queue has been completed.

[0023] Further suggestions are made regarding additional jobs arriving at specific points in time during the continuation of the current job, and these additional jobs are added to the job set. An advantage is that this method can also handle dynamic settings due to the increased flexibility of the job scheduler. To consider arrival time in job selection, each job can be defined by another feature that characterizes a timestamp when it arrives in the job queue.

[0024] Further suggestions include optimizing the parameterization of graph neural networks through reinforcement learning, particularly through deep Q-networks. Preferably, a simulated factory, referred to as a digital twin, is used to reflect the dynamics of a real factory during reinforcement learning, more precisely, how the factory's state changes when a task begins. In essence, the simulated factory corresponds to the environment in which the reinforcement learning algorithm acts as an agent and optimizes its policy.

[0025] A further recommendation is to apply Monte Carlo Tree Search (MCTS) for selecting subsequent jobs. Monte Carlo Tree Search iteratively builds a search tree starting from the machine's current state and the set of jobs, where the output of a graph neural network is used as a search heuristic to expand the search tree. Depending on the search tree, subsequent jobs are selected and / or the Monte Carlo Tree Search outputs job scheduling.

[0026] The advantages of combining graph neural networks with Monte Carlo tree search result in scheduling that comes very close to optimal. Because of GNNs, there is no need to simulate the machines, especially the factory.

[0027] A further suggestion is to select actions based on a confidence upper bound to expand the search tree. This implies selecting the action with the highest reward based on how frequently the corresponding action has already been selected.

[0028] Further recommendations include pruning the search tree, specifically by considering only the pre-defined number that yields the highest reward in at least one rollout. k This is a task. The advantage lies in the reduced complexity of the search tree, which, through excellent optimization, leads to improved search behavior in MCTS.

[0029] The machine can be a production machine, and the logistics robot can be used to supply the production machine with the production materials required for the current operation of the corresponding machine, wherein the logistics robot is controlled according to the scheduled operation.

[0030] The production machine can be a manufacturing machine, preferably a manufacturing system, such as a stamping tool, cutting tool, or gun drill, for example, as part of one or more production lines. It is further suggested that the control system controls the machine based on the scheduled operations.

[0031] In a second aspect of the invention, the machine corresponds to a warehouse or delivery location for goods, wherein a transportation route for the goods is determined depending on the set of jobs. For this aspect of the invention, the job scheduler is applied to logistics problems (e.g., planning routes for couriers through a city, wherein additional pick-up / drop-off points are added while the courier is en route). Note that the job scheduler is not limited to these specific applications and can be applied to a wide range of different online or offline scheduling or combinatorial optimization problems in diverse industrial sectors, such as packaging. Attached Figure Description

[0032] Embodiments of the invention will be discussed in more detail with reference to the following figures. The figures illustrate:

[0033] Figure 1 A flowchart of the method implemented by the job scheduler; and

[0034] Figure 2 Training system. Detailed Implementation

[0035] The following describes a job scheduler built upon reinforcement learning and global search, specifically by combining graph convolutional DQN with MCTS, allowing one to benefit from the other. This job scheduler is capable of directly computationally solving a scheduling problem represented as a graph without the aid of an external solver or simulation environment.

[0036] First, the settings for the problems that the job scheduler can solve should be defined.

[0037] Generally, a job scheduler should be able to solve online scheduling, specifically online combinatorial problems. Note that the job scheduler can be easily extended to other online combinatorial problems, such as the traveling salesman problem and the vehicle routing problem, where destinations are added online. Online combinatorial optimization problems include online logistics problems (e.g., planning routes for couriers through a city, where additional pick-up / drop-off points are added while the courier is on the road).

[0038] In the following embodiment, the scheduling problem is considered—where in a specific scheduling in a production plant, many waiting jobs need to be handled by a similar set of machines (the order of processing goods is determined at different processing steps), and scheduling in other industrial sectors, such as packaging. The machines can be any kind of machine, such as production machines, transport machines, etc.

[0039] Jobs can belong to different classes. Whenever two jobs of different classes are processed consecutively on the same machine, the machine settings need to be changed, incurring a time penalty due to machine reconfiguration. Two consecutive jobs of the same class do not incur such a penalty. In addition to their class, jobs are also characterized by their processing time, priority by their weight (i.e., importance), and priority by the time they arrive in the queue.

[0040] Total weighted completion time (TWCT) can be defined as

[0041]

[0042] in J It is a set of assignments. w j It's homework. j The weights, and C j It is its completion time, that is, in the current j The difference between the time it takes to complete processing and the time it arrives in the queue.

[0043] The job scheduler is configured to minimize TWCT.

[0044] A state can be characterized by the set of waiting jobs as defined above and their characteristics (e.g., completion time, job weight, etc.), as well as the current state of the machines. Specifically, each machine is represented by: its current class for which it is set; whether it is currently occupied by a job, and if so, the remaining processing time. The set-change time matrix is ​​also observable. Note that a state is considered a decision point if it contains at least one idle machine and one waiting job.

[0045] Additionally, received sensor signals—such as those from built-in monitoring sensors in plant processing machines (e.g., current load, maintenance requirements) and sensors monitoring the location and status of operations within the plant (e.g., silicon wafers)—can be used to characterize the state. Alternatively, the current state of the materials supply chain can be used to characterize that state.

[0046] Actions can be represented as tuples ( j;m ) and corresponding to the machine m The starting job on j When this action is performed, an event indicating a status update is registered for the time when the job terminates. Alternatively, no action can be performed. noop This action does not initiate a job on any idle machine. In both cases, the simulation is iteratively fast-forwarded to the next event until the next decision point is reached.

[0047] Static and dynamic setups can exist. In a static setup, all relevant jobs are visible in the initial state, for example, at the beginning of the method before the job scheduler is started. In a dynamic setup, additional jobs can arrive at specific points in time (i.e., within an arrival interval), where the number of jobs arriving at the beginning of each interval can be estimated by a Poisson distribution of all classes, each with its own rate. The frequency of job classes can follow Zipf's law. The phase terminates when all jobs have arrived and been processed.

[0048] Instant rewards can be given as The current time of the unfinished task is used to replace the completion time. Therefore, the total reward for this stage is:

[0049]

[0050] Note that the current time is The value in the middle is 0, therefore And all assignments are in s T Completed in the middle, therefore only TWCT Target value.

[0051] Explanation of the choice of Deep Q Network (DGN): Q Graph Neural Network.

[0052] The online scheduling problem described above can be viewed as a Markov Decision Process (MDP), where... S and A These are the state space and the action space, respectively. Indicate when the action is applied From state to state The transition probabilities are given. To solve the MDP formula, a reinforcement learning algorithm is applied.

[0053] Preferably, the Deviation-Strategy DQN algorithm can be used, which learns the biased strategy DQN algorithm. The approximation of the function, The function represents the expression for in s Application a And then follow a cumulative discounted reward based on some action strategy, which may be greedy or exploratory. Regarding... Q For more information on the function, readers can refer to the document: Bertsekas, D., “Dynamic programming and optimal control”, Athena Scientific Belmont, MA, 2005.

[0054] Due to the large state space S andQ Function complexity, both, Parameterized neural networks are Q A common choice for function approximation. This approximation can be trained to minimize the square of the time difference, as indicated by the following formula:

[0055]

[0056] With discount factor and instant rewards .

[0057] In the implementation of the DQN algorithm, GNN is used as Q Function approximation. Generative Neural Networks (GNNs) are a common name for a framework in which a neural network or ensemble of neural networks is repeatedly applied to a graph representation of the input data. In the context of combinatorial optimization, GNNs possess several important properties: they are permutationally equivariant and capable of handling observations of varying scales (i.e., different numbers of nodes and edges) as long as the features remain constant in dimensionality. Furthermore, GNNs can exploit relational biases in observations through multi-step propagation of information within the graph representation, i.e., message passing.

[0058] For the scheduling problem, each observation is represented by a bipartite undirected graph, which consists of tuples. express.

[0059] In the proposed encoding, U and V This corresponds to two disjoint, independent sets of nodes, where u i It is a machine i eigenvectors, and v j It's homework. j The feature vectors of machines and jobs are composed of the characteristics described above. Each node and The actual feature vectors are the connections between machine and job features, where irrelevant dimensions can be masked with zeros. The sets of machine and job nodes are connected by the set of edges. Interconnection, where e k It is the edge feature vector. m k It is the index of the machine node, and j k It is by the side k The index of the connected job node. Edge feature pairs in the corresponding machine. m k Upload processing j k The required setup time is encoded. Additionally, the diagram illustrates the parameters. wIt is the global information vector. This parameter w It is called a global feature, and is initialized to 0 for each observation, and is used for information propagation during message passing.

[0060] Preferably, an encoded message-passing decoding GNN architecture is used, as described in "Relational inductive biases, deep learning, and graph networks" by Battaglia, P. W et al. (arXiv preprint arXiv:1806.01261, 2018).

[0061] The output of a GNN has the same structure but with updated nodes. u i and v j ,side e k and global w A graph of eigenvectors. Edges k Features e k Explained as machine m k Upper scheduling operation j k The Q-value and global features w Explain as noop The Q-value of the action.

[0062] In order to learn Q The function employs a machine simulator, modeling the problem as discrete event simulations. Preferably, a simulated factory, or "digital twin," is used, reflecting the dynamics of a real factory—how the factory state changes when operations are started on the machines, and how additional tasks arrive over time. In alternative embodiments, recorded data from the machines or the factory can be used for learning. Q The training data, or recorded data, of the function can be used to build a simulator. As stated above, DGN learning is preferably used for learning. Q Functions and training GNNs.

[0063] After training the GNN, it can be directly applied to scheduling jobs across multiple machines. For example, if a decision point occurs, the above input to the GNN is a continuation of the current situation at the time the decision point occurs. Depending on the output of the GNN, the job with the highest priority is selected from the set of waiting jobs. Q The next task is then performed. This process can be applied each time a decision point arises. Interestingly, the process is neither limited to static nor dynamic settings.

[0064] Monte Carlo Tree Search (MCTS) can be used for situations where a decision point arises, or where the complete schedule of all waiting jobs should be scheduled into a single scheduler or queue. MCTS is an online planning method for sequential and (potentially) stochastic decision problems. MCTS searches for the optimal decision to be made at each state by sampling trial runs iteratively: a sequence of steps that starts in the current state and ends in a terminating state or a state reached at the end of the planning range. These trials are used to construct a search tree that stores information about the states visited. Such information includes… Q Value estimation It is used to guide the search in subsequent trial runs and finally outputs a decision for the current state when a timeout is reached.

[0065] While MCTS can find high-quality decisions without incorporating any domain-specific knowledge, the availability of good domain-specific heuristics can significantly improve the search process, leading to better decisions in a shorter deliberation time. Preferably, a variant of the MCTS algorithm, known as UCT, is applied in conjunction with a trained GNN (see below: "Bandit based monte-carloplanning" by Kocsis, L. and Szepesvari, C., in the European Conference on Machine Learning, pp. 282-293. Springer, 2006). Specifically, trials are sampled by selecting actions based on a confidence upper bound (UCB) policy for the states inside the search tree (i.e., those added to the tree in previous trials), and otherwise actions are selected from the trained GNN. These actions are chosen based on the state inside the search tree (i.e., those added to the tree in previous trials) and otherwise based on the confidence upper bound (UCB, see below for more details: see also Kocsis et al. or Liu et al., “Adapting improved upper confidence bounds for Monte-Carlo tree search,” in: Advances in Computer Games, Springer, Cham, 2015, pp. 53-64). Q Value estimation The actions. UCB balances exploration and exploitation by first trying all actions once and then selecting the action that maximizes value:

[0066]

[0067] in Marking actions a In state s The number of times the sample is taken. ,and It is an exploratory factor.

[0068] Each state Q The value is estimated as:

[0069]

[0070] in It's a sample (trial run). i From state s Cumulative rewards until the end of the trial run.

[0071] Since the number of actions can be very large in some cases, often leading to poor performance, a pruning mechanism of GNNs can be applied in an additional way. That is, during the search, only actions with the highest... Q Value estimation of k One action.

[0072] Given the current state, MCTS can be applied to generate a search tree, from which job scheduling is derived, for example, selecting the branch of the search tree that leads to the shortest TWCT.

[0073] MCTS can run for as much time as is available for optimization (e.g., seconds to minutes), or until the optimal job queue is found.

[0074] Figure 1 A flowchart (10) is shown for one embodiment of training the job scheduler and implementing the job scheduler.

[0075] The method begins with step 11. In this step, the factory simulator is initialized.

[0076] In subsequent step 12, the GNN is trained, in which reinforcement learning algorithms, particularly DGN, are applied. The reinforcement algorithms are implemented on a simulator to explore the simulator and its behavior for a given action in a given state.

[0077] If the training of the GNN terminates, step 13 follows. For a given machine state and job set, a job queue is determined. As mentioned above, the queue is determined by MCTS.

[0078] Note that if all or some machines are idle, or if a decision point is reached for a machine, a job queue can be initiated via MCTS.

[0079] After obtaining the queue, in step 14, the machine then continues with the job. For example, this may be after the job queue has been determined by MCTS or the next job has been determined. QOnce the value is determined, the next job is assigned to the corresponding machine. The machine is then controlled to execute the newly assigned job.

[0080] If the job queue continues, and a decision point is reached, MCTS can be applied again, and the next job is selected not from the queue order, but via an updated queue determined by the MCTS search tree, or solely depending on the GNN. Q The value is used to select the next job, or to further utilize the order of the queue.

[0081] In another embodiment, a control signal is determined depending on the next identified job. This control signal is used to control a machine and / or a physical system, such as a computer-controlled robot, which loads jobs into available machines and / or reconfigures the machines to make the necessary settings for their next job.

[0082] Figure 2 An embodiment of a training system (20) for training a GNN (22) is shown. The training module (21) provides training data for the GNN (22), and the parameters... The parameters are stored in the storage device (23), and are preferably updated by DGN reinforcement learning depending on the output of GNN (22).

[0083] The computer-readable storage (24) contains instructions that, when executed on a computing device (25), start the computer to perform GNN training.

[0084] The term "computer" encompasses any device used to process predefined computational instructions. These computational instructions can be in the form of software, hardware, or a hybrid of both.

[0085] To further understand, processes can not only be fully implemented in software as described, but also in hardware, or in a hybrid of software and hardware.

Claims

1. A method for scheduling a set of jobs across multiple machines, each job being defined by at least one feature characterizing the processing time of the job, the method comprising: When any one of the multiple machines is an idle machine: Select a job from the job set to be executed by the idle machine. Selected jobs are scheduled on the idle machines, where each of the multiple machines is a cutting tool or a gun drill used for drilling in the manufacturing system. Choose the following assignments: Graph neural networks take a bipartite and undirected graph as input, the graph comprising two disjoint independent sets of nodes and edges, wherein the first set of nodes includes the current state of at least the idle machines, and the second set of nodes includes a set of jobs; and the edges represent the connections between nodes in the first set and nodes in the second set and are undirected. The graph neural network outputs the reward for each job launched on at least one of multiple machines, whose state is input into the graph neural network, and The selection of jobs for the idle machine is based on the reward output of the graph neural network to minimize the total completion time, wherein the parameterization of the graph neural network is optimized by learning through a deep Q-network. Furthermore, to select subsequent jobs, a Monte Carlo tree search is applied, wherein the Monte Carlo tree search iteratively constructs a search tree starting from the current state and job set of each of multiple machines, wherein the output reward of a graph neural network is used as a search heuristic to expand the search tree, wherein subsequent jobs involve the physical execution of the task on the idle machine, and wherein subsequent jobs are selected based on the search tree. The generated control signals are used to control the idle machine to perform physical actions such as cutting or drilling in subsequent operations. The idle machine performs the physical execution of the subsequent task, wherein the idle machine includes a robot, wherein control signals are generated according to the subsequent task, and wherein the execution includes applying the control signals to cause the robot to perform the physical execution of the subsequent task.

2. The method of claim 1, wherein the job is defined by additional features representing one of a plurality of classes, wherein each of the plurality of classes represents a corresponding setup of a plurality of machines required to perform the corresponding job, wherein the current state of each machine also represents its current setup, and wherein the graph neural network receives additional input, which is a time penalty, when a machine must be reconfigured by changing the setup of any machine due to two immediately consecutive jobs for different classes of the same machine.

3. The method of claim 1, wherein the additional job arrives at a specific time point and is added to the job set.

4. The method of claim 1, wherein the parameterization of the graph neural network is optimized by reinforcement learning. .

5. The method of claim 1, wherein, in order to expand the search tree, the action is selected based on the upper confidence limit.

6. The method of claim 1, wherein the search tree is obtained by considering only those trees with the highest search rate in at least one trial run. Q The predetermined number of jobs were pruned.

7. The method of claim 1, wherein the state of each of the plurality of machines is determined based on received sensor signals, wherein the state of each of the plurality of machines further includes the current load state and / or parameters characterizing the maintenance requirements of the corresponding one of the plurality of machines.

8. A non-transitory machine-readable storage medium having a computer program stored thereon for scheduling a set of jobs on a plurality of machines, each job being defined by at least one feature characterizing the processing time of the job, the computer program causing the processor to execute when executed by the processor: When any one of the multiple machines is an idle machine: Select a job from the job set to be executed by the idle machine. The selected job is scheduled on the idle machine, wherein... Each of the multiple machines is either a cutting tool used for cutting or a gun drill used for drilling within the manufacturing system. Choose the following assignments: Graph neural networks take a bipartite and undirected graph as input, the graph comprising two disjoint independent sets of nodes and edges, wherein the first set of nodes includes the current state of at least the idle machines, and the second set of nodes includes a set of jobs; and the edges represent the connections between nodes in the first set and nodes in the second set and are undirected. The graph neural network outputs the reward for each job launched on at least one of multiple machines, whose state is input into the graph neural network, and The selection of jobs for the idle machine is based on the reward output of the graph neural network to minimize the total completion time, wherein the parameterization of the graph neural network is optimized by learning through a deep Q-network. Furthermore, to select subsequent jobs, a Monte Carlo tree search is applied, wherein the Monte Carlo tree search iteratively constructs a search tree starting from the current state and job set of each of multiple machines, wherein the output reward of a graph neural network is used as a search heuristic to expand the search tree, wherein subsequent jobs involve the physical execution of the task on the idle machine, and wherein subsequent jobs are selected based on the search tree. The generated control signals are used to control the idle machine to perform physical actions such as cutting or drilling in subsequent operations. The idle machine performs the physical execution of the subsequent task, wherein the idle machine includes a robot, wherein control signals are generated according to the subsequent task, and wherein the execution includes applying the control signals to cause the robot to perform the physical execution of the subsequent task.

9. A system configured to schedule a set of jobs across multiple machines, each job being defined by at least one feature characterizing the processing time of the job, said system being configured to: When any one of the multiple machines is an idle machine: Select a job from the job set to be executed by the idle machine. The selected job is scheduled on the idle machine, wherein... Each of the multiple machines is either a cutting tool used for cutting or a gun drill used for drilling within the manufacturing system. Choose the following assignments: Graph neural networks take a bipartite and undirected graph as input, the graph comprising two disjoint independent sets of nodes and edges, wherein the first set of nodes includes the current state of at least the idle machines, and the second set of nodes includes a set of jobs; and the edges represent the connections between nodes in the first set and nodes in the second set and are undirected. The graph neural network outputs the reward for each job launched on at least one of multiple machines, whose state is input into the graph neural network, and The selection of jobs for the idle machine is based on the reward output of the graph neural network to minimize the total completion time, wherein the parameterization of the graph neural network is optimized by learning through a deep Q-network. Furthermore, to select subsequent jobs, a Monte Carlo tree search is applied, wherein the Monte Carlo tree search iteratively constructs a search tree starting from the current state and job set of each of multiple machines, wherein the output reward of a graph neural network is used as a search heuristic to expand the search tree, wherein subsequent jobs involve the physical execution of the task on the idle machine, and wherein subsequent jobs are selected based on the search tree. The generated control signals are used to control the idle machine to perform physical actions such as cutting or drilling in subsequent operations. The idle machine performs the physical execution of the subsequent task, wherein the idle machine includes a robot, wherein control signals are generated according to the subsequent task, and wherein the execution includes applying the control signals to cause the robot to perform the physical execution of the subsequent task.

Images