Scheduling method and device for flexible job shop

Abstract

The application discloses a scheduling method for a flexible job shop. It comprises the following steps: step one, obtaining a dynamic heterogeneous graph of the flexible job shop, including the scheduling state of the flexible job shop; step two, obtaining the characteristics of the scheduling state; step three, determining the final machine embedding according to the machine node characteristics, operation-machine pair characteristics and machine-machine pair characteristics; step four, determining the final operation embedding according to the operation node characteristics, predecessor node characteristics, successor node characteristics and final machine embedding; step five, determining the global state embedding according to the final machine embedding and the final operation embedding; step six, determining the probability distribution of the actionable according to the global state embedding; step seven, determining the target action according to the probability distribution; step eight, updating the scheduling state to obtain the updated dynamic heterogeneous graph according to the target action; and step nine, repeating the steps two to eight until all operations are assigned to the corresponding machines to complete the scheduling, and generating an information-rich embedding.

Description

Technical Field

[0001] This invention belongs to the field of workshop production scheduling and resource optimization technology, and specifically relates to a scheduling method and apparatus for flexible operation workshops. Background Technology

[0002] Currently, methods for solving the Flexible Job Shop Scheduling Problem (FJSP) generally fall into two main categories: the first is exact methods, which seek the optimal solution; the second is approximate methods, which pursue higher-quality solutions with higher computational efficiency. Exact methods, such as mathematical programming and constraint programming, while guaranteeing the optimal solution, have long computation times, especially in large-scale instances. Therefore, approximate methods are more commonly used in practical applications because they achieve a better balance between optimal solution and computational time. These methods mainly include heuristics, metaheuristics, and machine learning techniques. Mainstream metaheuristic methods, such as evolutionary algorithms (e.g., Genetic Algorithms (GA)) and swarm intelligence algorithms (e.g., Particle Swarm Optimization (PSO) and Artificial Bee Colony (ABC)), have proven to be very effective in solving FJSP. A dual-population cooperative genetic algorithm is proposed. This method uses two subpopulations with different decoding rules to expand the solution space and employs co-evolutionary operations to improve overall search performance. However, this method is time-consuming and cannot be applied to large-scale practical problems. A Learning-Based Genetic Algorithm (LGA) is proposed. This algorithm first uses a deep learning model to understand and simplify the complex problem. Then, the genetic algorithm operates on this simplified version to find the optimal solution more efficiently. However, this method is also time-consuming and cannot be applied to large-scale practical problems. Deep Reinforcement Learning (DRL) has attracted much attention in recent years due to its promising and scalable solution to the scheduling problem, especially suitable for problems with common structural features. Unlike traditional methods such as metaheuristics, the core advantage of DRL lies in learning general scheduling policies, rather than specific solutions for individual problems. While these methods can find high-quality solutions, they are computationally expensive and must be re-run for each new problem instance. A novel deep reinforcement learning framework for FJSP is proposed, which uses graph neural networks to learn effective scheduling policies. This framework employs a multi-pointer graph network (MPGN) architecture and a multi-proximal policy optimization (multi-PPO) training algorithm to handle the multi-action characteristics of FJSP, where the agent schedules job operations on a compatible machine at each time step. However, this framework is not good enough for information extraction and state representation.A novel deep reinforcement learning (DRL) approach is developed to solve the Free-JSP problem in real time. This method enhances deep reinforcement learning for FJSP by improving the Markov Decision Process (MDP) model, using an improved version of the Heterogeneous Graph Attention Network (HGATv2) for more efficient information extraction, and restricting the action space to high-performance scheduling rules. The method further leverages Bayesian optimization to tune hyperparameters and uses the K-nearest neighbor algorithm to select from a diverse set of generation policies, achieving superior performance, especially on large-scale problems, compared to state-of-the-art methods. The DRL model, through intensive offline training, learns complex scheduling logic. Once trained, it can make high-quality decisions in real-time applications almost immediately based on the current state. This one-time training and widespread applicability overcomes the scalability limitations of traditional methods, making it particularly effective in dynamic production environments requiring rapid decision-making. However, multi-policy generation using this method is time-consuming. Summary of the Invention

[0003] To address the aforementioned problems, embodiments of the present invention provide a scheduling method and apparatus for flexible work workshops.

[0004] In a first aspect, embodiments of the present invention provide a scheduling method for flexible workshops, comprising: Step 1: Obtain the dynamic heterogeneity diagram of the flexible workshop. The dynamic heterogeneity diagram includes the scheduling status of the flexible workshop, which includes operation nodes, machine nodes, operation-machine pairs, and machine-machine pairs. Step 2: Obtain the features of the scheduling state at time t. The features of the scheduling state include operation node features, machine node features, operation-machine pair features, and machine-machine pair features. Step 3: Based on the machine node features, operation-machine pair features, machine-machine pair features, and the deep learning network corresponding to each machine node, determine the final machine embedding corresponding to each machine node at time t. Step 4: Based on the operation node features corresponding to each operation node, the predecessor node features corresponding to each operation node, the successor node features corresponding to each operation node, the final machine embedding corresponding to each processable machine node corresponding to each operation node, and the deep learning network, determine the final operation embedding corresponding to each operation node at time t. Step 5: Determine the global state embedding at time t based on the final machine embeddings corresponding to all machine nodes at time t and the final operation embeddings corresponding to all operation nodes at time t. Step 6: Based on the global state embedding and reinforcement learning network, determine the probability distribution of all possible actions at time t. The possible actions are operation-machine pairs that satisfy preset conditions, namely, the preceding operation of the operation has been processed, the operation can be processed at time t, and the machine that can process the operation is in an idle state. Step 7: Based on the probability distribution of all possible actions, determine the target action, which is to be performed in the environment of the flexible work workshop, and generate an immediate reward to be sent to the reinforcement learning network; Step 8: Update the scheduling state according to the target action to obtain the updated dynamic heterogeneous graph; Step nine: Repeat steps two through eight until all operations are assigned to the appropriate machines and the scheduling is completed.

[0005] In one possible implementation, determining the final machine embeddings corresponding to all machine nodes at time t based on the machine node features, operation-machine pair features, machine-machine pair features, and the deep learning network in the features of the scheduling state includes: The first combined feature is determined by concatenating the machine node features corresponding to each machine node and the machine-machine pair features corresponding to the machine nodes connected to that machine node. Based on the machine node features and the first combined features, and using the deep learning network, determine the embedded representation of the competitive relationship between any machine node and any machine node in its neighborhood. The second combined feature is determined by concatenating the operation node feature corresponding to each operation node with the operation-machine pair feature corresponding to the machine node connected to that operation node. Based on the machine node features and the second combined features, and using the deep learning network, determine the embedded representation of the processing relationship between any machine node and the operation nodes that the machine node can process; Based on the competition relationship embedding representation and the processing relationship embedding representation, determine the final machine embedding corresponding to each machine node at time t.

[0006] In one possible implementation, determining the competition relationship embedding representation between any machine node and any machine node in its neighborhood, based on the machine node features and the first combined features and the deep learning network, includes: Based on the machine node features and the first combined features, and based on the deep learning network, a first attention coefficient is determined between any machine node and any machine node in its neighborhood. Based on the machine node characteristics and the deep learning network, the second attention coefficient of the machine node itself is determined. Normalize the first attention coefficient and the second attention coefficient to obtain the normalized first attention coefficient and the normalized second attention coefficient; Based on the normalized first attention coefficient, the normalized second attention coefficient, the machine node features, and the first combined features, the competition relationship embedding representation between any machine node and any machine node in the neighborhood of that machine node is determined.

[0007] In one possible implementation, determining the processing relationship embedding representation between any machine node and the operation nodes that can be processed by that machine node, based on the machine node features and the operation-machine pair features and the deep learning network, includes: Based on the machine node features and the second combined features, and using the deep learning network, a third attention coefficient is determined between any machine node and the operational nodes that the machine node can process. Based on the machine node characteristics and the deep learning network, the fourth attention coefficient of the machine node itself is determined. The third attention coefficient and the fourth attention coefficient are normalized to obtain the normalized third attention coefficient and the normalized fourth attention coefficient; Based on the normalized third attention coefficient, the normalized fourth attention coefficient, the machine node features, and the second combined features, the processing relationship embedding representation between any machine node and the operation nodes that the machine node can process is determined.

[0008] In one possible implementation, determining the final machine embeddings corresponding to all machine nodes at time t based on the competition relationship embedding representation and the processing relationship embedding representation includes: The competition relationship embedding representation and the processing relationship embedding representation are concatenated using a multilayer perceptron to determine the final machine embedding corresponding to each machine node at time t.

[0009] In one possible implementation, determining the final operation embeddings for all operation nodes at time t based on the operation node features corresponding to each operation node, the predecessor node features corresponding to each operation node, the successor node features corresponding to each operation node, the final machine embeddings corresponding to each processable machine node corresponding to each operation node, and the deep learning network includes: Based on the operation node features corresponding to all operation nodes, the predecessor node features corresponding to all operation nodes, the successor node features corresponding to all operation nodes, the final machine embedding corresponding to the processable machine node corresponding to all operation nodes, and the deep learning network, determine the target operation embedding corresponding to all operation nodes at time t. Based on the target operation embeddings and the obtained residual connection embeddings corresponding to all operation nodes at time t, the final operation embeddings corresponding to all operation nodes at time t are determined.

[0010] In one possible implementation, determining the target operation embeddings corresponding to all operation nodes at time t based on the operation node features corresponding to all operation nodes, the predecessor node features corresponding to all operation nodes, the successor node features corresponding to all operation nodes, the final machine embeddings corresponding to the processable machine nodes corresponding to all operation nodes, and the deep learning network includes: The predecessor node features corresponding to multiple predecessor nodes of each operation node are aggregated to obtain the predecessor aggregation vector corresponding to each operation node. The features of the successor nodes corresponding to multiple successor nodes of each operation node are aggregated to obtain the successor aggregation vector corresponding to each operation node. Aggregate the final machine embeddings corresponding to the machine nodes connected to each operation node to obtain the machine aggregation vector for each operation node; Based on the operation node features corresponding to each operation node, the predecessor aggregation vector corresponding to each operation node, the successor aggregation vector corresponding to each operation node, and the machine aggregation vector corresponding to each operation node, the target operation embeddings corresponding to all operation nodes at time t are determined.

[0011] Secondly, embodiments of the present invention provide a scheduling device for flexible workshops, comprising: The dynamic heterogeneous graph acquisition module is used to acquire the dynamic heterogeneous graph of the flexible work shop. The dynamic heterogeneous graph includes the scheduling status of the flexible work shop, and the scheduling status includes operation nodes, machine nodes, operation-machine pairs and machine-machine pairs. The scheduling state feature acquisition module is used to acquire the features of the scheduling state at time t. The features of the scheduling state include operation node features, machine node features, operation-machine pair features, and machine-machine pair features. The final machine embedding determination module is used to determine the final machine embedding of each machine node at time t based on the machine node features, operation-machine pair features, machine-machine pair features, and the deep learning network. The final operation embedding determination module is used to determine the final operation embedding of all operation nodes at time t based on the operation node features, the predecessor node features, the successor node features, the final machine embedding of the processable machine nodes, and the deep learning network. The global state embedding determination module is used to determine the global state embedding at time t based on the final machine embeddings corresponding to all machine nodes at time t and the final operation embeddings corresponding to all operation nodes at time t. The probability distribution determination module is used to determine the probability distribution of all possible actions at time t based on the global state embedding and reinforcement learning network. The possible actions are operation-machine pairs that satisfy preset conditions, namely, the preceding operation of the operation has been processed, the operation can be processed at time t, and the machine that can process the operation is in an idle state. The target action determination module is used to determine the target action based on the probability distribution of all possible actions. The target action is to be performed in the environment of the flexible work workshop and generate an immediate reward to be sent to the reinforcement learning network. The scheduling state update module is used to update the scheduling state according to the target action to obtain the updated dynamic heterogeneous graph. The repetition processing module is used to repeat the functions of the aforementioned scheduling status feature acquisition module to the scheduling status update module until all operations are assigned to the corresponding machines to complete the scheduling.

[0012] Thirdly, embodiments of the present invention provide an electronic device, comprising: The system includes a memory and a processor, which communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, which can invoke the program instructions to perform the steps of the method described in the first aspect and various possible implementations.

[0013] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect and various possible implementations.

[0014] Fifthly, embodiments of the present invention provide a computer program product containing instructions that, when the computer program product is run on a computer, cause the steps of the method described in the first aspect and various possible implementations to be executed by the computer.

[0015] The beneficial effects of the technical solution provided by the embodiments of the present invention are as follows: a dynamic heterogeneous graph is introduced to represent the entire scheduling state of the flexible job shop. This graph not only effectively captures the priority relationships between operations and the processing relationships between operations and machines, but also captures the key competitive relationships between machines. Based on the competition relationship embedding representation and the processing relationship embedding representation, the final machine embeddings corresponding to all machine nodes at time t are determined, and these final machine embeddings are used as inputs to determine the final operation embeddings. This aims to generate highly expressive and information-rich embeddings for machine nodes and operation nodes. Attached Figure Description

[0016] Figure 1 A schematic diagram of the training framework provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a scheduling method for a flexible workshop provided in an embodiment of the present invention; Figure 3a This is a conventional parsing graph provided in the embodiments of the present invention; Figure 3b The heterogeneous diagram provided in this embodiment of the invention does not introduce inter-machine competition. Figure 3c The present invention provides a dynamic heterogeneous diagram that introduces competition between machines in its embodiments. Figure 4 This is another flowchart illustrating a scheduling method for a flexible workshop, provided as an embodiment of the present invention. Figure 5 A schematic block diagram of a scheduling device for a flexible workshop provided in an embodiment of the present invention. Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0019] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0020] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if monitoring (the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when monitoring (the stated condition or event)," or "in response to monitoring (the stated condition or event)."

[0021] This invention provides a scheduling method for flexible workshops. This method is applicable to, for example... Figure 1 The training framework is shown below. A flowchart of the method is shown below. Figure 2 As shown. Figure 2 The method may include the following steps: Step 201: Obtain the dynamic heterogeneous graph of the flexible job shop. The dynamic heterogeneous graph includes the scheduling status of the flexible job shop, which includes operation nodes, machine nodes, operation-machine pairs, and machine-machine pairs.

[0022] Step 202: Obtain the features of the scheduling state at time t. The features of the scheduling state include operation node features, machine node features, operation-machine pair features, and machine-machine pair features.

[0023] Step 203: Based on the machine node features, operation-machine pair features, machine-machine pair features, and deep learning network corresponding to each machine node, determine the final machine embedding corresponding to each machine node at time t.

[0024] Step 204: Based on the operation node features corresponding to each operation node, the predecessor node features corresponding to each operation node, the successor node features corresponding to each operation node, the final machine embedding corresponding to each processable machine node corresponding to each operation node, and the deep learning network, determine the final operation embedding corresponding to each operation node at time t.

[0025] Step 205: Determine the global state embedding at time t based on the final machine embeddings corresponding to all machine nodes at time t and the final operation embeddings corresponding to all operation nodes at time t.

[0026] Step 206: Based on the global state embedding and reinforcement learning network, determine the probability distribution of all possible actions at time t. The possible actions are operation-machine pairs that meet preset conditions, namely, the preceding operation of the operation has been processed, the operation can be processed at time t, and the machine that can process the operation is in an idle state.

[0027] Step 207: Based on the probability distribution of all possible actions, determine the target action, which is to be performed in the environment of the flexible workshop, and generate an immediate reward to be sent to the reinforcement learning network.

[0028] Step 208: Update the scheduling state according to the target action to obtain the updated dynamic heterogeneous graph.

[0029] Step 209: Repeat steps 202 to 208 until all operations are assigned to the appropriate machines and the scheduling is completed.

[0030] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments of the present invention. It should be noted that the terms "first" and "second" involved in the embodiments of the present invention do not have limitations in terms of size, order, or quantity, but are only used to distinguish them in name. For example, "first attention coefficient" and "second attention coefficient" are used to distinguish two different attention coefficients.

[0031] First, in conjunction with the embodiments of the present invention, a detailed description will be given of step 201 above, namely, "obtaining a dynamic heterogeneous graph of the flexible workshop, the dynamic heterogeneous graph including the scheduling state of the flexible workshop, the scheduling state including operation nodes, machine nodes, operation-machine pairs and machine-machine pairs".

[0032] In embodiments of the present invention, from such Figure 3a The traditional disjunctive graph shown (which cannot represent the complex processing constraints and semantic relationships in FJSP) has evolved into something like... Figure 3b The diagram shown depicts a heterogeneous system without inter-machine competition, while the diagram with inter-machine competition introduced is as follows: Figure 3c The diagram shows a dynamic heterogeneity. (Example:) Figure 3c In the dynamic heterogeneous graph shown, the node Start is the starting node, and it is an empty node. For example... Figure 3c In the dynamic heterogeneous graph shown, the node End is the end node, and it is also an empty node. For example... Figure 3c The operational nodes in the dynamic heterogeneous graph shown include O 11 O 12 O 13 O 21 O 22 O 31 O 32 O33 .like Figure 3c The machine nodes in the dynamic heterogeneous graph shown include M1, M2, and M3. For example... Figure 3c The operation-machine pair in the dynamic heterogeneous graph shown is O. 11 -M2、O 12 -M1、O 12 -M3、O 13 -M2、O 13 -M3、O 21 -M1、O 22 -M3、O 31 -M1、O 32 -M1、O 32 -M2、O 33 -M2、O 33 -M3. For example... Figure 3c The machine-machine pairs shown in the dynamic heterogeneous graph are M1-M2, M2-M3, and M1-M3. Machine-machine pairs represent a competitive relationship between machines. For example, M1-M2 indicates a competitive relationship between machine M1 and machine M2.

[0033] The following describes step 202, namely, "obtaining the features of the scheduling state at time t, wherein the features of the scheduling state include operation node features, machine node features, operation-machine pair features, and machine-machine pair features", in detail with reference to the embodiments of the present invention.

[0034] In this embodiment of the invention, it should be noted that the tasks in the flexible workshop and the machines that complete the tasks are defined. The task set J = {J1, J2, ..., J...} n1}, machine set M''={M1, M2, ..., M n2}, Operation set O = {O 11 O 12 ..., O ij}, where n1, n2, i, j are positive integers, and J g M represents any job in the job set. k O represents any machine in the machine set. ij Let represent any operation in the operation set, where i represents the workpiece number, j represents the operation number, g∈[1, n1], k∈[1, n2]. Operation node characteristics include: state s ij Processing time p ijk Operation node O ij The number of adjacent machines N t (O) ij ), start time b ij The remaining number of unscheduled operations (us) g Homework completion time c g Operation node Oij Remaining workload of the workpiece Shortest processing time Relative position index ij represents the node number, k represents the machine number, t represents the time step, us represents unscheduled, g represents the job number, and s represents the status. ij It is a binary value used to represent the operation node O. ij Has it been scheduled at time step t, and processing time p? ijk Used to characterize operation node O ij In machine M k The actual or estimated processing time, start time (used to characterize the estimated or actual start time based on the scheduling status), and number of adjacent machines N. t (O) ij ) represents the node O that can be processed in a dynamic heterogeneous graph. ij Number of machine nodes, number of operation nodes O i Remaining workload of the workpiece Used to characterize this operation node O ij The sum of the average processing times of all unscheduled operations in the workpiece, and the shortest processing time. Used to characterize this operation node O ij At this operation node O ij Shortest processing time on all available machines, relative location index Used to characterize this operation node O ij At this operation node O ij The normalized position within the operation sequence of the workpiece. Machine node characteristics include: available time T. k Machine M k Number of adjacent operations N t (M) k ) Utilization rate U k Workload to be processed Available time T k Used to characterize machine M k The time it takes to complete all operations assigned to it and be able to accept new operations is the utilization rate U. k Used to characterize machine M k The ratio of processing time to total time, where the total time is the machine time (M). k The sum of processing time and idle time, and the number of adjacent operations N t (M) k ) is a dynamic heterogeneous graph that can be machine-defined. k Number of processing operations, amount of work to be processed Used to represent all that can be generated by machine M kThe sum of the average processing times of operations that have been processed but not yet scheduled. Operation-machine pair features are used to characterize operation node O. ij In machine M k The actual processing time. Machine-machine pair features are used to quantify the competition intensity between two machines; their feature values ​​(i.e., the features of the edges between machines) represent the number of operations that both machines can jointly process and that have not yet been scheduled at a given time. For example... Figure 3c As shown, the number of operations that machines M1 and M3 can jointly process that have not yet been scheduled at a certain time is 1, i.e., operation node O. 12 The number of operations that machines M1 and M2 can jointly process that have not yet been scheduled at any given time is 1, i.e., operation node O. 32 The number of operations that machines M2 and M3 can jointly process that have not yet been scheduled at any given time is 2, i.e., operation node O. 13 and O 33 It should be noted that, if... Figure 3c The number between M1 and M2 is 0, indicating that there is no edge between them, meaning there is no competition between them. Therefore, the number of operations that M1 and M2 can jointly process and that have not yet been scheduled at any given time is 0.

[0035] The following describes in detail step 203, namely, "determining the final machine embedding corresponding to all machine nodes at time t based on the machine node features, operation-machine pair features, machine-machine pair features, and deep learning network corresponding to all machine nodes", in conjunction with the embodiments of the present invention.

[0036] In this embodiment of the invention, the deep learning network is a Relation-Aware Heterogeneous Graph Neural Network (RAHGNN).

[0037] In one possible implementation of this invention, the machine node features corresponding to each machine node and the machine-machine pair features corresponding to the machine nodes connected to that machine node are concatenated to determine a first combined feature. Based on the machine node features and the first combined feature, and using the Graph Attention Network (GAT) in RAHGNN, a competition relationship embedding representation between any machine node and any machine node in its neighborhood is determined. The operation node features corresponding to each operation node and the operation-machine pair features corresponding to the machine nodes connected to that operation node are concatenated to determine a second combined feature. Based on the machine node features and the second combined feature, and using the GAT in RAHGNN, a processing relationship embedding representation between any machine node and the operation nodes that that machine node can process is determined. Based on the competition relationship embedding representation and the processing relationship embedding representation, the final machine embeddings corresponding to all machine nodes at time t are determined. This invention provides an independent, dedicated GAT module for each relationship in the dynamic heterogeneous graph (such as priority relationships between operations, processing relationships between operations and machines, and competition relationships between machines).

[0038] Specifically, based on machine node features and a first combined feature, and using the Gaussian Attention Parameter (GAT) in RAHGNN, the competitive relationship embedding representation between any machine node and any machine node in its neighborhood is determined. This can be achieved by: determining the first attention coefficient between any machine node and any machine node in its neighborhood based on the machine node features and the first combined feature, and using the GAT in RAHGNN; determining the second attention coefficient of the machine node itself based on the machine node features and the GAT in RAHGNN; normalizing the first and second attention coefficients to obtain normalized first and second attention coefficients; and determining the competitive relationship embedding representation between any machine node and any machine node in its neighborhood based on the normalized first and second attention coefficients, the machine node features, and the first combined feature.

[0039] For example, in, This is the first combination of features. For machine node M k Corresponding machine node characteristics For machine node M k With machine node M k The machine-machine pair features are composed of machine-machine pairs, and l is the l-th layer of RAHGNN. in, The first attention coefficient, Here, is the activation function, 'a' is the learnable parameter matrix, 'W' is the linear transformation matrix, and 'M' represents the corresponding machine node M. k M' is used to represent the corresponding machine node M k '. in, This is the second attention coefficient.

[0040] in, Embedding representation for competitive relationships. For activation function, The normalized second attention coefficient, The first attention coefficient after normalization. Represents machine node M k The set of neighboring machine nodes.

[0041] Based on machine node features and operator-machine pair features, and using the Gaussian Attention Parameter (GAT) in RAHGNN, the processing relationship embedding representation between any machine node and the operator nodes that the machine node can process is determined. Specifically, this can be achieved as follows: Based on machine node features and the second combined features, and using the GAT in RAHGNN, the third attention coefficient between any machine node and the operator nodes that the machine node can process is determined; based on machine node features and the GAT in RAHGNN, the fourth attention coefficient of the machine node itself is determined; the third and fourth attention coefficients are normalized to obtain normalized third and fourth attention coefficients; and based on the normalized third and fourth attention coefficients, the machine node features, and the second combined features, the processing relationship embedding representation between any machine node and the operator nodes that the machine node can process is determined.

[0042] For example, in, This is the second combination feature. For operation node O ij Corresponding operation node characteristics, For operation node O ij With machine node M k The operation-machine pair corresponds to the operation-machine pair feature. in, The third attention coefficient is the third attention coefficient, and the fourth attention coefficient is the second attention coefficient.

[0043] in, Embedded representation for processing relationships = , is the normalized fourth attention coefficient. The normalized third attention coefficient, Represents machine node M k The set of neighboring operation nodes.

[0044] Based on the competition relationship embedding representation and the processing relationship embedding representation, the final machine embedding corresponding to each machine node at time t can be determined by concatenating the competition relationship embedding representation and the processing relationship embedding representation through a multilayer perceptron (MLP) to determine the final machine embedding corresponding to each machine node at time t.

[0045] For example, the final machine embedding .

[0046] The following describes in detail step 204, namely, "determining the final operation embeddings corresponding to all operation nodes at time t based on the operation node features corresponding to all operation nodes, the predecessor node features corresponding to all operation nodes, the successor node features corresponding to all operation nodes, the final machine embeddings corresponding to the processable machine nodes corresponding to all operation nodes, and the deep learning network," in conjunction with embodiments of the present invention.

[0047] In one embodiment of the invention, as a possible implementation, the target operation embeddings for all operation nodes at time t are determined based on the operation node features, the predecessor node features, the successor node features, the final machine embeddings for the processable machine nodes, and the GAT in the RAHGNN. It should be noted that residual connections are integrated in each layer of the RAHGNN to alleviate the oversmoothing problem in deep graph neural networks, ensuring stable propagation of information and gradients across multiple layers, thereby supporting deeper and more effective representation learning. The final operation embeddings for all operation nodes at time t are determined based on the target operation embeddings and the obtained residual connection embeddings.

[0048] Specifically, the target operation embeddings for all operations at time t are determined based on the operation node features, predecessor node features, successor node features, final machine embeddings for the processable machine nodes, and the GAT in RAHGNN. This can be achieved by: aggregating the predecessor node features of multiple predecessor nodes for each operation node to obtain a predecessor aggregation vector for each operation node; aggregating the successor node features of multiple successor nodes for each operation node to obtain a successor aggregation vector for each operation node; aggregating the final machine embeddings of the machine nodes connected to each operation node to obtain a machine aggregation vector for each operation node; and finally, based on the operation node features, predecessor aggregation vectors, successor aggregation vectors, and machine aggregation vectors for each operation node, the target operation embeddings for all operations at time t are determined. It should be noted that the aforementioned aggregation is performed using GAT. By using this divide-and-conquer and then fusion approach, the problem of "averaging" semantic information of different natures during the aggregation process is effectively avoided, ensuring the integrity of the semantics and ultimately obtaining node representations that have a deep understanding of operation priority, operation-machine matching degree, and machine competition situation.

[0049] For example, in, Embedded for target operation, Used to characterize nodes that can be processed and operated. machine nodes, Used to characterize operation nodes The predecessor node, Used to characterize operation nodes The successor node, Used to characterize operation nodes .

[0050] in, Embedded for final operation, To obtain the residual connection, i.e., operation node O ij The corresponding operation node characteristics.

[0051] The following describes step 205, namely, "determining the global state embedding at time t based on the final machine embeddings corresponding to all machine nodes at time t and the final operation embeddings corresponding to all operation nodes at time t", in conjunction with embodiments of the present invention.

[0052] One possible implementation is to perform average pooling on the final operation embeddings for all operation nodes and the final machine embeddings for all machine nodes, resulting in two d-dimensional vectors, where d is a positive integer. Then, concatenating these two d-dimensional vectors yields the final global state embedding.

[0053] Specifically, in, For global state embedding, L is the total number of layers in RAHGNN, and POOL represents average pooling.

[0054] The following describes in detail step 206, namely, "determine the probability distribution of all possible actions at time t based on the global state embedding and reinforcement learning network, and the possible actions are operation-machine pairs that satisfy preset conditions, namely, the preceding operation of the operation has been processed, the operation can be processed at time t, and the machine that can process the operation is in an idle state," in conjunction with the embodiments of the present invention.

[0055] In one embodiment of the invention, as a possible implementation, the reinforcement learning network is a proximal policy optimization algorithm based on the actor-critic framework.

[0056] As one possible implementation, determining the probability distribution of all possible actions at time t based on the global state embedding and reinforcement learning network can be specifically as follows: using an actor-based policy network, a score is obtained based on the final machine embedding corresponding to each machine, the final operation embedding corresponding to each operation, and the global state embedding; the probability distribution of all possible actions at time t is determined based on the score and the softmax function.

[0057] Specifically, The score is: The probability distribution of all possible actions at time t is as follows: in, Represents the current available action out of all available actions. Represents any one of all possible actions. It represents the set of all possible actions.

[0058] It should be noted that infeasibility masking is used for infeasible actions (i.e., illegal actions, which are operations-machine pairs that do not meet preset conditions).

[0059] The following describes in detail step 207, namely, "determining the target action based on the probability distribution of all possible actions, the target action being executed in the environment of a flexible workshop, and generating an immediate reward to be sent to the reinforcement learning network," in conjunction with an embodiment of the present invention.

[0060] In this embodiment of the invention, a target action is determined based on the probability distribution of all possible actions. The target action is used to perform in the environment of a flexible workshop, and an immediate reward is generated and sent to the reinforcement learning network.

[0061] It should be noted that the global state is embedded into the input of the critic's value network to predict the scheduling state value, which is used to characterize the quality of the action. A positive prediction result indicates a good action, and a negative prediction result indicates a poor action.

[0062] The following describes step 208, namely "updating the scheduling state according to the target action and obtaining the updated dynamic heterogeneous graph", in detail with reference to the embodiments of the present invention.

[0063] In this embodiment of the invention, after the FJSP environment performs an action, it transitions to the next scheduling state and provides an immediate reward.

[0064] In other words, in order to achieve the minimum possible final completion time Define the immediate reward at each time step as the target action.

[0065] Specifically, the instant reward is: In other words, the final completion time at time t minus the final completion time at time t+1 is the instant reward. If the final completion time decreases, a positive instant reward will be obtained; otherwise, a negative instant reward will be obtained.

[0066] The cumulative rewards are: By setting discount factors The cumulative rewards are simplified as follows: Due to the initial completion time The cumulative reward is constant for a given instance, therefore maximizing the cumulative reward is possible. Mathematically, this is equivalent to minimizing the final completion time. This equivalence ensures direct consistency with the optimization objective.

[0067] Regarding scheduling strategies, for any state probability distribution Define as an action set The conditional probability distribution. At each time step t, according to the probability distribution... Choose one operation-machine pair as the target action. Deep reinforcement learning will be designed to generate a probability distribution of the action. And adjust its parameters to maximize cumulative rewards. .

[0068] The following describes step 209, namely, "repeating steps 202 to 208 until all operations are assigned to the corresponding machines and the scheduling is completed," in conjunction with an embodiment of the present invention.

[0069] In this embodiment of the invention, steps 202 to 208 are repeated until all operations are assigned to the corresponding machines and the scheduling is completed.

[0070] The beneficial effects of the technical solution provided by the embodiments of the present invention are as follows: a dynamic heterogeneous graph is introduced to represent the entire scheduling state of the flexible job shop. This graph not only effectively captures the priority relationships between operations and the processing relationships between operations and machines, but also captures the key competitive relationships between machines. Based on the competition relationship embedding representation and the processing relationship embedding representation, the final machine embeddings corresponding to all machine nodes at time t are determined, and these final machine embeddings are used as inputs to determine the final operation embeddings. This aims to generate highly expressive and information-rich embeddings for machine nodes and operation nodes.

[0071] Figure 4 Another flowchart illustrating a scheduling method for flexible workshops is shown. Figure 4 As shown, this method can be implemented through the following steps: First, based on the traditional disjunctive graph, a competition relationship between machines is introduced to form a dynamic heterogeneous graph. This dynamic heterogeneous graph includes the scheduling state of the flexible job shop. This scheduling state includes operator nodes, machine nodes, operator-machine pairs, and machine-machine pairs.

[0072] Secondly, the characteristics of the scheduling state are obtained. These characteristics include the characteristics of the operation nodes (see example...). Figure 4 OO relation), machine node characteristics (see example) Figure 4 The MM relation shown), operation-machine pair features (see example) Figure 4 The OM relation or MO relation shown) and machine-machine pair features (see example) Figure 4 (The MM relation shown). It should be noted that, as Figure 4The MM relation shown includes not only machine-machine pair features (i.e., machine-machine pair features between M1 and M2, M2 and M3, and M1 and M3), but also the individual machine node features (i.e., the machine node features corresponding to machine node M1, machine node M2, and machine node M3). Similarly, the OM relation, MO relation, and OO relation are also described in the same way.

[0073] Then, the machine node features corresponding to each machine node and the machine-machine pair features corresponding to the machine nodes connected to that machine node are concatenated to determine the first combined feature. Based on the machine node features and the first combined feature, and using GAT in RAHGNN, the competition relationship embedding representation between any machine node and any machine node in its neighborhood is determined, i.e. Figure 4 In That is, the one mentioned above The operation node features corresponding to each operation node and the operation-machine pair features corresponding to the machine nodes connected to that operation node are concatenated to determine the second combined feature. Based on the machine node features and the second combined feature, and using GAT in RAHGNN, the processing relationship embedding representation between any machine node and the operation nodes that any machine node can process is determined, i.e. Figure 4 In That is, the one mentioned above Based on the competition relationship embedding representation and the processing relationship embedding representation, the final machine embedding corresponding to each machine node is determined according to MLP. =h m .

[0074] Furthermore, the predecessor node features corresponding to multiple predecessor nodes of each operation node are aggregated to obtain the predecessor aggregation vector corresponding to each operation node, i.e. Figure 4 In ; Aggregate the successor node features corresponding to multiple successor nodes for each operation node to obtain the successor aggregation vector corresponding to each operation node, i.e. Figure 4 In ; Aggregate the final machine embeddings corresponding to the machine nodes connected to each operation node to obtain the machine aggregation vector corresponding to each operation node, i.e. Figure 4 In Based on the characteristics of each operation node, i.e. Figure 4 in Based on MLP, the predecessor aggregation vector, successor aggregation vector, and machine aggregation vector for each operation node are used to determine the target operation embedding for each operation node. The final operation embedding is then determined based on the target operation embedding and the obtained residual connections. Figure 4 h o .

[0075] Finally, the final operation embedding, i.e., h, is defined for each operation node. o And the final machine embedding corresponding to each of the machine nodes, i.e., h m Perform average pooling separately to obtain two d-dimensional vectors. Then, concatenate the two d-dimensional vectors to obtain the final global state embedding.

[0076] According to another embodiment, a scheduling device for a flexible workshop is provided. Figure 5 A schematic block diagram of a scheduling device for a flexible workshop according to one embodiment is shown. Figure 5 As shown, the device 500 may include: a dynamic heterogeneous graph acquisition module 501, a scheduling state feature acquisition module 502, a final machine embedding determination module 503, a final operation embedding determination module 504, a global state embedding determination module 505, a probability distribution determination module 506, a target action determination module 507, a scheduling state update module 508, and a repetition processing module 509. The main functions of each component module are as follows: The dynamic heterogeneous graph acquisition module 501 is used to acquire the dynamic heterogeneous graph of the flexible work shop. The dynamic heterogeneous graph includes the scheduling status of the flexible work shop, and the scheduling status includes operation nodes, machine nodes, operation-machine pairs and machine-machine pairs. The scheduling state feature acquisition module 502 is used to acquire the features of the scheduling state at time t. The features of the scheduling state include operation node features, machine node features, operation-machine pair features, and machine-machine pair features. The final machine embedding determination module 503 is used to determine the final machine embedding of each machine node at time t based on the machine node features, operation-machine pair features, machine-machine pair features and the deep learning network. The final operation embedding determination module 504 is used to determine the final operation embedding of all operation nodes at time t based on the operation node features corresponding to all operation nodes, the predecessor node features corresponding to all operation nodes, the successor node features corresponding to all operation nodes, the final machine embedding corresponding to the processable machine nodes corresponding to all operation nodes, and the deep learning network. The global state embedding determination module 505 is used to determine the global state embedding at time t based on the final machine embeddings corresponding to all machine nodes at time t and the final operation embeddings corresponding to all operation nodes at time t. The probability distribution determination module 506 is used to determine the probability distribution of all possible actions at time t based on the global state embedding and reinforcement learning network. The possible actions are operation-machine pairs that meet preset conditions, namely, the preceding operation of the operation has been processed, the operation can be processed at time t, and the machine that can process the operation is in an idle state. The target action determination module 507 is used to determine a target action based on the probability distribution of all possible actions, the target action being performed in the environment of the flexible work workshop, and generating an immediate reward to be sent to the reinforcement learning network; The scheduling status update module 508 is used to update the scheduling status according to the target action to obtain the updated dynamic heterogeneous graph; The repeat processing module 509 is used to repeat the functions of the aforementioned scheduling status feature acquisition module to the scheduling status update module until all operations are assigned to the corresponding machines to complete the scheduling.

[0077] In one possible implementation, the final machine embedding determination module 503 includes: a first combined feature determination module, a competitive relationship embedding representation determination module, a second combined feature determination module, a processing relationship embedding representation determination module, and a first processing module; The first combined feature determination module is used to concatenate the machine node features corresponding to each machine node and the machine-machine pair features corresponding to the machine nodes connected to that machine node to determine the first combined feature. The competition relationship embedding representation determination module is used to determine the competition relationship embedding representation between any machine node and any machine node in the neighborhood of any machine node, based on the machine node features and the first combined features and the deep learning network. The second combined feature determination module is used to concatenate the operation node feature corresponding to each operation node and the operation-machine pair feature corresponding to the machine node connected to that operation node to determine the second combined feature. The processing relationship embedding representation determination module is used to determine the processing relationship embedding representation between any machine node and the operation node that can be processed by any machine node based on the machine node features and the second combined features, and on the basis of the deep learning network. The first processing module is used to determine the final machine embedding corresponding to all machine nodes at time t based on the competition relationship embedding representation and the processing relationship embedding representation.

[0078] In one possible implementation, the competition relationship embedding representation determination module is specifically configured to: determine a first attention coefficient between any machine node and any machine node in its neighborhood based on the machine node features and the first combined features, using the deep learning network; determine a second attention coefficient for the machine node itself based on the machine node features and the deep learning network; normalize the first attention coefficient and the second attention coefficient to obtain normalized first attention coefficient and normalized second attention coefficient; and determine a competition relationship embedding representation between any machine node and any machine node in its neighborhood based on the normalized first attention coefficient, the normalized second attention coefficient, the machine node features, and the first combined features.

[0079] In one possible implementation, the processing relationship embedding representation determination module is specifically configured to: determine a third attention coefficient between any machine node and the operation node that the machine node can process, based on the machine node features and the second combined features, using the deep learning network; determine a fourth attention coefficient of the machine node itself, based on the machine node features and the deep learning network; normalize the third attention coefficient and the fourth attention coefficient to obtain normalized third attention coefficient and normalized fourth attention coefficient; and determine a processing relationship embedding representation between any machine node and the operation node that the machine node can process, based on the normalized third attention coefficient, the normalized fourth attention coefficient, the machine node features, and the second combined features.

[0080] In one possible implementation, the first processing module is specifically used to concatenate the competition relationship embedding representation and the processing relationship embedding representation through a multilayer perceptron to determine the final machine embedding corresponding to all machine nodes at time t.

[0081] In one possible implementation, the final operation embedding determination module 504 includes: a target operation embedding determination module and a second processing module; The target operation embedding determination module is used to determine the target operation embeddings of all operation nodes at time t based on the operation node features, the predecessor node features, the successor node features, the final machine embeddings of the processable machine nodes, and the deep learning network. The second processing module is used to determine the final operation embedding corresponding to each operation node at time t based on the target operation embedding and the obtained residual connection embedding for each operation node at time t.

[0082] In one possible implementation, the target operation embedding determination module is specifically used to aggregate the predecessor node features corresponding to multiple predecessor nodes of each operation node to obtain a predecessor aggregation vector corresponding to each operation node; aggregate the successor node features corresponding to multiple successor nodes of each operation node to obtain a successor aggregation vector corresponding to each operation node; aggregate the final machine embeddings corresponding to the machine nodes connected to each operation node to obtain a machine aggregation vector corresponding to each operation node; and determine the target operation embeddings corresponding to all operation nodes at time t based on the operation node features corresponding to each operation node, the predecessor aggregation vector corresponding to each operation node, the successor aggregation vector corresponding to each operation node, and the machine aggregation vector corresponding to each operation node.

[0083] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0084] In addition, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0085] And an electronic device, comprising: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0086] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0087] in, Figure 6An exemplary architecture of an electronic device is shown, which may specifically include a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, and a memory 620. The processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620 can communicate with each other via a communication bus 630.

[0088] The processor 610 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0089] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store the operating system 621 for controlling the operation of the electronic device 600, and the basic input / output system (BIOS) 622 for controlling the low-level operations of the electronic device 600. Additionally, it can store a web browser 623, a data storage management system 624, and a scheduling device 625 for flexible workshops, etc. The aforementioned scheduling device 625 for flexible workshops can be the application program that specifically implements the aforementioned steps in this embodiment of the invention. In summary, when the technical solution provided in this embodiment of the invention is implemented through software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0090] Input / output interface 613 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0091] Network interface 614 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0092] Bus 630 includes a pathway for transmitting information between various components of the device, such as processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620.

[0093] It should be noted that although the above-described device only shows the processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, memory 620, bus 630, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the embodiments of the present invention have been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scheduling method for flexible workshops, characterized in that, include: Step 1: Obtain the dynamic heterogeneity diagram of the flexible workshop. The dynamic heterogeneity diagram includes the scheduling status of the flexible workshop, which includes operation nodes, machine nodes, operation-machine pairs, and machine-machine pairs. Step 2: Obtain the features of the scheduling state at time t. The features of the scheduling state include operation node features, machine node features, operation-machine pair features, and machine-machine pair features. Step 3: Based on the machine node features, operation-machine pair features, machine-machine pair features, and the deep learning network corresponding to each machine node, determine the final machine embedding corresponding to each machine node at time t. Step 4: Based on the operation node features corresponding to each operation node, the predecessor node features corresponding to each operation node, the successor node features corresponding to each operation node, the final machine embedding corresponding to each processable machine node corresponding to each operation node, and the deep learning network, determine the final operation embedding corresponding to each operation node at time t. Step 5: Determine the global state embedding at time t based on the final machine embeddings corresponding to all machine nodes at time t and the final operation embeddings corresponding to all operation nodes at time t. Step 6: Based on the global state embedding and reinforcement learning network, determine the probability distribution of all possible actions at time t. The possible actions are operation-machine pairs that satisfy preset conditions, namely, the preceding operation of the operation has been processed, the operation can be processed at time t, and the machine that can process the operation is in an idle state. Step 7: Based on the probability distribution of all possible actions, determine the target action, which is to be performed in the environment of the flexible work workshop, and generate an immediate reward to be sent to the reinforcement learning network; Step 8: Update the scheduling state according to the target action to obtain the updated dynamic heterogeneous graph; Step nine: Repeat steps two through eight until all operations are assigned to the appropriate machines and the scheduling is completed.

2. The method according to claim 1, characterized in that, The step of determining the final machine embeddings corresponding to all machine nodes at time t based on the machine node features, operation-machine pair features, machine-machine pair features, and the deep learning network in the features of the scheduling state includes: The first combined feature is determined by concatenating the machine node features corresponding to each machine node and the machine-machine pair features corresponding to the machine nodes connected to that machine node. Based on the machine node features and the first combined features, and using the deep learning network, determine the embedded representation of the competitive relationship between any machine node and any machine node in its neighborhood. The second combined feature is determined by concatenating the operation node feature corresponding to each operation node with the operation-machine pair feature corresponding to the machine node connected to that operation node. Based on the machine node features and the second combined features, and using the deep learning network, determine the embedded representation of the processing relationship between any machine node and the operation nodes that the machine node can process; Based on the competition relationship embedding representation and the processing relationship embedding representation, determine the final machine embedding corresponding to each machine node at time t.

3. The method according to claim 2, characterized in that, The step of determining the competition relationship embedding representation between any machine node and any machine node in its neighborhood based on the machine node features and the first combined features, using the deep learning network, includes: Based on the machine node features and the first combined features, and based on the deep learning network, a first attention coefficient is determined between any machine node and any machine node in its neighborhood. Based on the machine node characteristics and the deep learning network, the second attention coefficient of the machine node itself is determined. Normalize the first attention coefficient and the second attention coefficient to obtain the normalized first attention coefficient and the normalized second attention coefficient; Based on the normalized first attention coefficient, the normalized second attention coefficient, the machine node features, and the first combined features, the competition relationship embedding representation between any machine node and any machine node in the neighborhood of that machine node is determined.

4. The method according to claim 2, characterized in that, The step of determining the processing relationship embedding representation between any machine node and the operation nodes that can be processed by that machine node, based on the machine node features and the operation-machine pair features and the deep learning network, includes: Based on the machine node features and the second combined features, and based on the deep learning network, a third attention coefficient is determined between any machine node and the operation nodes that the machine node can process. Based on the machine node characteristics and the deep learning network, the fourth attention coefficient of the machine node itself is determined. The third attention coefficient and the fourth attention coefficient are normalized to obtain the normalized third attention coefficient and the normalized fourth attention coefficient; Based on the normalized third attention coefficient, the normalized fourth attention coefficient, the machine node features, and the second combined features, the processing relationship embedding representation between any machine node and the operation nodes that the machine node can process is determined.

5. The method according to claim 2, characterized in that, The step of determining the final machine embeddings corresponding to all machine nodes at time t based on the competition relationship embedding representation and the processing relationship embedding representation includes: The competition relationship embedding representation and the processing relationship embedding representation are concatenated using a multilayer perceptron to determine the final machine embedding corresponding to each machine node at time t.

6. The method according to claim 1, characterized in that, The step of determining the final operation embeddings for all operation nodes at time t based on the operation node features, the predecessor node features, the successor node features, the final machine embeddings for the processable machine nodes, and the deep learning network for each operation node includes: Based on the operation node features corresponding to all operation nodes, the predecessor node features corresponding to all operation nodes, the successor node features corresponding to all operation nodes, the final machine embedding corresponding to the processable machine node corresponding to all operation nodes, and the deep learning network, determine the target operation embedding corresponding to all operation nodes at time t. Based on the target operation embeddings and the obtained residual connection embeddings corresponding to all operation nodes at time t, the final operation embeddings corresponding to all operation nodes at time t are determined.

7. The method according to claim 6, characterized in that, The step of determining the target operation embeddings corresponding to all operation nodes at time t based on the operation node features corresponding to all operation nodes, the predecessor node features corresponding to all operation nodes, the successor node features corresponding to all operation nodes, the final machine embeddings corresponding to the processable machine nodes corresponding to all operation nodes, and the deep learning network includes: The predecessor node features corresponding to multiple predecessor nodes of each operation node are aggregated to obtain the predecessor aggregation vector corresponding to each operation node. The features of the successor nodes corresponding to multiple successor nodes of each operation node are aggregated to obtain the successor aggregation vector corresponding to each operation node. Aggregate the final machine embeddings corresponding to the machine nodes connected to each operation node to obtain the machine aggregation vector for each operation node; Based on the operation node features corresponding to each operation node, the predecessor aggregation vector corresponding to each operation node, the successor aggregation vector corresponding to each operation node, and the machine aggregation vector corresponding to each operation node, the target operation embeddings corresponding to all operation nodes at time t are determined.

8. A scheduling device for flexible work workshops, characterized in that, include: The dynamic heterogeneous graph acquisition module is used to acquire the dynamic heterogeneous graph of the flexible work shop. The dynamic heterogeneous graph includes the scheduling status of the flexible work shop, and the scheduling status includes operation nodes, machine nodes, operation-machine pairs and machine-machine pairs. The scheduling state feature acquisition module is used to acquire the features of the scheduling state at time t. The features of the scheduling state include operation node features, machine node features, operation-machine pair features, and machine-machine pair features. The final machine embedding determination module is used to determine the final machine embedding of each machine node at time t based on the machine node features, operation-machine pair features, machine-machine pair features, and the deep learning network. The final operation embedding determination module is used to determine the final operation embedding of all operation nodes at time t based on the operation node features, the predecessor node features, the successor node features, the final machine embedding of the processable machine nodes, and the deep learning network. The global state embedding determination module is used to determine the global state embedding at time t based on the final machine embeddings corresponding to all machine nodes at time t and the final operation embeddings corresponding to all operation nodes at time t. The probability distribution determination module is used to determine the probability distribution of all possible actions at time t based on the global state embedding and reinforcement learning network. The possible actions are operation-machine pairs that satisfy preset conditions, namely, the preceding operation of the operation has been processed, the operation can be processed at time t, and the machine that can process the operation is in an idle state. The target action determination module is used to determine the target action based on the probability distribution of all possible actions. The target action is to be performed in the environment of the flexible work workshop and generate an immediate reward to be sent to the reinforcement learning network. The scheduling state update module is used to update the scheduling state according to the target action to obtain the updated dynamic heterogeneous graph. The repetition processing module is used to repeat the functions of the aforementioned scheduling status feature acquisition module to the scheduling status update module until all operations are assigned to the corresponding machines to complete the scheduling.

9. An electronic device, characterized in that, include: The memory and the processor communicate with each other via a bus; The memory stores program instructions that can be executed by the processor, which can invoke the program instructions to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images