Method for optimizing workshop scheduling based on bottleneck-sensitive heterogeneous graph neural network

Abstract

This invention discloses a workshop scheduling optimization method based on a bottleneck-sensitive heterogeneous graph neural network. The method includes constructing a heterogeneous graph model containing process nodes and machine nodes to explicitly represent complex resource constraints. For the constructed heterogeneous graph, a corresponding graph neural network architecture is designed, integrating global workshop load information into the node interaction process, enabling the agent to dynamically identify and alleviate system bottlenecks. A mini-batch A2C neural network framework is used to implement related reinforcement learning training. This invention overcomes the limitation of traditional methods in handling flexible constraints by explicitly representing complex constraints between processes and machines through a heterogeneous graph structure. Integrating global load information into a graph attention mechanism enables the network to dynamically identify bottleneck resources in the production system and prioritize the scheduling of key processes. It also exhibits strong generalization ability; through the structured representation of the graph neural network, the learned strategy can be generalized to workshop environments of different sizes and configurations.

Description

Technical Field

[0001] This invention relates to the field of intelligent workshop scheduling methods, and in particular to a workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural networks. Background Technology

[0002] The flexible job shop scheduling problem, as a classic NP-hard combinatorial optimization problem, has significant application value in the manufacturing industry. Traditional solutions mainly include exact algorithms, heuristic algorithms, and metaheuristic algorithms. However, these methods have the following limitations: exact algorithms can only solve small-scale problems and have high computational complexity; heuristic algorithms rely on human experience and are difficult to adapt to dynamically changing production environments; metaheuristic algorithms require extensive parameter tuning, are computationally time-consuming, and are prone to getting trapped in local optima.

[0003] In recent years, Deep Reinforcement Learning (DRL) has provided a new approach to scheduling problems. However, existing DRL methods still face challenges when dealing with Functional JSP (FJSP), struggling to effectively model complex process-machine constraints; lacking dynamic awareness of bottlenecks in the production system; and most methods employ Graph Convolutional Networks (GCNs) or basic Graph Attention Networks (GATs) for node feature aggregation, a message-passing mechanism typically limited to the local neighborhood of a node. However, scheduling problems exhibit a significant bottleneck effect, where the maximum completion time of the entire system often depends on a few bottleneck machines on the critical path. Under local aggregation mechanisms, agents struggle to capture long-range dependencies, i.e., they fail to perceive global load distribution and potential bottlenecks. Therefore, agents tend to optimize the utilization of local machines while neglecting global load balancing, ultimately limiting the overall quality of the scheduling scheme. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a workshop scheduling optimization method based on a bottleneck-sensitive heterogeneous graph neural network; it is used to perceive workshop bottlenecks, optimize the agent's decision-making process, and enable the agent to obtain the optimal scheduling scheme.

[0005] Technical solution: The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network of the present invention includes:

[0006] S1. Construct a heterogeneous graph model containing process nodes and machine nodes to explicitly represent complex resource constraints.

[0007] S2. For the constructed heterogeneous graph, design the corresponding graph neural network architecture, integrate the global load information of the workshop into the node interaction process, so that the agent can dynamically identify and alleviate the system bottleneck.

[0008] S3. Employ a mini-batch A2C neural network framework to implement relevant reinforcement learning training.

[0009] Further, step S1 includes:

[0010] S11. Constructing a heterogeneous graph ,

[0011] in, Represents the nodes in the graph. ,node It consists of two types of node sets: process node sets and process node sets. This represents all the processes in the operation; and the set of machine nodes. This represents all available machines in the workshop; This represents an edge in a heterogeneous graph;

[0012] S12. Construct the feature vector of each process node. Defined as:

[0013] ,

[0014] in, Whether the process is completed; Whether the process is ready, Represents the relative position of the process in the workpiece operation;

[0015] S13, Construct machine nodes Node feature vectors Defined as:

[0016] ,

[0017] in, This represents the relative busyness of the machine. This indicates the percentage of tasks that the machine has completed. This represents the percentage of the total number of processes that the machine can still process.

[0018] S14, Edge Weight Construction, edge weights Processing time set to normalization ,in For workpiece process In the machine Processing time Let be the scaling factor. Scaling factor for the average completion time of all processes. .

[0019] Furthermore, the edges in the heterogeneous graph in step S11 include two types of connection relationships: connecting edges, which are used to connect adjacent processes of the same workpiece. One type of relationship represents the sequential constraint between processes; another type of relationship is the disjunctive edge, used to connect the relationship between processes and machines. , indicating machine With process The ability to perform processing.

[0020] Further, step S2 includes:

[0021] S21. By using two independent linear transformation layers, the process node features and machine node features are mapped to a unified hidden space. For node features... and The transformation formula is as follows:

[0022] ,

[0023] ,

[0024] in, , The weight matrix is ​​a learnable matrix. , The bias term is ReLU, and the activation function is ReLU.

[0025] S22. Extract the global context vector based on the heterogeneous graph design completed in step S1. For the feature matrix of machine nodes Perform aggregation:

[0026] ,

[0027] in, For max pooling operation, For average pooling operation, The feature matrix of machine nodes, This represents a vector concatenation operation. It is a multilayer perceptron;

[0028] S23. Features of adjacent nodes after embedding and Pay attention The computation is performed using a bottleneck-aware graph attention mechanism, based on global context vectors. It modifies the strength between connections to generate scheduling operations suitable for the current load situation;

[0029] S24. Multi-stage bidirectional message passing is achieved through a graph attention layer that perceives three bottlenecks: the first stage propagates from machine to process; the second stage propagates from process to machine; and the third stage propagates again from machine to process.

[0030] S25. By aggregating all node information through attention-based pooling layers, and utilizing global context vectors... Key features are extracted from both process and machine perspectives to identify the key processes and machines that have the greatest impact on the current scheduling objectives.

[0031] Furthermore, step S23 also employs residual connections to prevent the gradient information of the deep network from disappearing. The feature update formula for node u is:

[0032]

[0033] in, and Adjacent nodes for The neighborhood group, This is a learnable attention vector.

[0034] Further, step S25 includes:

[0035] First, extract the key vectors of the nodes. The formula is as follows:

[0036] ,

[0037] ,

[0038] ,

[0039] ,

[0040] in: For attention score, For normalized attention weights, The final state representation vector is the aggregated global pooled features. Based on the global characteristics of the process Machine global characteristics and the original global context vector It is pieced together to form a complete description of the current state of the workshop.

[0041] Furthermore, step 3 includes mask matrix design, policy network design, value network design, reward function design, and loss function design.

[0042] Furthermore, the value network design is based on global feature estimation of state value, and value estimation... as follows:

[0043] ,

[0044] The critic is a two-layer MLP network.

[0045] Furthermore, the reward function design employs a differential reward function based on the maximum completion time, which calculates the differential minimum completion time for each step, i.e. ,in This refers to the completion time after the previous action is completed, followed by... The negative value is used as the immediate reward at each time step, i.e. Once all operations have been scheduled, the cumulative reward is the negative of the maximum completion time. .

[0046] Furthermore, the loss function design employs a mini-batch update strategy, and the total loss function includes policy loss, value loss, and entropy regularization term.

[0047] ,

[0048] Where B is the number of trajectories collected during mini-batch training. For the first The loss from training the actor once.

[0049] ,

[0050] The difference between the actual return and the review model. For the first Training The loss,

[0051] ,

[0052] This is the entropy regularization loss.

[0053] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0054] (1) This invention uses a heterogeneous graph structure to explicitly characterize the complex constraint relationship between processes and machines, thus overcoming the limitations of traditional methods in handling flexible constraints.

[0055] (2) This invention integrates global load information into the graph attention mechanism, enabling the network to dynamically identify bottleneck resources in the production system and prioritize the scheduling of key processes;

[0056] (3) The present invention guides the agent to escape local optima based on the differential reward function of the maximum completion time, and achieves balanced utilization of resources while minimizing the maximum completion time;

[0057] (4) The present invention has strong generalization ability. Through the structured representation of graph neural network, the learned strategy can be generalized to workshop environments of different sizes and configurations. Attached Figure Description

[0058] Figure 1 This is a flowchart of the algorithm of the present invention;

[0059] Figure 2 This is an iterative curve diagram of the present invention;

[0060] Figure 3 This is the Gantt chart for scheduling the algorithm of this invention. Detailed Implementation

[0061] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0062] Example 1:

[0063] The basic algorithm flowchart of this invention is as follows: Figure 1 As shown, taking the Brandimarte standard test set Mk03 as an example, this instance contains 15 workpieces, 8 machines, and a total of 150 operations. The implementation steps are as follows:

[0064] Using Python, the Mk03 instance data is first read, including the process sequence, available machine set, and processing time for each workpiece. A heterogeneous graph is constructed based on the current scheduling state. Then, embedding and global information extraction are performed based on the heterogeneous graph. The embedded vector is concatenated with the global information and fed into the bottleneck perception layer to obtain the bottleneck-perceived vector information. Finally, global context pooling is used to obtain the global bottleneck perception feature vector. Based on the extracted feature vector, the Actor and Crtic networks are input for state and action estimation. The difference between the estimated and actual values ​​is corrected using backpropagation to adjust the neural network parameters. This process is repeated until an effective solution is obtained.

[0065] The specific implementation plan for this algorithm is as follows:

[0066] S1, Heterogeneous Graph Construction

[0067] Based on the existing nodes, extract the corresponding feature vectors of the process nodes and machine nodes, which are calculated in real time according to the current scheduling status. First, extract the feature vectors of the process nodes. , Define it as , in: Whether the process is completed; This indicates whether the process is ready, meaning whether the preceding process has been completed and the process itself has not yet been processed. This represents the relative position of the process within the workpiece operation. Next, the machine node feature vector is extracted. Define it as , in This represents the relative busyness of the machine, that is, the remaining time the machine is occupied relative to the current time. This represents the percentage of tasks the machine has completed, used to indicate the machine's historical throughput. This represents the percentage of the total number of processes that the machine can still handle. It is used to measure the importance of the machine. The edge weights are then recalculated. Calculate the average completion time of the process. Scaling factor Through formula Calculate edge weights ,in For workpiece process In the machine The processing time on the surface.

[0068] S2, Heterogeneous Graph Neural Construction:

[0069] Processing is performed based on the extracted heterogeneous graph feature vectors.

[0070] First, node embedding is performed based on the following transformation formula.

[0071] ,

[0072] ,

[0073] Obtain the embedded node features and .in, , The weight matrix is ​​a learnable matrix. , Let be the bias term matrix, and ReLU be the activation function.

[0074] Secondly, global context extraction is performed based on formulas. For machine node feature matrix Aggregation is performed to obtain the global context vector. ,in: This is a max-pooling operation used to identify the machine currently under the highest load, i.e., the dynamic bottleneck. This is an average pooling operation used to sense the overall load level of the workshop. The feature matrix of machine nodes, This represents a vector concatenation operation. It is a multilayer perceptron.

[0075] Secondly, based on the features of neighboring nodes after embedding and The calculation for bottleneck awareness in the global context is as follows:

[0076] ,

[0077] ,

[0078] in, For an improved version of the ReLU activation function, learn weights ,vector , and Adjacent nodes for The neighborhood group, This is a learnable attention vector.

[0079] Next, based on the graph attention mechanism, the feature vector of node u is updated, referring to the formula.

[0080] Finally, global context pooling is used to aggregate information from all nodes to obtain the key vector. The formula is as follows: For each node in the graph ,in Include all process nodes and machine nodes, and calculate their attention scores. The formula is as follows:

[0081] ,

[0082] in, This is a learnable attention weight vector. For node feature transformation matrix and This is the global context transformation matrix. The hyperbolic tangent activation function is used. The following is done using the formula... Attention score Normalization is performed to obtain normalized attention weights. Finally, global features are aggregated based on attention weights, referring to the formula. The aggregated global pooling features are obtained. The final state representation vector Based on the global characteristics of the process Machine global characteristics and the original global context vector It is pieced together to form a complete description of the current state of the workshop, that is... .

[0083] S3, Deep Reinforcement Learning Training

[0084] First, a mask matrix is ​​created based on the existing heterogeneous graph. The specific formula is as follows. ,

[0085] ,

[0086] Secondly, the mask matrix is ​​combined with the workshop state description. Input the actor network to obtain the probability distribution of actions. for: ,in In the state Make The probability of an action. That is, the machine operation pair (O,M) of the process.

[0087] Secondly, provide a complete description of the workshop status. Inputting the data into the commentator network will yield the state value. , By using random sampling, agent actions are extracted, and the differential minimum completion time is calculated. ,in This refers to the completion time after the previous action is completed, followed by... The negative value is used as the immediate reward at each time step, i.e. The process of selecting actions is repeated until all scheduling tasks in the workshop are completed, and all rewards are recorded. Received the record reward for this event. .

[0088] Finally, for the recorded data, loss is calculated using a mini-batch update strategy, with mini-batch B set to 10, and the average loss is calculated. The specific formula is as follows: Where: B is the number of trajectories collected during mini-batch training. For the first The loss of training the actor once. , The difference between the actual return and the review model. For the first Training The loss. , The loss is regularized to entropy. The loss is backpropagated to correct the previous learning parameters. The process is repeated 5000 times to complete the training.

[0089] S4, Experimental Verification

[0090] Based on the above scheme, the Brandimarte standard test set was used for testing. Taking Mk03 as an example, the mini-batch update B was set to 10, and the greedy policy was evaluated every 20 rounds. The iteration curve of the algorithm proposed in this invention is as follows: Figure 2 As shown, the optimal solution 204 was found in approximately 100 rounds, and remained stable at 204 even in the later stages of training, demonstrating its high level of sophistication. The completed scheduling Gantt chart is shown below. Figure 3 As shown.

[0091] Table 1: Comparison of Multiple Algorithms

[0092]

[0093] Meanwhile, a comparison was made with several existing algorithms. As shown in Table 1, the algorithm of this invention achieved optimal or near-optimal performance in most cases, especially demonstrating significant advantages in large-scale cases such as MK09 and MK10. The algorithm of this invention plays a crucial role in handling complex load balancing, avoiding the performance degradation caused by local perspective issues in traditional methods, and is therefore more advanced.

Claims

1. A workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural networks, characterized in that, include: S1. Construct a heterogeneous graph model containing process nodes and machine nodes to explicitly represent complex resource constraints. S2. For the constructed heterogeneous graph, design the corresponding graph neural network architecture, integrate the global load information of the workshop into the node interaction process, so that the agent can dynamically identify and alleviate the system bottleneck. S3. Employ a mini-batch A2C neural network framework to implement relevant reinforcement learning training.

2. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 1, characterized in that, Step S1 includes: S11. Constructing a heterogeneous graph , in, Represents the nodes in the graph. ,node It consists of two types of node sets: process node sets and process node sets. This represents all the processes in the operation; and the set of machine nodes. This represents all available machines in the workshop; This represents an edge in a heterogeneous graph; S12. Construct the feature vector of each process node. Defined as: ， in, Whether the process is completed; Whether the process is ready, Represents the relative position of the process in the workpiece operation; S13, Construct machine nodes Node feature vectors Defined as: ， in, This represents the relative busyness of the machine. This indicates the percentage of tasks that the machine has completed. This represents the percentage of the total number of processes that the machine can still process. S14, Edge Weight Construction, edge weights Processing time set to normalization ,in For workpiece process In the machine Processing time Let be the scaling factor. Scaling factor for the average completion time of all processes. .

3. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 2, characterized in that, The edges in the heterogeneous graph in step S11 include two types of connection relationships: connecting edges, which are used to connect adjacent processes of the same workpiece. One type of relationship represents the sequential constraint between processes; another type of relationship is the disjunctive edge, used to connect the relationship between processes and machines. , indicating machine With process The ability to perform processing.

4. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 1, characterized in that, Step S2 includes: S21. By using two independent linear transformation layers, the process node features and machine node features are mapped to a unified hidden space. For node features... and The transformation formula is as follows: ， ， in, , The weight matrix is ​​a learnable matrix. , The bias term is ReLU, and the activation function is ReLU. S22. Extract the global context vector based on the heterogeneous graph design completed in step S1. For the feature matrix of machine nodes Perform aggregation: ， in, For max pooling operation, For average pooling operation, The feature matrix of machine nodes, This represents a vector concatenation operation. It is a multilayer perceptron; S23. Features of adjacent nodes after embedding and Pay attention The computation is performed using a bottleneck-aware graph attention mechanism, based on global context vectors. It modifies the strength between connections to generate scheduling operations suitable for the current load situation; S24. Multi-stage bidirectional message passing is achieved through a graph attention layer that perceives three bottlenecks: the first stage propagates from machine to process; the second stage propagates from process to machine; and the third stage propagates again from machine to process. S25. By using an attention-based pooling layer to aggregate all node information, and utilizing the global context vector... Key features are extracted from both process and machine perspectives to identify the key processes and machines that have the greatest impact on the current scheduling objectives.

5. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 4, characterized in that, Step S23 also employs residual connections to prevent the gradient information of the deep network from disappearing. The feature update formula for node u is: ， in, and Adjacent nodes for The neighborhood group, This is a learnable attention vector.

6. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 4, characterized in that, Step S25 includes: First, extract the key vectors of the nodes. The formula is as follows: ， ， ， ， in: For attention score, For normalized attention weights, The final state representation vector is the aggregated global pooled features. Based on the global characteristics of the process Machine global characteristics and the original global context vector It is pieced together to form a complete description of the current state of the workshop.

7. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 1, characterized in that, Step 3 includes mask matrix design, policy network design, value network design, reward function design, and loss function design.

8. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 7, characterized in that, The value network design is based on global feature estimation of state value, and value estimation... as follows: ， The critic is a two-layer MLP network.

9. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 7, characterized in that, The reward function design employs a differential reward function based on the maximum completion time, which calculates the differential minimum completion time for each step, i.e. ,in This refers to the completion time after the previous action is completed, followed by... The negative value is used as the immediate reward at each time step, i.e. Once all operations have been scheduled, the cumulative reward is the negative of the maximum completion time. .

10. The workshop scheduling optimization method based on bottleneck-sensitive heterogeneous graph neural network according to claim 7, characterized in that, The loss function design employs a mini-batch update strategy, and the total loss function includes policy loss, value loss, and entropy regularization term. ， Where B is the number of trajectories collected during mini-batch training. For the first The loss from training the actor once. ， The difference between the actual return and the review model. For the first Training The loss, ， This is the entropy regularization loss.

Images