Multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph converter, storage medium and terminal thereof

Abstract

The invention discloses a multi-dynamic flexible job shop scheduling optimization method based on a heterogeneous graph converter, which belongs to the technical field of shop scheduling optimization, and is characterized in that the heterogeneous graph converter is used as a basic framework of a strategy network, a dynamic scheduling environment is matched with strategy optimization, and a proper workpiece and machine combination is selected in a scheduling process. Firstly, in consideration of complexity of a scheduling scene, a heterogeneous graph converter is used for extracting and enhancing features of workpieces and machines, meanwhile, a corresponding reasonable combination of the workpieces and the machines is given, secondly, a reinforcement learning mechanism of a strategy optimization method is adopted to improve exploration capability and search efficiency, and then an efficient dynamic event processing method is provided. According to the multi-dynamic flexible job scheduling algorithm, the current dynamic event is processed in real time, so that the anti-disturbance capability of the scheduling algorithm on the dynamic event is enhanced, finally, the performance of the multi-dynamic flexible job scheduling algorithm is verified on a test set, and the robustness and effectiveness of the algorithm are verified by an experimental result.

Description

Technical Field

[0001] This invention belongs to the technical field of production scheduling for flexible operations in the manufacturing industry. Specifically, it relates to a method for optimizing multi-dynamic flexible job shop scheduling based on a heterogeneous graph converter. Background Technology

[0002] In modern manufacturing, with the continuous changes in scale customization and production demands, the Flexible Job Shop Scheduling Problem (FJSP) has become particularly important. FJSP can quickly adapt to market changes, meet the needs of high-volume, diversified production, and enhance the flexibility and adaptability of the production process. This has led to widespread attention from enterprises and researchers regarding FJSP. In the context of highly customized manufacturing, dynamic production models are more in line with the actual production environment and have become a focus of attention for enterprises and researchers. Dynamic production models can better cope with uncertainties in the production process, such as machine failures and the dynamic insertion of new jobs, which is crucial for the production efficiency and stability of manufacturing. The Multi-Dynamic Flexible Job Shop Scheduling Problem (MDFJSP) is an extension of the traditional FJSP. Its core is to allocate jobs to different machines and ensure high productivity and resource utilization. However, dynamic events can disturb the scheduling process. FJSP has been proven to be an NP-hard problem. MDFJSP takes into account more dynamic events on top of the flexible workshop, therefore, MDFJSP is also an NP-hard problem.

[0003] The Multi-Dynamic Flexible Job Shop Scheduling Problem (MDFJSP) faces significant challenges in both theoretical research and algorithm design due to its inherent complexity. This complexity manifests primarily in several aspects: uncertainty, variability, large scale, strong constraints, multiple objectives, and nonlinearity. Because of the complexity of the MDFJSP, exact algorithms struggle to obtain solutions within a finite number of events. When solving complex combinatorial optimization problems, exact algorithms are limited to solving small-scale problems due to the large solution space and long computation time. For large-scale problems, heuristic, metaheuristic, and hybrid algorithms are widely used. Heuristic algorithms offer the advantage of rapidly obtaining scheduling solutions in a relatively short time, but the quality of the solutions is often difficult to guarantee. Due to the long computation time of exact and heuristic algorithms for solving large-scale problems, current mainstream methods primarily rely on approximation algorithms such as metaheuristics. In recent years, reinforcement learning, with its dynamic adaptability to the environment, has been increasingly applied to solving scheduling problems. Reinforcement learning involves an agent continuously interacting with the environment, learning through trial and error and adapting actions under different environmental states to generate a policy network. After training, an optimal policy network is finally formed. Compared to metaheuristic algorithms, reinforcement learning can dynamically adjust its scheduling strategy in real time based on the scheduling status in the workshop, making it particularly suitable for dynamically changing production environments. However, reinforcement learning faces challenges such as long training times and difficulties in setting up the scheduling environment and processing the action space.

[0004] MDFJSP allows modeling in various production scheduling systems. In certain processing stages of manufacturing systems, flexible production processes pose a significant challenge. How to rationally arrange workpieces and machines to accelerate the scheduling process is crucial. Meanwhile, the occurrence of multiple dynamic events can significantly disrupt the scheduling process, substantially delaying manufacturing completion time. Therefore, it is necessary to minimize the impact of dynamic events on the scheduling process. Minimizing completion time is a key practical goal in the manufacturing industry. In dynamic production environments, how to unify and balance resource usage among production enterprises, including reducing the impact of machine failures on the manufacturing process and the impact of new job arrivals on existing scheduling processes, is considered of practical significance. Therefore, it is necessary to expand existing research to obtain effective scheduling methods to solve scheduling problems in production environments. Summary of the Invention

[0005] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0006] To solve the above problems, the present invention adopts the following technical solution.

[0007] A method for optimizing multi-dynamic flexible job shop scheduling based on heterogeneous graph transformers includes the following steps:

[0008] Step 1: Simulate and model the scheduling environment of the multi-dynamic flexible workshop;

[0009] Step 2: Fit a reinforcement learning-based action selection policy network into the model built in Step 1. The reinforcement learning-based action selection policy network adaptively optimizes action selection based on the environmental state and actions.

[0010] Step 3: Dynamic event handling mechanism. For sudden dynamic events during the scheduling process, it can be processed in real time, and the event information can be fed back and the scheduling environment status information can be updated in a timely manner.

[0011] Preferably, in the above scheduling optimization method, the action selection strategy network based on reinforcement mechanism in step 2 includes a heterogeneous graph transformer action selection strategy module and a reinforcement learning action selection strategy module.

[0012] Preferably, in the above-described scheduling optimization method, the heterogeneous graph transformer enhances and extracts the features of the workpiece and the machine, and then calculates the matching degree between the workpiece and the machine to provide downstream action selection; the specific process is as follows.

[0013] 1) Feature enhancement:

[0014] a) Normalization processing: Normalize the original features;

[0015] b) Feature embedding: Depending on the scheduling environment requirements, either splicing or direct embedding into the feature space can be enabled;

[0016] 2) Feature extraction:

[0017] a) Multi-layer encoder structure: Multiple workpiece and machine encoders are used to encode features in multiple layers, while multi-head attention mechanism is used in the encoder to extract the relationship between various entities;

[0018] b) Fusion layer: In each layer, arc features and node features are fused through a linear layer;

[0019] 3) Matching probability calculation;

[0020] a) Post-encoding feature processing: After encoding, the final machine features and process features are obtained, and the features are further transformed through a linear layer;

[0021] b) Similarity matrix calculation: Calculate the feature similarity of the currently selectable processes and machines, and normalize the similarity matrix to prevent numerical instability;

[0022] c) Masking: Set the similarity of non-executable actions to negative infinity to ensure they are not selected in softmax;

[0023] d) Softmax probability calculation: Flatten the similarity matrix and apply the softmax function to obtain the probability distribution of the action.

[0024] Preferably, in the above scheduling optimization method, in step 2, an improved ε-greedy strategy is adopted, which explores new behaviors with a higher probability during the interaction with the environment. As the training time increases, the agent shifts from exploring new actions to selecting actions that can currently obtain the greatest reward.

[0025] Preferably, in the above scheduling optimization method, the improved ε-greedy strategy is expressed as follows:

[0026]

[0027] Where ε represents the probability of the greedy strategy, t represents the number of iterations, and T max The stopping criterion is represented by μ(a), which is the total number of iterations. t |s t ) represents the state s at time t. t Take action a t The probability, P rand Let a represent a sample value that follows a standard normal distribution. * Indicates that in state S t The action with the largest Q value, A(S) t ) represents state S t The set of all optional actions in Reward t f represents the agent's reward value at time t. new f represents the completion time of the new moment after the selected action. old This indicates the completion time at the current moment before the action is selected.

[0028] Preferably, in the above-mentioned scheduling optimization method, the dynamic event handling mechanism in step 3 is divided into new job processing and machine fault processing.

[0029] Preferably, in the above-described scheduling optimization method, the new job processing determines the difference between the arrival time of a new job and the current environment time. If there is a job that needs to be inserted, the new job is inserted into the existing scheduling environment by expanding the job list and the corresponding status record list in the existing scheduling environment.

[0030] Preferably, in the above scheduling optimization method, machine fault handling is used to handle the occurrence of machine fault events. If a machine fault is detected, the fault information is recorded, the current processing step is interrupted, the processing timer on the faulty machine is cleared, the machine availability mask is updated, and the environmental status information is updated based on the machine fault information.

[0031] In another aspect, the present invention provides a storage medium for receiving user input programs, wherein the stored computer programs enable electronic devices to execute the multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph converters as described above.

[0032] In addition, the present invention also provides an information data processing terminal, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer as described above.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] (1) This invention constructs a model of a scheduling environment that is more closely aligned with actual production. Modeling the scheduling environment is a key step in the scheduling process. Modeling a scheduling environment that is more closely aligned with actual production processes can improve the generalization and robustness of the algorithm.

[0035] (2) PPO reinforcement learning algorithm is used for dynamic selection of workpieces and machines. The policy network based on HGT can integrate and enhance the information of various entities in the scheduling environment in real time, and at the same time give the corresponding workpiece and machine matching degree, so that reinforcement learning can dynamically select the combination of workpieces and machines to further balance exploration and development capabilities.

[0036] (3) Dynamic event handling mechanism: It can handle sudden dynamic events in the scheduling process in real time, and promptly feed back event information and update the scheduling environment status information.

[0037] (4) The present invention is simple in logic, easy to implement and easy to extend, and can extend the optimizer to meet most scheduling problems in the current field of intelligent manufacturing production. Attached Figure Description

[0038] Figure 1 It is a Gantt chart of a multi-dynamic flexible job shop scheduling problem.

[0039] Figure 2 This is a combined model diagram of heterogeneous graph transformer and reinforcement learning in this invention.

[0040] Figure 3 This is a schematic diagram of the reinforcement learning module in this invention.

[0041] Figure 4 This is a flowchart of the multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer in this invention.

[0042] Figure 5 This is a completion time distribution diagram comparing the method of this invention with other methods.

[0043] Figure 6 The method in this invention uses a convergence graph. Detailed Implementation

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0045] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0046] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments. The present invention provides the following embodiments.

[0047] This embodiment provides a multi-dynamic flexible job shop scheduling optimization method based on a heterogeneous graph transformer, used to handle the multi-dynamic flexible job shop scheduling problem. The Gantt graph of this multi-dynamic flexible job shop scheduling problem is shown below. Figure 1 As shown, this method, by fitting a heterogeneous graph transformer and an action selection strategy network based on reinforcement learning mechanism into a multi-dynamic flexible job shop scheduling method, can reduce the impact of machine failures on the scheduling process and machine idle time in the manufacturing job shop, improve the overall effect of the production system, enhance the competitiveness of the manufacturing industry, and provide strong theoretical support and practical guidance for realizing multi-dynamic and flexible requirements in production.

[0048] Specifically, the scheduling optimization method in this embodiment includes the following steps.

[0049] Step 1: Simulate and model the scheduling environment of the multi-dynamic flexible workshop. Specifically, generate simulation models of each entity according to the scheduling environment, and initialize the state record list of each state in the environment to improve the accuracy and stability of the state feature record of the environment. The pseudocode for environment initialization is shown in Table 1.

[0050] Table 1: Simulation code for the scheduling environment of a multi-dynamic flexible workshop

[0051]

[0052]

[0053] Step 2: Fit a reinforcement-based action selection policy network into the model built in Step 1. This action selection policy network includes a heterogeneous graph transformer (HGT) action selection policy module and a reinforcement learning (RL) action selection policy module. The combined model of the heterogeneous graph transformer action selection policy module and the reinforcement learning action selection policy module is as follows: Figure 2 As shown, in this embodiment, the action selection strategy network based on reinforcement learning mechanism adaptively optimizes action selection according to the environmental state and the historical experience of actions with the environment. Compared with traditional heuristic algorithms, it improves scheduling decision-making ability by calculating the matching degree of each entity state based on the real-time state of entities in the scheduling environment.

[0054] In addition, in step 2, a heterogeneous graph transformer (HGT) is used to enhance and extract features of the workpiece and the machine, and then the matching degree between the workpiece and the machine is calculated to provide information for downstream action selection. The specific process is as follows.

[0055] 1) Feature enhancement

[0056] a) Normalization processing: Normalize the original features.

[0057] b) Feature embedding: Depending on the scheduling environment requirements, splicing or direct embedding into the feature space can be enabled.

[0058] 2) Feature extraction

[0059] a) Multi-layer encoder structure: Multiple workpiece and machine encoders are used to encode features in multiple layers, while multi-head attention mechanism is used in the encoder to extract the relationship between the various entities.

[0060] b) Fusion layer: In each layer, arc features and node features are fused through a linear layer.

[0061] 3) Matching probability calculation

[0062] a) Post-encoding feature processing: After encoding, the final machine features and process features are obtained, and the features are further transformed through a linear layer.

[0063] b) Similarity matrix calculation: Calculate the feature similarity of the currently selectable processes and machines, and normalize the similarity matrix to prevent numerical instability.

[0064] c) Masking: Set the similarity of non-executable actions to negative infinity to ensure they are not selected in softmax.

[0065] d) Softmax probability calculation: Flatten the similarity matrix and apply the softmax function to obtain the probability distribution of the action.

[0066] Furthermore, in step 2, an improved ε-greedy strategy is employed, exploring new behaviors with higher probability during interaction with the environment. As training time increases, the agent shifts from exploring new actions to selecting actions that currently yield the greatest reward. The advantage of this strategy is that it ensures the agent can explore a wider search area while retaining a certain depth-exploration capability. The improved strategy is specifically described below:

[0067]

[0068] Where ε represents the probability of the greedy strategy, t represents the number of iterations, and T max The stopping criterion is represented by μ(a), which is the total number of iterations. t |s t ) represents the state s at time t. t Take action a t The probability, P rand Let a represent a sample value that follows a standard normal distribution. * Indicates that in state S t The action with the largest Q value, A(S) t ) represents state S t The set of all optional actions in Reward t f represents the agent's reward value at time t. new f represents the completion time of the new moment after the selected action. old This indicates the completion time at the current moment before the action is selected.

[0069] Specifically, the meanings of the above formulas are as follows.

[0070]

[0071] Step 3: Dynamic event handling mechanism. For sudden dynamic events during the scheduling process, it can handle them in real time, and promptly feed back event information and update the scheduling environment status information.

[0072] Dynamic events are divided into new job processing and machine failure processing. New job processing determines the difference between the arrival time of a new job and the current environmental time. If a job needs to be inserted, it is done by expanding the existing job list and corresponding status record list in the scheduling environment. Simultaneously, for machine failure events, if a machine failure is detected, the failure information is recorded, the currently processing step is interrupted, the processing timer on the failed machine is cleared, the machine availability mask is updated, and the environmental status information is updated based on the machine failure information.

[0073] The multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformers in this embodiment mainly consists of three parts: simulation modeling that closely matches the actual scheduling environment, a policy network based on reinforcement learning and HGT, and a real-time dynamic event processing mechanism. In the environment initialization phase, a rich masking mechanism and a state record list are used to record the environment state in real time. In the policy network, an HGT-based policy generation network combined with a reinforcement learning mechanism optimizes the selection of workpieces and machines. The dynamic event processing mechanism processes and records occurring events in real time. The specific process is as follows: Figure 4 As shown in the figure, the specific pseudocode of the multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph converter in this embodiment is shown in Table 2.

[0074] Table 2: Pseudocode for Multi-Dynamic Flexible Job Shop Scheduling Optimization Method Based on Heterogeneous Graph Transformer

[0075]

[0076] In addition, to verify the effectiveness of the method, this embodiment will experiment with the multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer (HGTIEL) and other scheduling methods, including the following:

[0077] SPT: Shortest processing time priority. Based on the processing time of each process, the process with the shortest processing time is given priority for processing.

[0078] MWKR: The workpiece with the most remaining workload is selected for priority processing.

[0079] MOR: The workpiece with the most remaining operations is processed first.

[0080] LST: Latest Start Time, sorted based on the latest start time of each workpiece to ensure that all workpieces can be completed on time;

[0081] LPT: Longest processing time first. In contrast to SPT, this rule prioritizes the task with the longest processing time for processing.

[0082] LB: Load balancing, attempts to balance the load among machines so that the workload of each machine is as even as possible;

[0083] FIFO: First In First Out, processes workpieces in the order they arrive, regardless of their specific properties or requirements.

[0084] EFT: Earliest Finish Time. Select the machine that can complete a task the earliest to process it, in order to minimize the overall system completion time.

[0085] The result is as follows Figure 5 As shown, by Figure 5 As can be seen, the completion time of the HGTIEL method in this embodiment is superior to other methods. Furthermore, this embodiment also provides the convergence of the method at different scales, as shown in the following results. Figure 6 As shown, f001, f002, and f003 represent instances with different dynamic event distributions, m is the number of machines, and j is the number of workpieces. That is, f001_m5j15 means that in an instance with distribution f001, there are 5 machines and 15 workpieces that need to be scheduled for processing. Figure 6 The results show that the method in this embodiment can meet the needs of modern manufacturing enterprises for production workshop scheduling.

[0086] In addition, this embodiment also provides a storage medium for receiving user input of the aforementioned computer program, i.e., code. The stored computer program enables an electronic device to execute the aforementioned multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph converter. Furthermore, the aforementioned storage medium is assembled in an information data processing terminal, which includes a memory and a processor. The memory stores the aforementioned computer program. When the computer program is executed by the processor, the processor executes the aforementioned multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph converter to solve the job shop scheduling problem.

[0087] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection defined by the claims submitted herein.

Claims

1. A method for optimizing multi-dynamic flexible job shop scheduling based on heterogeneous graph transformers, characterized in that, Includes the following steps: Step 1: Simulate and model the scheduling environment of the multi-dynamic flexible workshop; Step 2: Fit a reinforcement-based action selection policy network into the model built in Step 1. The reinforcement learning-based action selection policy network adaptively optimizes action selection based on the environmental state and actions. The reinforcement learning mechanism uses a policy optimization method for optimization. Step 3: Dynamic event handling mechanism. For sudden dynamic events during the scheduling process, it can be processed in real time, and the event information can be fed back and the scheduling environment status information can be updated in a timely manner.

2. The multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer according to claim 1, characterized in that, The action selection policy network based on reinforcement mechanism in step 2 includes a heterogeneous graph transformer action selection policy module and a reinforcement learning action selection policy module.

3. The method for optimizing multi-dynamic flexible job shop scheduling based on heterogeneous graph transformers according to claim 2, characterized in that, The heterogeneous graph transformer is used to enhance and extract the features of the workpiece and the machine, and then calculates the matching degree between the workpiece and the machine to provide downstream action selection; the specific process is as follows. 1) Feature enhancement: a) Normalization processing: Normalize the original features; b) Feature embedding: Depending on the scheduling environment requirements, either splicing or direct embedding into the feature space can be enabled; 2) Feature extraction: a) Multi-layer encoder structure: Multiple workpiece and machine encoders are used to encode features in multiple layers, while multi-head attention mechanism is used in the encoder to extract the relationship between various entities; b) Fusion layer: In each layer, arc features and node features are fused through a linear layer; 3) Matching probability calculation; a) Post-encoding feature processing: After encoding, the final machine features and process features are obtained, and the features are further transformed through a linear layer; b) Similarity matrix calculation: Calculate the feature similarity of the currently selectable processes and machines, and normalize the similarity matrix to prevent numerical instability; c) Masking: Set the similarity of non-executable actions to negative infinity to ensure they are not selected in softmax; d) Softmax probability calculation: Flatten the similarity matrix and apply the softmax function to obtain the probability distribution of the action.

4. The method for optimizing multi-dynamic flexible job shop scheduling based on heterogeneous graph transformers according to claim 2, characterized in that, In step 2, an improved ε-greedy strategy was adopted, which explores new behaviors with a higher probability during interaction with the environment. As the training time increases, the agent shifts from exploring new actions to selecting actions that can currently obtain the greatest reward.

5. The multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer according to claim 4, characterized in that, The improved ε-greedy strategy is expressed as follows: Where ε represents the probability of the greedy strategy, t represents the number of iterations, and T max The stopping criterion is represented by μ(a), which is the total number of iterations. t |s t ) represents the state s at time t. t Take action a t The probability, P rand Let a represent a sample value that follows a standard normal distribution. * Indicates that in state S t The action with the largest Q value, A(S) t ) represents state S t The set of all optional actions in Reward t f represents the agent's reward value at time t. new f represents the completion time of the new moment after the selected action. old This indicates the completion time at the current moment before the action is selected.

6. The multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer according to claim 1, characterized in that, In step 3, the dynamic event handling mechanism is divided into new job processing and machine fault processing.

7. The method for optimizing multi-dynamic flexible job shop scheduling based on heterogeneous graph transformers according to claim 6, characterized in that, The new job processing method determines the difference between the arrival time of a new job and the current environment time. If there is a job to be inserted, the new job is inserted into the existing scheduling environment by expanding the job list and the corresponding status record list in the existing scheduling environment.

8. The method for optimizing multi-dynamic flexible job shop scheduling based on heterogeneous graph transformers according to claim 6, characterized in that, Machine fault handling is used to handle machine fault events. If a machine fault is detected, the fault information is recorded, the current processing step is interrupted, the processing timer on the faulty machine is cleared, the machine availability mask is updated, and the environmental status information is updated based on the machine fault information.

9. A storage medium for receiving user input, characterized in that, The stored computer program causes the electronic device to execute the multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer as described in any one of claims 1-8.

10. An information data processing terminal, characterized in that, The terminal includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor performs the multi-dynamic flexible job shop scheduling optimization method based on heterogeneous graph transformer as described in any one of claims 1-8.

Images