Scheduling method for flexible job shop considering transportation time

Abstract

The invention discloses a scheduling method for a flexible job shop considering transportation time. The scheduling method comprises the following steps: constructing a corresponding heterogeneous graph model according to a current shop environment; feature extraction is carried out on the heterogeneous graph model based on the relation among processes, machines and AGVs, and node features and joint features are obtained; the node features are input into a multiple attention network for GAT coding, and new features are obtained; and combining the new features with the joint features, performing MLP decoding by using an Actor network for decision in a decision module with trained network parameters to obtain corresponding actions and probability distribution thereof, selecting and executing the action with the highest probability, updating the current workshop environment according to the executed action, and performing repeated iteration until all scheduling processes are completed.

Description

Technical Field

[0001] The present invention specifically relates to a scheduling method for a flexible job shop considering transportation time, and specifically relates to a heterogeneous graph neural network scheduling optimization method integrating multiple attention mechanisms, and belongs to the technical field of industrial intelligent simulation. Background Art

[0002] With the widespread application of automation technology in enterprises, automatic guided vehicles (AGVs) have become indispensable transportation equipment in smart factories. AGVs can not only realize efficient and accurate transportation of workpieces between different processing links, but also realize automation and intelligence of the production process through seamless integration with the production system. Compared with traditional manpower or fixed transportation tools, AGVs have the advantages of high flexibility, intelligent scheduling, and strong real-time performance. They can dynamically adjust paths and tasks according to production plans, improve production efficiency and flexible manufacturing capabilities. In addition, the application of AGVs can effectively reduce errors and safety hazards caused by human operations, and improve the safety and stability of the production process.

[0003] At present, the main methods for solving the Flexible Workshop Scheduling Problem for Automated Guided Vehicles (FJSP-AGV) are divided into two categories: exact methods and approximate methods. Exact methods include mathematical programming methods represented by mixed integer linear programming, Lagrangian relaxation method and decomposition method, as well as branch and bound method. Exact methods solve the scheduling problem by transforming it into multiple constraint combinations, which leads to a long running time on medium and large-scale cases, or even failure to obtain a solution. Approximate methods include metaheuristic algorithms, deep reinforcement learning algorithms, etc. Metaheuristic algorithms include genetic algorithms, wolf pack algorithms, taboo search algorithms, and artificial bee colony algorithms. Although these algorithms perform well in some cases, they usually require manual adjustment of parameters and require a large number of iterations to obtain the best solution. Therefore, there are problems such as unstable solution quality, easy to fall into local optimality, low efficiency in solving large-scale cases, and weak robustness. At the same time, when the case changes, the traditional metaheuristic algorithm can only recalculate the scheduling and cannot achieve the goal of real-time response. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the background technology, provide a scheduling method for a flexible job shop considering the transportation time, and solve the technical problem of scheduling a flexible job shop considering the transportation time of an automated guided vehicle.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] In a first aspect, a flexible job shop scheduling method considering transportation time is provided, comprising:

[0007] Construct the corresponding heterogeneous graph model according to the current workshop environment;

[0008] Extract features from heterogeneous graph models based on the relationships among processes, machines, and AGVs to obtain node features and joint features.

[0009] Input the node features into the multi-attention network for GAT encoding to obtain new features;

[0010] Combine the new features with the joint features, use the Actor network for decision-making in the decision module with trained network parameters to perform MLP decoding, obtain the corresponding actions and their probability distribution, select the action with the highest probability and execute it;

[0011] Update the current workshop environment according to the executed actions, and repeat the above steps to complete all scheduling processes.

[0012] In combination with the first aspect, further, the decision module includes an Actor network and a Critic network for evaluating the quality of actions selected by the Actor network, and the training process of the decision module includes:

[0013] The training data related to the workshop actions are input into the Actor network, and the probability distribution of each action is obtained through GAT encoding and MLP decoding;

[0014] Based on the action sampling strategy, an action is selected from the probability distribution of the action, the reward of the action is obtained and the state is transferred, and the action tuple is placed in the experience replay pool;

[0015] When the number of action tuples in the experience replay pool reaches the preset value, action tuples are randomly extracted from the experience replay pool to update the Critic network parameters and the Actor network parameters;

[0016] Assign the updated Actor network parameters to the Actor network used for decision-making to obtain a trained Actor network for decision-making.

[0017] Furthermore, the probability distribution of actions is shown as follows:

[0018]

[0019] In the formula, π θ (a t ∣s t ) means in state s t Select action a t The probability of μ(a t |s t) indicates action a t corresponds to a scalar, where They represent the L-th level process, machine, AGV and global features respectively. Represents the process-machine joint characteristics, Represents the joint features of process and AGV, MLP θ represents a multi-layer perceptron, μ(b t |s t ) indicates action b t corresponds to a scalar; represents the action space set, and || represents the merge connection operation.

[0020] Furthermore, the Critic network parameters and the Actor network parameters are updated through the gradient descent and adaptive learning rate Adam algorithm, as shown in the following formula:

[0021] Loss(θ,φ)=c p L CLIP (θ)+c v L MSE (φ)+c e L E (θ),

[0022] Where Loss(θ,φ) represents the total loss of the policy function, θ and are the parameters of the Actor network and the Critic network, L CLIP (θ) is the near-end ratio loss, L MSE (φ) is the value function loss, L E (θ) is the entropy loss, c p 、c v and c e They are the proximal ratio clipping loss coefficient, the value function loss coefficient, and the entropy loss coefficient, respectively.

[0023] Furthermore, the node features include: process node features, machine node features and AGV node features; the joint features include: process-machine pair features, process-AGV pair features.

[0024] Furthermore, the multiple attention network classifies the input node features and processes them through a process information attention module, a machine information attention module or an AGV information attention module to obtain new features.

[0025] Furthermore, the process information attention module is used to perform the following steps:

[0026] For the input node features, the attention coefficient of the current process and its connected processes is calculated, and the SoftMax function is used to normalize the attention coefficient to obtain the normalized attention coefficient;

[0027] According to the normalized attention coefficient, the correlation degree between the current process and its previous and next processes is output. The calculation formula is as follows:

[0028]

[0029] In the formula, Represents the new feature output by the process information attention module, α i,j,p represents the normalized attention coefficient, σ represents the nonlinear activation function, W represents the linear transformation, Represents the input characteristics of the current process, O ip represents the p-th process of the i-th workpiece, and p=j-1 means that the parameter p starts from j-1.

[0030] Furthermore, the machine information attention module is used to perform the following steps:

[0031] Filter the node features input into the process information attention module;

[0032] Preprocess the feature attributes input into the machine information attention module and the filtered node features respectively, and then concatenate them to obtain the competitive strength between the current machine and other machines;

[0033] Calculate the machine information attention coefficient based on the competition intensity between the current machine and other machines;

[0034] Normalize the machine information attention coefficient and output the processed machine features;

[0035] The calculation formula of the machine information attention coefficient is as follows:

[0036]

[0037] In the formula, represents the machine information attention coefficient, are weight matrices, is a linear transformation, are the input features of machine k and machine q respectively, Represents the competing process M k and M q A measure of competitiveness between Indicates the machine competition process M k and M q A collection of .

[0038] Furthermore, the AGV information attention module is used to perform the following steps:

[0039] Filter the node features input into the process information attention module;

[0040] The node features input into the AGV information attention module and the filtered node features are preprocessed respectively, and then spliced ​​to obtain the competition intensity between the current AGV and other AGVs;

[0041] According to the competition intensity between the current AGV and other AGVs, the AGV information attention coefficient is calculated;

[0042] The attention coefficient of AGV information is normalized, and the output is that multiple AGVs compete to transport unfinished workpieces;

[0043] The calculation formula of the AGV information attention coefficient is as follows:

[0044]

[0045] In the formula, represents the AGV information attention coefficient, are weight matrices, is a linear transformation, For automatic guided vehicle A k With A q A collection of competing transport artifacts, A measure of competitiveness between AGVs, They represent the input features of the qth AGV and the kth AGV respectively, and LeakyReLU represents an activation function.

[0046] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned flexible job shop scheduling methods considering transportation time.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention provides a scheduling method for a flexible job shop considering transportation time, which utilizes a heterogeneous graph neural network integrating multiple attention mechanisms, and considers the joint features of process-machine pair features and process-AGV pair features while considering the node features of each process, machine and AGV, so as to more accurately represent its state space;

[0049] The present invention inputs node features into a multi-attention network for GAT encoding to obtain new features; combines the new features with joint features, uses the Actor network for decision-making in a decision module with trained network parameters to perform MLP decoding, selects the action with the highest probability and executes it to complete the state transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of a scheduling method for a flexible job shop considering transportation time provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the overall framework of a multiple attention network provided by an embodiment of the present invention;

[0052] Figure 3 Actor network architecture diagram provided for an embodiment of the present invention;

[0053] Figure 4 A flowchart of the training phase and the testing phase of the decision module provided by an embodiment of the present invention;

[0054] Figure 5 A standard case makespan comparison line chart provided for an embodiment of the invention. DETAILED DESCRIPTION

[0055] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0056] Embodiment 1:

[0057] like Figure 1 As shown, it is a flow chart of the scheduling method of the flexible job shop considering the transportation time provided by the present invention, including a heterogeneous graph model integrating the workshop state, extracting the node features from the heterogeneous graph model and inputting the corresponding attention module to obtain new node features. The new node features are input into the decision module, and an action is selected through an Actor network to obtain a reward for state transfer. In this way, a training data can be obtained and put into the experience replay pool. When the number of data in the experience replay pool reaches the set value, a certain batch of data is randomly extracted from it to update the parameters of another Actor network and the Critic network. After a certain step, the updated network parameters are assigned to the Actor network originally used for decision-making, and it is continuously iterated until the training is completed.

[0058] The heterogeneous graph model includes process nodes, machine nodes, AGV nodes, process-process edges, process-machine edges, and process-AGV edges. The process-process edge represents the processing sequence of the workpiece, the process-machine edge represents that the connected machine can process the process, and the process-AGV edge represents that the connected AGV can transport the process.

[0059] The corresponding feature information extracted from the heterogeneous graph model includes: process node features, machine node features, AGV node features, process-machine features and process-AGV edge features.

[0060] The present invention provides a heterogeneous graph neural network scheduling optimization method integrating multiple attention mechanisms to solve the technical problem of flexible job shop scheduling considering the transportation time of automated guided vehicles. The method includes:

[0061] Step 1: construct a corresponding heterogeneous graph model according to the current workshop environment;

[0062] Step 2: Extract features from the heterogeneous graph model to obtain corresponding node features and joint features;

[0063] Step 3: Input the node features into the multi-attention network for GAT encoding to obtain new features;

[0064] Step 4: Combine the new features with the joint features, use the Actor network for decision-making in the decision module with trained network parameters to perform MLP decoding, select the action with the highest probability and execute it;

[0065] Step 5: Update the current workshop environment according to the execution, and return to step 1 to iterate until all scheduling processes are completed.

[0066] The present invention provides a scheduling method for a flexible operation workshop taking into account transportation time to solve complex workshop scheduling problems. The proposed scheduling method has strong environmental adaptability and autonomous learning ability, and can effectively improve the production efficiency and real-time scheduling capability of the workshop.

[0067] The node features include: process node features, machine node features and AGV node features; the joint features include: process-machine pair features, process-AGV pair features.

[0068] The characteristics of process nodes include:

[0069] The proportion of machines that can process this process; decision variables: 0 if not scheduled, otherwise 1; the number of processes that have not been completed for the current workpiece; the sum of the average processing time of the processes that have not been completed for the current workpiece; waiting time: the time from the ready time to the current moment and the remaining processing time; the time from the current moment to the completion of the processing of the workpiece.

[0070] Characteristics of machine nodes include:

[0071] The number of unscheduled processes that the current machine can process; the number of candidate workpieces that the current machine can process; machine idle time; decision variables: 0 if the machine is idle, otherwise 1; machine waiting time: the time from the machine idle moment to the current moment and the remaining processing time; the time from the current moment to the next idle moment.

[0072] Characteristics of process-machine pairs include:

[0073] Processing time; the ratio of the current processing time to the maximum processing time; the ratio of the current processing time to the maximum processing time that the machine can process for the candidate workpiece; the ratio of the current processing time to the maximum processing time in the unscheduled process; the ratio of the current processing time to the maximum processing time of the unscheduled process that the machine can process; the ratio of the current processing time to the maximum processing time of the workpiece-machine edge of the candidate workpiece; the ratio of the current processing time to the remaining processing time of the current workpiece and the sum of the waiting time of the machine and the process.

[0074] The characteristic attributes of the process-AGV pair include the sum of the idle time and the loaded time of the AGV transporting the current process.

[0075] The combination of all optional process-machine-AGV pairs defines the action space at time t. As more and more processes are scheduled, the action space will become smaller and smaller.

[0076] The multi-attention network can extract the features of the process, machine, AGV itself, and the features of the process-machine and process-AGV edges. It decomposes the complex relationship between the process, machine, and AGV into three parts and uses multiple different attention networks to process them, namely the process information attention module, the machine information attention module, and the AGV information attention module. Figure 2 , which is a schematic diagram of the overall framework of the multiple attention network provided in an embodiment of the present invention.

[0077] Process information attention module: It is used to associate the processes of the same workpiece. For each input feature, the process information attention module finds the most important process by calculating the attention coefficient of the current process and its connected process (if any), where the process information attention coefficient e i,j,p The calculation method of is shown in formula (2). Since some processes may have no related processes before and after, this application dynamically masks the processes before and after these processes.

[0078] The input features of the process information attention module are first processed by the linear layer, and the process is combined with its previous and next processes and then the attention coefficient e between the process and its previous and next processes is calculated. i,j,p , using the SoftMax function to i,j,p Perform normalization to obtain the normalized attention coefficient αi,j,p Finally, the process output feature vector is obtained by weighted linear combination of the transformed input features and adding the nonlinear activation function σ as shown in formula (3). By connecting the process information attention modules one by one, the jth process O of the i-th workpiece can be ij The information of is propagated to the i-th workpiece J i all processes.

[0079]

[0080]

[0081] In the formula, e i,j,p is the process information attention coefficient, and W represents a linear transformation, represents the input features of the current process, and || represents the merge connection operation; Represents the new features output by the process information attention module; ip represents the p-th process of the i-th workpiece, LeakyReLU represents an activation function; p=j-1 represents that the parameter p starts from j-1; σ represents a nonlinear activation function.

[0082] Machine Information Attention Module: It is used to characterize the unfinished processing steps that multiple machines compete for. As production progresses, this competitive relationship will change dynamically. The machine information attention module uses the competitiveness measurement between machines to calculate the machine information attention coefficient. For the input machine features, the machine information attention coefficient The calculation method of is shown in formula (4):

[0083]

[0084] In the formula, is the weight matrix, is a linear transformation, are the input features of machines k and q respectively, Indicates machine M k and M q The set of competing processing operations, Indicates machine M k With M q A measure of competitiveness between them.

[0085] The original input features of the process are filtered to obtain the filtered process features, and the original input features of the machine are put into the linear layer for preliminary processing;

[0086] The filtered process features and the processed machine features are spliced ​​together;

[0087] The competition intensity between the current machine and other machines is calculated based on the input machine features, and the machine information attention coefficient is calculated based on the competition intensity and the features after splicing through formula (4);

[0088] Normalizing the machine information attention coefficient to obtain a normalized machine information attention coefficient;

[0089] New machine features are calculated based on the obtained normalized attention coefficient and the competition intensity between machines.

[0090] AGV Information Attention Module: Similar to the Machine Information Attention Module, multiple AGVs compete to transport unfinished workpieces. As production progresses, this competitive relationship will change dynamically. The AGV Information Attention Module uses the competitiveness measure between AGVs to calculate the AGV Information Attention Coefficient For the input AGV feature, the calculation method of its attention coefficient is shown in formula (5):

[0091]

[0092] In the formula, is the weight matrix, is a linear transformation, For AGVA k With A q The collection of competing transport artifacts, A measure representing the competitiveness between AGVs; Represent the input features of AGVk and AGV q respectively.

[0093] The original input features of the process are filtered to obtain the filtered process features, and the original input features of the AGV are put into the linear layer for preliminary processing;

[0094] The filtered process features and the processed AGV features are spliced ​​together;

[0095] The competition intensity between the current AGV and other AGVs is calculated based on the input AGV features, and the machine information attention coefficient is calculated based on the competition intensity and the features after splicing through formula (5);

[0096] The AGV information attention coefficient is normalized to obtain a normalized AGV information attention coefficient;

[0097] The new AGV feature is calculated based on the obtained normalized attention coefficient and the competition intensity between AGVs.

[0098] The new features output by the attention module are input into the decision module. The present invention adopts the actor-critic algorithm to make decisions. Two multi-layer perceptrons (MLPs) are used as actor and critic networks respectively. θ and Both have two 64-dimensional hidden layers, θ and These are the parameters of the Actor network and the Critic network, respectively, using tanh as the activation layer.

[0099] like Figure 3 As shown, the goal of the actor network is to generate a policy function π in two steps θ (a t |s t ): First, for each action a t Corresponding to a scalar μ(a t |s t ), scalar μ(a t |s t ) is calculated as shown in formula (6). And use the SoftMax function to output the expected action probability distribution, which is related to action a t =(O ij ,M k ,A l ) are concatenated into a single vector, which is then fed into the MLP θ , calculated as shown in formula (7):

[0100]

[0101] In the formula, They represent the L-th level process, machine, AGV and global features respectively. are the joint features of process-machine pairs and process-AGV pairs respectively; π θ (a t ∣s t ) means in state s t Select action a t The probability of μ(b t |s t ) indicates action b t The corresponding scalar; b t represents an action in the action space, represents the action space set, and || is the merge connection operation.

[0102] The decision module is trained using proximal policy optimization with a clipping proxy objective, e.g. Figure 4As shown in the figure, during the training phase, the prepared training data is input into the Actor network, and the probability distribution of each action is obtained through the GAT encoder and the MLP decoder. The action sampling strategy is adopted to sample an action a from the action distribution, so that the decision module can fully explore during the training process. After the workshop takes the current action, it will obtain the reward r of the action and perform a state transition (s→s'), and then put the obtained tuple (s, a, r, s') into the experience replay pool.

[0103] This application sets that when the number of tuples in the experience replay pool reaches 10,000, tuples are randomly selected from the experience replay pool to update the parameters of another set of Actor-Critic networks, where the Critic network is used to evaluate the quality of the Actor, and the network parameters are updated through gradient descent and adaptive learning rate Adam algorithm, and iterates continuously until the training is completed.

[0104] During the testing phase, the Actor network directly uses the trained network parameters. It only needs to input the test data. The Actor network selects the action with the highest probability according to the current workshop status through GAT encoding and MLP decoding, repeats it continuously, and finally completes the scheduling.

[0105] Policy function π θ (a t |s t ) is updated by minimizing the proximal ratio clipping loss, the value function loss, and the entropy loss, as shown in formula (8):

[0106] Loss(θ,φ)=c p L CLIP (θ)+c v L MSE (φ)+c e L E (θ) (8)

[0107] In the formula, c p 、c v and c e are the proximal ratio clipping loss coefficient, the value function loss coefficient, and the entropy loss coefficient, respectively. Loss(θ,φ) represents the total loss of the strategy function. CLIP (θ) is the near-end ratio loss, L MSE (φ) is the value function loss, L E (θ) is the entropy loss.

[0108] Specifically, the hyperparameter settings of this application are shown in Table 1.

[0109] Table 1: Method training hyperparameters

[0110] Hyperparameters value Iterations 2000 learning rate 0.0003 Discount factor γ 1 PPO Update Times 10 <![CDATA[Proximal ratio clipping loss coefficient c p > 1 <![CDATA[Value function loss coefficient c v > 0.5 <![CDATA[Entropy loss coefficient c e > 0.01 Clipping rate ε 0.2 Actor network layers 3 Critic network layers 3 Network hidden layer dimensions 64 Attention Layers 2 Number of attention heads 4 Attention to the output dimension of the first layer of the head 32 Attention head second layer output dimension 8 GAE Parameters 0.98

[0111] Embodiment 2:

[0112] In order to verify the effectiveness of heterogeneous graph neural networks integrating multiple attention mechanisms in solving the flexible job shop scheduling problem considering AGV transportation time, this application uses 100 randomly generated FJSP-AGV examples as the training set, and applies the decision module after training to a small-scale dataset designed by Deroussi and Norre et al. (Deroussi, L., & Norre, S. (2010). Simultaneous scheduling of machines and vehicles for the flexible job shop problem.) containing 10 flexible job shop scheduling problems considering transportation time for verification. The download address of the dataset is https: / / fastmanufacturingproject.wordpress.com.

[0113] This application uses 100 examples of a 20x10x2 scale for training. In order to further prove the effectiveness of the method proposed in the present invention, the maximum completion time (makespan) of the method proposed in the present invention is compared with the known optimal results in the existing literature, and the gap between each algorithm and the optimal result is given, as shown in Table 2. In the table, the optimal makespan other than the optimal result is bolded, and the first number in each cell is the makespan of the current algorithm in the current case, and the second number is the gap between the current solution and the optimal solution.

[0114] Table 2: Comparison of standard case results

[0115]

[0116]

[0117] Among them: CDR1 is MONR+SPT+EETA, which means: select the workpiece with the largest number of remaining processes, select the machine with the shortest processing time, and select the AGV with the earliest idle time;

[0118] CDR2 is MONR+EETM+EETA, which means: select the workpiece with the largest number of remaining processing steps, select the machine with the earliest idle time, and select the AGV with the earliest idle time;

[0119] CDR3 is MWKR+SPT+EETA, which means: select the workpiece with the longest remaining processing time, select the machine with the shortest processing time, and select the AGV with the earliest idle time;

[0120] CDR4 is MWKR+EETM+EETA, which means: select the workpiece with the longest remaining processing time, select the machine with the earliest idle time, and select the AGV with the earliest idle time.

[0121] In order to more clearly show the differences between the algorithms, this application draws a line graph of the running results, such as Figure 5 As shown in Table 3 and Figure 5 It can be seen that the makespan obtained by this method is mostly smaller than the existing algorithm and scheduling rule combination, and the results are relatively stable. The example with the smallest gap is fjsp8, with a gap of 2.25%. The average gap is 8.87%, which has good stability.

[0122] From the perspective of running time, the method of the present invention takes less than 0.6s to solve the ten examples fjsp1 to fjsp10, with an average time of 0.36s. Unlike GA, PSO, ABC and other meta-heuristic algorithms that require continuous iteration and multiple training for different cases, the method of the present invention can effectively cope with cases of different scales without re-adjustment after training, has good generalization performance, and can effectively balance the solution quality and solution speed when deployed in actual workshops.

[0123] Embodiment 3:

[0124] In order to further verify the generalization of the heterogeneous graph neural network integrating multiple attention mechanisms to solve the flexible job shop scheduling problem considering AGV transportation time, this application applies the trained model to 9 random cases of different scales, with the scales of 10x5x2, 15x10x2, 40x10x2 (the process of each workpiece is the same); 10x5x2various, 15x10x2various, 40x10x2various, 20x10x3various, 40x10x3various, 40x10x4various (the process of each workpiece is different). Each scale test set contains 100 independent examples, the makespan of the 100 examples is averaged and the gap between each scheduling rule combination and the method of the present invention is given, and the results are shown in Table 3.

[0125] Table 3: Comparison of results of various algorithms for randomly generated data sets

[0126]

[0127] It can be seen from Table 4 that the average makespan obtained by the method proposed in the present invention in the test set of the above four scale examples is smaller than that of the combined scheduling rule. In addition, with the continuous increase of the scale of the example and the increase of the number of AGVs, the performance of the four combined scheduling rules will continue to fluctuate, while the performance of the method of the present invention is stable and there is no performance degradation phenomenon, which effectively verifies the stability and generalization of the heterogeneous graph neural network scheduling optimization method proposed in the present invention that integrates multiple attention mechanisms.

[0128] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, a flexible job shop scheduling method considering transportation time is implemented.

[0129] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A flexible job shop scheduling method considering transportation time, It is characterized in that include: Construct the corresponding heterogeneous graph model according to the current workshop environment; Extract features from heterogeneous graph models based on the relationships among processes, machines, and AGVs to obtain node features and joint features. Input the node features into the multi-attention network for GAT encoding to obtain new features; Combine the new features with the joint features, use the Actor network used for decision-making in the decision module with trained network parameters to perform MLP decoding, obtain the corresponding actions and their probability distribution, select the action with the highest probability and execute it; Update the current workshop environment according to the executed actions, and repeat the above steps to complete all scheduling processes.

2. The scheduling method according to claim 1, It is characterized in that The decision module includes an Actor network and a Critic network for evaluating the quality of actions selected by the Actor network. The training process of the decision module includes: The training data related to the workshop actions are input into the Actor network, and the probability distribution of each action is obtained through GAT encoding and MLP decoding; Based on the action sampling strategy, an action is selected from the probability distribution of the action, the reward of the action is obtained and the state is transferred, and the action tuple is placed in the experience replay pool; When the number of action tuples in the experience replay pool reaches the preset value, action tuples are randomly extracted from the experience replay pool to update the Critic network parameters and the Actor network parameters; Assign the updated Actor network parameters to the Actor network used for decision-making to obtain a trained Actor network for decision-making.

3. The method according to claim 2, It is characterized in that The probability distribution of the action is shown as follows: In the formula, π θ (a t ∣s t ) means in state s t Select action a t The probability of μ(a t |s t ) indicates action a t corresponds to a scalar, where They represent the L-th level process, machine, AGV and global features respectively. Represents the process-machine joint characteristics, Represents the joint features of process and AGV, MLP θ represents a multi-layer perceptron, μ(b t |s t ) indicates action b t corresponds to a scalar; represents the action space set, and || represents the merge connection operation.

4. The method according to claim 2, It is characterized in that The Critic network parameters and the Actor network parameters are updated through the gradient descent and adaptive learning rate Adam algorithm, as shown in the following formula: Loss(θ,φ)=c p L CLIP (θ)+c v L MSE (φ)+c e L E (i), Where Loss(θ,φ) represents the total loss of the policy function, θ and are the parameters of the Actor network and the Critic network, L CLIP (θ) is the near-end ratio loss, L MSE (φ) is the value function loss, L E (θ) is the entropy loss, c p 、c v and c e They are the proximal ratio clipping loss coefficient, the value function loss coefficient, and the entropy loss coefficient, respectively.

5. The method according to claim 1, It is characterized in that The node features include: process node features, machine node features and AGV node features; the joint features include: process-machine pair features, process-AGV pair features.

6. The method according to claim 1, It is characterized in that The multiple attention network classifies the input node features and processes them through a process information attention module, a machine information attention module or an AGV information attention module to obtain new features.

7. The method according to claim 6, It is characterized in that The process information attention module is used to perform the following steps: For the input node features, the attention coefficient of the current process and its connected processes is calculated, and the SoftMax function is used to normalize the attention coefficient to obtain the normalized attention coefficient; According to the normalized attention coefficient, the correlation degree between the current process and its previous and next processes is output. The calculation formula is as follows: In the formula, Represents the new feature output by the process information attention module, α i,j,p represents the normalized attention coefficient, σ represents the nonlinear activation function, W represents the linear transformation, Represents the input characteristics of the current process, O ip represents the p-th process of the i-th workpiece, and p=j-1 means that the parameter p starts from j-1.

8. The method according to claim 6, It is characterized in that The machine information attention module is used to perform the following steps: Filter the node features input into the process information attention module; Preprocess the feature attributes input into the machine information attention module and the filtered node features respectively, and then concatenate them to obtain the competitive strength between the current machine and other machines; Calculate the machine information attention coefficient based on the competition intensity between the current machine and other machines; Normalize the machine information attention coefficient and output the processed machine features; The calculation formula of the machine information attention coefficient is as follows: In the formula, represents the machine information attention coefficient, and are weight matrices, is a linear transformation, are the input features of machine k and machine q respectively, Represents the competing process M k and M q A measure of competitiveness between Indicates the machine competition process M k and M q A collection of .

9. The method according to claim 6, It is characterized in that The AGV information attention module is used to perform the following steps: Filter the node features input into the process information attention module; The node features input into the AGV information attention module and the filtered node features are preprocessed respectively, and then spliced ​​to obtain the competition intensity between the current AGV and other AGVs; According to the competition intensity between the current AGV and other AGVs, the AGV information attention coefficient is calculated; The attention coefficient of AGV information is normalized, and the output is that multiple AGVs compete to transport unfinished workpieces; The calculation formula of the AGV information attention coefficient is as follows: In the formula, represents the AGV information attention coefficient, and are weight matrices, is a linear transformation, For automatic guided vehicle A k With A q A collection of competing transport artifacts, A measure of competitiveness between AGVs, They represent the input features of the qth AGV and the kth AGV respectively, and LeakyReLU represents an activation function.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, a flexible job shop scheduling method considering transportation time as described in any one of claims 1 to 9 is implemented.

Images