Workshop scheduling method considering AGV transportation time based on near-end strategy optimization algorithm

Abstract

The invention discloses a workshop scheduling method considering AGV transportation time based on a near-end strategy optimization algorithm, and the method comprises the steps: obtaining an action space, a process local state, a machine local state and an AGV local state according to a current workshop state, carrying out the coding in sequence, and obtaining a process state feature, a machine state feature and an AGV state feature; inputting the state characteristics into a trained strategy network for decoding to obtain action values of the process action, the machine action and the AGV action; the method comprises the following steps: performing normalization processing on an action value, selecting and executing a process selection action, a machine selection action and an AGV selection action with the highest probability in an action space, updating a current workshop environment based on the executed actions, and repeating the steps until all scheduling processes are completed, so that the expansibility of an algorithm and the overall operation efficiency are improved.

Description

Technical Field

[0001] The present invention specifically relates to a workshop scheduling method based on a proximal strategy optimization algorithm taking into account AGV transportation time, and belongs to the technical field of industrial intelligent simulation. Background Art

[0002] With the popularization of enterprise automation technology, the Automatic Guided Vehicle (AGV) plays an important role in the efficient operation of intelligent manufacturing systems as an important transportation equipment for the transformation and upgrading of smart factories. Under the condition that the transportation capacity is not limited, we only need to pay attention to the processing of the workpiece on the machine; however, in the actual production workshop, the transportation capacity of the transportation tool is often limited, resulting in constraints between the transportation tool and the machine in different time and space, so the flow process of the workpiece between different machines cannot be ignored.

[0003] At present, the solution to the problem of introducing the scheduling of automatic guided vehicles into the flexible job shop scheduling problem (Flexible Job Shop Scheduling Problem with Automated Guided Vehicle, referred to as FJSP-AGV) is mainly concentrated on intelligent algorithms. Although deep reinforcement learning has achieved some results on FJSP-AGV, there are still some technical problems that cannot be solved. For example, how to design a general and efficient state space to represent the complex environmental state in FJSP-AGV and improve the scalability and overall operation efficiency of the algorithm. In addition, FJSP-AGV is a multi-action problem, and how to design an efficient action space is also a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the background technology and provide a workshop scheduling method based on a proximal strategy optimization algorithm taking into account the AGV transportation time, thereby improving the scalability of the algorithm and the overall operating efficiency.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] In a first aspect, the present invention provides a workshop scheduling method based on a proximal strategy optimization algorithm taking into account AGV transportation time, comprising:

[0007] According to the current workshop status, the action space, process local status, machine local status and AGV local status are obtained;

[0008] Encode the local state of the process, the local state of the machine and the local state of the AGV to obtain the process state characteristics, the machine state characteristics and the AGV state characteristics respectively;

[0009] Input the process state features, machine state features and AGV state features into the trained strategy network for decoding to obtain the action values ​​of process actions, machine actions and AGV actions;

[0010] Normalize each action value to get the corresponding probability distribution, select the process selection action, machine selection action and AGV selection action with the highest probability in the action space and execute them;

[0011] Update the current workshop environment according to the executed actions, and repeat the above steps until all scheduling processes are completed.

[0012] In combination with the first aspect, further, the training process of the strategy network includes:

[0013] The training data is encoded and input into the policy network to obtain the probability distribution of actions;

[0014] According to the action sampling strategy, an action is selected from the probability distribution of actions and executed, the reward of the action is obtained and the state is transferred, and the action tuple is placed in the experience replay pool;

[0015] Repeat the above steps until the experience replay pool reaches the preset value, randomly extract several action tuples from it, use the Critic network to evaluate the selected actions, and calculate the total loss function based on the evaluation results;

[0016] By using Adam and the gradient descent method on the total loss function, the policy network parameters and the Critic network parameters are updated to obtain a trained policy network.

[0017] Furthermore, the training data is encoded and input into the strategy network to obtain the probability distribution of the action; an action is sampled and selected from the probability distribution of the action according to the action sampling strategy and executed, the reward of the action is obtained and the state is transferred, including:

[0018] Mask the completed processes, and input the masked process features into the multi-layer GIN for encoding; perform MLP decoding and softmax normalization on the encoded process state features to obtain the probability distribution of process actions, and sample the process actions based on the probability distribution;

[0019] Mask the machines that cannot process the current process, input the masked machine features into the fully connected layer and perform batch normalization encoding; perform pooling on the encoded process state features, fuse the pooled process state features with the encoded machine state features and perform MLP decoding, and output the probability distribution of the machine action through the softmax layer, and sample the machine action according to the probability distribution;

[0020] Mask the AGV that cannot transport the current workpiece ， The masked AGV features are input into the fully connected layer and batch normalized. The encoded process state features are pooled. The pooled process state features are fused with the encoded AGV state features and MLP decoded. The probability distribution of AGV actions is output through the softmax layer, and AGV actions are sampled based on the probability distribution.

[0021] According to the action sampling strategy, an action is sampled from the probability distribution of the action and executed, the reward of the action is obtained and the state is transferred.

[0022] Execute the process actions, machine actions and AGV actions obtained at that moment, obtain action rewards and perform state transition to enter the next moment action selection.

[0023] Furthermore, the strategy network includes a process sub-strategy network, a machine sub-strategy network and an AGV sub-strategy network.

[0024] Furthermore, the action space includes:

[0025] The process that can currently be processed selects an action set, the machine that can process the current workpiece selects an action set, and the AGV that can transport the current workpiece selects an action set.

[0026] Furthermore, a graph isomorphism network is used to encode the local state of the process;

[0027] The local state of the machine and the local state of the AGV are encoded through the fully connected layer and batch normalization processing.

[0028] Furthermore, each process node in the process local state The local features are represented by a two-dimensional vector [LB t (O ij ),I t (O ij )], wherein O ij represents the jth process of the i-th workpiece, represents the set of all processes, LB t (O ij ) indicates the processing is completed. ij The completion time or unprocessed process O ij The estimated minimum completion time, I t (O ij ) is process O ij Indicates whether the scheduling is completed. If the scheduling is completed, it is equal to 1, and if it is not scheduled, it is equal to 0;

[0029] Furthermore, the local state of a machine consists of the local states of all machines, [CTM k (t),pt ijk ] represents the local state of each machine, where CTM k (t) is the machine M in the decision process of step t k Completion time, pt ijk For process O ij On machine M k If the processing time of machine M k Unable to process process O ij , then pt ijk By process O ij Compatible machine collection The average processing time is expressed as;

[0030] Furthermore, the local state of an AGV is composed of the local states of all AGVs. l (t),UA l (t)] represents the local state of each AGV, where CTA l (t) is the completion time of the lth AGV in decision step t, UA l (t) is the utilization rate of the lth AGV in decision step t.

[0031] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any of the above-mentioned workshop scheduling methods based on a proximal strategy optimization algorithm taking into account AGV transportation time.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention uses a graph isomorphic network to encode the local state of the process, better extract the current workpiece state characteristics, and designs a hierarchical action selection structure. The action space consists of a process selection action set, a machine selection action set, and an AGV selection action set. The available solution space is larger and the possibility of obtaining the optimal solution is higher.

[0034] At the same time, multiple strategy networks are used for action selection, which makes the decision more reliable and the operation results better, providing technical support for the realization of flexible job shop scheduling considering the AGV transportation time. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the structure of a hierarchical action space provided by an embodiment of the present invention;

[0036] Figure 2 An action selection flow chart for the test phase provided by an embodiment of the present invention;

[0037] Figure 3 A flowchart of training and testing a policy network provided by an embodiment of the present invention;

[0038] Figure 4 A scheduling Gantt chart of the fjsp9 example provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0040] Embodiment 1:

[0041] The present invention provides a flexible job shop scheduling method based on a proximal strategy optimization algorithm and taking into account AGV transportation time, wherein the training process includes:

[0042] Step 1: According to the current workshop status, obtain the local status of the process, the local status of the machine, and the local status of the AGV, as well as the action set of the process that can be processed, the action set of the machine that can process the current workpiece, and the action set of the AGV that can transport the current workpiece. These three sets constitute the current action space.

[0043] Step 2: Encode the obtained process local state, machine local state and AGV local state. Use graph isomorphism network to encode process local state and obtain process state features; encode machine local state and AGV local state through full connection layer and batch normalization to obtain machine state features and AGV state features respectively.

[0044] Step 3: decode the encoded process state features, machine state features, and AGV state features through three strategy networks to obtain the action values ​​of process actions, machine actions, and AGV actions;

[0045] Normalize each action value to get the corresponding probability distribution, and select the process selection action, machine selection action and AGV selection action according to the rule sampling in the action space and execute them;

[0046] Step 4: Update the current workshop environment according to the executed actions, and update the three decision networks according to the obtained results. Repeat the above steps and complete the training when the preset number of iterations is reached.

[0047] Partial status of process Machine local status and AGV local status Composition state space Each process node The local features are represented by a two-dimensional vector [LB t (O ij ),I t (O ij )], wherein O ij represents the jth process of the i-th workpiece, LB t (O ij ) indicates the completed process O ij The completion time or unprocessed process O ij The estimated minimum completion time, I t (O ij ) is process O ij Indicates whether the scheduling is completed. If the scheduling is completed, it is equal to 1, and if it is not scheduled, it is equal to 0.

[0048] Machine local status It is composed of the states of all machines, each of which consists of a two-dimensional vector [CTM k (t),pt ijk ] indicates that CTM k (t) is the machine M in the decision process of step t k Completion time, pt ijk For process O ij On machine M k If the processing time of machine M k Unable to process process O ij , then pt ijk By process O ij Compatible machine collection The average processing time is expressed as

[0049] AGV local status It is composed of the states of all AGVs, and each AGV state consists of a two-dimensional vector [CTA l (t),UA l (t)] indicates that CTA l (t) is the completion time of the lth AGV in decision step t, UA l (t) is the utilization rate of the lth AGV in decision step t.

[0050] The feature encoding method in step 2 is: for process features, GIN is used for encoding, and for machine features and AGV features, full connection plus batch normalization is used for encoding.

[0051] In step 3, the strategy network is divided into process sub-strategies Machine Sub-Strategy and AGV sub-strategy where θ o ,θm and θ a They are the process selection strategy neural network parameters, machine selection strategy neural network parameters and AGV selection strategy neural network parameters, and s is the current workshop status.

[0052] In step 4, after the workshop executes the process, machine and AGV action, it will obtain the reward r of the action and perform state transition (s→s'). Then, the obtained tuple (s, a, r, s') is put into the experience replay pool. When the experience replay pool reaches a certain size, a certain batch is extracted from it, and the selected actions are evaluated using the Critic network. The three policy network parameters θ are adjusted by Adam and gradient descent methods. o ,θ m ,θ a And the Critic network parameters φ are trained and updated.

[0053] Three sub-strategies The total loss function takes into account the proximal ratio clipping loss Loss o (θ o ,φ), value function loss Loss m (θ m ,φ) and entropy loss Loss a (θ a ,φ), as shown in formula (1), formula (2) and formula (3) respectively:

[0054]

[0055] Among them, c p 、c v and c e They are the proximal ratio clipping loss coefficient, the value function loss coefficient, and the entropy loss coefficient. The three sub-strategies use shared parameters to set the same value. They represent the proximal proportional losses of the process, machine and AGV strategies, respectively, and L MSE represents the value function loss, They represent the entropy loss of the process, machine and AGV respectively.

[0056] In this algorithm, three sub-strategies These are three independent probability distributions that do not affect each other. Finally, the Adam optimizer is used to update the parameters θ of the three policy networks using the gradient descent method. o ,θ m ,θ a And the parameters φ of the Critic network.

[0057] The action space described in step 3 is as follows Figure 1 As shown, the action space It is composed of a hierarchical action space network structure. In the action selection process, action a is composed of process actions Machine Action and AGV action Composition, of which is the set of available processes under the workshop status. For processing operation a o A collection of compatible machines, To transport process action a o A collection of AGVs corresponding to the workpiece.

[0058] The overall decision-making process of the algorithm is as follows Figure 3 As shown in the figure, during the training phase, the process features are GIN encoded, and the machine and AGV features are fully connected encoded. The decision network is trained using proximal policy optimization with a clipping proxy objective, and the encoded features are input into the corresponding policy network. The policy network uses MLP decoding to obtain the probability distribution of each process, machine, and AGV. The action sampling strategy is used to sample an action from the action distribution, so that the decision module can fully explore during the training process.

[0059] Embodiment 2:

[0060] The present invention provides a flexible job shop scheduling method based on a proximal strategy optimization algorithm and taking into account AGV transportation time, and the testing process thereof includes:

[0061] Step 1: According to the current workshop state, the local state of the process, the local state of the machine, and the local state of the AGV are obtained, as well as the action set of the process that can be processed, the action set of the machine that can process the current workpiece, and the action set of the AGV that can transport the current workpiece. These three sets constitute the current action space. The state space and action space in the test phase are the same as those in the training phase.

[0062] Step 2: Encode the obtained process local state, machine local state, and AGV local state. Use graph isomorphism network to encode process local state and obtain process state features; encode machine local state and AGV local state through full connection layer and batch normalization to obtain machine state features and AGV state features respectively. The encoding method in the test phase is the same as that in the training phase.

[0063] Step 3: decode the encoded process state features, machine state features, and AGV state features through three trained strategy networks to obtain the action values ​​of process actions, machine actions, and AGV actions;

[0064] Normalize each action value to get the corresponding probability distribution, select the process selection action, machine selection action and AGV selection action with the highest probability in the action space and execute them;

[0065] Step 4: Update the current workshop environment according to the executed actions, and repeat the above steps until the test case scheduling is completed.

[0066] In the testing phase of the policy network, according to the process sub-strategy Machine Sub-Strategy and AGV sub-strategy The obtained probabilities select process action a respectively o 、Machine Action m and AGV action a a In the testing phase, the process action a with the highest probability is selected according to the greedy strategy. o 、Machine Action m and AGV action a a .

[0067] The action selection process in step 3 is as follows Figure 2 As shown in the figure, at the decision step t=1, the process action is selected first: the completed process is masked, the masked process features are input into the multi-layer GIN encoding, and the encoded process features are decoded by MLP and normalized by softmax to obtain the probability distribution of the process action;

[0068] According to the obtained probability distribution, the processing procedure with the highest probability is selected, and then the machine action is selected:

[0069] Mask the machines that cannot process the current process, input the masked machine features into the fully connected layer and perform batch normalization encoding;

[0070] Pooling operation is performed on the process features after multi-layer GIN encoding;

[0071] The pooled process features are fused with the encoded machine features and then decoded by MLP. The decoded machine features are output through the softmax layer to obtain the probability distribution of the machine actions, from which the machine action with the highest probability is selected.

[0072] Finally, the AGV action is selected, the AGV features are input into the fully connected layer and batch normalized and encoded; the pooled process features are fused with the encoded AGV features and MLP decoded; the decoded machine features are output through the softmax layer to output the probability distribution of the AGV action, from which the AGV action with the highest probability is selected.

[0073] The obtained process action, machine action and AGV action are taken as the action executed at time t=1 and the state is transferred to the decision step t=2, and the above action selection process is repeated until the scheduling is completed.

[0074] The overall decision-making process of the algorithm is as follows Figure 3 As shown in the figure, in the test phase, the process, machine and AGV features are encoded in the same way as in the training phase and input into the corresponding decision network to obtain the probability distribution of each process, machine and AGV, and select the action with the highest probability to complete the optimal scheduling. Repeat until the scheduling is completed.

[0075] Embodiment 3:

[0076] In order to verify the effectiveness of a flexible job shop scheduling method based on proximal strategy optimization algorithm considering AGV transportation time, 6400 randomly generated FJSP-AGV examples were used as training data sets. The processing time pt of each process in the generated examples was randomly generated by a random function and obeyed the uniform distribution pt~U(5,99). During training, the transportation time tt from AGV to different machines was randomly generated by a random function and obeyed the uniform distribution tt~U(5,10). During training, all cases used the same transportation schedule.

[0077] For FJSP-AGV algorithms of different scales, the batch size is selected according to the scale of the example. The number of network layers in the graph isomorphism network GIN is 2, and the multilayer perceptron in each layer of GIN is a three-layer fully connected layer with 128 neurons. In addition, the process decoder Machine Decoder AGV Decoder and Critic Network MLP φ The network structures are the same, both of which consist of two layers of 128 neurons forming a fully connected layer. Other hyperparameter settings are shown in Table 1.

[0078] Table 1: Algorithm training hyperparameters

[0079] Hyperparameters value The number of layers of GIN is L 3 Learning rate 0.001 Experience pool capacity buffer size 100000 Discount factor γ 0.96 PPO Update Times Es 10 <![CDATA[Proximal ratio clipping loss coefficient c p > 2 <![CDATA[Value function loss coefficient c v > 1 <![CDATA[Entropy loss coefficient c e > 0.01 Clipping rate ε 0.2

[0080] The model obtained after training is applied to a small-scale dataset designed by Deroussi and Norre et al., which contains 10 flexible job shop scheduling problems considering transportation time. The download address is: https: / / fastmanufacturingproject.wordpress.com.

[0081] The present invention adopts 6400 examples of 8×8×2 scale for training, and then applies the model obtained after training to the public dataset proposed by Deroussi and Norre et al. to test the generalization performance.

[0082] Through experiments on these data sets, the generalization performance of the model trained based on randomly generated examples on small-scale data sets was verified.

[0083] In order to further prove the effectiveness of the proposed algorithm, the present invention compares the makespan of the proposed algorithm with the known optimal results in the existing literature, and gives the gap between each algorithm and the optimal result, as shown in Table 2. The optimal makespan other than the optimal result in the table is indicated in bold.

[0084] Table 2: Comparison results with other algorithms

[0085]

[0086]

[0087] Wherein CDR1~CDR6 are represented as follows:

[0088] (1) CDR1: MONR+SPT+EETA, selects the workpiece with the largest number of remaining processing steps, selects the machine with the shortest processing time, and selects the AGV with the earliest idle time;

[0089] (2) CDR2: MONR+EETM+EETA, select the workpiece with the largest number of remaining processing steps, select the machine with the earliest idle time, and select the AGV with the earliest idle time;

[0090] (3) CDR3: MWKR+SPT+EETA, select the workpiece with the longest remaining processing time, select the machine with the shortest processing time, and select the AGV with the earliest idle time;

[0091] (4) CDR4: MWKR+EETM+EETA, select the workpiece with the longest remaining processing time, select the machine with the earliest idle time, and select the AGV with the earliest idle time;

[0092] (5) CDR5: LWKR+SPT+EETA, select the workpiece with the shortest remaining processing time, select the machine with the shortest processing time, and select the AGV with the earliest idle time;

[0093] (6) CDR6: LWKR+EETM+EETA, select the workpiece with the shortest remaining processing time, select the machine with the earliest idle time, and select the AGV with the earliest idle time.

[0094] As can be seen from Table 2, the makespan obtained by the algorithm of the present invention is mostly smaller than that of the combined scheduling rules and the result value is stable, and the gap with the optimal makespan known in the latest literature is basically less than 20%. The smallest gap is in the calculation example fjsp9, with the smallest gap of 12.50%. The scheduling Gantt chart is shown in Figure 4 As shown, the makespan is 162; the largest gap does not exceed 25%, and the average gap accounts for 17.44%, which has good stability.

[0095] In addition, in the public dataset, the best performing combined scheduling rules are CDR2 and CDR4, with an average difference of about 25%, which is contrary to the best performance of CDR1 and CDR3. This shows that a single scheduling rule or a single scheduling rule combination cannot be applicable to all workshop environments. The algorithm of the present invention has a good performance both on randomly generated datasets and on classic public datasets.

[0096] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A workshop scheduling method based on proximal strategy optimization algorithm considering AGV transportation time, It is characterized in that include: According to the current workshop status, the action space, process local status, machine local status and AGV local status are obtained; Encode the local state of the process, the local state of the machine and the local state of the AGV to obtain the process state characteristics, the machine state characteristics and the AGV state characteristics respectively; Input the process status features, machine status features and AGV status features into the trained strategy network for decoding to obtain the action values ​​of process actions, machine actions and AGV actions; Normalize each action value to get the corresponding probability distribution, select the process selection action, machine selection action and AGV selection action with the highest probability in the action space and execute them; Update the current workshop environment according to the executed actions, and repeat the above steps until all scheduling processes are completed.

2. The method according to claim 1, It is characterized in that The training process of the policy network includes: The training data is encoded and input into the policy network to obtain the probability distribution of actions; According to the action sampling strategy, an action is selected from the probability distribution of actions and executed, the reward of the action is obtained and the state is transferred, and the action tuple is placed in the experience replay pool; Repeat the above steps until the experience replay pool reaches the preset value, randomly extract several action tuples from it, use the Critic network to evaluate the selected actions, and calculate the total loss function based on the evaluation results; By using Adam and the gradient descent method on the total loss function, the policy network parameters and the Critic network parameters are updated to obtain a trained policy network.

3. The method according to claim 2, It is characterized in that After encoding the training data, input it into the strategy network to obtain the probability distribution of the action; according to the action sampling strategy, sample an action from the probability distribution of the action and execute it, obtain the reward of the action and perform state transfer, including: Mask the completed processes, and input the masked process features into the multi-layer GIN for encoding; perform MLP decoding and softmax normalization on the encoded process state features to obtain the probability distribution of process actions, and sample the process actions based on the probability distribution; Mask the machines that cannot process the current process, input the masked machine features into the fully connected layer and perform batch normalization encoding; perform pooling on the encoded process state features, fuse the pooled process state features with the encoded machine state features and perform MLP decoding, and output the probability distribution of the machine action through the softmax layer, and sample the machine action according to the probability distribution; Mask the AGV that cannot transport the current workpiece ， The masked AGV features are input into the fully connected layer and batch normalized. The encoded process state features are pooled. The pooled process state features are fused with the encoded AGV state features and MLP decoded. The probability distribution of AGV actions is output through the softmax layer, and AGV actions are sampled based on the probability distribution. According to the action sampling strategy, an action is sampled from the probability distribution of the action and executed, the reward of the action is obtained and the state is transferred; Execute the process actions, machine actions and AGV actions obtained at that moment, obtain action rewards and perform state transition to enter the next moment action selection.

4. The method according to any one of claims 1 to 3, It is characterized in that The strategy network includes a process sub-strategy network, a machine sub-strategy network and an AGV sub-strategy network.

5. The method according to claim 1, It is characterized in that The action space includes: The process that can currently be processed selects an action set, the machine that can process the current workpiece selects an action set, and the AGV that can transport the current workpiece selects an action set.

6. The method according to claim 1, It is characterized in that A graph isomorphic network is used to encode the local state of the process; The local state of the machine and the local state of the AGV are encoded through the fully connected layer and batch normalization processing.

7. The method according to claim 1, It is characterized in that Each process node in the process local state The local features are represented by a two-dimensional vector [LB t (O ij ),I t (O ij )], wherein O ij represents the jth process of the i-th workpiece, represents the set of all processes, LB t (O ij ) indicates the processing is completed. ij The completion time or unprocessed process O ij The estimated minimum completion time, I t (O ij ) is process O ij Indicates whether the scheduling is completed. If the scheduling is completed, it is equal to 1, and if it is not scheduled, it is equal to 0.

8. The method according to claim 1, It is characterized in that Machine Local State [CTM k (t),pt ijk ] consists of the local states of all machines, [CTM k (t),pt ijk ] represents the local state of each machine, where CTM k (t) is the machine M in the decision process of step t k Completion time, pt ijk For process O ij On machine M k If the processing time of machine M k Unable to process process O ij , then pt ijk By process O ij Compatible machine collection The average processing time is expressed as 9. The method according to claim 1, It is characterized in that AGV local status [CTA l (t),UA l (t)] consists of the local states of all AGVs, [CTA l (t),UA l (t)] represents the local state of each AGV, where CTA l (t) is the completion time of the lth AGV in decision step t, UA l (t) is the utilization rate of the lth AGV in decision step t.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, a workshop scheduling method based on a proximal strategy optimization algorithm and taking into account AGV transportation time as described in any one of claims 1 to 9 is implemented.

Images