Production workshop multi-AGV dynamic scheduling method based on digital twinning

By constructing a virtual environment in the production workshop, establishing a Markov scheduling model, and training the scheduling model using the SAC algorithm, the problem of low efficiency in traditional AGV scheduling methods is solved, achieving efficient and stable dynamic scheduling of multiple AGVs and reducing transportation costs.

CN118915727APending Publication Date: 2024-11-08TONGJI UNIV +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410949164.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Traditional AGV scheduling methods in production workshops have long solution times and poor optimization effects, failing to meet dynamic scheduling requirements and resulting in low transportation efficiency and high costs.

Method used

A dynamic scheduling method for multiple AGVs in a production workshop based on digital twins is proposed. This method constructs a virtual production workshop, simulates the physical production process, establishes a Markov scheduling decision model, and trains the scheduling model using the SAC algorithm to achieve collaborative scheduling between the virtual and physical production workshops.

Benefits of technology

It improves the dynamic scheduling efficiency of AGVs, reduces transportation costs, enhances the robustness and efficiency of workshop logistics, and adapts to the real-time scheduling needs in large-scale, highly dynamic, and highly complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915727A_ABST
    Figure CN118915727A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of workshop intelligent scheduling, in particular to a digital twinning-based production workshop multi-AGV dynamic scheduling method, which comprises the following steps: S1, constructing a virtual production workshop according to a physical production workshop, and simulating the production process of the physical production workshop in the virtual production workshop to obtain sample data of AGV scheduling; s2, constructing a Markov scheduling decision model for AGV scheduling; s3, training the Markov scheduling decision model by using an SAC algorithm based on the sample data; and S4, generating an AGV scheduling result according to the current physical production workshop state and a Markov scheduling decision model, sending the AGV scheduling result to the virtual production workshop for simulation, generating a scheduling instruction by the virtual production workshop according to the simulation result, sending the scheduling instruction to the physical production workshop, and sending the scheduling instruction to the physical production workshop. The physical production workshop generates sample data of AGV scheduling after executing the scheduling instruction, and returns to the step S3 until a termination condition is reached; compared with the prior art, real-time scheduling of multiple AGVs in a large-scale, strong-dynamic and high-complexity environment can be realized, the robustness of workshop logistics transportation is enhanced, the workshop logistics cost is reduced, and the workshop logistics efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent workshop scheduling, and in particular to a method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins. Background Art

[0002] At present, many production workshops have multiple AGVs to transport materials between processes. For example, the solar cell production workshop is characterized by large scale, strong dynamics, and high complexity. There are hundreds or even thousands of AGVs in the workshop to transport cells between processes. In addition, the material release time in each processing stage and the transportation time between two processes are uncertain. The traditional AGV scheduling method has a long solution time and poor optimization effect, which cannot meet the requirements of dynamic scheduling, resulting in low transportation efficiency and high transportation costs. In recent years, digital twin technology with the characteristics of bidirectional mapping, dynamic interaction, and real-time connection can realize the comprehensive perception of the workshop status and lay the foundation for dynamic scheduling. Therefore, how to provide a multi-AGV dynamic scheduling method for production workshops based on digital twins, which can realize the dynamic scheduling of multiple AGVs, improve transportation efficiency, and reduce transportation costs, has become a research direction for technicians in this field. Summary of the invention

[0003] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a dynamic scheduling method for multiple AGVs in a production workshop based on digital twins, which can realize dynamic scheduling of multiple AGVs, improve transportation efficiency and reduce transportation costs.

[0004] The purpose of the present invention can be achieved by the following technical solution: A method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins, characterized in that it includes the following steps:

[0005] S1. Build a virtual production workshop based on the physical production workshop, and simulate the production process of the physical production workshop in the virtual production workshop to obtain sample data for AGV scheduling;

[0006] S2, build a Markov scheduling decision model for AGV scheduling;

[0007] S3, training the Markov scheduling decision model using the SAC algorithm based on the sample data;

[0008] S4. Generate AGV scheduling results based on the current physical production workshop status and the Markov scheduling decision model, and send the AGV scheduling results to the virtual production workshop for simulation. The virtual production workshop generates scheduling instructions based on the simulation results, and sends the scheduling instructions to the physical production workshop. After executing the scheduling instructions, the physical production workshop generates sample data for AGV scheduling and returns to step S3 until the termination condition is reached.

[0009] In some of the embodiments, in step S2, the Markov scheduling decision model includes a state model, an action model and a reward model.

[0010] In some of the embodiments, in step S2, the state model is established from four aspects: AGV state, feed port state, feed port state and machine state;

[0011] The AGV status includes the real-time location of the AGV, the number of tasks to be completed by the AGV, and the estimated idle time of the AGV;

[0012] The unloading port status includes the number of batches to be transported and whether there are new tasks at the current scheduling time;

[0013] The feeding port status is the number of batches to be processed;

[0014] The machine status is the processing progress of the current material on the machine.

[0015] In some embodiments, in step S2, the action model is an action model for selecting a scheduling rule; the scheduling rule includes:

[0016] The recently idle AGV has priority over the NIN rule;

[0017] Minimum number of transport tasks, idle AGV priority MTTI rule;

[0018] Maximum waiting time idle AGV priority MWT rule;

[0019] The MUT rule of minimum utilization idle AGV priority;

[0020] The MPT rule prioritizes the AGV with the most power available.

[0021] In some embodiments, in step S2, the feedback model is the inverse of the average waiting time for transportation of materials at the material discharge port of the processing equipment, and the calculation formula of the feedback model is as follows:

[0022]

[0023] Where nn represents the number of new tasks to be transported from time t-1 to time t, T as_kn Indicates AGV k The time of arriving at the starting location of the transport task, T re_kn Represents task n Assigned to AGV k Moment. x kn Is a 0-1 variable. If AGV k Transport task n , then xkn =1, otherwise x kn =0.

[0024] In some of the embodiments, the SAC algorithm includes:

[0025] S31, initialize the neural network parameters of Critic, Actor and target Critic;

[0026] S32, initializing the experience replay pool;

[0027] S33, determine whether the termination condition is reached: if yes, output the scheduling decision network model of the Actor; if no, select an action based on the current workshop status and strategy, execute the action, calculate the reward, and observe the workshop status at the next scheduling time;

[0028] S34, storing sample data including the current workshop status, action, feedback, and workshop status at the next scheduling decision time into the experience playback pool;

[0029] S35, assigning the workshop state at the next scheduling decision time to the workshop state at the current time;

[0030] S36, determine whether the training conditions are met: if not, go to S33, if satisfied, randomly select batchsize samples from the experience replay pool, update the Critic, Actor and target Critic network parameters and entropy regularization coefficient;

[0031] S37. Go to S33.

[0032] In some of the embodiments, in step S4, the step of generating the AGV scheduling result according to the current physical production workshop state and the Markov scheduling decision model specifically includes:

[0033] S41. Obtain the status of the physical production workshop at regular intervals;

[0034] S42, determine whether there are new tasks. If there are new tasks, use the Markov scheduling decision model to output scheduling rules according to the current physical production workshop state. If there are no new tasks, go to S41;

[0035] S43. Generate an AGV scheduling result corresponding to the newly added task according to the scheduling rules.

[0036] In some of the embodiments, in the step S4, data transmission between the physical production workshop and the virtual production workshop is performed based on a virtual-real interaction mechanism;

[0037] The virtual-reality interaction mechanism is to map the status of the physical production workshop to the virtual production workshop in real time, and to transmit the corresponding scheduling instructions of the virtual production workshop to the physical production workshop to guide the operation of the physical production workshop.

[0038] In some of the embodiments, the production workshop is a battery cell production workshop.

[0039] Compared with the prior art, the present invention has at least one of the following advantages or beneficial effects:

[0040] 1. The digital twin-based dynamic scheduling method for multiple AGVs in a production workshop disclosed in the present invention realizes the coordination of digital twins, scheduling decision model learning mechanism and online application mechanism. The scheduling decision model can be trained in advance in the virtual production workshop, and efficient scheduling can be achieved during the initial operation of the physical production workshop. It also realizes virtual-real coordination and enhances the scheduling decision-making ability.

[0041] 2. The digital twin-based dynamic scheduling method for multiple AGVs in a production workshop disclosed in the present invention takes into account the AGV status, the unloading port status, the loading port status and the machine tool status, uses five scheduling rules as actions, sets the average waiting time of materials as a reward, and designs a multi-AGV dynamic scheduling method based on Markov scheduling decision, which improves transportation efficiency and reduces transportation costs.

[0042] 3. The digital twin-based dynamic scheduling method for multiple AGVs in a production workshop disclosed in the present invention utilizes the SAC algorithm to train the Markov scheduling decision model. The algorithm accelerates the learning speed of the scheduling decision model and enhances the learning stability, thereby improving the final performance of the scheduling decision model.

[0043] 4. The present invention can realize the real-time scheduling of multiple AGVs in large-scale, highly dynamic and highly complex environments, enhance the robustness of workshop logistics transportation, reduce workshop logistics costs and improve workshop logistics efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention and its features, appearance and advantages will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings. The same reference numerals indicate the same parts in all the drawings. The drawings are not drawn to scale, and emphasis is placed on illustrating the subject matter of the present invention.

[0045] Figure 1 It is a flow chart of a method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins in an embodiment of the present invention;

[0046] Figure 2 Schematic diagram of a dynamic scheduling architecture of multiple AGVs in a production workshop based on digital twins in an embodiment of the present invention;

[0047] Figure 3Flow chart of the SAC algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention provides a method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins, realizes the pre-trained scheduling decision model of the virtual production workshop, and the efficient scheduling of the initial operation stage of the physical production workshop. The designed Markov scheduling decision model based on the production workshop scene realizes the dynamic scheduling of workshop AGVs, and the designed SAC algorithm improves the speed, stability and final performance of the scheduling decision model. The method enhances the robustness of workshop logistics transportation, reduces workshop logistics costs, and improves workshop logistics efficiency.

[0049] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] like Figure 1 As shown, a method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins is provided. The production workshop may be a battery cell production workshop, etc. Specifically, the method includes the following steps:

[0051] S1. Build a virtual production workshop based on the physical production workshop, and simulate the production process of the physical production workshop in the virtual production workshop to obtain sample data for AGV scheduling;

[0052] S2, build a Markov scheduling decision model for AGV scheduling;

[0053] S3, using the SAC algorithm to train the Markov scheduling decision model based on sample data;

[0054] S4. Generate AGV scheduling results based on the current physical production workshop status and Markov scheduling decision model, and send the AGV scheduling results to the virtual production workshop for simulation. The virtual production workshop generates scheduling instructions based on the simulation results and sends the scheduling instructions to the physical production workshop. After executing the scheduling instructions, the physical production workshop generates sample data for AGV scheduling and returns to step S3 until the termination condition is reached; the termination condition can be set manually or by default as the end of production.

[0055] Before implementing the above-mentioned digital twin production workshop multi-AGV dynamic scheduling method, Figure 2 As shown in the figure, a multi-AGV dynamic scheduling architecture can be constructed from three aspects: virtual-reality interaction mechanism, scheduling decision model construction and online application. Then, the multi-AGV dynamic scheduling method of the digital twin production workshop can be carried out based on the AGV dynamic scheduling architecture.

[0056] Specifically, data transmission between the physical production workshop and the virtual production workshop is carried out based on the virtual-real interaction mechanism; the virtual-real interaction mechanism maps the status of the physical production workshop to the virtual production workshop in real time, and the virtual production workshop transmits the corresponding scheduling instructions to the physical production workshop to guide the operation of the physical production workshop. Virtual-real interaction realizes comprehensive data perception in the physical production workshop, fast instruction transmission, and prevents data delay.

[0057] The construction of the above-mentioned scheduling decision model includes two stages: the first stage uses a virtual environment and virtual data to establish a Markov scheduling decision model for AGV scheduling and train the Markov scheduling decision model. Specifically, based on the high fidelity of the virtual production workshop, the battery production process is simulated. Sample data is generated using the relevant data obtained from the simulation, and these sample data are placed in the experience replay pool for training the Markov scheduling decision model. As the simulation progresses, more and more sample data are obtained, and the scheduling decision model is continuously iterated and optimized. By applying the scheduling decision model that has converged through training in the virtual production workshop, the physical production workshop can achieve efficient and stable production in the initial operation stage. The second stage: collaboration between virtual and real workshops; specifically, the AGV scheduling results are generated according to the current physical production workshop status and the current Markov scheduling decision model, and the AGV scheduling results are sent to the virtual production workshop for simulation. The virtual production workshop generates scheduling instructions based on the simulation results (the virtual production workshop generates scheduling instructions after correcting the AGV scheduling results based on the simulation results, and takes into account the evolution of workshop performance), and sends the scheduling instructions to the physical production workshop. After executing the scheduling instructions, the physical production workshop generates sample data for AGV scheduling, and puts these sample data into the experience replay pool for training the Markov scheduling decision model.

[0058] The above online application generates AGV scheduling results based on the current Markov scheduling decision model according to the current physical production workshop status obtained at the scheduling time, and directly outputs the AGV scheduling results to the virtual production workshop for simulation.

[0059] In the above step S2, the above Markov scheduling decision model includes a state model, an action model and a reward model; that is, the Markov scheduling decision model is constructed from three aspects: state S, action A and reward R.

[0060] Specifically, the state model is established from four aspects: AGV state, unloading port state, loading port state and machine state;

[0061] (1) For state S, from AGV state s t_AGV , Feeding port status t_BL , feeding port status t_BU and machine status t_M Four aspects to establish the state model, namely s t={s t_AGV ,s t_BL ,s t_BU ,s t_M}.

[0062] 1) The status of the above AGV t_AGV Including real-time location of AGV pos_t 、The number of tasks to be completed by AGV re_task_t , AGV estimated idle time T AGV_completed , that is, s t_AGV =(AGV pos_t AGV re_task_t , T AGV_completed );

[0063] 2) The above feeding port status s t_BL BL is the number of batches to be processed re_task_t ;

[0064] 3) The state of the above-mentioned feeding port s t_BU Including the number of batches to be transported BU re_task_t Whether there is a new task BU at the current scheduling time new_task_t , that is, s t_BU =(BU re_task_t , B.U. new_task_t );

[0065] 4) The above machine state s t_M is the processing progress M of the current material on the machine pro_t .

[0066] (2) The above action model is an action model for selecting a scheduling rule; for the action A, an action model for selecting a scheduling rule is constructed. The five scheduling rules include:

[0067] 1) The recently idle AGV has priority over the NIN rule;

[0068] 2) MTTI rule with the least number of transport tasks and idle AGVs first;

[0069] 3) Maximum waiting time idle AGV priority MWT rule;

[0070] 4) Minimum utilization idle AGV priority MUT rule;

[0071] 5) The MPT rule that prioritizes the AGV with the most idle power;

[0072] That is, A={NIN=1, MTTI=2, MWT=3, MUT=4, MPT=5}.

[0073] (3) The above feedback model is the inverse of the average waiting time for materials at the material discharge port of the processing equipment. The calculation formula of the feedback model is as follows:

[0074]

[0075] Where nn represents the number of new tasks to be transported from time t-1 to time t, T as_kn Indicates AGV k The time when (kth AGV) arrives at the starting position of the transportation task, T re_kn Represents task n (Task) is assigned to the AGV k Moment. x kn Is a 0-1 variable. If AGV k Transport task n , then x kn =1, otherwise x kn =0.

[0076] In step S3, if Figure 3 As shown, the above SAC algorithm can interact with the discrete action environment. The output of the policy network is modified to a softmax distribution on the discrete action space, and the value network directly receives the state and the distribution of the discrete action space as input. Specifically, the SAC algorithm includes:

[0077] S31. Initialize two Critic neural networks and Actor Neural Network π θ (s), and simultaneously copy the two critic neural network parameters to the target critic neural network.

[0078] S32, initializing the experience replay pool M;

[0079] S33, determine whether the termination condition is reached: if so, output the Actor's scheduling decision network model π θ (s), if not, then based on the current workshop status s t and strategies Select action a t , and perform action a t , calculate the reward r t , and observe the workshop status s at the next scheduling time t+1 ;

[0080] S34, the sample (s t , a t , r t ,s t+1 ) will be stored in the experience replay pool M;

[0081] S35, assign the workshop state at the next decision time to the workshop state s at the current time t ←s t+1 ;

[0082] S36, determine whether the training conditions are met: if not, go to S33, if satisfied, randomly select batchsize samples from the experience replay pool M {(s i , a i , r i ,s i+1 )} i=1,...,batchsize , update the Critic, Actor and target Critic network parameters and entropy regularization coefficients as follows:

[0083] For each tuple, compute

[0084] Based on minimizing the loss function for the two Critic networks Make updates;

[0085] Sampling Actions with the Reparameterization Technique Then use the loss function Update the current Actor network:

[0086] based on Update the coefficient α of the entropy regularization term, where represents the minimum expected entropy of the setting;

[0087] Update the target critic network

[0088] S37. Go to S33.

[0089] In a specific embodiment of the invention, in step S4, the step of generating the AGV scheduling result according to the current physical production workshop state and the Markov scheduling decision model specifically includes:

[0090] S41. Obtain the status of the physical production workshop at regular intervals;

[0091] S42, determine whether there are new tasks. If there are new tasks, use the Markov scheduling decision model to output scheduling rules according to the current physical production workshop state. If there are no new tasks, go to S41;

[0092] S43. Generate an AGV scheduling result corresponding to the newly added task according to the scheduling rules.

[0093] The above-mentioned related explanations and the description of the embodiments are for the convenience of those skilled in the art to understand and apply the present invention. It is obvious that those familiar with the art can easily make various modifications to these contents and apply the general principles described here to other embodiments without creative work. Therefore, the present invention is not limited to the above-mentioned related explanations and the description of the real-time embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the protection scope of the present invention.

Claims

1. A method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins, characterized in that: The following steps are involved: S1. Build a virtual production workshop based on the physical production workshop, and simulate the production process of the physical production workshop in the virtual production workshop to obtain sample data for AGV scheduling; S2, build a Markov scheduling decision model for AGV scheduling; S3, training the Markov scheduling decision model using the SAC algorithm based on the sample data; S4. Generate AGV scheduling results based on the current physical production workshop status and the Markov scheduling decision model, and send the AGV scheduling results to the virtual production workshop for simulation. The virtual production workshop generates scheduling instructions based on the simulation results, and sends the scheduling instructions to the physical production workshop. After executing the scheduling instructions, the physical production workshop generates sample data for AGV scheduling and returns to step S3 until the termination condition is reached.

2. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins according to claim 1, characterized in that: In step S2, the Markov scheduling decision model includes a state model, an action model and a reward model.

3. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins according to claim 2, characterized in that; In step S2, the state model is established from four aspects: AGV state, unloading port state, loading port state and machine state; The AGV status includes the real-time location of the AGV, the number of tasks to be completed by the AGV, and the estimated idle time of the AGV; The unloading port status includes the number of batches to be transported and whether there are new tasks at the current scheduling time; The feeding port status is the number of batches to be processed; The machine status is the processing progress of the current material on the machine.

4. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twin according to claim 2 is characterized in that: In step S2, the action model is an action model for selecting a scheduling rule; the scheduling rule includes: The recently idle AGV has priority over the NIN rule; Minimum number of transport tasks, idle AGV priority MTTI rule; Maximum waiting time idle AGV priority MWT rule; The minimum utilization idle AGV priority MUT rule; The MPT rule prioritizes the AGV with the most power available.

5. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twin according to claim 1, characterized in that: In step S2, the feedback model is the inverse of the average waiting time for transportation of materials at the material discharge port of the processing equipment. The calculation formula of the feedback model is as follows: Where nn represents the number of new tasks to be transported from time t-1 to time t, T as_kn Indicates AGV k The time of arriving at the starting location of the transport task, T re_kn Represents task n Assigned to AGV k Moment. x kn Is a 0-1 variable. If AGV k Transport task n , then x kn =1, otherwise x kn =0.

6. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twin according to claim 1, characterized in that: The SAC algorithm includes: S31, initialize the neural network parameters of Critic, Actor and target Critic; S32, initializing the experience replay pool; S33, determine whether the termination condition is reached: if yes, output the scheduling decision network model of the Actor; if no, select an action based on the current workshop status and strategy, execute the action, calculate the reward, and observe the workshop status at the next scheduling time; S34, storing sample data including the current workshop status, action, feedback, and workshop status at the next scheduling decision time into the experience playback pool; S35, assigning the workshop state at the next scheduling decision time to the workshop state at the current time; S36, determine whether the training conditions are met: if not, go to S33, if satisfied, randomly select batchsize samples from the experience replay pool, update the Critic, Actor and target Critic network parameters and entropy regularization coefficient; S37. Go to S33.

7. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins according to claim 1 is characterized in that: In step S4, the steps of generating the AGV scheduling result according to the current physical production workshop state and the Markov scheduling decision model specifically include: S41. Obtain the status of the physical production workshop at regular intervals; S42, determine whether there are new tasks. If there are new tasks, use the Markov scheduling decision model to output scheduling rules according to the current physical production workshop state. If there are no new tasks, go to S41; S43. Generate an AGV scheduling result corresponding to the newly added task according to the scheduling rules.

8. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins according to claim 1 is characterized in that: In the step S4, data transmission between the physical production workshop and the virtual production workshop is performed based on a virtual-real interaction mechanism; The virtual-reality interaction mechanism is to map the status of the physical production workshop to the virtual production workshop in real time, and to transmit the corresponding scheduling instructions of the virtual production workshop to the physical production workshop to guide the operation of the physical production workshop.

9. The method for dynamic scheduling of multiple AGVs in a production workshop based on digital twins according to claim 1, characterized in that: The production workshop is a battery cell production workshop.

Citation Information

Cited By

  • Supply chain logistics real-time scheduling system and method

    CN120782155A