Cigarette factory rolling and packing workshop AMR scheduling method and system based on intelligent agent deep reinforcement learning

Abstract

The embodiment of the invention provides a cigarette factory rolling and packing workshop AMR scheduling method and system based on intelligent agent deep reinforcement learning, and belongs to the technical field of cigarette production. The method comprises the following steps: determining environment information of a rolling and packaging workshop and basic information of each AMR; determining a state space, an action space and a reward function of each AMR according to the environment information and the basic information; and determining a scheduling scheme of the AMR by adopting an MADDPG algorithm pair. According to the method and the system, environment information of a roll and package workshop and basic information of AMRs are combined, each AMR is regarded as an intelligent agent, a state space, an action space and a reward function of each intelligent agent are determined according to the current environment information, and finally, a scheduling scheme of each AMR is determined by adopting an MADDPG algorithm. Compared with the prior art, the method and the system provided by the embodiment of the invention have the advantages that the dynamic and high-efficiency scheduling of each AMR is realized, and the production efficiency of a rolling and packaging workshop is improved.

Description

Technical Field

[0001] This invention relates to the field of cigarette production technology, specifically to an agent-based deep reinforcement learning-based AMR scheduling method and system for cigarette factory packaging workshops. Background Technology

[0002] With the deepening reforms and improved manufacturing levels in the tobacco industry, the production process in cigarette workshops has been largely automated. However, some production logistics remain bottlenecks restricting the full automation of cigarette production. In particular, the distribution and loading of disc-shaped auxiliary materials such as transparent film, white cardboard, and aluminum foil still rely on manual operation. Traditional manual material calling and fixed-path handling methods are difficult to meet the flexible scheduling needs in complex and dynamic environments.

[0003] In existing technologies, research has attempted to apply multi-agent systems and deep reinforcement learning methods to manufacturing system scheduling optimization. For example, the paper "Multi-agent deep reinforcement learning-based approach for dynamic flexible assembly job shop scheduling with uncertain processing and transport times" (DOI: 10.1016 / j.eswa.2025.126441) proposes a scheduling method based on multi-agent deep reinforcement learning. In a flexible assembly job shop environment, it achieves coordinated scheduling and optimized control of multiple types of resources to address the uncertainty of processing and transport times. Although this method has some reference value in scheduling strategy design, it mainly focuses on modeling assembly processes and does not consider the key production characteristics of the cigarette packaging workshop scenario, such as the wide variety of disc-shaped auxiliary materials, fast consumption cycle, high-frequency dynamic triggering of tasks, and extremely low fault tolerance. Therefore, it is difficult to directly apply it to the intelligent scheduling needs of actual tobacco production workshops.

[0004] Another paper, "A framework of cloud-edge collaborated digital twin for flexible job shop scheduling with conflict-free routing" (DOI: 10.1016 / j.rcim.2023.102672), employs a cloud-edge collaborative digital twin scheduling framework to construct a flexible job shop modeling method that integrates processing scheduling and AGV path avoidance. This method achieves real-time response to abnormal disturbances by introducing space-time network modeling, an improved time-window Dijkstra path algorithm, and a rolling window-based edge rescheduling mechanism, effectively optimizing completion time and energy consumption in a simulated automotive welding workshop. However, this method primarily targets multi-variety, small-batch processing workshops and does not consider the highly time-sensitive scheduling challenges in actual workshops caused by differences in material handling cycle times, dynamic task generation, vehicle conflicts, and path constraints. Therefore, it is difficult to directly apply to the dynamic task scheduling and path conflict optimization requirements of multi-agent handling systems in tobacco packaging workshops.

[0005] Furthermore, Chinese patent CN202110006953.9, "A Multi-Agent Deep Reinforcement Learning Scheduling Method for Textile Fabric Dyeing Workshops," proposes a scheduling mechanism based on MA-RPPO. This mechanism models the batching and cylinder arrangement sub-problems in the dyeing workshop as two heterogeneous agents, achieving historical state fusion and policy collaboration between agents through a global Critic and LSTM structure, thus improving the dynamic scheduling response capability for orders. However, this scheme is mainly applicable to dyeing workshop environments with obvious batch processing characteristics and clear process sequences. The task structure, resource allocation, and path mechanism involved differ significantly from the automated handling robot scheduling scenario in the tobacco packaging workshop of this project. Its scheduling modeling does not address key factors such as transportation path optimization, material diversity, and extremely low cycle time tolerance, making it difficult to directly apply to the multi-agent collaborative handling scheduling problem in actual tobacco packaging workshops, which has complex handling path constraints and high timeliness requirements.

[0006] In summary, existing scheduling methods either lack detailed modeling of the actual handling task status or have limited ability to handle dynamic task generation and environmental uncertainties. This is particularly true in cigarette factory packaging workshops, where the following problems urgently need to be addressed: 1. The types of disc-shaped auxiliary materials are diverse and vary in specifications, and the consumption rates of various auxiliary materials differ significantly, requiring high timeliness and accuracy in task scheduling. 2. When multiple robots work together, their paths frequently intersect, which can easily lead to conflicts and blockages, affecting the overall handling efficiency. 3. Dynamic scheduling optimization was not performed by combining multi-dimensional dynamic information from the packaging workshop (such as environmental conditions, robot load status, and material loading urgency). Summary of the Invention

[0007] The purpose of this invention is to provide an agent-based deep reinforcement learning-based AMR scheduling method and system for cigarette factory packaging workshops, which can improve the production efficiency of the packaging workshops.

[0008] To achieve the above objectives, embodiments of the present invention provide an agent-based deep reinforcement learning-based AMR scheduling method for a cigarette factory packaging workshop, comprising: Determine the environmental information of the packaging workshop and the basic information of each AMR; The state space, action space, and reward function of each AMR are determined based on the environmental and basic information. The MADDPG algorithm is used to determine the scheduling scheme for the AMR.

[0009] Optionally, the state space, action space, and reward function of each AMR are determined based on the environmental information and basic information, including: The reward function is determined according to formula (1): (1) in, For the first Taiwan AMR The reward function for the time step, For the first Taiwan AMR's task delay penalty function, For the first The efficiency-related reward function of Taiwan AMR, For the first The resource-related reward function of Taiwan AMR, , These are adjustable weighting coefficients.

[0010] Optionally, the MADDPG algorithm is used to determine the scheduling scheme for the AMR, including: The status inputs for each AMR are determined, including current location, current load status, task execution status, task queue summary, path node occupancy status, and scheduling time information. Determine the task type and urgency information for the current task; An attention mechanism weighting operation is performed based on the task type and urgency information. Based on the result of the weighted operation, select AMR to execute the current task.

[0011] Optionally, the MADDPG algorithm is used to determine the scheduling scheme for the AMR, including: The scheduling scheme is generated using a preset policy network model, wherein the policy network model includes: The backbone network is used for parameter sharing; The Actor network is connected to the backbone network and corresponds one-to-one with the AMRs, and is used to output the corresponding AMR action strategies in real time. The Critic network, connected to the backbone network and corresponding one-to-one with the AMR, is used to determine the current state value function of the AMR based on the current state and the current action strategy.

[0012] Optionally, an attention mechanism weighting operation is performed based on the task type and urgency information, including: The task type and urgency information are represented by one-hot embedding encoding. The result of the one-hot embedding encoding representation operation is weighted by an attention mechanism.

[0013] Optionally, the AMR is selected to execute the current task based on the result of the weighted operation, including: Given the current AMR task, determine the path node to be reached. The backbone network is used to determine whether the path node is occupied by another AMR. If the path node is occupied by another AMR, stop moving until the path node becomes available.

[0014] Optionally, the MADDPG algorithm is used to determine the scheduling scheme for the AMR, including: Initialize the Actor network and Critic network for each AMR; Initialize the global experience replay pool; Based on the input task training set, the Actor network and Critic network of each AMR are used for processing to obtain the corresponding state value, action value and reward function value of each AMR. The obtained state value, action value, and reward function value are stored in the global experience replay pool to update the global experience replay pool; The Actor network and Critic network are trained by sampling batches from the updated global experience replay pool to update the Actor network and Critic network; Determine whether the current training termination condition is met; If the training termination condition is met, return to the step of processing the input task training set using the Actor network and Critic network of each AMR to obtain the corresponding state value, action value and reward function value of each AMR. If the current training termination condition is met, output the policy network model that has completed training.

[0015] Optionally, the method further includes: Formulas (2) and (3) are used as the objective functions of the Critic network: (2) in, The objective function of the Critic network is... In a united state, For the first The Q-value output by the Critic network of an AMR. This is the first action performed. For the first The action to be performed; (3) in, For the first The reward value for this action. As a discount factor, For the first The target Q-value output by the Critic network of an AMR. For the target joint state, The target action is executed for the first time. For the first The target action to be executed next; Formula (4) is used as the objective function of the Actor network: (4) in, Let be the objective function of the Actor network. For the expectation operator, This is the initial action. For the first -1 action executed, For the first +1 action executed, For the first The actions output by the Actor network of each AMR; Formulas (5) and (6) are used as the network parameter update functions for the policy network model: (5) in, For the updated number Parameters of an Actor network, This is the soft update coefficient. For the previous version Parameters of an Actor network; (6) in, For the updated number The parameters of a Critic network, For the previous version The parameters of a Critic network.

[0016] On the other hand, the present invention also provides an AMR scheduling system for a cigarette factory packaging workshop based on agent deep reinforcement learning. The system includes a main controller and multiple AMRs, wherein the main controller is connected to each of the AMRs and is used to cooperate with the AMRs to execute any of the methods described above.

[0017] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the methods described above.

[0018] Through the above technical solutions, this invention provides an AMR scheduling method and system for a cigarette factory packaging workshop using agent deep reinforcement learning. This method and system combine environmental information from the packaging workshop with basic information about the AMRs, treating each AMR as an agent. Based on the current environmental information, the state space, action space, and reward function of each agent are determined. Finally, the MADDPG algorithm is used to determine the scheduling scheme for each AMR. Compared to existing technologies, the method and system provided by this invention achieve dynamic and efficient scheduling of each AMR, improving the production efficiency of the packaging workshop.

[0019] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an agent-based deep reinforcement learning AMR scheduling method for a cigarette factory packaging workshop according to an embodiment of the present invention. Figure 2This is a flowchart of a method for determining an AMR scheduling scheme according to an embodiment of the present invention; Figure 3 This is a structural block diagram of a policy network model according to an embodiment of the present invention; Figure 4 This is a flowchart of a method for training a policy network model according to an embodiment of the present invention. Detailed Implementation

[0021] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0022] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0023] like Figure 1 The diagram shows a flowchart of an agent-based deep reinforcement learning AMR scheduling method for a cigarette factory packaging workshop according to an embodiment of the present invention. Figure 1 In this method, the steps may include: In step S10, the environmental information of the packaging workshop and the basic information of each AMR (Autonomous Mobile Robot) system are determined. In step S11, the state space, action space, and reward function of each AMR are determined based on environmental information and basic information; In step S12, the MADDPG (Multi-Agent Deep Deterministic Policy Gradient) algorithm is used to determine the scheduling scheme of AMR.

[0024] In such Figure 1 In the method shown, step S10 can be used to determine the environmental information of the packaging workshop and the basic information of each AMR. The environmental information may include the transport route, shelf distribution, and workstation distribution of the packaging workshop. The basic information of the AMR may include the load capacity, load type, current location, and travel speed of each AMR.

[0025] Step S11 can be used to determine the state space, action space, and reward function of each AMR based on environmental and basic information. The state space can represent the current position, current load status, task execution status, task queue summary, path node occupancy status, scheduling time information, etc. of each AMR, while the action space can represent the execution actions of each AMR, such as discrete decision actions (selecting a task, moving to a path node, loading / unloading, waiting, etc.).

[0026] In one example of this invention, the reward function can be represented by the following formula (1): (1) in, For the first Taiwan AMR The reward function for each time step. For the first Taiwan AMR's task delay penalty function, For the first The efficiency-related reward function of Taiwan AMR, For the first The resource-related reward function of Taiwan AMR, , These are adjustable weighting coefficients.

[0027] In formula (1), the delay penalty mechanism ( This can be used to significantly penalize delays in completing critical tasks, guiding the system to prioritize timely material delivery.

[0028] In formula (1), the system calculates the difference between the actual completion time of the task and its deadline, which is set as the delay time. If a task is severely delayed (e.g., past the deadline), a substantial penalty will be imposed. If the delay is slight, apply a unit delay penalty. Otherwise, the reward will be 0.

[0029] In formula (1), efficiency-based reward mechanisms ( This can be used to incentivize task processing speed, path efficiency, and response time. It includes task processing time penalties (Makespan), path travel distance penalties, and task response time penalties.

[0030] In formula (1), the resource utilization reward mechanism ( It can be used to measure system-level load balancing and resource utilization efficiency. It includes task scheduling load balancing and idle penalty mechanisms.

[0031] Step S12 can be used to determine the scheduling scheme for the AMR using the MADDPG algorithm. In one example of the present invention, step S12 may further include, for example... Figure 2 The method shown in the diagram. Figure 2 In this process, step S12 may further include the following methods: In step S20, the status input for each AMR is determined. This status input may include current location, current load status, task execution status, task queue summary, path node occupancy status, and scheduling time information.

[0032] In step S21, the task type and urgency information of the current task are determined.

[0033] In step S22, an attention mechanism weighting operation is performed based on the task type and urgency information. Since task type and urgency information cannot be directly weighted, in one example of this invention, step S22 may involve first performing, for example, a one-hot embedding encoding representation operation, and then performing an attention mechanism weighting operation on the result of the one-hot embedding encoding representation operation.

[0034] In step S23, the AMR is selected to execute the current task based on the result of the weighted operation.

[0035] The specific steps of the MADDPG algorithm can take many forms known to those skilled in the art. Considering that when this invention is implemented in a packaging workshop, it is necessary to ensure that the operation of each AMR does not conflict with each other and can work simultaneously, in one example of this invention, the MADDPG algorithm can generate a scheduling scheme using a preset policy network model. This policy network model can include, for example: Figure 3 The structure shown. In this Figure 3 The policy network model can include a backbone network 1, an actor network 2, and a criterion network 3. The backbone network 1 can be used for parameter sharing. Specifically, the backbone network 1 can uniformly encode the input states, allowing actor networks 2 and criterion networks 3 to directly share parameters through the encoded information. Actor network 2 can be connected to the backbone network 1 and corresponds one-to-one with each actor role (AMR), used to output the action policy of the corresponding AMR in real time. Critic network 3 can be connected to the backbone network 1 and corresponds one-to-one with each actor role (AMR), used to determine the current state value function of the AMR based on the current state and the current action policy. Furthermore, the deployment method of this policy network model can be any of those known in the art. In one example of this invention, the policy network model can be implemented using the PyTorch framework and deployed in the VSCode programming environment.

[0036] In one embodiment of the present invention, to avoid collisions between AMRs, in one example, when selecting an AMR to perform a task, the path node to be reached can be determined first, and then the backbone network 1 can be used to determine whether the path node is occupied by another AMR. If the path node is occupied by another AMR, the AMR can stop moving until the path node becomes available.

[0037] In one embodiment of the present invention, the training process of the policy network model can take various forms known to those skilled in the art. In one example of the present invention, considering that the policy network model is a distributed decision-making model, the Actor network 2 and Critic network 3 on each AMR directly make decisions on task execution, while the backbone network 1 is mainly used for information encoding and parameter sharing. Therefore, the training process of the policy network model can include, for example: Figure 4 The method shown in the diagram. Figure 4 In this process, the training process may include the following steps: In step S30, the Actor network of each AMR is initialized. and Critic Network ,in, ; In step S31, the global experience replay pool D is initialized; In step S32, based on the input task training set, the Actor network and Critic network of each AMR are used for processing to obtain the corresponding state value, action value, and reward function value of each AMR. In this step S32, the processing time of each AMR can be synchronized, which facilitates improved overall scheduling efficiency. This processing can be performed by each AMR based on its local observations. Select action: And execute the action, and the environment returns to a united state. Joint actions Rewards for each intelligent agent and the next joint state .

[0038] In step S33, the obtained state value, action value and reward function value are stored in the global experience replay pool to update the global experience replay pool; In step S34, the Actor network and Critic network are trained by sampling batches from the updated global experience replay pool to update the Actor network and Critic network. The objective function of the Actor network can be one of the following formulas (2) to (4): (2) in, The objective function of the Critic network is... In a united state, For the first The Q-value output by the Critic network of an AMR. This is the first action performed. For the first The action to be performed; (3) in, For the first The reward value for this action. As a discount factor, For the first The target Q-value output by the Critic network of an AMR. For the target joint state, The target action is executed for the first time. For the first The target action to be executed next; Formula (4) is used as the objective function of the Actor network: (4) in, Let be the objective function of the Actor network. For the expectation operator, This is the initial action. For the first -1 action executed, For the first +1 action executed, For the first The action output by the Actor network of each AMR.

[0039] While the parameter update methods for the Actor network and Critic network can take many forms known to those skilled in the art, considering the special operating conditions of the Actor network and Critic network according to the present invention, formulas (5) and (6) can be used as the network parameter update functions for the Actor network and Critic network: (5) in, For the updated number Parameters of an Actor network, This is the soft update coefficient. For the previous version Parameters of an Actor network; (6) in, For the updated number The parameters of a Critic network, For the previous version The parameters of a Critic network.

[0040] In step S35, it is determined whether the training termination condition is met. If the training termination condition is met, return to the step of processing the input task training set using the Actor network and Critic network of each AMR to obtain the corresponding state value, action value and reward function value of each AMR. In step S36, if the training termination condition is met, the trained policy network model is output.

[0041] On the other hand, the present invention also provides an AMR scheduling system for a cigarette factory packaging workshop based on agent deep reinforcement learning. The system includes a main controller and multiple AMRs, wherein the main controller is connected to each of the AMRs and is used to cooperate with the AMRs to execute any of the methods described above.

[0042] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the methods described above.

[0043] Through the above technical solutions, this invention provides an AMR scheduling method and system for a cigarette factory packaging workshop using agent deep reinforcement learning. This method and system combine environmental information from the packaging workshop with basic information about the AMRs, treating each AMR as an agent. Based on the current environmental information, the state space, action space, and reward function of each agent are determined. Finally, the MADDPG algorithm is used to determine the scheduling scheme for each AMR. Compared to existing technologies, the method and system provided by this invention achieve dynamic and efficient scheduling of each AMR, improving the production efficiency of the packaging workshop.

[0044] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0048] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0049] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0050] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0051] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0052] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for scheduling AMR (Automatic Reinforcement Learning) in a cigarette factory packaging workshop using deep reinforcement learning, characterized in that, include: Determine the environmental information of the packaging workshop and the basic information of each AMR; The state space, action space, and reward function of each AMR are determined based on the environmental and basic information. The MADDPG algorithm is used to determine the scheduling scheme for the AMR.

2. The method according to claim 1, characterized in that, Based on the aforementioned environmental and basic information, the state space, action space, and reward function of each AMR are determined, including: The reward function is determined according to formula (1): ，（1） in, For the first Taiwan AMR The reward function for the time step, For the first Taiwan AMR's task delay penalty function, For the first The efficiency-related reward function of Taiwan AMR, For the first The resource-related reward function of Taiwan AMR, , These are adjustable weighting coefficients.

3. The method according to claim 1, characterized in that, The MADDPG algorithm is used to determine the scheduling scheme for the AMR, including: The status inputs for each AMR are determined, including current location, current load status, task execution status, task queue summary, path node occupancy status, and scheduling time information. Determine the task type and urgency information for the current task; An attention mechanism weighting operation is performed based on the task type and urgency information. Based on the result of the weighted operation, select AMR to execute the current task.

4. The method according to claim 3, characterized in that, The MADDPG algorithm is used to determine the scheduling scheme for the AMR, including: The scheduling scheme is generated using a preset policy network model, wherein the policy network model includes: The backbone network is used for parameter sharing; The Actor network is connected to the backbone network and corresponds one-to-one with the AMRs, and is used to output the corresponding AMR action strategies in real time. The Critic network, connected to the backbone network and corresponding one-to-one with the AMR, is used to determine the current state value function of the AMR based on the current state and the current action strategy.

5. The method according to claim 3, characterized in that, The attention mechanism performs weighted operations based on the task type and urgency information, including: The task type and urgency information are represented by one-hot embedding encoding. The result of the one-hot embedding encoding representation operation is weighted by an attention mechanism.

6. The method according to claim 4, characterized in that, Based on the result of the weighted operation, the AMR is selected to execute the current task, including: Given the current AMR task, determine the path node to be reached. The backbone network is used to determine whether the path node is occupied by another AMR. If the path node is occupied by another AMR, stop moving until the path node becomes available.

7. The method according to claim 3, characterized in that, The MADDPG algorithm is used to determine the scheduling scheme for the AMR, including: Initialize the Actor network and Critic network for each AMR; Initialize the global experience replay pool; Based on the input task training set, the Actor network and Critic network of each AMR are used for processing to obtain the corresponding state value, action value and reward function value of each AMR. The obtained state value, action value, and reward function value are stored in the global experience replay pool to update the global experience replay pool; The Actor network and Critic network are trained by sampling batches from the updated global experience replay pool to update the Actor network and Critic network; Determine whether the current training termination condition is met; If the training termination condition is met, return to the step of processing the input task training set using the Actor network and Critic network of each AMR to obtain the corresponding state value, action value and reward function value of each AMR. If the current training termination condition is met, output the policy network model that has completed training.

8. The method according to claim 7, characterized in that, The method further includes: Formulas (2) and (3) are used as the objective functions of the Critic network: ，（2） in, The objective function of the Critic network is... In a united state, For the first The Q-value output by the Critic network of an AMR. This is the first action performed. For the first The action to be performed; ，（3） in, For the first The reward value for this action. As a discount factor, For the first The target Q-value output by the Critic network of an AMR. For the target joint state, The target action is executed for the first time. For the first The target action to be executed next; Formula (4) is used as the objective function of the Actor network: ，（4） in, Let be the objective function of the Actor network. For the expectation operator, This is the initial action. For the first -1 action executed, For the first +1 action executed, For the first The actions output by the Actor network of each AMR; Formulas (5) and (6) are used as the network parameter update functions for the policy network model: ，（5） in, For the updated number Parameters of an Actor network, This is the soft update coefficient. For the previous version Parameters of an Actor network; ，（6） in, For the updated number The parameters of a Critic network, For the previous version The parameters of a Critic network.

9. A deep reinforcement learning-based AMR scheduling system for a cigarette factory packaging workshop, characterized in that, The system includes a main controller and multiple AMRs, wherein the main controller is connected to each of the AMRs and is used to cooperate with the AMRs to perform the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that are read by a machine to cause the machine to perform the method as described in any one of claims 1 to 8.

Images