Multi-AGV path planning method, system, device and product

Abstract

The invention belongs to the technical field of intelligent logistics, and aims to provide a multi-AGV path planning method, system, equipment and product. The method comprises the following steps: establishing an environment map of a warehousing operation area; a multi-AGV path planning problem based on the environment map is converted into a Markov decision model; obtaining current environment state information of each AGV based on the Markov decision model, and obtaining an initial path of each AGV based on a deep reinforcement learning method according to the current environment state information of each AGV; performing path conflict detection on the initial path of each AGV, and when a path conflict occurs, performing path updating processing on the initial path of the corresponding AGV based on a priority obstacle avoidance strategy so as to obtain a final path of each AGV; and generating a motion instruction according to the final path of each AGV, and controlling each AGV to run according to the corresponding final path based on the motion instruction. The method can be applied to a multi-AGV collaborative operation scene, and the task execution efficiency and safety are higher.

Description

Technical Field

[0001] This invention belongs to the field of intelligent logistics technology, specifically relating to a multi-AGV path planning method, system, equipment, and product. Background Technology

[0002] With the rapid development of China's e-commerce industry and the continuous growth in demand for intelligent logistics, Automated Guided Vehicles (AGVs), as an important component of intelligent warehousing, have been widely used in various warehousing and logistics scenarios, especially playing a key role in Robotic Mobile Fulfillment Systems (RMFS). In RMFS, AGVs are mainly responsible for transporting mobile shelves containing goods to the picking station. Pickers complete the picking according to the order instructions and then transport the shelves back to the storage area, thereby achieving efficient order response.

[0003] To improve overall operational efficiency, it is typically necessary to coordinate multiple AGVs to perform transport tasks within a limited space, and to plan the optimal path for each AGV while ensuring collision-free operation. Current AGV path planning methods mainly include metaheuristic algorithms (such as genetic algorithms and ant colony algorithms) and graph search algorithms (such as Dijkstra's algorithm and A* algorithm). These methods perform well in static or low-dynamic environments, but in highly dynamic warehouse environments involving multiple AGVs working collaboratively, the following problems exist: In multi-vehicle path planning, existing methods often rely on static obstacle avoidance or path reservation for conflict avoidance, lacking a unified priority mechanism and dynamic obstacle avoidance strategy. This makes it difficult to coordinate conflicts among multiple AGVs, leading to frequent problems such as path blocking and deadlock, resulting in low task execution efficiency and difficulty in ensuring the safety of multi-AGV operation. Summary of the Invention

[0004] The present invention aims to solve the above-mentioned technical problems to at least a certain extent, and provides a multi-AGV path planning method, system, equipment and product.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multi-AGV path planning method, comprising: Create an environmental map of the warehouse operation area; The multi-AGV path planning problem based on the environmental map is transformed into a Markov decision model. Based on the Markov decision model, the current environmental state information of each AGV is obtained, and based on the current environmental state information of each AGV, the initial path of each AGV is obtained using a deep reinforcement learning method. The initial path of each AGV is checked for path conflict. When a path conflict occurs, the initial path of the corresponding AGV is updated based on the priority obstacle avoidance strategy in order to obtain the final path of each AGV. Motion commands are generated based on the final path of each AGV, and each AGV is controlled to travel along its corresponding final path based on the motion commands.

[0006] In one possible design, a grid method is used to create an environmental map of the warehouse operation area; wherein, the warehouse operation area includes a rack storage area, a picking station, an AGV docking area, and aisles, and correspondingly, in the environmental map, the rack storage area, the picking station, and the AGV docking area are marked as 1, and the aisles are marked as 0.

[0007] In one possible design, the Markov decision model is represented as a quintuple ( S , A , P , R , c ); in, S The state space represents the current position of the AGV in the environment map, the target position of the AGV, the position of obstacles, and the position of map boundaries. A The action space refers to the set of actions of the AGV in each state. The set of actions in any state includes the basic actions that the AGV can select in that state, including forward, backward, left and / or right. R The reward function is used to assign an immediate reward to each state-action pair; wherein the immediate reward includes penalties for exceeding map boundaries, penalties for obstacle collisions, penalties for turning, distance rewards, and rewards for reaching the target. P This represents the state transition probability, which includes the probability that the AGV will transition to the next state after performing an action in any state. c This represents a discount factor, which is used to balance the AGV's focus on immediate rewards and long-term future rewards during path planning. c ∈[0,1].

[0008] In one possible design, based on the current environmental state information of any AGV, the initial path of any AGV is obtained using a deep reinforcement learning method, including: Construct a neural network model based on the deep learning method; The current environmental state information of any AGV is input into the neural network model to obtain the Q value of all possible actions of any AGV; Based on the Q value of all possible actions of any AGV, an action selection is performed on any AGV to obtain the next action of any AGV. Based on the next action, update the position of any AGV to obtain new environmental state information for any AGV; Based on the new environmental status information, determine whether any AGV has reached the target location; If so, all actions of any of the sequentially selected AGVs are combined to obtain the initial path of any of the AGVs; If not, the current environmental state information of any AGV is updated to the new environmental state information, and then the current environmental state information of any AGV is input into the neural network model until the initial path of any AGV is obtained.

[0009] In one possible design, the neural network model includes a convolutional layer, a fully connected layer, and an output layer connected in sequence; correspondingly, the current environmental state information of any AGV is input into the neural network model to obtain the Q-values ​​of all possible actions of any AGV, including: The convolutional layer extracts spatial structural feature information from the current environmental state information to obtain high-dimensional spatial features. The high-dimensional spatial features are nonlinearly combined and dimensionality reduced through the fully connected layer to obtain the Q-values ​​of all possible actions of any AGV, and then output through the output layer.

[0010] In one possible design, adopt e The -greedy strategy selects actions for any of the AGVs.

[0011] In one possible design, the method further includes: The AGV's travel trajectory is displayed in real time.

[0012] Secondly, the present invention provides a multi-AGV path planning system, comprising: The environmental modeling module is used to create environmental maps of the warehouse operation area. The path planning module is communicatively connected to the environment modeling module and is used to convert the multi-AGV path planning problem based on the environment map into a Markov decision model; it is also used to obtain the current environmental state information of each AGV based on the Markov decision model, and obtain the initial path of each AGV based on the current environmental state information of each AGV using a deep reinforcement learning method. The path update module is communicatively connected to the path planning module. It is used to detect path conflicts in the initial path of each AGV, and when a path conflict occurs, it updates the initial path of the corresponding AGV based on the priority obstacle avoidance strategy so as to obtain the final path of each AGV. The execution control module is communicatively connected to the path update module and is used to generate motion instructions based on the final path of each AGV, and control each AGV to travel along its corresponding final path based on the motion instructions.

[0013] Thirdly, the present invention provides an electronic device, comprising: Memory, used to store computer program instructions; and, A processor is configured to execute the computer program instructions to perform the operation of a multi-AGV path planning method as described in any of the preceding descriptions.

[0014] Fourthly, the present invention provides a computer program product, including a computer program or instructions, wherein the computer program or instructions, when executed by a computer, implement a multi-AGV path planning method as described in any of the above.

[0015] The beneficial effects of this invention are as follows: This invention discloses a multi-AGV path planning method, system, device, and product, applicable to multi-AGV collaborative operation scenarios, offering higher task execution efficiency and safety. In implementation, this invention transforms the multi-AGV path planning problem based on the environmental map into a Markov decision model, and obtains the initial paths of each AGV using deep reinforcement learning. Path planning through deep reinforcement learning technology gives this invention excellent generalization and real-time response capabilities. Furthermore, after obtaining the initial paths of each AGV, conflict detection is performed on each AGV beforehand. When a path conflict is detected, the initial paths of the corresponding AGVs are updated based on a priority obstacle avoidance strategy. This dynamically adjusts the actions or paths of low-priority AGVs, ensuring that high-priority AGVs have priority passage, reducing path conflicts, and enabling AGVs to quickly and accurately avoid obstacles and other AGVs in dynamic environments. This reduces deadlock problems caused by conflicts, achieving multi-AGV obstacle avoidance, thereby improving the collaborative operation capability of multi-AGV systems in complex warehousing environments, resulting in higher task execution efficiency and safety.

[0016] Other beneficial effects of the present invention will be further explained in the specific embodiments. Attached Figure Description

[0017] Figure 1 This is a flowchart of a multi-AGV path planning method in one embodiment; Figure 2This is a block diagram of a multi-AGV path planning system in one embodiment; Figure 3 This is a block diagram of an electronic device in one embodiment. Detailed Implementation

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0019] Example 1: This embodiment discloses a multi-AGV path planning method, which can be executed by, but is not limited to, a computer device or virtual machine with certain computing resources, such as a personal computer, smartphone, personal digital assistant or wearable device, or by a virtual machine.

[0020] like Figure 1 As shown, a multi-AGV path planning method may include, but is not limited to, the following steps: S1. Create an environmental map of the warehouse operation area.

[0021] In step S1, an environmental map of the warehouse operation area is established using a grid method. The warehouse operation area includes a shelf storage area, a picking station, an AGV docking area, and aisles. Correspondingly, in the environmental map, the shelf storage area, the picking station, and the AGV docking area are marked as 1, that is, 1 in the environmental map represents an impassable area, and the aisles are marked as 0, that is, 0 in the environmental map represents a passable area.

[0022] S2. The multi-AGV path planning problem based on the environmental map is transformed into a Markov decision model.

[0023] In this embodiment, the Markov decision model is represented as a quintuple ( S , A , P , R , c ); in, SThe state space represents the set of states in the current environment. The state space includes the current position of the AGV in the environment map, the target position of the AGV, the positions of obstacles (such as shelves and picking stations), and the map boundary positions. It should be noted that each state in the state space... s ∈ S It can be encoded as a state matrix and used as input data for a neural network model.

[0024] A The action space refers to the set of actions the AGV performs in each state. The action set for any state includes the basic actions the AGV can select in that state, including forward, backward, leftward, and / or rightward movements. In this embodiment... This includes moving forward, backward, left, and right.

[0025] R The reward function is used to assign an immediate reward to each state-action pair; wherein the immediate reward includes a penalty for exceeding the map boundary. R 1. Obstacle collision penalty R 2. Turning punishment R 3 (When the current action direction is inconsistent with the previous action direction, such as changing from "right" to "up", it is considered a "turn") Distance reward R 4 and rewards for reaching the target R 5. Used to guide AGVs to choose a better path; P This represents the state transition probability, which includes the probability that the AGV will transition to the next state after performing an action in any state. c This represents a discount factor, which is used to balance the AGV's focus on immediate rewards and long-term future rewards during path planning. c ∈[0,1].

[0026] In this embodiment, the reward function of the Markov decision model R By introducing a steering penalty R3. Embedded as part of the immediate reward in the overall reward function, during implementation, the directional change of a single AGV's actions between two consecutive time steps is used as the criterion. If the current action direction is inconsistent with the previous action direction (e.g., changing from "right" to "up"), it is considered an "inflection point," and this behavior will be assigned a negative reward. In this way, the subsequent neural network model will gradually tend to choose paths with consistent direction and fewer turns during training, thereby effectively avoiding unnecessary movement patterns such as zigzag and spiral motions, and reducing path redundancy. Based on this, unnecessary turns during single AGV operation can be effectively reduced, the driving trajectory optimized, and the continuity of the path and execution efficiency improved.

[0027] S3. Based on the Markov decision model, obtain the current environmental state information of each AGV, and based on the current environmental state information of each AGV, obtain the initial path of each AGV using a deep reinforcement learning method.

[0028] In step S3, based on the current environmental state information of any AGV, the initial path of any AGV is obtained using a deep reinforcement learning method, including: S301. Construct a neural network model based on the deep learning method.

[0029] S302. Input the current environmental state information of any AGV into the neural network model to obtain the Q value of all possible actions of any AGV, which is used to provide the basis for action selection of AGV in the path planning process.

[0030] It should be noted that the AGV has global perception capabilities during operation, and can acquire environmental elements including scene layout information, its current position, and the position of the task target point. The environmental state information is used to accurately represent the current environmental state of the AGV during operation. In this embodiment, the current environmental state information of any AGV includes a two-dimensional AGV position matrix, a target position matrix, and an obstacle matrix, which are used to describe the current environmental state of any AGV. The obstacle matrix is ​​used to represent the spatial distribution of static obstacles (such as shelves, picking stations, etc.) in the current environmental map, and the matrix element corresponding to the location of the obstacle is assigned a value of 1, while the other positions are assigned a value of 0. The AGV position matrix is ​​assigned a value of 1 only at the grid corresponding to the current position of any AGV, and a value of 0 at other positions, which is used to represent the position state of any AGV. The target position matrix is ​​assigned a value of 1 only at the grid where the task target position (e.g., a specified shelf or picking station) is located, and a value of 0 at other positions, which is used to guide the path strategy to converge toward the target point.

[0031] In this embodiment, the neural network model includes a convolutional layer, a fully connected layer, and an output layer connected in sequence; correspondingly, in step S302, the current environmental state information of any AGV is input into the neural network model to obtain the Q-values ​​of all possible actions of any AGV, including: S3021. The spatial structure feature information of the current environment state information is extracted through the convolutional layer to obtain high-dimensional spatial features; it should be noted that the convolutional layer is used to identify spatial features such as obstacle boundaries, path connectivity and target guidance direction.

[0032] S3022. The high-dimensional spatial features are nonlinearly combined and dimensionality reduced through the fully connected layer to obtain the Q value of all possible actions of any AGV, and then output through the output layer.

[0033] In this embodiment, the Q values ​​of all possible actions of any AGV are output in vector form, in the form of: ,in, This indicates the current state of any of the AGVs. Next action Expected returns i ∈{1,2,3,4}, action a 1. a 1. a 3 and a 4 represents the current state of each of the AGVs. The actions of moving up, down, left, and right.

[0034] In this embodiment, the Q value of each action is calculated using a preset target update function, which is: ,in, This represents an immediate reward, which comprehensively considers factors in the Markov decision model such as penalties for exceeding map boundaries, obstacle collisions, turning, distance rewards, and rewards for reaching the target. c This represents a discount factor used to balance current immediate rewards with future long-term benefits. Indicates the state Next action The expected return; the neural network model optimizes parameters by minimizing the mean square error between the predicted Q value and the target Q value.

[0035] S303. Based on the Q value of all possible actions of any AGV, select an action for any AGV to obtain the next action of any AGV.

[0036] In step S303, to achieve a balance between exploration and utilization during path planning, the following is adopted: eThe -greedy strategy selects actions for any of the AGVs.

[0037] It should be noted that in this embodiment, in the action decision for each step of any AGV, the following is adopted: e The -greedy strategy is used for action selection to enhance exploration capabilities. Specifically, when any AGV is in a certain state... First, the state is input into the neural network model, and an action Q-value vector is output. After obtaining the action Q-value vector, the system automatically generates a random number between 0 and 1. If the random number is less than the currently set exploration probability ,Right now If so, it enters "Exploration" mode and randomly selects an action that is currently executable; otherwise, if Then it enters "exploitation" mode, and will automatically select the action with the highest Q value, that is: .

[0038] S304. Update the position of any AGV according to the next action to obtain the new environmental state information of any AGV.

[0039] Specifically, in this embodiment, the position of any AGV is updated by executing the next action on any AGV in the environment where the environmental map is located, and its new environmental state information is obtained based on its updated position. During this process, the reward value of the next action can be calculated according to the reward function in the Markov decision model. Then, the combination of the current state, the selected next action, the reward value, and the next state is stored as a state transition sample in the experience replay pool. Subsequently, data is randomly extracted from the experience replay pool, and the parameters of the neural network model are updated using the mean squared error loss function to enable subsequent training of the neural network model.

[0040] S305. Determine whether any of the AGVs has reached the target location based on the new environmental status information.

[0041] S306. If so, all actions of any of the sequentially selected AGVs are combined to obtain the initial path of any AGV. In this embodiment, the path planning strategy is determined by the Q value, and actions are selected for execution based on the current state of the AGV, thereby forming a path with the goal of maximizing the Q value at each step.

[0042] S307. If not, update the current environmental state information of any AGV to the new environmental state information, and then input the current environmental state information of any AGV into the neural network model until the initial path of any AGV is obtained.

[0043] S4. Perform path conflict detection on the initial path of each AGV, and when a path conflict occurs, update the initial path of the corresponding AGV based on the priority obstacle avoidance strategy in order to obtain the final path of each AGV.

[0044] In this embodiment, the types of path conflicts include opposing conflicts, cross conflicts, and static occupancy conflicts. Among them, opposing conflicts refer to two AGVs traveling from opposite directions to the same grid node, resulting in a head-on encounter. Cross conflicts refer to two AGVs intersecting at a path node, with the risk of simultaneously occupying the intersection point. Static occupancy conflicts refer to the target position being occupied by another AGV in a stationary state, preventing the current AGV from entering the target node normally.

[0045] Correspondingly, during the path update process for the initial path of the corresponding AGV based on the priority obstacle avoidance strategy, a waiting mechanism is adopted when a cross-collision occurs. That is, the priority between the conflicting AGVs is obtained, and the AGV with higher priority is allowed to pass first, while the AGV with lower priority waits in place at its current position and initiates the path action again after a delay of one or more time steps. When a reversing conflict or a static occupancy conflict occurs, a path replanning mechanism is adopted. That is, the AGV with higher priority is allowed to execute the original path normally, while the AGV with lower priority will call the neural network model again according to its current environmental state information in order to generate a new path that can avoid the conflict grid, thereby realizing path update.

[0046] S5. Generate motion instructions based on the final path of each AGV, and control each AGV to travel along its corresponding final path based on the motion instructions.

[0047] S6. Display the AGV's driving trajectory in real time.

[0048] This embodiment is applicable to multi-AGV collaborative operation scenarios, offering higher task execution efficiency and safety. During implementation, the multi-AGV path planning problem based on the environmental map is transformed into a Markov decision model. Initial paths for each AGV are obtained using deep reinforcement learning. Path planning through deep reinforcement learning technology gives this embodiment good generalization and real-time response capabilities. Furthermore, after obtaining the initial paths for each AGV, conflict detection is performed on each AGV beforehand. When path conflicts are detected, the initial paths of the corresponding AGVs are updated based on a priority obstacle avoidance strategy. This dynamically adjusts the actions or paths of low-priority AGVs, ensuring that high-priority AGVs have priority passage, reducing path conflicts, and enabling AGVs to quickly and accurately avoid obstacles and other AGVs in dynamic environments. This reduces deadlock problems caused by conflicts, achieving multi-AGV obstacle avoidance, thereby improving the collaborative operation capability of the multi-AGV system in complex warehousing environments, resulting in higher task execution efficiency and safety.

[0049] Example 2: This embodiment discloses a multi-AGV path planning system for implementing the multi-AGV path planning method in Embodiment 1; such as Figure 2 As shown, the multi-AGV path planning system includes: The environmental modeling module is used to create environmental maps of the warehouse operation area. The path planning module is communicatively connected to the environment modeling module and is used to convert the multi-AGV path planning problem based on the environment map into a Markov decision model; it is also used to obtain the current environmental state information of each AGV based on the Markov decision model, and obtain the initial path of each AGV based on the current environmental state information of each AGV using a deep reinforcement learning method. The path update module is communicatively connected to the path planning module. It is used to detect path conflicts in the initial path of each AGV, and when a path conflict occurs, it updates the initial path of the corresponding AGV based on the priority obstacle avoidance strategy so as to obtain the final path of each AGV. The execution control module is communicatively connected to the path update module and is used to generate motion instructions based on the final path of each AGV, and control each AGV to travel along its corresponding final path based on the motion instructions. The simulation visualization module is communicatively connected to the execution control module and is used to display the AGV's driving trajectory in real time.

[0050] It should be noted that the working process, working details and technical effects of the multi-AGV path planning system provided in this embodiment 2 can be found in embodiment 1, and will not be repeated here.

[0051] Example 3: Based on Embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smartphone, tablet computer, laptop computer, or desktop computer, etc. The electronic device may be referred to as a user terminal, portable terminal, desktop terminal, etc. Figure 3 As shown, the electronic device includes: Memory, used to store computer program instructions; and, A processor is configured to execute the computer program instructions to perform the operation of a multi-AGV path planning method as described in any of Embodiment 1.

[0052] Specifically, processor 301 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 301 may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). Processor 301 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 301 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen.

[0053] The memory 302 may include one or more computer-readable storage media, which may be non-transitory. The memory 302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 302 are used to store at least one instruction, which is executed by the processor 301 to implement the multi-AGV path planning method provided in Embodiment 1 of this application.

[0054] In some embodiments, the terminal may also optionally include a communication interface 303 and at least one peripheral device. The processor 301, memory 302, and communication interface 303 can be connected via a bus or signal line. Each peripheral device can be connected to the communication interface 303 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 304, a display screen 305, and a power supply 306.

[0055] The communication interface 303 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the communication interface 303 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0056] The radio frequency (RF) circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 304 communicates with communication networks and other communication devices via electromagnetic signals.

[0057] Display screen 305 is used to display the UI (User Interface). The UI may include any combination of graphics, text, icons, and video.

[0058] Power supply 306 is used to supply power to various components in electronic devices.

[0059] Example 4: Based on any one of Embodiments 1 to 3, this embodiment discloses a computer program product, including a computer program or instructions, which, when executed by a computer, implements a multi-AGV path planning method as described in any one of Embodiments 1. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0060] Obviously, those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-AGV path planning method, characterized in that, include: Create an environmental map of the warehouse operation area; The multi-AGV path planning problem based on the environmental map is transformed into a Markov decision model. Based on the Markov decision model, the current environmental state information of each AGV is obtained, and based on the current environmental state information of each AGV, the initial path of each AGV is obtained using a deep reinforcement learning method. The initial path of each AGV is checked for path conflict. When a path conflict occurs, the initial path of the corresponding AGV is updated based on the priority obstacle avoidance strategy in order to obtain the final path of each AGV. Motion commands are generated based on the final path of each AGV, and each AGV is controlled to travel along its corresponding final path based on the motion commands.

2. The multi-AGV path planning method according to claim 1, characterized in that, An environmental map of the warehouse operation area is established using a grid method; wherein, the warehouse operation area includes a rack storage area, a picking station, an AGV docking area, and aisles. Correspondingly, in the environmental map, the rack storage area, the picking station, and the AGV docking area are marked as 1, and the aisles are marked as 0.

3. The multi-AGV path planning method according to claim 1, characterized in that, The Markov decision model is represented as a quintuple ( S , A , P , R , γ ); in, S The state space represents the current position of the AGV in the environment map, the target position of the AGV, the position of obstacles, and the position of map boundaries. A The action space refers to the set of actions of the AGV in each state. The set of actions in any state includes the basic actions that the AGV can select in that state, including forward, backward, left and / or right. R The reward function is used to assign an immediate reward to each state-action pair; wherein the immediate reward includes penalties for exceeding map boundaries, penalties for obstacle collisions, penalties for turning, distance rewards, and rewards for reaching the target. P This represents the state transition probability, which includes the probability that the AGV will transition to the next state after performing an action in any state. γ This represents a discount factor, which is used to balance the AGV's focus on immediate rewards and long-term future rewards during path planning. γ ∈[0,1].

4. The multi-AGV path planning method according to claim 1, characterized in that, Based on the current environmental state information of any AGV, the initial path of any AGV is obtained using a deep reinforcement learning method, including: Construct a neural network model based on the deep learning method; The current environmental state information of any AGV is input into the neural network model to obtain the Q value of all possible actions of any AGV; Based on the Q value of all possible actions of any AGV, an action selection is performed on any AGV to obtain the next action of any AGV. Based on the next action, update the position of any AGV to obtain new environmental state information for any AGV; Based on the new environmental status information, determine whether any AGV has reached the target location; If so, all actions of any of the sequentially selected AGVs are combined to obtain the initial path of any of the AGVs; If not, the current environmental state information of any AGV is updated to the new environmental state information, and then the current environmental state information of any AGV is input into the neural network model until the initial path of any AGV is obtained.

5. The multi-AGV path planning method according to claim 4, characterized in that, The neural network model includes a convolutional layer, a fully connected layer, and an output layer connected in sequence; correspondingly, the current environmental state information of any AGV is input into the neural network model to obtain the Q-values ​​of all possible actions of any AGV, including: The convolutional layer extracts spatial structural feature information from the current environmental state information to obtain high-dimensional spatial features. The high-dimensional spatial features are nonlinearly combined and dimensionality reduced through the fully connected layer to obtain the Q-values ​​of all possible actions of any AGV, and then output through the output layer.

6. The multi-AGV path planning method according to claim 4, characterized in that, use ε The -greedy strategy selects actions for any of the AGVs.

7. The multi-AGV path planning method according to claim 1, characterized in that, The method further includes: The AGV's travel trajectory is displayed in real time.

8. A multi-AGV path planning system, characterized in that, include: The environmental modeling module is used to create environmental maps of the warehouse operation area. The path planning module is communicatively connected to the environment modeling module and is used to convert the multi-AGV path planning problem based on the environment map into a Markov decision model; it is also used to obtain the current environmental state information of each AGV based on the Markov decision model, and obtain the initial path of each AGV based on the current environmental state information of each AGV using a deep reinforcement learning method. The path update module is communicatively connected to the path planning module. It is used to detect path conflicts in the initial path of each AGV, and when a path conflict occurs, it updates the initial path of the corresponding AGV based on the priority obstacle avoidance strategy so as to obtain the final path of each AGV. The execution control module is communicatively connected to the path update module and is used to generate motion instructions based on the final path of each AGV, and control each AGV to travel along its corresponding final path based on the motion instructions.

9. An electronic device, characterized in that, include: Memory is used to store computer program instructions; as well as, A processor is configured to execute the computer program instructions to perform the operation of a multi-AGV path planning method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instructions are executed by the computer, they implement a multi-AGV path planning method as described in any one of claims 1 to 7.

Images