AGV path planning method, electronic device and machine-readable storage medium

Abstract

The present application discloses an AGV path planning method, an electronic device and a machine-readable storage medium, and relates to the field of material transportation technology. The method includes obtaining the operation requirements and the AGV information of all warehouses, the operation requirements include the starting position, the end position, the positions of all first-level batching areas and the first-level cache area, the positions of all second-level batching areas and the second-level cache area, and the operation process, and the AGV information includes the loading status of the AGV and the position of the AGV; determining the target AGV based on the starting position and all AGV information; screening the AGV to save resources for AGV scheduling between multiple warehouses. Construct an operation time function based on the operation requirements and the position of the target AGV; determine the reward function of the target path planning algorithm based on the operation time function; perform AGV path planning based on the target path planning algorithm to obtain the target AGV path. Remove the restrictions on AGV scheduling between warehouses and improve the adaptability to AGV scheduling between multiple warehouses.

Description

Technical Field

[0001] The present application relates to the technical field of material transportation, and in particular to an AGV path planning method, an electronic device, and a machine-readable storage medium. Background Art

[0002] AGV (Automated Guided Vehicle) is an industrial vehicle that loads goods automatically or manually, and automatically drives along a set route or pulls a cargo trolley to a designated location to load and unload goods. It is widely used in industrial production lines, warehousing and logistics, and other fields. As a subsystem in the management system, WMS (Warehouse Management System) mainly involves three functional modules: inbound management, in-warehouse management, and outbound management. In the warehouse replenishment operation, various equipment needs to be integrated. The WMS mainly sends the handling tasks to the downstream integration system, and the AGV performs the handling actions. In order to improve the operating efficiency of each equipment in the warehouse, the AGV needs to be reasonably scheduled to reduce the transportation time. The existing AGV scheduling method is mainly based on the internal scheduling of a single warehouse, and the scheduling control adaptability between multiple warehouses is low. With the increasing complexity of material transportation needs, how to cover the AGV operation path planning between multiple warehouses has become a problem that needs to be solved urgently. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the purpose of the embodiments of the present application is to provide an AGV path planning method, an electronic device and a machine-readable storage medium.

[0004] In order to achieve the above objectives, the present application provides, in a first aspect, an AGV path planning method, comprising:

[0005] Obtain the operation requirements and AGV information of all warehouses, where the operation requirements include the starting position, the end position, the positions of all primary batching areas and primary buffer areas, the positions of all secondary batching areas and secondary buffer areas, and the operation process; the AGV information includes the loading status of the AGV and the position of the AGV; the primary batching area and the primary buffer area are correspondingly set in the same warehouse; the secondary batching area and the secondary buffer area are correspondingly set in the same warehouse; the primary batching area and the secondary batching area are set in different warehouses;

[0006] Determine the target AGV based on the starting point location and all AGV information;

[0007] Construct the operation time function based on the operation requirements and the location of the target AGV;

[0008] Determine the reward function of the target path planning algorithm based on the operation time function;

[0009] AGV path planning is performed based on the target path planning algorithm to obtain the target AGV path.

[0010] In the embodiment of the present application, an operation time function is constructed based on the operation requirements and the position of the target AGV, including:

[0011] Determine a target primary batching area position and a target primary cache area position among all primary batching area positions and all primary cache area positions, and a target secondary batching area position and a target secondary cache area position among all secondary batching area positions and all secondary cache area positions based on the operation process;

[0012] Determine a waiting time function based on a target primary batching area position and a target primary buffer area position, and a target secondary batching area position and a target secondary buffer area position;

[0013] Determine the transportation time function based on the starting position, the end position, the target primary batching area position and the target primary buffer area position, the target secondary batching area position and the target secondary buffer area position, the target AGV position and the operation process;

[0014] The sum of the waiting time function and the transportation time function is determined as the operation time function.

[0015] In the embodiment of the present application, the AGV path planning method further includes:

[0016] Optimize the operation time function based on preset constraints, where the preset constraints include AGV loading threshold, AGV transportation range, AGV transportation distance minimum, buffer area capacity threshold, and operation area capacity threshold;

[0017] The reward function of the target path planning algorithm is determined based on the operation time function, including:

[0018] The reward function of the target path planning algorithm is determined based on the optimized operation time function.

[0019] In the embodiment of the present application, the reward function of the target path planning algorithm is determined based on the operation time function, including:

[0020] Determine the penalty value based on the preset loss factor and the operation time function;

[0021] The reward function of the target path planning algorithm is constructed based on the difference between the preset reward value and the penalty value.

[0022] In the embodiment of the present application, the steps of constructing the target path planning algorithm include:

[0023] Take each AGV as an agent, establish the environment set, action set, reward function, MADDPG network architecture and the state space of each agent, where the state space is composed of the state of the agent at each moment, including the loading state of the AGV and all preset AGV positions, the relative position of the AGV and the working area, the current capacity of the buffer area and the current capacity of the working area. The environment set includes the working requirements, obstacle information and the AGV information of all AGVs. The action set includes moving in the preset direction, stopping, loading and unloading. The reward function is determined based on the working time function.

[0024] Determine the target action based on the preset strategy, state space, environment set and action set, and obtain the new state of the agent after executing the target action;

[0025] Determine the reward for the new state of the agent based on the reward function, and obtain the state transition sequence of each agent;

[0026] Store the state transition sequence of all agents at each moment into the experience pool;

[0027] Random sampling is performed in the experience pool;

[0028] Update the local network parameters in the MADDPG network architecture based on the back-propagation algorithm;

[0029] And update the target network parameters in the MADDPG network architecture based on the updated local network parameters and the soft update method;

[0030] The MADDPG network architecture is iteratively optimized and trained based on the preset number of iterations to obtain the target path planning algorithm.

[0031] In the embodiment of the present application, the target AGV is determined based on the starting position and all AGV information, including:

[0032] Determine the unloaded AGV based on the loading status of all AGVs;

[0033] Based on the starting position and the positions of all unloaded AGVs, the unloaded AGV closest to the starting position among all unloaded AGVs is determined as the target AGV.

[0034] In the embodiment of the present application, based on the starting position and the AGV positions of all the unloaded AGVs, determining the unloaded AGV closest to the starting position among all the unloaded AGVs as the target AGV includes:

[0035] Determine the Manhattan distance between the starting position and the position of each empty AGV respectively;

[0036] The empty AGV with the smallest Manhattan distance among all empty AGVs is determined as the target AGV.

[0037] In the embodiment of the present application, the AGV path planning method further includes:

[0038] Rasterize all areas of all operation warehouses to obtain a grid space;

[0039] Establish a Cartesian coordinate system based on the grid space;

[0040] Based on the Cartesian coordinate system, the starting position, the end position, the positions of all first-level batching areas, the positions of all first-level buffer areas, the positions of all second-level batching areas, the positions of all second-level buffer areas and the position of the AGV are determined.

[0041] A second aspect of the present application provides an electronic device, including:

[0042] a memory configured to store instructions;

[0043] The processor is configured to call instructions from the memory and implement the AGV path planning method as described in the above embodiment when executing the instructions.

[0044] A third aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the AGV path planning method as described in the above embodiment.

[0045] Through the above technical solution, by obtaining the operation requirements and AGV information of all warehouses, wherein the operation requirements include the starting position, the end position, the positions of all first-level batching areas and the first-level buffer area, the positions of all second-level batching areas and the second-level buffer area, and the operation process, the AGV information includes the loading status of the AGV and the position of the AGV; and the target AGV is determined based on the starting position and all AGV information; by screening the AGV, the resources for scheduling AGVs between multiple warehouses are saved. And the operation time function is constructed based on the operation requirements and the position of the target AGV; the reward function of the target path planning algorithm is determined based on the operation time function; the AGV path is planned based on the target path planning algorithm to obtain the target AGV path. When the target AGV performs the task corresponding to the operation requirements based on the target AGV path, the task execution time can be reduced on the premise of meeting the operation requirements. And through the above technical solution, the restrictions on AGV scheduling between warehouses are lifted, ensuring that AGV can obtain AGV information and operation requirements of different warehouses in real time, and is no longer limited to the AGV scheduling within a single warehouse, and realizes the AGV operation path planning covering multiple warehouses, and improves the adaptability to AGV scheduling between multiple warehouses.

[0046] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0048] Figure 1 A schematic diagram of a process flow of an AGV path planning method according to an embodiment of the present application is schematically shown;

[0049] Figure 2 A schematic diagram of a warehouse scene according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0051] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0052] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0053] Figure 1 The following schematically shows a flow chart of an AGV path planning method according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides an AGV path planning method, which may include the following steps:

[0054] Step 100, obtaining the operation requirements and AGV information of all warehouses, wherein the operation requirements include the starting position, the end position, the positions of all primary batching areas and primary buffer areas, the positions of all secondary batching areas and secondary buffer areas, and the operation process, and the AGV information includes the loading status of the AGV and the position of the AGV, the primary batching area and the primary buffer area are correspondingly set in the same warehouse, the secondary batching area and the secondary buffer area are correspondingly set in the same warehouse, and the primary batching area and the secondary batching area are set in different warehouses;

[0055] In this embodiment, it should be noted that the warehouse is used to store materials and perform activities such as material distribution, and the AGV moves in the warehouse and between warehouses. According to different material uses, single or multiple sorting may be required before the materials are transported to the target location. Each warehouse includes a batching area for sorting materials for pallets. According to different batching sorting degrees, different levels of sorting activities can be carried out in different warehouses. For example, after the first batching sorting is carried out in the batching area in warehouse A, the materials are transported to the batching area of ​​warehouse B for secondary batching sorting by AGV. In one embodiment, more levels of batching sorting can also be included. In this embodiment, taking two levels of batching sorting as an example, the batching area for the first batching sorting is used as the primary batching area, and the batching area for the secondary batching sorting is used as the secondary batching area. It can be understood that in order to improve the efficiency of material sorting, each warehouse includes multiple batching areas, which can realize synchronous batching operations. When all batching areas in the warehouse are in operation, if there are still materials that need to be batched that are being transported by AGVs, the materials can be placed in the buffer area, so that when the batching area is idle, the materials in the buffer area can be further transported to the batching area for batching sorting activities. In the buffer area, the AGV can unload the materials, or it can wait in the buffer area in a loaded state without unloading the materials. One or more buffer areas can be set for each batching area. In warehouses with rich batching area resources, multiple batching areas can share one buffer area. It can be understood that the buffer area associated with the first-level batching area is the first-level buffer area, and the buffer area associated with the second-level batching area is the second-level buffer area. The first-level batching area and the first-level buffer area are correspondingly set in the same warehouse, the second-level batching area and the second-level buffer area are correspondingly set in the same warehouse, and the first-level batching area and the second-level batching area are set in different warehouses. Reference Figure 2 , Figure 2 The figure shows an operation scenario between two warehouses, including warehouse A and warehouse B, and the passage between warehouses A and B. Warehouse A includes a primary batching area and a primary buffer area; warehouse B includes a secondary batching area and a secondary buffer area. Both warehouses A and B can include AGVs and stored materials. Warehouse B can also set up an AGV charging area, and the location of obstacles is shown. Obstacles can be fixed objects, such as workshop doors, production areas, etc., which are areas that AGVs cannot reach.

[0056] The operation requirement refers to the transportation task information that the AGV needs to perform in the current application scenario, including the specified starting point and end point, all first-level batching area locations and first-level buffer area locations that can perform batching and sorting, all second-level batching area locations and second-level buffer area locations that can perform batching and sorting, and the operation process, wherein the operation process includes the relevant operation process that the AGV needs to perform to transport materials required by the current operation requirement. For example, if the material needs to be sorted by the first and second batching trays, the material needs to be transported from the starting point to the first batching area, and the first batching tray sorting is performed in the first batching area. If the first batching area is currently in operation, the material needs to be waited in the first buffer area; after the first batching tray sorting is completed, the material is transported to the secondary batching area, and the secondary batching tray sorting is performed in the secondary batching area. If the secondary batching area is currently in operation, the material needs to be waited in the secondary buffer area until all the batching tray sorting is completed and the material is transported to the end point.

[0057] It should be noted that the parking location of the AGV can be in a single warehouse, multiple warehouses, or some warehouses. AGV information includes the loading status and the location of the AGV. The loading status can be a loaded state or an empty state. In this embodiment, the transportation speed of all AGVs is fixed and the carrying capacity is the same. The location of the AGV can be located in real time through a positioning system such as GPS (Global Positioning System).

[0058] refer to Figure 2 Specifically, in one embodiment, the AGV path planning method further includes:

[0059] Rasterize all areas of all operation warehouses to obtain a grid space;

[0060] Establish a Cartesian coordinate system based on the grid space;

[0061] Based on the Cartesian coordinate system, the starting position, the end position, the positions of all first-level batching areas, the positions of all first-level buffer areas, the positions of all second-level batching areas, the positions of all second-level buffer areas and the position of the AGV are determined.

[0062] It should be noted that rasterization is to divide the continuous space into a series of regular grids or units, which can be squares, cubes, hexagons or other shapes. Each grid is called a grid. After rasterization, the original continuous space or object is converted into a discrete grid space. Based on the grid space, a Cartesian coordinate system is established. Based on the origin of the Cartesian coordinate system, the starting position, the end position, the positions of all first-level batching areas, the positions of all first-level cache areas, the positions of all second-level batching areas, the positions of all second-level cache areas, and the position of the AGV are determined in the form of Cartesian coordinates.

[0063] Step 200, determining the target AGV based on the starting position and all AGV information;

[0064] It should be noted that the target AGV refers to the AGV used to perform the current task. In order to save the scheduling resources of the AGV and improve the task execution efficiency of the task corresponding to the operation demand, in this embodiment, the AGV with the lowest cost to run to the starting position is used as the target AGV. In this embodiment, the running distance is used as a reference for the cost of the AGV running to the starting position. It can be understood that in one embodiment, other indicators can also be used as a reference for the cost of the AGV running to the starting position.

[0065] Specifically, in one embodiment, determining the target AGV based on the starting position and all AGV information includes:

[0066] Determine the unloaded AGV based on the loading status of all AGVs;

[0067] Based on the starting position and the positions of all unloaded AGVs, the unloaded AGV closest to the starting position among all unloaded AGVs is determined as the target AGV.

[0068] It should be noted that in order to avoid affecting the AGVs that are currently performing tasks and reduce the ineffective scheduling of AGVs, in this embodiment, the empty AGVs that are not currently performing tasks are screened to ensure that the target AGV has the capacity to perform the current handling task. In this embodiment, the distance is calculated based on the location and the location of all empty AGVs, and the empty AGV closest to the starting point among all the empty AGVs is taken as the target AGV, so as to ensure the task execution capability of the AGV while minimizing the scheduling cost.

[0069] Further, in one embodiment, based on the starting position and the AGV positions of all the unloaded AGVs, determining the unloaded AGV closest to the starting position among all the unloaded AGVs as the target AGV includes:

[0070] Determine the Manhattan distance between the starting position and the position of each empty AGV respectively;

[0071] The empty AGV with the smallest Manhattan distance among all empty AGVs is determined as the target AGV.

[0072] It should be noted that the Manhattan distance is used to measure the sum of the absolute axis distances of two points in the standard coordinate system. For two points A (x1, y1) and B (x2, y2) on a two-dimensional plane, the Manhattan distance between them can be calculated by the sum of the absolute values ​​of the coordinate differences of the two points in the X-axis and Y-axis directions. The specific formula is as follows:

[0073]

[0074] Among them, d represents the Manhattan distance; i represents the starting point; j represents the jth unloaded AGV.

[0075] After calculating the Manhattan distances between all empty AGVs and the starting position, the empty AGV with the smallest corresponding Manhattan distance among all empty AGVs is determined as the target AGV.

[0076] Step 300, constructing an operation time function based on the operation requirements and the location of the target AGV;

[0077] It should be noted that the operation time function is used to calculate the time it takes for the AGV to perform a task. Based on the operation requirements, the designated locations that the AGV needs to pass through to perform the task can be determined, such as the location of the first-level batching area and the location of the first-level buffer area, the location of the second-level batching area and the location of the second-level buffer area. The operation requirements can also determine the time that may be required to wait for material sorting in the distribution area and the waiting time in the buffer area when performing the task. Based on the position of the target AGV, the transportation time for the target AGV to transport materials to each designated location when performing the task can be determined. Combining all waiting times and transportation times can determine the operation time function corresponding to the target AGV performing the task under the current operation requirements.

[0078] Step 400, determining a reward function of a target path planning algorithm based on the operation time function;

[0079] Step 500, performing AGV path planning based on a target path planning algorithm to obtain a target AGV path.

[0080] It should be noted that the target path planning algorithm is used to plan the path that the target AGV runs when performing the transportation task corresponding to the job demand. The reward function of the target path planning algorithm is used to provide reward feedback to the target path planning algorithm to continuously optimize the target path planning algorithm and improve the effectiveness and accuracy of the target path planning algorithm. By determining the reward function of the target path planning algorithm based on the job time function, it is possible to limit the time required for the target AGV path output by the target path planning algorithm, so that when the target AGV performs the task corresponding to the job demand based on the target AGV path, the task execution time can be reduced while meeting the job demand.

[0081] In this embodiment, by obtaining the operation requirements and AGV information of all warehouses, wherein the operation requirements include the starting position, the end position, the positions of all first-level batching areas and the first-level cache area, the positions of all second-level batching areas and the second-level cache area, and the operation process, the AGV information includes the loading status of the AGV and the position of the AGV; and the target AGV is determined based on the starting position and all AGV information; by screening the AGV, the resources for scheduling AGVs between multiple warehouses are saved. And based on the operation requirements and the position of the target AGV, an operation time function is constructed; based on the operation time function, the reward function of the target path planning algorithm is determined; based on the target path planning algorithm, the AGV path is planned to obtain the target AGV path. When the target AGV performs the task corresponding to the operation requirements based on the target AGV path, the time for task execution can be reduced on the premise of meeting the operation requirements, the restrictions on AGV scheduling between warehouses are lifted, and the adaptability to AGV scheduling between multiple warehouses is improved.

[0082] In one embodiment, a work time function is constructed based on work requirements and the location of the target AGV, including:

[0083] Determine a target primary batching area position and a target primary cache area position among all primary batching area positions and all primary cache area positions, and a target secondary batching area position and a target secondary cache area position among all secondary batching area positions and all secondary cache area positions based on the operation process;

[0084] Determine a waiting time function based on a target primary batching area position and a target primary buffer area position, and a target secondary batching area position and a target secondary buffer area position;

[0085] Determine the transportation time function based on the starting position, the end position, the target primary batching area position and the target primary buffer area position, the target secondary batching area position and the target secondary buffer area position, the target AGV position and the operation process;

[0086] The sum of the waiting time function and the transportation time function is determined as the operation time function.

[0087] It should be noted that the operation process in the operation requirements includes the relevant operation processes that the AGV needs to perform to transport materials in the current operation requirements, including the designated locations that need to be reached and the order of arrival at each designated location. Each warehouse includes multiple first-level batching area locations, multiple first-level cache area locations, multiple second-level batching area locations, and multiple second-level cache area locations; the multiple first-level batching area locations and second-level batching area locations can also be distributed in multiple different warehouses. Based on the operation process, the target first-level batching area location is screened out from all the first-level batching area locations; the target first-level cache area location is screened out from all the first-level cache area locations; the target second-level batching area location is screened out from all the second-level cache area locations; and the target second-level cache area location is screened out from all the second-level cache area locations.

[0088] It should be noted that after the AGV transports the materials to the batching area, the AGV needs to wait for the materials to be sorted by the distribution tray; and when the AGV transports the materials to the buffer area, it needs to wait in the buffer area so that when the batching area is idle, the materials in the buffer area can be transported to the batching area to perform the distribution tray sorting activity. That is, during the task execution process of the AGV, the time the AGV waits in the batching area and the buffer area will affect the overall task completion time of the AGV. In this embodiment, the waiting time function is determined based on the target first-level batching area position and the target first-level buffer area position, as well as the target second-level batching area position and the target second-level buffer area position, and the waiting time of the AGV in executing the task is included in the scheduling considerations. It can be understood that when the target first-level batching area position or the target first-level buffer area position or the target second-level batching area position or the target second-level buffer area position is not specified in the operation process, the waiting time parameter value corresponding to the position not specified by the operation process in the waiting time function is 0.

[0089] When the AGV is performing a task, the time spent in the transportation process will also affect the overall task completion time of the AGV. In this embodiment, based on the starting position, the end position, the target first-level batching area position and the target first-level cache area position, the target second-level batching area position and the target second-level cache area position, and the position of the target AGV, the transportation time spent by the target AGV in the process of performing the task from the position of the target AGV to the starting position, the end position, the target first-level batching area position, the target first-level cache area position, the target second-level batching area position and the target second-level cache area position in sequence is determined to construct a transportation time function.

[0090] The sum of the waiting time function and the transportation time function is the operation time function. For details, please refer to the following formula:

[0091]

[0092] in, T 总Indicates the total operation time of AGV to perform a single task; Indicated in m The waiting time at m ∈{target first-level batching area position, target first-level cache area position, target second-level batching area position, target second-level cache area position}; t n Indicated in n The transit time in the subroutes, n ∈{target AGV position, starting position, end position, target primary batching area position, target primary cache area position, target secondary batching area position, and the path between any two positions in the target secondary cache area position}.

[0093] In this embodiment, the operation time function is constructed by considering the waiting time and transportation time of the AGV, thereby reducing the time spent by the AGV in executing a single task and improving the scheduling efficiency of the AGV.

[0094] In one embodiment, the AGV path planning method further includes:

[0095] Optimize the operation time function based on preset constraints, where the preset constraints include AGV loading threshold, AGV transportation range, AGV transportation distance minimum, buffer area capacity threshold, and operation area capacity threshold;

[0096] The reward function of the target path planning algorithm is determined based on the operation time function, including:

[0097] The reward function of the target path planning algorithm is determined based on the optimized operation time function.

[0098] In this embodiment, it should be noted that the preset constraints include the AGV loading threshold, the AGV transportation range, the minimum AGV transportation distance, the buffer area capacity threshold, and the operation area capacity threshold, specifically:

[0099]

[0100] in, Indicates the AGV loading status at time t, and the AGV loading thresholds include 0 and 1. When it is 0, it means that the AGV loading state is empty at time t. When it is 1, it means that the AGV loading state at time t is the loading state.

[0101]

[0102] in, d Indicates the transport distance of the AGV at time t, and the minimum value of the AGV transport distance is 0. That is, the transport distance of the AGV must be a positive number.

[0103]

[0104]

[0105] in, represents the coordinates of the i-th AGV at time t, and I represents the number of AGVs; xmin, xmax, ymin, and ymax are used to limit the coordinate values ​​of the AGV, that is, to represent the restrictions on the transportation range of the AGV.

[0106]

[0107]

[0108] in, Indicates the cache capacity at time t. The cache capacity thresholds include 0 and M. M indicates the maximum capacity of the cache. Indicates the capacity of the operating area at time t. The operating area capacity thresholds include 0 and N; N represents the maximum capacity of the cache area.

[0109] It should be noted that the operation time function is optimized based on the preset constraint conditions so as to minimize the operation time function while satisfying the constraint conditions.

[0110] It should be noted that the reward function of the target path planning algorithm is determined based on the optimized operation time function, that is, when calculating the reward function, if the operation time parameter is considered, the above-mentioned preset constraints also need to be considered.

[0111] In this embodiment, the operation time function is constrained based on preset constraints to ensure the adaptability of the operation time function to actual application scenarios and improve the effectiveness of the operation time function.

[0112] In one embodiment, determining a reward function of a target path planning algorithm based on an operation time function includes:

[0113] Determine the penalty value based on the preset loss factor and the operation time function;

[0114] The reward function of the target path planning algorithm is constructed based on the difference between the preset reward value and the penalty value.

[0115] It should be noted that the preset loss factors include the penalty factor of transportation time, the time loss factor caused by potential collision risk, and the penalty factor of waiting time. The penalty object is determined based on the operation time function, including waiting time and transportation time. Specifically, the calculation formula of the reward function is:

[0116]

[0117] in, R represents the reward function value; j represents the jth task; J represents the total number of tasks executed; α represents the reward obtained by the AGV when it completes a complete transportation task, and α is equal to the preset reward value; β represents the penalty factor for transportation time; t n represents the transportation time in the n subpaths; γ represents the time loss factor caused by potential collision risk; δ represents the penalty factor for waiting time; ω m represents the waiting time at m.

[0118] In this embodiment, by setting the reward function taking into account the time that the AGV may need to spend to perform the transportation task, the target AGV path is constructed with the goal of minimizing the total task execution time, thereby improving the effectiveness of the target AGV path.

[0119] In one embodiment, the steps of constructing the target path planning algorithm include:

[0120] Take each AGV as an agent, establish the environment set, action set, reward function, MADDPG network architecture and the state space of each agent, where the state space is composed of the state of the agent at each moment, including the loading state of the AGV and all preset AGV positions, the relative position of the AGV and the working area, the current capacity of the buffer area and the current capacity of the working area. The environment set includes the working requirements and the AGV information of all AGVs. The action set includes moving in the preset direction, stopping, loading and unloading. The reward function is determined based on the working time function.

[0121] Determine the target action based on the preset strategy, state space, environment set and action set, and obtain the new state of the agent after executing the target action;

[0122] Determine the reward for the new state of the agent based on the reward function, and obtain the state transition sequence of each agent;

[0123] Store the state transition sequence of all agents at each moment into the experience pool;

[0124] Random sampling is performed in the experience pool;

[0125] Update the local network parameters in the MADDPG network architecture based on the back-propagation algorithm;

[0126] And update the target network parameters in the MADDPG network architecture based on the updated local network parameters and the soft update method;

[0127] The MADDPG network architecture is iteratively optimized and trained based on the preset number of iterations to obtain the target path planning algorithm.

[0128] In this embodiment, it should be noted that the MADDPG network architecture is the basic architecture of the MADDPG (Multi-Agent DeepDeterministic Policy Gradient) algorithm. In this embodiment, each AGV is regarded as an independent agent. And define the environment set, action set, reward function and state space of each agent; wherein the state space is composed of the state of the agent at each moment, the state space includes the loading state of the AGV and all preset AGV positions, the relative position of the AGV and the working area, the current capacity of the buffer area and the current capacity of the working area, the environment set includes the working requirements, obstacle information and AGV information of all AGVs, the action set includes moving in the preset direction, stopping, loading and unloading, and the reward function is The MADDPG network architecture includes an Actor network and a Critic network. The Actor network is responsible for outputting continuous actions based on the current state; all agents share or have their own Critic network, which is used to evaluate the quality of actions, that is, to calculate the Q value. The key to MADDPG is that the Critic network can consider the strategies and behaviors of other agents when evaluating the Q value. Build a corresponding target network for each Actor and Critic network.

[0129] The experience replay mechanism is used to collect the experience of each agent and store it in the experience pool. The experience of the agent includes the state transition sequence of all agents at each moment, and the state transition sequence is (state, target action, reward, new state of the agent). A batch of samples are randomly selected from the experience pool for network training to increase the validity and stability of the data.

[0130] For each agent, use its critic network to evaluate these actions, considering the impact of other agents: the input of the critic network includes the states and actions of other agents. And update the network parameters. The target network is a stable version of the local network, which is used to provide a relatively fixed target during training to reduce fluctuations in the learning process. Both the actor network and the critic network have their own local network and target network.

[0131] For local network parameter update: In each iteration, the local network parameters of the Actor network and the Critic network are updated using the back-propagation algorithm based on the samples drawn from the experience pool. This is usually achieved by minimizing the loss function (for the Critic network) and maximizing the expected return (for the Actor network, through gradient ascent).

[0132] For target network update: After the local network parameters are updated, the target network parameters are not directly replaced by the local network parameters, but gradually approach the local network parameters through a soft update strategy. The soft update strategy usually uses a small update rate, such as 0.01, to weight the local network parameters and the target network parameters to maintain the stability of the target network. Assume that at a certain time step, the local network parameters of the Actor network are θ actor，eval , the target network parameter is θ actor，targ ; The local network parameter of the Critic network is θ critic，eval , the target network parameter is θ critic，targ In the update process: First, use the back-propagation algorithm to update θ actor，eval and θ critic，eval . Then, the target network parameters are updated using the soft update strategy:

[0133] θ actor，targ ←τθ actor，eval +(1−τ)θ actor，targ

[0134] θ critic，targ ←τθ critic，eval +(1−τ)θ critic，targ

[0135] where τ is the update rate, which is usually a small positive number.

[0136] It should be noted that when updating the target network parameters, the target network parameters of the Actor network and the Critic network are updated simultaneously and independently, but they all follow the same soft update strategy. This mechanism helps stabilize the training process, reduce fluctuations during the learning process, and improve the overall performance of the algorithm.

[0137] In this embodiment, a target path planning algorithm is constructed by using the MADDPG algorithm to improve the effectiveness of the target AGV path output, thereby improving the operating efficiency of the AGV.

[0138] The present application also provides an electronic device, including:

[0139] a memory configured to store instructions;

[0140] The processor is configured to call instructions from the memory and implement the AGV path planning method as described in the above embodiment when executing the instructions.

[0141] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the AGV path planning method as described in the above embodiment.

[0142] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0143] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] In a typical configuration, an electronic device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0147] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0148] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by electronic devices. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0149] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0150] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An AGV path planning method, characterized in that: include: Obtaining the operation requirements and AGV information of all warehouses, wherein the operation requirements include the starting position, the end position, the positions of all primary batching areas and primary buffer areas, the positions of all secondary batching areas and secondary buffer areas, and the operation process, and the AGV information includes the loading status of the AGV and the position of the AGV, the primary batching area and the primary buffer area are correspondingly set in the same warehouse, the secondary batching area and the secondary buffer area are correspondingly set in the same warehouse, and the primary batching area and the secondary batching area are set in different warehouses; Determine a target AGV based on the starting point position and all the AGV information; Constructing an operation time function based on the operation requirements and the position of the target AGV; Determining a reward function of a target path planning algorithm based on the operation time function; Perform AGV path planning based on the target path planning algorithm to obtain the target AGV path; Wherein, constructing the operation time function based on the operation requirement and the position of the target AGV includes: Determine, based on the operation flow, a target primary batching area position and a target primary cache area position among all the primary batching area positions and all the primary cache area positions, and a target secondary batching area position and a target secondary cache area position among all the secondary batching area positions and all the secondary cache area positions; Determine a waiting time function based on the target primary batching area position and the target primary cache area position, and the target secondary batching area position and the target secondary cache area position; Determine a transportation time function based on the starting position, the end position, the target primary batching area position and the target primary buffer area position, the target secondary batching area position and the target secondary buffer area position, the target AGV position, and the operation process; The sum of the waiting time function and the transportation time function is determined as the operation time function.

2. The AGV path planning method according to claim 1, characterized in that: Also includes: Optimizing the operation time function based on preset constraints, wherein the preset constraints include an AGV loading threshold, an AGV transport range, a minimum AGV transport distance, a buffer area capacity threshold, and an operation area capacity threshold; The reward function of the target path planning algorithm is determined based on the operation time function, including: The reward function of the target path planning algorithm is determined based on the optimized operation time function.

3. The AGV path planning method according to claim 1, characterized in that: The reward function of the target path planning algorithm is determined based on the operation time function, including: Determining a penalty value based on a preset loss factor and the operation time function; A reward function of the target path planning algorithm is constructed based on the difference between the preset reward value and the penalty value.

4. The AGV path planning method according to claim 1, characterized in that: The steps of constructing the target path planning algorithm include: Taking each AGV as an agent, establish an environment set, an action set, a reward function, a MADDPG network architecture and a state space of each agent, wherein the state space is composed of the state of the agent at each moment, the state space includes the loading state of the AGV and all preset AGV positions, the relative position of the AGV and the working area, the current capacity of the buffer area and the current capacity of the working area, the environment set includes the working requirements, obstacle information and AGV information of all AGVs, the action set includes moving in a preset direction, stopping, loading and unloading, and the reward function is determined based on the working time function; Determine a target action based on a preset strategy, the state space, the environment set, and the action set, and obtain a new state of the agent after executing the target action; Determine the reward obtained by the new state of the agent based on the reward function, and obtain the state transition sequence of each of the agents; The state transition sequences of all the agents obtained at each moment are stored in the experience pool; Performing random sampling within the experience pool; Updating local network parameters in the MADDPG network architecture based on a back-propagation algorithm; And updating the target network parameters in the MADDPG network architecture based on the updated local network parameters and the soft update method; The MADDPG network architecture is iteratively optimized and trained based on a preset number of iterations to obtain a target path planning algorithm.

5. The AGV path planning method according to claim 1, characterized in that: The determining of the target AGV based on the starting point position and all the AGV information includes: Determining an unloaded AGV based on the loading status of all of the AGVs; Based on the starting position and the positions of all the unloaded AGVs, the unloaded AGV closest to the starting position among all the unloaded AGVs is determined as the target AGV.

6. The AGV path planning method according to claim 5, characterized in that: The step of determining, based on the starting position and the AGV positions of all the unloaded AGVs, the unloaded AGV closest to the starting position among all the unloaded AGVs as the target AGV comprises: Determine the Manhattan distance between the starting point and the position of each of the unloaded AGVs respectively; The empty AGV with the smallest Manhattan distance among all empty AGVs is determined as the target AGV.

7. The AGV path planning method according to claim 1, characterized in that: Also includes: Rasterize all areas of all operation warehouses to obtain a grid space; Establishing a Cartesian coordinate system based on the grid space; The starting position, the end position, all the first-level dispensing area positions, all the first-level buffer area positions, all the second-level dispensing area positions, all the second-level buffer area positions and the position of the AGV are determined based on the Cartesian coordinate system.

8. An electronic device, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the AGV path planning method according to any one of claims 1 to 7 when executing the instructions.

9. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for enabling a machine to execute the AGV path planning method according to any one of claims 1 to 7.

Images