Warehouse order batch picking path planning method based on multi-agent cooperation

Abstract

The application discloses a warehouse order batch picking path planning method based on multi-agent cooperation and relates to the technical field of warehouse logistics. The method comprises the following steps: acquiring a plurality of picking order sets to be processed in a current wave and a warehouse layout topology graph; calling a pre-trained order-bin correlation model to perform commodity correlation analysis processing on the picking order sets, and generating an inter-order correlation matrix; constructing a multi-agent cooperative task allocation network based on the inter-order correlation matrix, and allocating each picking order to a corresponding agent; generating an initial picking sequence of each agent through the multi-agent cooperative task allocation network, and performing multi-agent path conflict detection processing on the initial picking sequence to obtain a cooperative obstacle avoidance path set; and inputting the cooperative obstacle avoidance path set into a pre-trained reinforcement learning cooperative strategy model for joint path optimization processing to generate a globally optimal batch picking path planning scheme.

Description

Technical Field

[0001] This invention relates to the field of warehousing and logistics technology, specifically to a method for planning batch picking routes for warehousing orders based on multi-agent collaboration. Background Technology

[0002] In large-scale warehousing and logistics centers, wave picking operations need to process a large number of customer orders simultaneously. Existing batch picking path planning methods typically combine multiple orders into a fixed batch, and then a single picker or machine sequentially traverses the storage locations to complete the picking. This centralized serial processing mode treats order batching and path planning as two independent stages. It first uses a basic clustering algorithm to statically group orders once, and then solves an independent traveling salesman problem for each group of orders to determine the walking path. This approach ignores the inherent correlation between different orders in terms of product storage location and SKU requirements. When the order set contains heterogeneous products distributed in different storage areas, the generated picking paths will have a large number of back-and-forth trips and intersections, significantly increasing the walking distance. Existing technical solutions have obvious drawbacks. Based on the static batching and single-machine path planning mode, the order grouping results depend entirely on preset clustering parameters, and the picking process of each batch after grouping is isolated from each other, making it impossible to detect possible multi-device intersections and congestion in the aisles. As the number of mobile robots or personnel operating in parallel within the warehouse increases, conflicts frequently erupt between their independently planned paths. The system is forced to take passive avoidance measures such as emergency stops or prolonged waiting times, causing the planned paths to deviate from expectations during actual execution and resulting in a sharp decline in overall picking efficiency. Furthermore, the later path execution phase lacks the ability to dynamically optimize the strategic interactions between multiple entities. Faced with a highly dynamic operating environment, it cannot coordinate the movements of each entity from a global perspective, only achieving locally feasible suboptimal solutions.

[0003] The problem this invention aims to solve is how to mine and discover the correlation between orders in terms of product location and SKU composition, replacing static batch division, and how to achieve global collaborative dynamic path optimization in a multi-entity environment with path conflicts, and output the optimal picking path planning scheme. Summary of the Invention

[0004] The purpose of this invention is to propose a warehouse order batch picking route planning method based on multi-agent collaboration, in order to solve the problems of existing technologies lacking intelligent task allocation based on the correlation between product location and SKU between orders, and lacking global collaborative dynamic adjustment capability in multi-executor route planning during batch order processing.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for batch picking route planning of warehouse orders based on multi-agent collaboration includes:

[0007] Obtain the set of multiple picking orders to be processed within the current wave and the warehouse layout topology;

[0008] An order-location correlation model is established and pre-trained. The pre-trained model is then used to analyze the product correlation of the picking order set, generating an inter-order correlation matrix. Based on this matrix, a multi-agent collaborative task allocation network is constructed to assign each picking order to a corresponding agent. Each agent is equipped with a local path planner. An initial picking sequence for each agent is generated through the multi-agent collaborative task allocation network, and multi-agent path conflict detection is performed on this sequence to obtain a collaborative obstacle avoidance path set. A reinforcement learning collaborative strategy model is established and pre-trained. The collaborative obstacle avoidance path set is input into the pre-trained model for joint path optimization, generating a globally optimal batch picking path planning scheme.

[0009] By weighted fusion of spatial distance between orders and product overlap, the correlation between picking tasks is accurately quantified, providing a reliable correlation matrix for task partitioning of the agent. As a technical solution of this invention, when calling the pre-trained order-location correlation model, location mapping is performed on the product SKUs in each picking order to obtain the location coordinate set of the corresponding order; the mean Euclidean distance between the location coordinate sets of any two picking orders is calculated as the initial spatial correlation; the number of product SKUs commonly included in any two picking orders is obtained, and the intersection ratio of this product SKU number with the total number of SKUs in each order is calculated as the product overlap; the initial spatial correlation and the product overlap are weighted and fused to generate the order correlation matrix. Preferably, the weight coefficients in the weighted fusion process are determined by entropy weighting or principal component analysis, so that the correlation measurement reflects both the physical proximity of locations and the overlapping characteristics of product picking, improving the rationality of subsequent task clustering.

[0010] When constructing a multi-agent collaborative task allocation network based on the order correlation matrix, each row vector in the order correlation matrix is ​​extracted as the correlation distribution feature of the current order. This correlation distribution feature is then input into a pre-trained task clustering model for order grouping, generating multiple order subsets. Each order subset is assigned an independent agent, and the multi-agent collaborative task allocation network is established. This automatically aggregates orders with high correlation into the same agent's task pool, making the picking area of ​​each agent relatively concentrated, reducing long-distance cross-regional movement, and overall reducing the probability of path overlap between multiple agents.

[0011] In multi-agent path conflict detection and processing, the local path planner corresponding to each agent is used to calculate the shortest path based on the location coordinates in the order subset assigned to that agent, generating an initial picking sequence. The initial picking sequences of each agent are then time-axis aligned to obtain the spatiotemporal occupancy trajectory of each agent in the warehouse layout. The system detects whether there are overlapping location coordinates of different agents at the same time point in the spatiotemporal occupancy trajectory; if so, these are marked as path conflict nodes. For time-axis alignment, a preferred implementation is to obtain the walking distance and theoretical walking speed between adjacent locations in each agent's initial picking sequence, calculate the theoretical timestamp of reaching each location, and bind the theoretical timestamp with the corresponding location coordinates to generate the spatiotemporal occupancy trajectory. This processing can accurately reconstruct the occupancy of each agent in the time and space dimensions, making conflict detection more accurate and reliable. For the path conflict nodes, priority avoidance strategies or speed adjustment strategies can be used to resolve the conflict, generating conflict-free cooperative obstacle avoidance paths, thereby eliminating the collision risk between multiple agents without affecting the picking task completion time.

[0012] When inputting the set of cooperative obstacle avoidance paths into a pre-trained reinforcement learning cooperative policy model for joint path optimization, the path of each agent in the set is used as the initial state of the model. The initial state is evaluated using the value network of the model to generate a state value vector. Based on this state value vector, a policy gradient algorithm is used to iteratively adjust the path nodes in the set, outputting the globally optimal batch picking path planning scheme. In this process, the value network concatenates the position information, remaining task quantity, and path length of each agent into a joint state feature vector, which is then nonlinearly mapped through multiple fully connected layers to output a joint state value scalar, thereby comprehensively evaluating the global merits of the current cooperative path scheme. During iterative adjustment, for each path node in the set, the action value corresponding to all candidate transition directions in its action space is calculated. The candidate transition direction with the highest action value is selected as the update direction for that path node, generating an iterative path node. This iterative path node replaces the original path nodes in the set, resulting in an updated path set. By employing the aforementioned reinforcement learning collaborative strategy, joint trade-offs can be made among multiple agents, thereby further reducing the total walking distance while eliminating conflicts, optimizing the overall picking efficiency of the warehouse, and ultimately obtaining a batch picking solution that balances task completion time and path smoothness.

[0013] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0014] By invoking a pre-trained order-location correlation model, the product correlation of the picking order set is analyzed, generating an order correlation matrix. This matrix serves as the foundation for constructing a multi-agent collaborative task allocation network, replacing traditional static batch partitioning. In this process, the correlation between any two picking orders is determined by the spatial proximity between location coordinates and the overlap ratio of common product SKUs, achieving a joint representation of orders in both spatial and product demand dimensions. Agent task allocation based on this correlation matrix enables the order set handled by the same agent to exhibit clustering in the warehouse physical layout while also possessing collaborative product picking capabilities. This allocation method directly reduces the number of ineffective round trips between different storage areas for the executing agent from the task source, compressing the total walking distance of a single task cycle and improving picking density and operational continuity. After generating the initial picking sequences for each agent, they are time-axis aligned to obtain spatiotemporal occupancy trajectories, and multi-agent path conflict detection is implemented. The resulting set of collaborative obstacle avoidance paths is used as the initial state input for the reinforcement learning collaborative strategy model. The value network of this reinforcement learning collaborative policy model evaluates the value of the joint state, which includes the location information of all agents, the remaining task volume, and the path length. Based on the evaluation results, a policy gradient algorithm is used to jointly iteratively adjust the path nodes of all agents. This mechanism is no longer limited to independently optimizing the path segments of a single agent, nor does it rely on preset traffic rules to passively resolve conflicts. Instead, it continuously searches and adjusts the action coordination methods among multiple agents in a high-dimensional joint action space, resolving temporal and spatial intersection conflicts, and finally outputs a globally optimal batch picking path planning scheme under the condition of mutual constraints among multiple agents. Attached Figure Description

[0015] Figure 1 This is a flowchart of a warehouse order batch picking route planning method based on multi-agent collaboration;

[0016] Figure 2 This is a schematic diagram illustrating the process of generating the correlation matrix between orders;

[0017] Figure 3 This is a probability density distribution diagram of spatial correlation between orders and product overlap;

[0018] Figure 4 It is a principal component analysis cluster diagram of the order correlation distribution characteristics;

[0019] Figure 5 This is a diagram showing the path iteration optimization results of the reinforcement learning collaborative strategy model. Detailed Implementation

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings and examples.

[0021] See Figure 1 The present invention provides a method for batch picking path planning of warehouse orders based on multi-agent collaboration, comprising:

[0022] Obtain the set of multiple picking orders to be processed within the current wave and the warehouse layout topology. Call the pre-trained order-location correlation model to perform product correlation analysis on the picking order set, generating an inter-order correlation matrix. Based on the inter-order correlation matrix, construct a multi-agent collaborative task allocation network to assign each picking order to its corresponding agent. Generate an initial picking sequence for each agent through the multi-agent collaborative task allocation network, and perform multi-agent path conflict detection on the initial picking sequence to obtain a set of collaborative obstacle avoidance paths. Input the set of collaborative obstacle avoidance paths into a pre-trained reinforcement learning collaborative strategy model for joint path optimization, generating a globally optimal batch picking path planning scheme.

[0023] Example 1:

[0024] In specific implementation, please refer to Figure 2 The order-location correlation model comprises four parts: location mapping, spatial correlation calculation, product overlap calculation, and weighted fusion. The implementation process of these four parts is described in detail below. First, location mapping is performed on the product SKUs in each picking order to obtain a set of location coordinates corresponding to each picking order. A one-to-one correspondence mapping table between product SKUs and warehouse locations is pre-constructed. Warehouse locations are represented by three-dimensional coordinates, where the first two dimensions represent the row and column numbers of the shelves in the warehouse plane, and the third dimension represents the shelf layer number. For each picking order, all product SKU identifiers contained in the picking order are extracted. The one-to-one correspondence mapping table between product SKUs and warehouse locations is queried one by one to obtain the location coordinates corresponding to each product SKU in the warehouse layout topology diagram. The location coordinates corresponding to all product SKUs in a single picking order are then aggregated into a single set of location coordinates. In the location coordinate set of a single picking order, the total number of location coordinates is equal to the total number of product SKUs included in the picking order. When multiple product SKUs in a picking order correspond to the same location coordinate, the duplicate location coordinates are retained in the location coordinate set.

[0025] Secondly, the mean Euclidean distance between the location coordinate sets of any two picking orders is calculated as the initial spatial correlation. Among all picking orders, the first picking order and the second picking order are selected sequentially. The location coordinate set of the first picking order is represented as follows: The set of location coordinates for the second picking order is represented as follows: superscript and superscript These are the index numbers of the picking orders. and Take different values. The number of cargo location coordinates included is denoted as , The number of cargo location coordinates included is denoted as .Will The Middle Coordinates of each storage location and The Middle The Euclidean distance between the coordinates of each storage location is marked as follows: subscript The value is to Integers between [a certain range], subscripts The value is to Integers between Euclidean distances. The calculation method is as follows: The Middle The three-dimensional components of the coordinates of each storage location and The Middle For each cargo location coordinate, calculate the difference between the three-dimensional components, square them, sum the squares of the three-dimensional components, and take the square root. For all... and Combination Find the arithmetic mean, which is the sum of all... After summing, divide by The resulting quotients are used as the arithmetic mean as the initial spatial correlation between the first and second picking orders. All picking order pairs are iterated over, and the initial spatial correlation is calculated for each pair.

[0026] Next, obtain the number of common SKUs in any two picking orders, and calculate the intersection ratio of the common SKU count with the total SKU count of each order. This ratio is used as the product overlap. For the first and second picking orders, extract the SKU set of the first picking order and the SKU set of the second picking order, and calculate the number of SKUs contained in the intersection of the two SKU sets. Divide the intersection number of SKUs by the sum of the cardinality of the SKU sets of the first and second picking orders, and then subtract the intersection number of SKUs. The resulting ratio is the product overlap between the first and second picking orders. This process is repeated for all picking order pairs, calculating the product overlap for each pair.

[0027] Finally, the initial spatial correlation degree and product overlap degree are weighted and fused to generate an order correlation degree matrix. The number of rows and columns in the order correlation degree matrix are equal to the total number of picking orders in the picking order set. The order correlation degree matrix is ​​then used to calculate the correlation degree between orders. Line 1 The elements of the column correspond to the first The picking order and the first The correlation value between picking orders, where and All are picking order index numbers. The weighted fusion processing is implemented by: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] The picking order and the first The initial spatial correlation between each picking order is multiplied by the first weighting coefficient to obtain the first weighting term; the first weighting term is then calculated by multiplying the initial spatial correlation between each picking order by the first weighting coefficient. The picking order and the first The product overlap between each picking order is multiplied by the second weighting coefficient to obtain the second weighting term; the first weighting term is added to the second weighting term, and the sum is used as the first weighting term in the order correlation matrix. Line 1 The element values ​​of the column. The expression for the weighted merging process is:

[0028]

[0029] in, The order correlation matrix represents the first order in the matrix. Line 1 The element of the column, i.e., the first The picking order and the first The correlation value between individual picking orders; This represents the first weighting coefficient. This represents the second weighting coefficient; Indicates the first The picking order and the first Initial spatial correlation between picking orders; Indicates the first The picking order and the first Product overlap between picking orders. First weighting coefficient. Second weighting coefficient All are non-negative real numbers, and and The sum of equals .

[0030] In some embodiments, the first weighting coefficient Second weighting coefficient The entropy weight method is used to determine the first weight coefficient. Second weighting coefficient The steps are as follows: First, construct an evaluation matrix consisting of the initial spatial correlation and product overlap of all picking order pairs. Each row of the evaluation matrix corresponds to a different picking order pair. The first column of the evaluation matrix stores the initial spatial correlation of each picking order pair, and the second column stores the product overlap of each picking order pair. Then, standardize the data in each column of the evaluation matrix using Min-Max standardization, linearly mapping each column of data to an interval. ; Calculate the proportion of each value in each column of data to the sum of all values ​​in that column after standardization, to obtain a proportion matrix; Calculate the information entropy corresponding to each column of data based on the proportion matrix. The information entropy is calculated as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Each weight value in the column is multiplied by its natural logarithm and then summed. The sum is then multiplied by... , This indicates the number of rows in the evaluation matrix; the difference coefficient for each column of data is calculated based on the information entropy, and the difference coefficient equals... Subtract the information entropy; divide the difference coefficient of each column by the sum of the difference coefficients of all columns to obtain the weight coefficient for each column. The normalized value of the difference coefficient in the first column of the evaluation matrix is ​​used as the first weight coefficient. The normalized values ​​of the difference coefficients in the second column of the evaluation matrix are used as the second weighting coefficients. .

[0031] In other embodiments, the first weighting coefficient Second weighting coefficient The first weight coefficients were determined using principal component analysis. Second weighting coefficient The steps are as follows: Construct a variable matrix consisting of the initial spatial correlation and product overlap of all picking order pairs. Rows in the variable matrix correspond to different picking order pairs, and the two columns correspond to the initial spatial correlation variable and the product overlap variable, respectively. Center the variable matrix by subtracting the mean of each column's data. Calculate the covariance matrix of the centered variable matrix. Solve for the eigenvalues ​​and eigenvectors of the covariance matrix, arranging the eigenvalues ​​in descending order. Select principal components whose cumulative variance contribution rate exceeds a predetermined threshold (set to 85%). Using the proportion of each selected principal component's variance contribution rate to the cumulative variance contribution rate of all selected principal components as weights, sum the loading coefficients corresponding to the initial spatial correlation variable in each principal component to obtain the comprehensive score coefficient of the initial spatial correlation variable. Similarly, sum the loading coefficients corresponding to the product overlap variable in the same way to obtain the comprehensive score coefficient of the product overlap variable. Normalize the comprehensive score coefficients of the initial spatial correlation variable and the comprehensive score coefficients of the product overlap variable so that their sum is... The normalized initial spatial correlation coefficient is used as the first weighting coefficient. The normalized composite score coefficient of the product overlap variable is used as the second weighting coefficient. .

[0032] See Figure 3 In the figure, the horizontal axis represents numerical values ​​(unit: meters), the left vertical axis represents the probability density, and the right vertical axis represents the product overlap value corresponding to the probability density. The figure contains three curves: the initial spatial correlation curve (blue solid line), the order correlation curve (orange dashed line), and the product overlap curve (green dashed line), which correspond to the correlation matrix construction process of the picking order set in Example 1.

[0033] The initial spatial correlation curve exhibits a unimodal distribution within the horizontal axis range of approximately 0 to 50 meters, with the peak value located at approximately 5 meters, corresponding to a probability density of approximately 0.078. This indicates that the average Euclidean distance of the location coordinate sets of most picking order pairs is concentrated in a relatively small interval, reflecting spatial clustering characteristics. The curve rises sharply to the left of the peak and declines gently to the right, indicating a clear concentration trend in the initial spatial correlation between orders, with the tail extending to approximately 50 meters.

[0034] The peak of the order correlation curve is slightly higher than the initial spatial correlation peak, located at approximately 4 meters, with a probability density of about 0.13. This indicates that after weighted fusion of the initial spatial correlation and product overlap, the order correlation is more concentrated and the peak value is increased. The overall shape of the curve is similar to the initial spatial correlation, but the density is higher in the low-distance range, reflecting the enhanced order correlation after fusion of product overlap.

[0035] The product overlap curve is concentrated in a very small range, with a peak around 0 and a probability density of nearly 30, indicating that most orders have relatively few overlapping product SKUs, and the overlap distribution is extremely right-skewed. This characteristic is reflected in the calculation of the correlation between orders, where the contribution of product overlap to the overall correlation is concentrated in the low-value range.

[0036] Example 2:

[0037] In practice, each row vector is extracted from the order correlation matrix as the correlation distribution feature of the current order. The order correlation matrix is ​​a square matrix, with both the number of rows and columns equal to the total number of picking orders in the picking order set. The total number of picking orders is denoted as... The order correlation matrix is ​​as follows: Line 1 The elements of the column represent the first... The picking order and the first The correlation value between picking orders, where and All are picking order index numbers. The range of values ​​is to integers, The range of values ​​is to Integers. For the th For each picking order, extract the order correlation matrix. The row contains all Each element, will The elements are arranged in ascending order of column index, forming a dimension of... The row vector is the row vector of the first row. The correlation distribution characteristics of the picking order. The correlation distribution characteristics describe the correlation distribution characteristics of the first picking order. The distribution of correlation between a picking order and all picking orders in the picking order set. Perform the above extraction operation on each picking order to obtain... Each correlation degree distribution feature vector.

[0038] The correlation distribution features are input into a pre-trained task clustering model to group orders, generating multiple order subsets. The core architecture of the task clustering model is a density-based clustering model, specifically constructed using the DBSCAN clustering algorithm. The task clustering model consists of two parts: a density estimation module and a cluster partitioning module. The density estimation module receives the input correlation distribution feature vector and calculates the neighborhood density in the feature space for each correlation distribution feature vector. The neighborhood density is calculated by statistically analyzing the neighborhood density centered on the correlation distribution feature vector and its radius. The clustering module determines the number of other correlation distribution feature vectors contained within the hypersphere of radius. Based on the neighborhood density information output by the density estimation module, the clustering module marks correlation distribution feature vectors with neighborhood densities exceeding the minimum sample size threshold as core points. Core points satisfying density connectivity are grouped into the same cluster, while correlation distribution feature vectors not satisfying density connectivity are marked as noise points or grouped into different clusters. Picking orders corresponding to correlation distribution feature vectors within the same cluster are assigned to the same order subset, and picking orders corresponding to different clusters are assigned to different order subsets.

[0039] Neighborhood radius The value of is determined using the k-distance curve method, specifically by: [evaluating all...] Given a set of correlation distribution feature vectors, calculate the correlation distribution feature vectors of each feature vector to the remaining features vectors. The Euclidean distance between the eigenvectors of the correlation degree distribution is taken, and the nearest one is selected. Each distance value is used as the k-distance value of the feature vector of this association degree distribution. The value is set to ,in This represents the rounding up operation; sort the k-distance values ​​of all correlation distribution feature vectors in descending order, plot the k-distance curve, and take the distance value corresponding to the inflection point of the k-distance curve with the largest curvature change as the neighborhood radius. The value of is set to . The minimum sample size threshold is set to . The value plus ,Right now .

[0040] In some embodiments, the training process of the task clustering model includes setting the neighborhood radius. Parameter optimization is performed on the minimum sample size threshold. The parameter optimization method is as follows: construct a sample set of correlation distribution features generated from historical picking order data, and use the silhouette coefficient as the evaluation index for clustering effect; in the neighborhood radius... In the candidate value set, each parameter combination is traversed, and the DBSCAN clustering algorithm is run to obtain the clustering results. The silhouette coefficients corresponding to the clustering results are calculated. The parameter combination with the maximum silhouette coefficient is selected as the optimal parameter, and the optimal parameter is configured into the task clustering model to complete the training. Neighborhood radius The candidate value set is Within the interval The set of discrete values ​​generated for the step size, and the candidate set of values ​​for the minimum sample size threshold are: The set of integers within the interval.

[0041] The task clustering model receives all input data. There are 3 correlation degree distribution feature vectors, and the dimension of each correlation degree distribution feature vector is 1. The task clustering model outputs clustering results, which contain information about multiple clusters. Each cluster consists of a cluster label and a list of correlation distribution feature vector indices belonging to that cluster. The cluster label is used to distinguish different order subsets. Noise points in the output data that do not belong to any cluster are labeled separately, and the picking orders corresponding to each noise point form an independent single-element order subset.

[0042] Optionally, the pre-trained task clustering model is constructed using GaussianMixtureModel. GaussianMixtureModel is... It consists of Gaussian distribution components. The value of is determined by the BIC criterion. The range of values ​​is to Integers, where This indicates the floor function. Each Gaussian distribution component contains three parameters: a mean vector, a covariance matrix, and mixing coefficients. The dimension of the mean vector is the same as the dimension of the correlation distribution feature vector, and the covariance matrix is ​​of dimension [missing information]. A square matrix with non-negative real coefficients and all The sum of the mixing coefficients of the components equals The GaussianMixtureModel is trained using the EM algorithm to iteratively update parameters until the log-likelihood function converges or the maximum number of iterations is reached. The maximum number of iterations is set to... After training, the posterior probability of each correlation distribution feature vector under each Gaussian distribution component is calculated. The component number corresponding to the Gaussian distribution component with the highest posterior probability is used as the cluster label of the correlation distribution feature vector. Picking orders with the same cluster label are assigned to the same order subset.

[0043] Assign an independent agent to each order subset and establish a multi-agent collaborative task allocation network. Let the number of order subsets output by the task clustering model be... Then initialize There are 1 intelligent agents, and the intelligent agents are numbered as follows: Intelligent agents With the A one-to-one correspondence is established between the subsets of orders, where The range of values ​​is to Integer. The first The picking tasks for all picking orders contained in a subset of orders are assigned to the agent. implement.

[0044] The multi-agent collaborative task allocation network is established as follows: each agent is configured with a communication interface module. These modules establish bidirectional data connections with the communication interface modules of other agents via a warehouse LAN or wireless communication link, forming a fully connected communication topology. In this fully connected topology, a direct communication link exists between any two agents. With intelligent agents The communication link between them is used to transmit real-time location coordinates, the sequence of cargo locations to be completed, and information on occupied spatiotemporal trajectory segments, among which and Each agent is assigned an index number. The communication interface module uses a publish-subscribe messaging model. After receiving its own state update, each agent publishes its state message to other agents via the communication link, and simultaneously subscribes to receive state messages published by other agents. The state information of all agents in the multi-agent collaborative task allocation network is aggregated into a collaborative state database, which is deployed on a central computing node or distributed and stored in the local storage modules of each agent.

[0045] See Figure 4 In the figure, the horizontal axis represents the first principal component of the distribution characteristics of the correlation between orders, and the vertical axis represents the second principal component. The clusters marked with blue solid dots represent the correlation distribution feature vector of order subset 1, while the gray crosses represent noise points. The horizontal axis values ​​range from approximately -25 to 35, and the vertical axis values ​​range from approximately -20 to 15. The points in the blue order subset 1 are mainly concentrated in the range of -5 to 0 on the horizontal axis and 3 to 5 on the vertical axis, forming a tight cluster. Noise points are more widely distributed, with both horizontal and vertical axis values ​​showing greater dispersion, reflecting the DBSCAN clustering algorithm's ability to identify points with low neighborhood density.

[0046] Example 3:

[0047] In practice, the local path planner corresponding to each agent is invoked to calculate the shortest path based on the location coordinates in the order subset assigned to the agent, generating an initial picking sequence. For the first agent in the multi-agent collaborative task allocation network... The first intelligent agent, the... Each agent assigns an order subset containing one or more picking orders. The location coordinates of all SKUs within each picking order subset are extracted, deduplicated, and used to construct a target location coordinate set. This target location coordinate set contains... Different cargo location coordinates greater than or equal to The local path planner uses the A* search algorithm. The input to the A* search algorithm is the warehouse layout topology map, the coordinates of the starting node, and the set of coordinates of the target storage locations. The output is an ordered sequence of nodes that returns to the starting node after traversing all storage location coordinates in the target storage location coordinate set starting from the starting node. The ordered sequence of nodes is the initial picking sequence.

[0048] The A* search algorithm is implemented by maintaining a list of open nodes and a list of closed nodes. Initially, the open node list contains only the starting node. The starting node is selected from the warehouse layout topology map, choosing the coordinates of the storage location or sorting station closest to the geometric center of the target storage location coordinate set. In each iteration of the A* search algorithm, a cost evaluation function is selected from the open node list. The node with the smallest value is selected as the current expansion node, and it is moved from the list of open nodes to the list of closed nodes. Cost evaluation function. Actual path cost and heuristic cost estimation The sum is obtained. This indicates the destination node starting from the starting node. The actual path length traversed, the path length between nodes is the edge weight between adjacent nodes in the warehouse layout topology graph, and the edge weight in the warehouse layout topology graph is the Euclidean distance between the coordinates of two adjacent nodes. Indicates from node Heuristic cost estimation for departure to unvisited target storage location coordinates Get Node The minimum Manhattan distance from the coordinates to all unvisited target storage locations, multiplied by the heuristic weighting factor. , The value is When all target location coordinates have been added to the list of closed nodes and the current expanding node returns to the starting node, the A* search algorithm terminates, and the parent node pointer chain is backtracked to obtain the complete traversal sequence as the initial picking sequence.

[0049] In some embodiments, the local path planner employs a Lin-Kernighan heuristic algorithm with capacity constraints. The set of target warehouse location coordinates is used as the set of city nodes in the TSP problem, and the shortest path length between any two target warehouse location coordinates in the warehouse layout topology is used as the element of the distance matrix between the two cities. An iterative local search strategy is employed, starting from a random initial solution, and... The -opt edge-swapping operation continuously improves the total path length. Values or Continue until the total path length no longer decreases, then output the optimal traversal sequence as the initial selection sequence.

[0050] The initial picking sequences of each agent are time-aligned to obtain the spatiotemporal occupancy trajectory of each agent in the warehouse layout. The walking distance and theoretical walking speed between adjacent storage locations in the initial picking sequence of each agent are obtained, and the theoretical timestamp for reaching each storage location is calculated. For the... The initial picking sequence of each agent, the sequence contains Given a set of ordered nodes, mark the nodes sequentially in traversal order as nodes. .node The theoretical timestamp for reaching the starting node. Set to time zero. For the first time in the sequence... Nodes , The range of values ​​is to Integers, nodes Corresponding theoretical timestamp The calculation method is as follows: obtain node With nodes The walking distance between them is determined based on the edge weights of the warehouse layout topology graph. The walking distance is marked as... ; Get the The theoretical walking speed of each agent is preset according to the type of mobile device corresponding to the agent, and the theoretical walking speed is marked as... , The unit is meters per second. Node Theoretical timestamp From the previous node Theoretical timestamp The time spent walking is added to the walking time, which is the distance walked. Divide by theoretical walking speed The expression is:

[0051]

[0052] in, Indicates the first In the initial picking sequence of the i-th agent, the th... The theoretical timestamps corresponding to each node are in seconds; Indicates the first In the initial picking sequence of the i-th agent, the th... The theoretical timestamps corresponding to each node; Indicates the first In the initial picking sequence of the i-th agent, the th... The node and the first The walking distance between nodes, in meters; Indicates the first The theoretical walking speed of an intelligent agent, measured in meters per second. The value is determined based on the type of picking equipment simulated by the agent. When the picking equipment is a manual picking cart... Pick When the picking equipment is an automated guided vehicle (AGV). Pick . Node Cargo location coordinates and theoretical timestamp Bind and generate a spacetime occupancy record. ,in They are nodes Three-dimensional coordinate components in warehouse layout. The first... All of the agents The time-space occupancy records are sorted in ascending order by timestamp to obtain the [number]. The spatiotemporal occupancy trajectories of each agent are obtained by performing the above operations on each agent in the multi-agent cooperative task allocation network.

[0053] Optional, walking distance between adjacent nodes Additional distance for channel turns was considered. When the node... With nodes When the path between these points passes through a turning node in the warehouse layout topology map, an additional turning distance is added to the straight-line distance. The additional turning distance is taken as the width of the warehouse aisle. times.

[0054] The system detects whether there are overlapping cargo location coordinates of different agents at the same time point in the spatiotemporal occupancy trajectory. If so, these are marked as path conflict nodes. The detection process discretizes the time axis using a preset time step. Values Seconds. Starting from zero, with time steps... Incrementing at each sampling time Traverse the spatiotemporal occupancy trajectories of all agents, and determine the sampling time of each agent through linear interpolation. The coordinates of the location of the cargo. For the first... The first intelligent agent and the first An intelligent agent. and For different agent index numbers, if at the sampling time Next, the The coordinates of the cargo location of the first agent are related to the first... The Euclidean distance between the coordinates of the storage location where the agent is located and the safe distance threshold is less than the safe distance threshold. If this occurs, a cargo location coordinate overlap event is determined. Safety distance threshold. Take the minimum distance between adjacent storage locations in the warehouse. The sampling time at which the cargo location coordinates coincide will occur. The first The first intelligent agent and the first The identifier of each agent and the coordinates of the cargo location where the conflict occurred are used to mark a path conflict node. The above detection process is repeated until all sampling times have been traversed, generating a set of path conflict nodes.

[0055] Path conflict nodes are resolved using either a priority avoidance strategy or a speed adjustment strategy to generate conflict-free cooperative obstacle avoidance paths. The priority avoidance strategy works as follows: each agent is pre-assigned a unique priority number, determined by the urgency or creation time of the assigned order subset. At path conflict nodes, the original spatiotemporal occupancy trajectory of the agent with the highest priority number remains unchanged. For agents with lower priority numbers, a waiting node is inserted before the node corresponding to the conflicting cargo location coordinates in their spatiotemporal occupancy trajectory. The cargo location coordinates of the waiting node are the same as the cargo location coordinates of the previous node for that agent, and the theoretical timestamp of the waiting node is set to the theoretical timestamp of the previous node plus the waiting time. , The value is the time step. Multiples of integers, by gradually increasing the waiting time. Recalculate the theoretical timestamps of subsequent nodes until the Euclidean distance between the location coordinates of the current agent at the moment of conflict and the location coordinates of the higher-priority agent is greater than or equal to the safe distance threshold. Replace the original spatiotemporal occupancy trajectory with the adjusted one.

[0056] The speed adjustment strategy resolves the issue by: addressing the path conflict at the first involved node. The first agent or the first In the spatiotemporal occupancy trajectory of each agent, select one or more adjacent road segments upstream of the conflict node to reduce the theoretical walking speed. The value of . Speed ​​adjustment factor The range of values ​​is The original theoretical walking speed Multiply by speed adjustment factor The adjusted theoretical walking speed ,use Recalculate the theoretical timestamps of the conflict node and all subsequent nodes. Iterate by selecting different velocity adjustment factors. , making the first The first intelligent agent and the first The Euclidean distance between the location coordinates of each agent at the moment of conflict sampling is greater than or equal to the safe distance threshold. If a single speed adjustment cannot resolve the conflict, the speed adjustment strategy is applied simultaneously to multiple agents involved in the conflict, or a collaborative resolution is achieved by inserting waiting nodes. After resolution, a set of conflict-free collaborative obstacle avoidance paths is obtained, in which the spatiotemporal occupancy trajectory of each agent does not overlap with the cargo location coordinates of other agents at any sampling time.

[0057] Example 4:

[0058] In practical implementation, the path of each agent in the cooperative obstacle avoidance path set is used as the initial state of the reinforcement learning cooperative strategy model. The core architecture of the reinforcement learning cooperative strategy model consists of two parts: a value network and a policy optimization module. Besides the core architecture's value network and policy optimization module, the reinforcement learning cooperative strategy model also includes four non-core but necessary components: an environment interaction interface, a reward calculation module, a state encoder, and a model scheduling controller. The environment interaction interface is responsible for receiving the update directions of each agent from the policy optimization module in each iteration, converting them into actual replacement operations for path nodes in the warehouse layout topology diagram, and updating the current location coordinates, remaining unvisited location sequence, and traveled path length for each agent. Simultaneously, it checks whether the updated path set satisfies the conflict-free condition, i.e., whether the Euclidean distance between the location coordinates of any two agents at any sampling time is greater than or equal to the safe distance threshold dsafe, and feeds the detection results back to the reward calculation module. The reward calculation module generates a reward signal according to the instant reward calculation rules based on the conflict-free detection results fed back by the environment interaction interface and the change in the total path length, and passes the reward signal to the experience playback buffer of the policy optimization module for storage. The state encoder is used to convert the original path node information in the multi-agent collaborative path set into a numerical feature vector that can be processed by the reinforcement learning model. The encoding process includes: for each agent, extracting the x-coordinate and y-coordinate of its current storage location coordinates, dividing the x-coordinate and y-coordinate by the maximum row number and maximum column number of the shelf in the warehouse layout topology map respectively for normalization, so that the normalized coordinate values ​​fall within the [0,1] interval; extracting the remaining task quantity, dividing it by the initial remaining task quantity for normalization; extracting the length of the already traveled path, dividing it by the total length of all feasible channels in the warehouse layout topology map for normalization; and concatenating the four normalized values ​​into a joint state feature vector according to the agent index order. The model scheduler controller is responsible for controlling the iteration process of the entire reinforcement learning collaborative strategy model, including setting the upper limit of the number of iterations Imax, the convergence threshold δ, and the learning rate decay strategy during training. After each iteration, the model scheduler controller determines whether the change in the joint state value is less than the convergence threshold δ or whether the number of iterations has reached the upper limit Imax. If either condition is met, the iteration is terminated and the current globally optimal batch picking path planning scheme is output; otherwise, the next iteration continues.

[0059] The value network, a deep neural network, is used to evaluate the value of the joint state of multiple agents. The policy optimization module, based on the policy gradient algorithm, iteratively adjusts path nodes using the state value information output by the value network. The policy network is a deep neural network whose input is the joint state feature vector output by the value network, and whose output is the probability distribution of each agent's optional transition directions in the warehouse layout topology map under the current joint state. The structure of the policy network includes an input layer, two hidden layers, and an output layer. The number of neurons in the input layer is the same as the dimension of the joint state feature vector, i.e., 4N. The first hidden layer contains 128 neurons, and the second hidden layer contains 64 neurons. Each hidden layer is followed by a ReLU activation function for nonlinear transformation. The output layer contains the same number of neurons as the total number of optional transition directions for all agents, and uses a Softmax activation function to convert the output values ​​into a probability distribution. The action selector receives the probability distribution output by the policy network and uses a combination of greedy strategy and random exploration to determine the update direction for each agent at the current path node. The greedy strategy selects the candidate transition direction with the highest probability of 1-ε, and randomly selects a direction uniformly from all candidate transition directions with probability ε. The initial value of ε is set to 0.3, and it decays exponentially with the number of iterations, with a decay rate of 0.995, until ε decays to 0.01 and remains constant. The experience replay buffer is used to store historical interaction experiences. Each experience record includes the current joint state feature vector, the actual transition direction selected by each agent, the immediate reward obtained after the transition, and the next joint state feature vector. The immediate reward is calculated as follows: after the transition, if the Euclidean distance between all agents at the next sampling time is greater than or equal to the safe distance threshold dsafe, the reward value is 0; otherwise, the reward value is -1. At the same time, if the total path length of all agents after the transition is reduced compared to the previous iteration, an additional positive reward of 0.1 is added. The experience replay buffer capacity is set to 10,000 records. During training, 64 experience records are randomly selected from the buffer each time. The network parameters of the policy network are updated using the policy gradient algorithm. The objective function of the policy gradient algorithm is the expected cumulative discount reward, the discount factor γ is set to 0.99, and the learning rate is set to 1×10⁻. 4 The Adam optimizer is used for parameter updates. The policy network and value network are trained alternately; after each parameter update of the value network, the parameter update of the policy network is performed.

[0060] The initial state includes the current coordinates of each agent's location in the cooperative obstacle avoidance path set, the sequence of remaining unvisited locations, and the length of the already traveled path. The total number of agents in the multi-agent cooperative task allocation network is denoted as […]. , greater than or equal to Integers. For the th An intelligent agent. The range of values ​​is to The integer, the current cargo location coordinates are represented as two-dimensional coordinates. ,in Indicates the first The horizontal coordinate of each agent in the warehouse layout plane Indicates the first The vertical coordinate of the agent in the warehouse layout plane; the remaining sequence of locations to be accessed is determined by the first... The coordinates of the unvisited locations in the collaborative obstacle avoidance path set are arranged in their original order, and the remaining task is... The number of location coordinates in the remaining unvisited location sequence; the length of the traveled path. For the first The total path length is the distance traveled by each agent from its starting node to its current storage location. This total path length is obtained by summing the edge weights of the traveled segments in the warehouse layout topology graph. The initial state is constructed as follows: the first path node of each agent is extracted from the collaborative obstacle avoidance path set as the current storage location coordinates; all path nodes after the first path node are considered as the remaining storage locations to be visited; and the length of the traveled path is initialized to [value missing]. .

[0061] The initial state is evaluated using a value network derived from a reinforcement learning collaborative policy model, generating a state value vector. The position information, remaining task load, and path length of each agent in the initial state are concatenated to form a joint state feature vector. The dimension of the joint state feature vector is... The splicing order is as follows: sequentially splice the first... The x-coordinate of each agent y-axis Remaining task volume Length of traveled route , No. The x-coordinate of each agent y-axis Remaining task volume Length of traveled route Until the The x-coordinate of each agent y-axis Remaining task volume Length of traveled route Connecting the first and last elements forms a one-dimensional vector. The values ​​of the horizontal and vertical axes use the same units as the node coordinates in the warehouse layout topology diagram. The remaining task quantity is an integer, and the length of the traveled path is in meters. The joint state feature vector is input into a multi-layer fully connected layer of the value network for non-linear mapping processing, outputting a scalar as the joint state value. A multi-layer fully connected layer refers to a neural network structure composed of multiple fully connected layers stacked sequentially. Each fully connected layer consists of several neurons, and each neuron between adjacent layers is interconnected through learnable weight and bias parameters. The working principle of a multi-layer fully connected layer is as follows: For the input layer, the joint state feature vector is received as input, and the number of neurons in the input layer is consistent with the dimension of the input vector; For the i-th hidden layer, it receives the output values ​​of all neurons in the previous layer, and calculates the output value of each neuron in this layer through linear transformation and non-linear activation function. The linear transformation is calculated by multiplying the output values ​​of all neurons in the previous layer by their corresponding weight parameters, summing them, and adding the bias parameters of the neurons in this layer to obtain a linear combination value. This linear combination value is then input into the non-linear activation function to obtain the output value of the neurons in this layer; The last fully connected layer is the output layer, which receives the output of the last hidden layer and outputs a scalar value as the joint state value through linear transformation. The specific implementation of the multi-layer fully connected layer is as follows: The multi-layer fully connected layer of the value network consists of one input layer, three hidden layers, and one output layer. The number of neurons in the input layer is 4N. The first hidden layer contains 256 neurons, the second hidden layer contains 128 neurons, and the third hidden layer contains 64 neurons. A ReLU activation function is applied after each hidden layer for nonlinear transformation. The mathematical expression of the ReLU activation function is f(x) = max(0,x), where x is the linear combination value. The output layer contains one neuron and does not use an activation function, directly outputting the linear combination value as the scalar value of the joint state value. The connection between adjacent layers is fully connected, that is, the output value of each neuron in the k-th layer is connected to the input of all neurons in the (k+1)-th layer. The weight parameters between the k-th layer and the (k+1)-th layer form a weight matrix of dimension (M_k × M_{k+1}), where M_k is the number of neurons in the k-th layer, M_{k+1} is the number of neurons in the (k+1)-th layer, and the bias parameters form a bias vector of dimension M_{k+1}. The weight matrix and bias vector are iteratively updated during the training of the value network using the backpropagation algorithm and the Adam optimizer, with a learning rate set to 1×10⁻. 4 The batch size is set to 32, and the training objective is to minimize the mean square error between the joint state value prediction value output by the value network and the state value label.

[0062] The multi-layer fully connected structure of the value network consists of one input layer, three hidden layers, and one output layer. The number of neurons in the input layer is... The first hidden layer contains The second hidden layer contains 10 neurons. The third hidden layer contains 1 neuron. Each hidden layer contains 10 neurons, and a ReLU activation function is applied after each hidden layer for non-linear transformation. The output layer contains 10 neurons. There are 10 neurons in the value network. The output layer does not use an activation function; the output scalar value is the joint state value. Adjacent layers are fully connected, and the weight matrix and bias vector are updated during the training process using the backpropagation algorithm.

[0063] The value network is trained offline using pre-training, with training samples constructed from historical picking path execution data. The steps for constructing training samples are as follows: Collect state records at each moment of the multi-agent collaborative picking process in historical waves. Each state record includes the position information of each agent, remaining task quantity, path length, and the total time actually consumed from that state to task completion; use the negative value of the total time as the reward signal, and normalize it to... The interval is used as the state value label; the joint state feature vector corresponding to each state record is used as the input feature, and the state value label is used as the supervision signal. The mean squared error loss function is used to train the value network. The mean squared error loss function is defined as the square of the difference between the joint state value prediction value output by the value network and the state value label. The training process uses the Adam optimizer, and the learning rate is set to... Batch size is Iterative training continues until the loss value on the validation set is continuous. Stop when the wheel has not descended.

[0064] Based on the state value vector, a policy gradient algorithm is used to iteratively adjust the path nodes in the cooperative obstacle avoidance path set, outputting the globally optimal batch picking path planning scheme. For each path node in the cooperative obstacle avoidance path set, the action value corresponding to all candidate transition directions in its action space is calculated. The action space is defined as the set of neighboring nodes that can transition to the current path node in the warehouse layout topology graph. The set of neighboring nodes consists of all nodes in the warehouse layout topology graph that have direct edges with the current path node, and the total number of candidate transition directions is the cardinality of the set of neighboring nodes. For each candidate transition direction, the current path node is simulated to be replaced by a candidate neighboring node, and the path information of the corresponding agent is updated to obtain a new joint state after executing the candidate transition direction. The joint state feature vector corresponding to the new joint state is input into the trained value network, and the joint state value scalar output by the value network is used as the action value corresponding to the candidate transition direction.

[0065] The expression for calculating action value is:

[0066]

[0067] in, Indicates the current path node, This represents the candidate neighboring nodes corresponding to the candidate transition direction. Indicates from path node Execution is transferred to candidate neighbor nodes. The value of the action; Represents the forward propagation function of the value network; This represents the encoding function for the joint state feature vector; This indicates that the path node Replace with candidate neighboring nodes The resulting new joint state. Joint state feature vector encoding function. The specific operation is as follows: according to the splicing order, the horizontal coordinate, vertical coordinate, remaining task quantity, and length of the traveled path of each agent in the new joint state are concatenated into a one-dimensional vector, and the horizontal and vertical coordinates are standardized during the encoding process. The standardization process adopts Z-score standardization, and the mean and standard deviation required for standardization are obtained statistically from the historical picking path execution data.

[0068] The candidate transition direction with the highest action value is selected as the update direction for the path node, generating an iterative path node. When multiple candidate transition directions have the highest and equal action values, the candidate transition direction that reduces the remaining task load of the corresponding agent the most is selected first; if the reduction in the remaining task load is still the same, the candidate transition direction with the smallest Manhattan distance between the corresponding adjacent nodes is selected. The iterative path node replaces the original path node in the cooperative obstacle avoidance path set, resulting in an updated path set. During the iteration process, all path nodes of all agents in the cooperative obstacle avoidance path set are adjusted sequentially in ascending order of timestamp. After completing one full traversal, the joint state value is recalculated. If the change in the joint state value compared to the previous round is less than the convergence threshold, the path is updated. , Values If the path set converges, the iteration stops and the updated path set is output as the globally optimal batch picking path planning scheme. If the number of iterations reaches the preset iteration limit... The situation has not yet improved. Values If the current updated path set is output, it will be used as the globally optimal batch picking path planning scheme.

[0069] See Figure 5In the figure, the horizontal axis represents the number of iterations of the reinforcement learning collaborative policy model, ranging from 0 to 500. The left vertical axis represents the joint state value, represented by a solid red line, with a value ranging from approximately -11 to -3; the right vertical axis represents the total path length (in meters), represented by a dashed blue line, with a value ranging from approximately 850 to 1120 meters.

[0070] As the number of iterations increases, the joint state value curve shows an overall upward trend. In the initial stage (from 0 to approximately 150 iterations), the joint state value increases rapidly from approximately -11 to approximately -5, indicating that the overall path value of the multi-agent joint state is significantly optimized. The joint state value tends to stabilize in the range of approximately 150 to 500 iterations, with small fluctuations, indicating that the reinforcement learning model gradually converges and the path planning scheme tends to be optimal.

[0071] The total path length curve shows an overall downward trend, rapidly decreasing from approximately 1100 meters to approximately 900 meters in the initial stage, indicating that path planning achieved significant path compression through iteration. Subsequently, the path length fluctuated slightly between 850 and 900 meters in the range of approximately 150 to 500 iterations, indicating that the path length gradually stabilized and the optimization effect tended to saturate.

[0072] Example 5:

[0073] In practice, after generating the initial picking sequence for each agent through a multi-agent collaborative task allocation network, the local path planner corresponding to each agent is called to calculate the shortest path based on the location coordinates in the order subset allocated to the agent, thereby generating the initial picking sequence.

[0074] For the first in a multi-agent cooperative task allocation network The first intelligent agent, the... Each agent assigns an order subset containing one or more picking orders. The location coordinates of all SKUs within each picking order subset are extracted. Duplicate location coordinates are removed to form a set of location coordinates to be visited. This set contains... Each cargo location has a unique coordinate. greater than or equal to The local path planner constructs a picking path subgraph using the set of coordinates of the locations to be visited as the node set and the feasible channels in the warehouse layout topology as the edge set. The picking path subgraph is constructed as follows: all location coordinates in the set of locations to be visited are used as nodes in the picking path subgraph; for any two nodes in the picking path subgraph, if there is one or more connected paths consisting of feasible channels between the two nodes in the warehouse layout topology, the shortest path length between the two nodes is calculated, and this shortest path length is used as the weight of the edge connecting these two nodes in the picking path subgraph; if there is no connected path between the two nodes in the warehouse layout topology, then these two nodes are not connected in the picking path subgraph. The node set of the picking path subgraph also includes a starting node corresponding to the agent's starting position. The starting node is either the coordinates of the sorting station or the warehouse entrance. The edge weights between the starting node and all location coordinates in the set of locations to be visited are calculated in the same way, and edges are added to connect them.

[0075] The local path planner solves the Traveling Salesman Problem (TSP) with a start and end point on the picking path subgraph, obtaining the shortest traversal sequence that starts from the starting node, traverses all the coordinates of the locations to be visited, and returns to the starting node. The order of the nodes in the shortest traversal sequence is the initial picking sequence. This is determined by the number of location coordinates in the set of locations to be visited. No more than At this time, the Held-Karp dynamic programming algorithm is used to solve the Traveling Salesman Problem exactly. The state of the Held-Karp dynamic programming algorithm is represented by a pair of tuples. ,in A subset of the set of visited warehouse coordinates. Given the current location's coordinates, the state recursion formula starts from the subset with a size of... Starting from the state, the subset size is gradually expanded until all cargo location coordinate nodes are covered, and finally the shortest traversal sequence is obtained through backtracking. This is when the number of cargo location coordinates in the set of cargo locations to be visited is... Greater than In this study, the Lin-Kernighan heuristic algorithm is used to approximate the solution to the Traveling Salesman Problem. The Lin-Kernighan heuristic starts with a randomly generated initial traversal sequence and iteratively shortens the total length of the traversal sequence by performing edge-swapping operations. These edge-swapping operations include... -opt and -opt has two neighborhood structures, when consecutive The algorithm terminates when no better solution is found in the next iteration, and outputs the optimal traversal sequence as the initial selection sequence.

[0076] The initial picking sequences of each agent are time-axis aligned to obtain the spatiotemporal occupancy trajectory of each agent in the warehouse layout. A unified time starting point is used as the reference, and the unified time starting point is set to time 1. Seconds. For the first The initial picking sequence of each agent contains... Given a set of ordered nodes, mark the nodes as nodes in traversal order. ,in As the starting node, This is the ending node after returning the starting node. For the nth node in the sequence... Nodes , The range of values ​​is to integers, the first Nodes The arrival timestamp is obtained through the previous node. The distance traveled between the two nodes is obtained by summing the arrival timestamps. From the nodes in the picking path subgraph With nodes The edge weights between them are determined, and the theoretical walking speed is... According to the Preset mobile device types for each agent. Node The expression for the arrival timestamp is:

[0077]

[0078] in, Indicates the first In the initial picking sequence of the i-th agent, the th... The arrival timestamps of each node, in seconds; Indicates the first In the initial picking sequence of the i-th agent, the th... Arrival timestamps of each node, initial value equal Second; Indicates the first In the initial picking sequence of the i-th agent, the th... The node and the first The walking distance between nodes, in meters; Indicates the first The theoretical walking speed of an intelligent agent, measured in meters per second. Determined based on mobile device type; when the mobile device is for a walking picker. Pick When the mobile device is an electric picking cart Pick When the mobile device is an automated guided vehicle Pick Each node The coordinates of the cargo location and its arrival timestamp Link them together to form a spatiotemporal occupancy record, and then... All of the agents The spacetime occupancy records are arranged in ascending order of arrival timestamp, forming the [number]th [item]. Spatiotemporal trajectory records of each agent. The above operation is performed on each agent in the multi-agent cooperative task allocation network to obtain a set of spatiotemporal trajectory records for each agent.

[0079] Detect whether there are overlapping events of cargo location coordinates between different agents at the same time point in the spatiotemporal occupancy trajectory. Use a preset time granularity step size. Discretize the time axis, with a time granularity step size. Values The value in seconds is determined based on the minimum safe sampling interval, which is a combination of the braking response time of mobile devices and the safe distance in a warehouse environment. From time... Starting in seconds, with time granularity steps For incremental sampling time, at each sampling time Iterate through the spatiotemporal trajectory records of all agents and determine the position of each agent at the sampling time using linear interpolation. The coordinates of the cargo location. The linear interpolation method is: find the arrival timestamp in the spatiotemporal occupancy trajectory record that is less than or equal to... The node corresponding to the maximum timestamp is used as the preceding anchor node, and the arrival timestamp is greater than or equal to... The node corresponding to the smallest timestamp is taken as the subsequent anchor node. If the cargo location coordinates of the preceding anchor node and the subsequent anchor node are the same, then the sampling time... The current cargo location coordinates are equal to the cargo location coordinates of the preceding anchor node; if the cargo location coordinates of the preceding and following anchor nodes are different, the cargo location coordinates are calculated by interpolation along the line connecting the two points according to the time ratio. For any two different agents, the corresponding sampling time is calculated. The Euclidean distance between the location coordinates of the next two agents is determined by the following condition: if the Euclidean distance is less than the safe distance threshold... Then it is determined that the two agents are at the sampling time. A cargo location coordinate overlap event occurred. Safety distance threshold. Take the minimum center-to-center distance between adjacent storage locations in the warehouse layout. The sampling time at which the cargo location coordinates coincide will occur. The identifiers of the two involved agents and the coordinates of the cargo location where the conflict occurred are aggregated and marked as a path conflict node. After processing all sampling times, a set of path conflict nodes is generated.

[0080] Path conflict nodes are resolved using either a priority avoidance strategy or a speed adjustment strategy. The priority avoidance strategy works as follows: Each agent is pre-assigned a unique priority number. The priority numbers are assigned in ascending order based on the creation time of the earliest order in the agent's assigned order subset; the earlier the creation time of the order subset, the smaller the priority number of the agent, and the higher the priority. At a path conflict node, the lower-priority agent with the smaller priority number and the higher-priority agent with the larger priority number are identified between the two agents involved in the conflict, while maintaining the spatiotemporal occupancy trajectory of the higher-priority agent. For the lower-priority agent, a waiting operation node is inserted after the nearest visited storage location coordinate node before the conflicting path node in its spatiotemporal occupancy trajectory. The storage location coordinates of the waiting operation node are the same as the storage location coordinates of the lower-priority agent's visited storage location coordinate node before the conflicting path node. The arrival timestamp of the waiting operation node is set to the arrival timestamp of the visited storage location coordinate node plus one waiting time unit. , The value is equal to the time granularity step. The arrival timestamps of all nodes following the visited cargo location coordinate node in the spatiotemporal occupancy trajectory of the low-priority agent are incremented. Regenerate the spatiotemporal occupancy trajectory records of low-priority agents. Re-execute conflict detection on the updated spatiotemporal occupancy trajectory records. If the conflict remains unresolved, continue to increase the waiting time units, increasing them each time. The distance between the location coordinates of the low-priority agent and the high-priority agent will continue until it is greater than or equal to the safe distance threshold. .

[0081] The resolution process of the speed adjustment strategy is as follows: At the path conflict node, at least one of the two agents involved in the conflict is selected as the speed adjustment object. In the spatiotemporal occupancy trajectory of the speed adjustment object, the location of the cargo position coordinates corresponding to the path conflict node in the original traversal sequence is determined. The first adjacent road segment upstream of this node is then used as the speed adjustment segment, and the original theoretical travel speed corresponding to this speed adjustment segment is calculated. Multiplied by a speed adjustment factor The adjusted theoretical walking speed was obtained. Speed ​​adjustment factor The initial value is set to ,by For step length in Adjust within the range, and use after each adjustment. The arrival timestamps of the speed-adjusted objects at all nodes in and after the speed-adjusted section are recalculated, the spatiotemporal occupancy trajectory records are updated, and conflict detection is re-performed on the updated spatiotemporal occupancy trajectory records of all agents. If adjusting the speed adjustment factor of a single agent within its value range cannot eliminate the conflict, the speed adjustment strategy is applied simultaneously to the two agents involved in the conflict for coordinated adjustment. After resolution, a set of conflict-free cooperative obstacle avoidance paths is obtained. This set of cooperative obstacle avoidance paths includes the final adjusted initial picking sequence and corresponding spatiotemporal occupancy trajectory of all agents, and the Euclidean distance between the cargo location coordinates of any two agents at any sampling time is greater than or equal to the safe distance threshold. .

[0082] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for batch picking route planning of warehouse orders based on multi-agent collaboration, characterized in that, The method includes: obtaining a set of multiple picking orders to be processed in the current wave and a warehouse layout topology map; Establish an order-location correlation model and perform pre-training; The pre-trained order-location correlation model is invoked to perform product correlation analysis on the picking order set, generating an inter-order correlation matrix; A multi-agent collaborative task allocation network is constructed based on the order correlation matrix to assign each picking order to the corresponding agent; each agent is configured with a corresponding local path planner. The multi-agent cooperative task allocation network generates an initial picking sequence for each agent, and performs multi-agent path conflict detection processing on the initial picking sequence to obtain a set of cooperative obstacle avoidance paths. Establish a reinforcement learning collaborative strategy model and pre-train it; The set of collaborative obstacle avoidance paths is input into a pre-trained reinforcement learning collaborative strategy model for joint path optimization, generating a globally optimal batch picking path planning scheme.

2. The warehouse order batch picking route planning method based on multi-agent collaboration as described in claim 1, characterized in that, The step of calling the pre-trained order-location correlation model to perform product correlation analysis on the picking order set and generating an order correlation matrix includes the following steps: Perform location mapping processing on the product SKUs in each picking order to obtain a set of location coordinates corresponding to each order; Calculate the mean Euclidean distance between the locations of any two picking orders as the initial spatial correlation. Obtain the number of product SKUs commonly included in any two picking orders, and calculate the intersection ratio of the product SKU count with the total number of SKUs in each order, as the product overlap. The initial spatial correlation degree and the product overlap degree are weighted and fused to generate the order correlation degree matrix.

3. The warehouse order batch picking route planning method based on multi-agent collaboration as described in claim 2, characterized in that, The weighting coefficients in the weighted fusion process are determined by the entropy weighting method or principal component analysis.

4. The warehouse order batch picking route planning method based on multi-agent collaboration according to claim 3, characterized in that, The step of constructing a multi-agent collaborative task allocation network based on the order correlation matrix and assigning each picking order to the corresponding agent includes the following steps: Extract each row vector from the order correlation matrix as the correlation distribution feature of the current order; Build a task clustering model and pre-train it; The correlation distribution features are input into a pre-trained task clustering model to perform order grouping processing, generating multiple order subsets. Each of the order subsets is assigned an independent agent, and all agents are interconnected to form a multi-agent collaborative task allocation network.

5. The warehouse order batch picking route planning method based on multi-agent collaboration according to claim 4, characterized in that, The process of generating an initial picking sequence for each agent through the multi-agent cooperative task allocation network, and performing multi-agent path conflict detection processing on the initial picking sequence to obtain a cooperative obstacle avoidance path set includes the following steps: The local path planner corresponding to each agent is invoked to calculate the shortest path based on the location coordinates in the location coordinate set in the order subset assigned to that agent, thereby generating the initial picking sequence; The initial picking sequence of each agent is time-axis aligned to obtain the spatiotemporal occupancy trajectory of each agent in the warehouse layout. The system detects whether there are overlapping events of cargo location coordinates of different agents at the same time point in the spatiotemporal occupancy trajectory. If such events exist, they are marked as path conflict nodes.

6. The warehouse order batch picking route planning method based on multi-agent collaboration according to claim 5, characterized in that, The process of aligning the initial picking sequences of each agent with a time axis to obtain the spatiotemporal occupancy trajectory of each agent in the warehouse layout includes the following steps: Obtain the walking distance and theoretical walking speed between adjacent locations in the initial picking sequence of each agent, and calculate the theoretical timestamp of arrival at each location; The theoretical timestamp is bound to the corresponding cargo location coordinates to generate the spatiotemporal occupancy trajectory.

7. The warehouse order batch picking route planning method based on multi-agent collaboration according to claim 5, characterized in that, The step of inputting the set of cooperative obstacle avoidance paths into a pre-trained reinforcement learning cooperative strategy model for joint path optimization to generate a globally optimal batch picking path planning scheme includes the following steps: The path of each agent in the cooperative obstacle avoidance path set is used as the initial state of the reinforcement learning cooperative policy model, which consists of a value network and a policy optimization module. The initial state is jointly evaluated by the value network of the reinforcement learning collaborative policy model to generate a state value vector. Based on the state value vector, the policy gradient algorithm is used to iteratively adjust the path nodes in the cooperative obstacle avoidance path set, and output the globally optimal batch picking path planning scheme.

8. The warehouse order batch picking route planning method based on multi-agent collaboration according to claim 7, characterized in that, The step of evaluating the joint state value of the initial state through the value network of the reinforcement learning collaborative policy model to generate a state value vector includes the following steps: The position information, remaining task quantity, and path length of each agent in the initial state are concatenated into a joint state feature vector. The value network is provided with multiple fully connected layers; The joint state feature vector is input into the multi-layer fully connected layer of the value network for nonlinear mapping processing, and a scalar is output as the state value vector.

9. The warehouse order batch picking route planning method based on multi-agent collaboration according to claim 7, characterized in that, The step of iteratively adjusting the path nodes in the cooperative obstacle avoidance path set based on the state value vector using a policy gradient algorithm to output the globally optimal batch picking path planning scheme includes: For each path node in the cooperative obstacle avoidance path set, calculate the action value corresponding to all candidate transfer directions in its action space; Select the candidate transition direction with the highest action value as the update direction of the path node, and generate an iterative path node; The iterative path nodes replace the original path nodes in the cooperative obstacle avoidance path set to obtain the updated path set.

10. The warehouse order batch picking route planning method based on multi-agent collaboration according to claim 5, characterized in that, The path conflict nodes are resolved using a priority avoidance strategy or a speed adjustment strategy to generate a conflict-free cooperative obstacle avoidance path.

Images