A high-density three-dimensional warehouse dynamic storage location allocation method for a WMS

Abstract

The application discloses a high-density stereoscopic warehouse dynamic storage space allocation method for WMS, and relates to the technical field of artificial intelligence. Stereoscopic warehouses are arranged along the direction of the roadway, and stackers, shuttles and hoists are arranged in the stereoscopic warehouses. Field programmable gate array acceleration cores are arranged in the edge nodes of the stereoscopic warehouses. The acceleration cores are interconnected with the WMS through a time-sensitive network. The method comprises the following steps: step 1, the acceleration core constructs a beat dependency graph based on the current carrying instruction issued by the WMS, wherein the carrying motion is taken as an event node, and the action connection relationship is taken as a directed edge; step 2, the acceleration core constructs a colored Petri net; and step 3, the acceleration core constructs a graph attention reinforcement learning decision module and takes a heterogeneous graph as input, and the WMS drives the corresponding carrying equipment to perform the actions of warehousing, moving and replenishment. The application can significantly improve the system throughput, reduce the equipment idle rate and beat jitter under the sub-millisecond end-to-end decision delay.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for dynamic location allocation in high-density three-dimensional warehouses for WMS (Warehouse Management System). Background Technology

[0002] High-density automated storage and retrieval systems (AS / RS) are a key infrastructure for modern intelligent logistics, widely used in e-commerce sorting centers, pharmaceutical cold chain warehouses, auto parts distribution centers, and food processing raw material warehouses. The inbound, relocation, replenishment, and outbound tasks issued by the warehouse management system need to be completed collaboratively among various heterogeneous handling equipment such as stacker cranes, shuttle cars, and elevators. The rationality of storage location allocation directly determines the system's throughput capacity and equipment utilization rate.

[0003] Several approaches have been explored in the field of dynamic warehouse location allocation. One type of method relies on heuristic rules for decision-making, such as proximity principles, ABC classification principles, random location strategies, and metaheuristic optimization methods based on genetic algorithms, particle swarm optimization, and ant colony optimization. These methods are simple to implement and easy to deploy in engineering, but the objective functions they use are mostly based on static distance or simple weighted indicators, which cannot characterize the coupling characteristics of equipment cycle time constraints in automated warehouses. This can easily lead to equipment waiting and task backlog during peak periods when task density surges. Another type of method introduces queuing theory models and Markov decision processes to model the dynamic behavior of the warehouse, achieving location decision-making by solving the optimal strategy table. This type of method performs well in small-scale warehouses, but the state space grows exponentially with the size of the location, encountering the curse of dimensionality and making it difficult to scale to real-world automated warehouses containing thousands or even tens of thousands of locations. In recent years, research on introducing deep reinforcement learning into location allocation has gradually increased, including discrete action strategies based on deep Q-networks and continuous control methods based on actor-critic structures. These methods possess the ability to adaptively optimize decisions from historical data, but they typically feed the warehouse status into a fully connected network as a tiling vector, failing to explicitly model the topological relationships between equipment, storage locations, tasks, and cycle times. Consequently, the learned strategies exhibit poor transferability to changes in warehouse topology. Furthermore, online training usually runs on general-purpose processors, and the decision latency cannot meet the sub-millisecond timing requirements of handling equipment. In practical deployments, this often degenerates into a simplified form of offline training plus online table lookups, losing its adaptability to real-time operating conditions. How to achieve adaptive, low-latency, and online-learnable storage location allocation decisions while explicitly characterizing equipment cycle time constraints remains a pressing technical challenge. Summary of the Invention

[0004] In view of this, the present invention provides a high-density three-dimensional warehouse dynamic location allocation method for WMS, which avoids the task backlog and equipment idleness caused by the neglect of cycle coupling in heuristic rule methods, and effectively avoids the dimensionality curse caused by state space explosion by incremental reachability expansion of colored Petri nets. By using the graph attention mechanism on heterogeneous graphs to non-destructively characterize the coupling relationship of heterogeneous entities, the learned strategy has good transferability to changes in warehouse size and topology, thereby significantly improving system throughput and reducing equipment idle rate and cycle jitter under sub-millisecond end-to-end decision latency.

[0005] The technical solution adopted in this invention is as follows:

[0006] A method for dynamic location allocation in a high-density automated warehouse oriented towards a WMS (Warehouse Management System), wherein stacker cranes, shuttle cars, and hoists are arranged along the aisle direction in the automated warehouse, and the edge nodes of the automated warehouse are equipped with field-programmable gate array (FPGA) acceleration cores, which are interconnected with the WMS via a time-sensitive network, characterized by the following steps:

[0007] Step 1: The acceleration core constructs a beat dependency graph based on the current transport instructions issued by WMS, with transport motion as event nodes and action connection relationships as directed edges, and parses the beat cycle and key beat loops on the beat dependency graph.

[0008] Step 2: Accelerate the construction of a colored Petri net. The colored Petri net includes storage locations, equipment-ready storage locations, batch queue storage locations, task handover storage locations, and handling transitions. The acceleration core takes the storage location where the token corresponding to the current WMS task is located as the root storage location, and performs incremental reachability expansion in a breadth-first manner according to preset enabling rules. During the expansion process, the storage locations reached by the acceleration core and the storage locations with empty status colors are collected into a candidate storage location set.

[0009] Step 3: Accelerate the construction of the graph attention reinforcement learning decision module and use the heterogeneous graph as input. Each grid in the candidate location set in the heterogeneous graph is used as a location node, the current task is used as a task node, each handling device is used as a device node, and each event node on the critical cycle loop is used as a cycle node. After performing multiple rounds of neighbor aggregation on the heterogeneous graph, the policy head outputs the discrete probability distribution on the candidate location set. The acceleration kernel selects the location with the highest probability in the discrete probability distribution as the location allocation result for this batch and writes it back to the WMS through the time-sensitive network. The WMS then drives the corresponding handling devices to perform inbound, transfer, or replenishment actions.

[0010] Furthermore, the stacker crane is equipped with a fork pose sensor, the shuttle is equipped with a track odometer, and the hoist is equipped with a floor encoder. The timestamped sampling sequences of the fork pose sensor, track odometer, and floor encoder are uploaded to the acceleration core along a time-sensitive network to share a unified spatiotemporal reference. The acceleration core includes a pulsation array unit, a lookup table unit, on-chip random access memory, and a near-end policy optimization coprocessor.

[0011] Furthermore, in step 1, the event nodes include stacker crane fork extension, stacker crane carrying movement, stacker crane releasing fork extension, hoist inter-level switching, shuttle car track movement, shuttle car fork extension, shuttle car releasing fork extension, and task handover events. The task handover event consists of one type corresponding to the task type of the handling instruction in the inbound port reception, transfer handover, replenishment handover, or outbound port release. All event nodes are numbered sequentially according to the delivery order declared in the handling instruction. The action connection relationship includes the chain connection from stacker crane fork extension to stacker crane carrying movement, stacker crane carrying movement to stacker crane releasing fork extension, stacker crane releasing fork extension to hoist inter-level switching, hoist inter-level switching to shuttle car track movement, shuttle car track movement to shuttle car fork extension, shuttle car fork extension to shuttle car releasing fork extension, shuttle car releasing fork extension to task handover event, and task handover event returning to stacker crane fork extension.

[0012] Furthermore, in step 1, the length of each directed edge is the greater of the corresponding action duration declared by the transport instruction and the corresponding actual action duration measured at the start and end times of the action based on the timestamped sampling sequence in the most recent complete action cycle. The adjacency list of the beat dependency graph is stored in on-chip random access memory.

[0013] Further, in step 1, the beat cycle and key beat loop are resolved by the pulsating array unit as follows: Let N be the total number of event nodes in the beat dependency graph; select the event node with the smallest number as the starting node; initialize the 0th round expansion value of each event node as follows: the 0th round expansion value of the starting node is 0, and the 0th round expansion values ​​of the other event nodes are preset minimum values; perform N rounds of iterative updates starting from the 1st round. In each iteration, the current round expansion value of each event node is the maximum of the sum of the previous round expansion value and the corresponding directed edge length of all its predecessor event nodes. The predecessor event node pointer points to the predecessor event node that makes the current round expansion value the maximum. When multiple predecessors have the largest values, the predecessor event node pointer points to the predecessor event node with the smallest number. After N iterations, for each event node, iterate through all values ​​of k from 0 to N minus 1, calculate the difference between the Nth round expansion value and the kth round expansion value, divide by N minus k, and retain the smallest quotient as the cyclic average estimate of this event node. Among all the cyclic average estimates of all event nodes, retain the largest one as the beat period. The closed sequence of event nodes formed by tracing back step by step from the event nodes corresponding to the beat period through the predecessor event node pointer is marked as the critical beat loop. The beat period is equal to the maximum cyclic average time that limits the system throughput in the beat dependency graph.

[0014] Furthermore, in step 2, each storage location corresponds one-to-one with each physical storage cell in the automated warehouse. Each storage location registers the layer height color, weight limit color, and status color of the corresponding physical storage cell. The status color takes one of the following: empty, occupied, or locked. The weight limit color corresponds to a preset weight level, and each weight level is mapped to a weight limit value. The equipment-ready storage location corresponds to the idle status of the stacker crane, shuttle, and elevator. Each equipment-ready storage location registers the color of the aisle it governs. The batch queue storage location records the pallets to be processed according to the order of pallet arrival. The task handover storage locations include the receiving storage location at the inbound port, the transfer handover storage location, the replenishment handover storage location, and the release storage location at the outbound port. The tokens circulating in the colored Petri net record the product identification color, batch arrival order color, pallet weight color, pallet layer height color, and target aisle color in a fixed-length field of no less than 32 bits. The delay of each handling change is assigned by the cycle time.

[0015] Furthermore, in step 2, the enabling rules are determined in the following order: 1. All input locations for the transport change have tokens; 2. The status color of the target location is empty; 3. The pallet height color of the input token belongs to the same height group as the registered height color of the target location; 4. The value of the pallet weight color of the input token is less than or equal to the weight limit value mapped by the registered weight limit color of the target location; 5. When the transport instruction is an inbound task, the batch arrival order color of the input token is at the head of the batch queue location; when the transport instruction is a transfer task or a replenishment task... When the batch arrival order color of the input token satisfies the batch priority constraint declared by WMS; sixth, the target aisle color of the input token is equal to the aisle color registered in the corresponding equipment ready warehouse; when WMS issues a new inbound, transfer, or replenishment task, the root warehouse is determined according to the following rules: when the task is an inbound task, the root warehouse is taken from the warehouse where the corresponding inbound pallet token is located in the batch queue warehouse; when the task is a transfer task, the root warehouse is taken from the warehouse where the corresponding pallet token is located in the original storage location warehouse; when the task is a replenishment task, the root warehouse is taken from the warehouse where the corresponding replenishment pallet token is located in the replenishment handover warehouse.

[0016] Furthermore, in step 2, incremental reachability deployment is performed as follows: The acceleration core maintains a first-in-first-out queue carried by a lookup table unit and an access bitmap. All transport transitions connected to the root warehouse via the output arc are enqueued as transport transitions to be determined and their corresponding positions are set in the access bitmap. Transport transitions to be determined are retrieved one by one from the head of the queue. The lookup table unit performs a match on each retrieved transport transition according to the determination order of the enabling rules. For transport transitions to be determined that have passed all rules, the subsequent warehouses connected to them via the output arc are read. If the subsequent warehouse is a storage location and its status color is empty, it is registered as a candidate storage location. If the subsequent warehouse is a warehouse of other types, the next layer transport transition connected to it via the output arc is taken as a transport transition to be determined when the access bit is in the initial state. The transport change is enqueued and the corresponding position is set; the initial expansion layer number of incremental reachability expansion is taken as the number of event nodes of the key cycle loop. When the number of candidate storage locations is less than the preset candidate number threshold and there are still transport changes to be determined in the first-in-first-out queue, the expansion continues according to the preset expansion step size until any one of the following conditions is met: candidate number threshold, reachable boundary or preset maximum search layer number; the acceleration core writes the layer height color position, weight limit color position, aisle steps from the corresponding workstation of the task handover storage location, batch arrival order color position of the storage location head of the batch queue, and the event node sequence number corresponding to the transport change triggered when this storage location is the target storage location as the cycle node sequence number of this storage location as the on-chip random access memory and outputs it to step 3.

[0017] Furthermore, in step 3, the edges of the heterogeneous graph are translated from the input-output relationship of the handling transition and supplemented with the following cross-type edges: the location node is connected to the equipment node with the same color position in its subordinate aisle; the equipment node is connected to the cycle node corresponding to the current event node number on the key cycle loop; the task node is connected to all location nodes in the candidate location set; the initial feature vector of each type of node contains continuous components and categorical components; the continuous components are normalized dimension by dimension according to the maximum value of each dimension; the continuous components are the aisle steps and cycle average estimate of the distance from the workstation corresponding to the task handover warehouse; the categorical components are expressed in one-hot encoding form; the categorical components are the layer height color position, weight limit color position, pallet weight color position, subordinate aisle color position, batch arrival order color position, current event node number, location number, and belonging cycle node number; the number of rounds of multi-round neighbor aggregation is equal to the number of event nodes in the key cycle loop. In each round of neighbor aggregation, the following operations are performed on each node: the current feature vector of this node. The query vector, key vector, and value vector are obtained through a three-way fully connected mapping that shares parameters according to node category. The pulsating array unit takes the sum of the component-wise products of the query vector of this node and the key vector of each neighbor node as the similarity score of the corresponding neighbor node. All similarity scores of this node are normalized by softmax along the neighbor dimension. The value vectors of all neighbor nodes of this node are linearly combined according to the corresponding normalization coefficients, and the result is added to the feature vector of this node in the previous round component by component and then passed through a linear rectified activation function to obtain the feature vector of this node in the current round. After multiple rounds of neighbor aggregation, the final feature vectors of all storage location nodes are averaged and pooled to obtain the graph-level state features. The graph attention reinforcement learning decision module has a strategy head and a value head. The strategy head concatenates the final feature vector of each storage location node in the candidate storage location set with a fully connected layer and a softmax layer to obtain a discrete probability distribution. The value head obtains the scalar value estimate of the current state by passing the graph-level state features through a fully connected layer.

[0018] By adopting the above technical solutions, this invention achieves the following beneficial effects: This invention fundamentally improves the rationality and real-time performance of dynamic storage location allocation in high-density automated warehousing by uniformly encoding equipment cycle time constraints, storage location state constraints, and task timing constraints into graph and color-coded structures that can be efficiently reasoned on edge acceleration cores. Firstly, through parallel solving of the Karp cyclic averaging algorithm on pulsating array units, the cycle time and key cycle loops can be resolved within milliseconds, enabling subsequent storage location decisions to have explicit bottleneck time perception capabilities, avoiding task accumulation and equipment idling caused by heuristic rule methods ignoring equipment cycle time coupling. Secondly, through the incremental reachability expansion mechanism of colored Petri nets, the candidate storage location set can adaptively converge to a reasonable size under various task types and various vacancy rate conditions, avoiding the dimensionality curse caused by state space explosion and ensuring candidate completeness in peak-hour sparse vacancy scenarios. Third, through the graph attention reinforcement learning decision module's multi-round neighbor aggregation of heterogeneous graphs, the coupling relationships between four types of heterogeneous entities—locations, equipment, tasks, and cycle time—are losslessly characterized. The learned strategy exhibits excellent transferability to changes in warehouse size and topology. Fourth, through hardware-level closed-loop training using a proximal policy optimization coprocessor and generalized advantage estimation, the decision module can continuously optimize online during actual operations, eliminating the need for a simplified form of offline training plus online table lookup. Fifth, relying on the collaborative deployment of field-programmable gate array (FPGA) acceleration cores and time-sensitive networks (TSNs), end-to-end decision latency can be strictly controlled to the sub-millisecond level. This closely coordinates with the start-up timing of stacker cranes, shuttles, and hoists, ultimately significantly improving system throughput while reducing equipment idle rate and cycle time jitter. Attached Figure Description

[0019] Figure 1 This is an analytical schematic diagram of the beat dependency graph and key beat loop in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram showing the longitudinal section of the three-dimensional storage aisle and the distribution relationship of the color positions of the cargo locations in an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the topological structure of the heterogeneous graph and the information flow direction of graph attention aggregation in an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram illustrating the evolution of the number of candidate storage locations as the number of deployment layers occurs during the incremental accessibility deployment process in this embodiment of the invention. Detailed Implementation

[0023] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] When WMS issues a handling instruction, the instruction is a set of structured fields that include a task identifier, pallet identifier, start and end positions, equipment scheduling priority, and a series of action duration declarations. In a typical embodiment, the handling instruction includes the following fields: Task Number Value Pallet markings Value The starting point is the warehouse entrance. The endpoint is the high-bay rack. Ranked Liede Layered storage compartments. In this handling instruction, the WMS also specifies the time required for the stacker crane's fork extension to reach the goods. Seconds, stacker crane carrying and traveling time Seconds, stacker crane fork extension time Seconds, time for switching between elevator floors Seconds, shuttle car track travel time Seconds, time for shuttle car to extend fork to pick up goods Seconds, time for shuttle car to place and extend forks Seconds and task handover event duration seconds, of which Indicates the first The nominal time for each action type is in seconds. Each time value is an engineering estimate pre-calibrated by WMS based on the equipment's factory parameters and historical scheduling records, with the aim of providing an initial baseline for the side length of the cycle time dependency graph.

[0025] The process accelerates the verification of the transport action field by byte-by-byte scanning and parsing, extracting the action type identifier and the time consumption value in pairs, and numbering each of the eight event nodes according to the submission order. Using the values ​​from the previous embodiment, the numbering is as follows: to The event nodes sequentially correspond to the stacker crane fork extension (picking up), stacker crane carrying movement, stacker crane fork extension (discharging), hoist inter-level switching, shuttle car track movement, shuttle car fork extension (picking up), shuttle car fork extension (discharging), and task handover events. The specific form of the task handover event is determined by the task type field of the handling instruction: when the task type field value is... Timely receipt at the inlet port, for Timely transfer and handover, for Timely replenishment and handover, for The system retrieves items from the storage port and releases them at specific times. Four possible values ​​correspond one-to-one with four actual operational modes in automated warehousing. Task handover is abstracted into a single type of event node, with specific forms distinguished by color. This allows the cycle time analysis logic to handle only one unified structure, saving hardware resources of the pulse array unit.

[0026] The directed edges between event nodes are established according to the action connection relationship declared in the handling instruction. In the aforementioned embodiment, the connection method is as follows: stacker crane picking forks extend to stacker crane carrying movement, stacker crane carrying movement to stacker crane placing forks extend, stacker crane placing forks extend to elevator inter-level switching, elevator inter-level switching to shuttle car track movement, shuttle car track movement to shuttle car picking forks extend, shuttle car picking forks extend to shuttle car placing forks extend, shuttle car placing forks extend to task handover event, and task handover event returns to stacker crane picking forks extend. The directed edges returning to stacker crane picking forks extend from task handover event form a closed loop, making the cycle time dependency graph form a directed loop graph, which is the premise for defining the cycle time period; the actual operation of the automated warehouse is a periodic cyclic process, and after each handling cycle is completed, the equipment returns to the initial state to prepare for the next cycle.

[0027] The length of each directed edge is the greater of the declared action duration and the actual action time. The actual action time is calculated based on a timestamped sampling sequence. In one optional implementation, the fork pose sensor uses... Hertz sampling rate is used to upload timestamped pose samples along a time-sensitive network for orbital odometry. Upload position pulse count samples at Hertz sampling rate, layer encoder at... The Hertz sampling rate uploads the absolute value samples of each layer. The acceleration core maintains a sliding window cache of all samples from the most recent complete action cycle, identifying the start and end times of each action type and calculating the time difference. For example, for a stacker crane carrying goods, the start time... From the velocity component in the fork pose sample Jump to threshold The time is determined, the end time is determined. The rate of change of position in the orbital odometry sample fell back to If the timing is determined, then the actual action time for that cycle is [not specified]. The unit is seconds. Continuing with the previous embodiment, let's assume the measured actual action times are as follows: , , , , , , , If the time interval is 1 second, then the lengths of the directed edges are taken sequentially. ,Right now , , , , , , and The larger value (seconds) allows the beat dependency graph to depict the bottleneck time under steady-state conditions, avoiding overly optimistic estimates that could lead to smaller subsequent beat cycles, thus providing a conservative safety margin for subsequent warehouse location selection and decision-making.

[0028] refer to Figure 1 In this embodiment of the invention, the cycle dependency graph contains 8 event nodes, which are evenly arranged along the circumference at equal angles and numbered sequentially from 1 to 8. These nodes correspond to events such as stacker crane picking fork extension, stacker crane carrying goods movement, stacker crane placing fork extension, elevator inter-layer switching, shuttle car track movement, shuttle car picking fork extension, shuttle car placing fork extension, and task handover. Node 8 returns to node 1 via a directed edge to form a closed loop. The lengths of each directed edge are labeled as 4.2 seconds, 19.4 seconds, 4.5 seconds, 7.1 seconds, 12.4 seconds, 3.9 seconds, 4.3 seconds, and 2.0 seconds, respectively. Figure 1 The output of the beat cycle is indicated by a dashed circle in the center, with a value of 7.225 seconds. Figure 1 The key beat loop corresponding to the 8-node closed-loop structure shown is the main loop itself, which is composed of all event nodes numbered 1 to 8.

[0029] The adjacency list of the beat-dependent graph is stored in on-chip random access memory. One optional storage format is a compressed sparse row layout, where each event node maintains a start pointer pointing to the starting position of all its outgoing edges in an edge length array, which is compactly arranged in order of event node numbers. In the aforementioned 8-node embodiment, the adjacency list occupies approximately [space missing]. Even if the number of event nodes expands to dozens, the adjacency list can still be controlled to the level of hundreds of bytes, and can all reside in on-chip random access memory, avoiding latency jitter introduced by accessing off-chip storage.

[0030] The acceleration kernel performs parsing on the beat-dependent graph using a systolic array of cells. The algorithm employed is a hardware-parallelized implementation of the Karp cyclic averaging algorithm. The algorithm objective can be formally expressed as solving... ;in For the set of event nodes in the beat dependency graph, The total number of event nodes. For nodes In the The expanded value after rounds of iterations, This is the desired cycle time. The geometric meaning of this formula is: In a cycle dependency graph, there are multiple directed cycles. The total time of each directed cycle divided by the number of event nodes it contains gives the cycle average time of that cycle. The maximum value among the cycle average times of all directed cycles is the time corresponding to the bottleneck cycle limiting system throughput, i.e., the cycle time. The Karp algorithm cleverly utilizes… By iteratively expanding layer by layer, the exponential complexity of enumerating all directed cycles is reduced to polynomial complexity.

[0031] The specific iteration process is implemented as follows. The initial expansion value of each event node is configured according to the following rules: the event node with the smallest number is selected as the starting node, and its number is... Wheel expansion value The remaining event nodes The wheel expansion value is a preset minimum value. In one alternative implementation, take The purpose of introducing a minimum initial value is to ensure that during iteration... Operations naturally suppress contributions from nodes not yet reached by the starting node via directed paths, thus avoiding interference with the calculation of the maximum cyclic average. From the... Execution in rotation Iterative updates are performed layer by layer, and each event node is updated in each iteration. The current round of expansion value is taken ;in for The set of all preceding event nodes, For directed edges The side length, For the preceding event node The expansion value in the previous round. This recurrence relation means: starting from the starting node, passing through exactly... Reaching a node by stepping on a directed edge The length of the path with the longest cumulative time among all paths is . The predecessor event node pointer points to the predecessor event node that maximizes the current round's expansion value; when multiple predecessors have the same maximum value, the predecessor event node pointer points to the predecessor event node with the smallest number. This method of breaking the parallel is deterministic, meaning that the same input will inevitably result in the same pointer direction, facilitating the efficient implementation of the subsequent backtracking process in hardware using a branchless judgment method.

[0032] The parallelism of the systolic array unit is reflected in the simultaneous computation of all event nodes within each iteration. One possible hardware implementation is based on the number of event nodes. Configure an equal number of processing units, each responsible for calculating the current round's expanded value for one event node. Processing units transmit the previous round's expanded value to each other via a pulsed data path. (Continue using...) In the embodiments, the entire Round iteration only requires The entire parallel computation can be completed in a single clock cycle-level pulse, achieving a speedup of up to [percentage missing] compared to the serial implementation of a general-purpose processor. This is an order of magnitude greater than the previous level. This is the key benefit of sinking beat analysis down to the edge acceleration core.

[0033] After each round of iterations, for each event node Seeking ;in This is the cyclic average estimate of the event node. The value of reflects the path taken from the starting node. Steps and Passages The two paths converge at... The minimum cycle average time of the directed cycle that must be contained between them. For all Take the minimum because The minimum estimated time for each cycle must be the lower bound of the average time spent in all possible loops to ensure that the result obtained when maximizing the time spent in the node dimension corresponds to the maximum value of the average time spent in the entire graph. In the aforementioned 8-node closed-loop embodiment, since the beat dependency graph itself contains only one main loop, the average estimated time for all event nodes converges to the same value. Substituting the side length... Summation seconds, then divide by the number of nodes The cyclic average estimate is approximately Second.

[0034] The largest cyclic average estimate among all event nodes is retained as the cycle time. In the aforementioned embodiment, the cycle time is 7.225 seconds, meaning that under the current equipment configuration and workflow, it takes at least 7.225 seconds to complete one full handling cycle in the automated storage and retrieval system. In more complex embodiments, if the cycle time dependency graph contains secondary loops generated by shared resources in addition to the main loop—for example, a resource constraint loop formed by multiple stacker cranes sharing the same aisle—the cyclic average estimates of each event node will no longer be the same, and the cycle time will correspond to the secondary loop with the tightest resource constraint.

[0035] A closed sequence of event nodes formed by backtracking step-by-step from the event nodes corresponding to the beat cycle through the pointers of the predecessor event nodes is marked as a critical beat loop. The backtracking process is implemented using a stack structure: the event node corresponding to the beat cycle is pushed onto the bottom of the stack; the current node is popped from the top of the stack, and the process is backtracked one step up from the predecessor event node pointed to by its predecessor event node pointer, pushing the predecessor event node onto the top of the stack; when a predecessor event node already exists in the stack, the backtracking stops, and the subsequence from the predecessor event node to the current top of the stack constitutes the critical beat loop. Each backtracking process requires only one memory access, with a time overhead on the sub-microsecond scale. In the aforementioned embodiment, the critical beat loop is the numbered... to The main loop itself, which consists of all the event nodes.

[0036] Each storage location corresponds one-to-one with a physical storage cell in the automated warehouse. In a typical implementation of an automated warehouse, the automated warehouse is... Row List The high-bay racking array has a total of [number] storage locations. Each storage location is registered with its corresponding physical storage cell, including its floor height color, maximum weight limit color, and status color. The floor height color is... The optional implementation method is to record the layer height of the storage cell, which is longer than a certain field. The physical layer is divided into The first layer height group: To the The layer is a low-order group, the first To the The layers are grouped into middle and low-level groups, the first... To the The layers are grouped into middle and high-level groups, the first... To the The layers are grouped at higher levels, based on differences in energy consumption and stability during equipment transport to each layer. Weight limits are determined by color. A field longer than a certain length records the weight class of the cargo compartment. An optional implementation method divides the weight class into... Levels, each level mapped to A weight limit value, for example Level mapping to kilogram, Level mapping to kilogram, Level mapping to kilogram, Level mapping to kilogram, Level mapping to Kilograms. The mapping relationship is pre-calibrated based on the load-bearing test data of the three-dimensional warehouse steel structure and remains static during the operation of the colored Petri net. The state color position is based on... Record the current status of the storage compartment using a field of a specified length, and retrieve the status from empty, occupied, or locked. There are three states for a storage space: Empty (no pallet) and Occupied (pallet already in use); Locked (pallet already in use); and Locked (pallet already in use). The Locked state prevents duplicate allocation of the same storage space during concurrent multi-task scheduling. This can be achieved through Boolean checks on the status color, without requiring an additional locking mechanism.

[0037] The idle status of the stacker crane, shuttle car, and elevator corresponding to the equipment-ready warehouse. In the aforementioned automated storage and retrieval system embodiment, equipped with... Stacker crane Taiwan shuttle bus The hoist, then the equipment ready warehouse has a total of Each equipment readiness unit registers the color position of its assigned roadway. The color position of the assigned roadway is determined by... The longer field records the lane number primarily served by the device; the lane number is taken from... to Scope. When a device is in an idle state, the corresponding device ready library contains [number of devices]. Each token has a lane color number that corresponds to the physical lane affiliation of the device. When a device enters a working state, the token in its corresponding device ready library is moved and removed. The removal of a token indicates that the device resource has been occupied.

[0038] The batch queue warehouse records pending pallets in the order they arrive. In one optional implementation, each batch queue warehouse corresponds to one inbound port. Whenever... When a pallet arrives at the inbound gate, the acceleration core allocates resources to that pallet. Each new token is pushed into the corresponding batch queue, and the batch arrival order of the tokens is determined by color. A field longer than a certain length records the global arrival sequence number of the pallet within the current business day. For example, if a pallet is the [number]th arrival of a certain day... When a batch of goods arrives at the warehouse entrance, its batch arrival sequence color value is [value missing]. The existence of batch arrival order color bits allows subsequent enabling rules to directly determine whether a token is at the head of the queue, without the need to maintain an additional queue pointer.

[0039] The task handover warehouse includes an inbound receiving warehouse, a transfer handover warehouse, a replenishment handover warehouse, and an outbound release warehouse. Each type of task handover warehouse corresponds to a specific physical workstation in the automated warehouse. In one optional implementation, the automated warehouse is equipped with... One warehouse entrance, One warehouse transfer handover area, A replenishment handover area There are one outbound port, so the task handover warehouse has a total of The introduction of the task handover warehouse allows colored Petri nets to uniformly depict inbound, outbound, replenishment, and restocking processes. This allows for workflows without the need to build a separate Petri net for each workflow.

[0040] The tokens circulating in the colored Petri net are no less than The fixed-length field of the token records the color code of the goods identification, the color code of the batch arrival order, the color code of the pallet weight, the color code of the pallet layer height, and the color code of the target aisle. In a specific implementation, the field layout of the token is: goods identification color code. Bit, batch arrival order, color bit Bit, tray weight, color bit Bit, tray layer height color bit Bit, target lane color bit Bit, plus Bit reservation, total Bit, by Memory is stored with fields aligned to a fixed length. Using fixed-length fields allows systolic array units to access on-chip random access memory in parallel steps, avoiding the overhead of variable-length decoding. The delay for each transport transition is assigned by the clock cycle, meaning that each excitation... Sub-transportation changes, advancement of three-dimensional warehousing. A complete work cycle.

[0041] The goal of breadth-first incremental reachability expansion is to start from the root location and filter out all reachable and available candidate storage locations according to enabling rules for the current task. The term "incremental" is introduced to distinguish it from the full-state-space reachability expansion of traditional Petri nets. Traditional methods encounter state-space explosion on large-scale Petri nets, especially when dealing with locations containing… In this embodiment of a single-location warehouse, the total space size can reach [size missing]. The magnitude is too large to be directly enumerated; the incremental approach only expands the local subgraph related to the current task, which can be completed in milliseconds.

[0042] The acceleration core uses the warehouse location of the token corresponding to the current WMS task as the root warehouse location. The root warehouse location is determined according to the following rules: when the task is an inbound task, the root warehouse location is the warehouse location of the corresponding inbound pallet token in the batch queue; when the task is a transfer task, the root warehouse location is the warehouse location of the corresponding pallet token in the original storage location; when the task is a replenishment task, the root warehouse location is the warehouse location of the corresponding replenishment pallet token in the replenishment handover warehouse. The physical meaning of the token starting point is different for the three types of tasks: the pallet for an inbound task appears in the batch queue corresponding to the inbound entrance; the pallet for a transfer task occupies an existing storage location; and the pallet for a replenishment task remains in the replenishment handover area. Therefore, different root warehouse locations are used to ensure that reachability deployment is consistent with the actual operation process.

[0043] Accelerate nuclear maintenance A first-in-first-out queue carried by a lookup table unit and Zhang accesses the bitmap. The first-in-first-out queue is implemented using a circular buffer of on-chip random access memory, with the depth preset according to the scale of the three-dimensional warehouse. Optional implementation methods include... The access bitmap records whether each transport transition has been queued in bitmap form, avoiding infinite loops or redundant calculations caused by duplicate queuing. The total number of bits in the access bitmap is equal to the number of transport transitions, which is approximately [value missing] for the aforementioned automated warehouse embodiment. Bit, approximately kilobytes, fully on-chip resident.

[0044] The specific steps of the unfolding process are as follows. First, all transport transitions connected by the output arcs traversed by the root library are enqueued as transport transitions to be determined, and the corresponding bits are set in the access bitmap. Transport transitions to be determined are retrieved one by one from the head of the queue, and the lookup table unit performs a line-by-line matching on the retrieved transport transitions according to the determination order of the enable rules. The hardware implementation of line-by-line matching is to... The enabling rule is coded as follows Each lookup table entry stores one rule's decision logic. For each retrieved transport transition, the table is searched sequentially, only when... All rules return Only after the transfer and change are completed will the next step of processing begin; as long as any one of them returns... Immediately skipping subsequent decisions significantly reduces unnecessary computation. The rules are arranged in ascending order of cost in the current embodiment, thus enabling rapid short-circuiting in cases with a high probability of failure.

[0045] For a transport transition that matches all rules, its successor warehouse connected via the output arc is read. If the successor warehouse is a storage location and its status color is empty, it is registered as a candidate storage location. If the successor warehouse is of other types, the next transport transition connected via the output arc is enqueued as a transport transition to be determined when the access bit is in the initial state, and the corresponding position is set. Storage locations are the destination of the transport task and are included in the candidate set once discovered; other types of warehouses are intermediate state nodes and require further expansion to reach the storage location.

[0046] refer to Figure 2 In this embodiment of the invention, the three-dimensional warehouse uses a high-level rack array of 10 layers and 12 columns to form a complete longitudinal section of the aisle. Figure 2 The left side has four levels of height grouping color bands, corresponding to low level group, low-middle level group, mid-high level group and high level group respectively. Figure 2 The bottom horizontal ground line represents the bearing base of the aisle. There is an inlet at the left end and an outlet at the right end along the ground line. The handling equipment arranged along the longitudinal section of the aisle is drawn directly according to its physical working position: the stacker crane is shown as a column that runs through the entire height of the rack, supporting a horizontal beam loading platform that can be raised and lowered along the column, with the loading platform at the 4th level; the shuttle is shown as a rectangular trolley, stationed on the horizontal track at the 6th level; the hoist is shown as an independent vertical aisle, located at the right end of the rack array, with its current level at the 8th level. Figure 2 Each cell in the system has a fill style corresponding to one state color: an unfilled cell is empty, a light-filled cell is occupied, and a dark-filled cell is locked. Figure 2The candidate storage locations obtained after expanding the current batch incremental accessibility are indicated by dashed circles outside the three storage locations, with numbers C1, C2, and C3, located in the 6th column of the 4th layer, the 9th column of the 6th layer, and the 4th column of the 8th layer, respectively.

[0047] The initial number of expansion layers for incremental reachability expansion is taken as the number of event nodes in the critical beat loop, following the example described above. The initial number of unfolded layers was chosen based on the fact that the number of event nodes in the critical cycle loop reflects the completion of the automated warehouse. The minimum number of operational stages required for a complete handling task; expanding according to this number of layers ensures at least [reaching / reaching] [the target]. The total number of warehouse locations reachable within a complete handling cycle is the reasonable initial value for the project. If the number of candidate warehouse locations is less than the preset candidate number threshold and there are still handling changes to be determined in the first-in-first-out queue, the expansion continues at a preset step size until any of the candidate number threshold, reachable boundaries, or preset maximum search layers are reached. Stop when one condition is met. In an optional implementation, the candidate number threshold is set to... Expand step size Maximum search depth .set up The use of multiple stopping conditions combined with "OR" logic is intended to simultaneously consider the candidate set size, graph structure boundaries, and computational budget. The constraints across multiple dimensions ensure that the deployment process is adaptive under different scales of automated warehouses and varying task densities. For example, during a peak business period, the vacancy rate of the automated warehouse is low, based on the initial... Layer expansion may only yield approximately There are several candidate warehouse locations; at this point, they are determined by step size. Continue to expand to the , Layers, until the number of candidates reaches Or trigger the maximum search depth Conversely, when the vacancy rate is high during off-peak periods, the initial... The system can reach hundreds of candidate storage locations within a single layer. At this point, the search should be stopped immediately based on the candidate quantity threshold to avoid unnecessary expansion.

[0048] The acceleration core writes the layer height color, weight limit color, aisle steps from the corresponding workstation in the task handover warehouse, batch arrival order color of the first queue of the batch queue to which it belongs, and the event node sequence number corresponding to the handling change triggered when this warehouse is the target warehouse, as the cycle time node sequence number of this warehouse, into the on-chip random access memory and outputs it to step 3. The design intent of the class attributes differs: the layer height color and weight limit color characterize the physical constraint attributes of the storage compartment; the aisle step count characterizes the spatial distance attribute of the storage compartment relative to the task handover station; the batch arrival order color characterizes the temporal priority attribute of the storage compartment in the current task context; and the associated cycle node number establishes an explicit association between the storage compartment and the cycle dependency graph, facilitating the graph attention reinforcement learning decision module in subsequent steps to simultaneously utilize space, time, and cycle on the heterogeneous graph. Dimensional information is used for comprehensive decision-making. Attributes are written to on-chip random access memory in a compact structure format, with an optional layout for each candidate compartment. Bytes, assuming the number of candidates is Then the total occupancy The delay in transmitting bytes to the next step can be controlled within a few microseconds.

[0049] refer to Figure 3 In this embodiment of the invention, the heterogeneous graph has four types of nodes: task nodes are shown as diamonds and denoted as T; each storage cell in the candidate storage location set is mapped to a storage location node and shown as a circle, denoted as S1 to S5; each handling device in the automated warehouse is mapped to a device node and shown as a square, denoted as D1 to D3, corresponding to stacker crane, shuttle car, and hoist, respectively; each event node on the critical cycle loop is mapped to a cycle node and shown as a hexagon, denoted as B1 to B4. Task node T is connected to all storage location nodes S1 to S5; storage location nodes S1 and S2 are connected to device node D1, storage location nodes S3 and S4 are connected to device node D2, and storage location node S5 is connected to device node D3; each device node is connected to the cycle node on the critical cycle loop corresponding to the current event node number; cycle nodes are connected according to the chain connection relationship of the critical cycle loop. Figure 3 Taking storage location node S3 as the central node, the information gathering arrows pointing to S3 from task node T, equipment node D2, and adjacent storage location nodes S2 and S4 during the graph attention aggregation process are shown. The normalization coefficients marked next to the arrows are 0.42, 0.31, 0.15, and 0.12, respectively, reflecting the differences in attention the graph attention mechanism pays to different neighboring nodes.

[0050] Heterogeneous graph co-location Class node. (The first...) The class is a storage location node, mapped from each storage cell in the candidate storage location set. Following the previous embodiment, the size of the candidate storage location set is... Then the number of storage location nodes is . No. The class is a task node, mapped from the current task, and only one is set for each decision. Each task node is responsible for collecting global information at the candidate set level. The class is a device node, mapped from various handling equipment in the automated warehouse, following the example described above. Stacker crane Taiwan shuttle bus Unit hoist, total Each device node. The class is a beat node, which is mapped from each event node on the critical beat loop, following the previous embodiment. Each beat node. Total number of class nodes The size is moderate, making it easy to complete all aggregation calculations on the acceleration core with millisecond-level latency.

[0051] The edges of the heterogeneous graph are divided into two parts. The first part is directly translated from the input-output relationships of the transport transitions in the colored Petri net. Specifically, for each transport transition, all its input locations and all its output locations are identified. If any of the input or output locations contain elements related to the above... For the entity corresponding to the class node, add [something] between the corresponding entity nodes. Edges. This translation method seamlessly transfers the causal dependencies depicted by colored Petri nets to heterogeneous graphs, enabling the graph attention reinforcement learning decision module to learn decision patterns from topologies that have already been filtered by enabling rules, thus avoiding the learning of redundant or infeasible actions.

[0052] Relying solely on edges derived from transport transitions may miss certain cross-type semantic relationships; therefore, the following cross-type edges are supplemented. Firstly, there are edges connecting storage location nodes to equipment nodes with the same color position in their assigned lanes. These edges characterize the physical reachability constraint of "which equipment can transport to which storage location." For example, the color position of a shuttle's assigned lanes is... Then the candidate storage location is located at the first position in the set. All cargo location nodes in the aisle are connected to the corresponding equipment node of the shuttle. Secondly, the equipment node is connected to the cycle node corresponding to the current event node number on the critical cycle loop. The current event node number refers to the event node number that this equipment is currently undertaking or will soon undertake in the cycle dependency graph. For example, if a stacker crane is in the stacker crane loading and traveling phase, then the equipment node corresponding to the stacker crane is connected to the number in the cycle dependency graph. The event nodes are connected to the corresponding tick nodes. These edges embed the real-time operating status of the equipment into the global view of the tick constraints. Thirdly, the task nodes are connected to all the location nodes in the candidate location set. The task nodes can act as aggregation centers, enabling... In the round aggregation, information is directly gathered from all candidate storage locations, making it easier for the strategy leader to evaluate the relative merits of each candidate storage location from a global perspective. In one optional implementation, similar edges can be added between cycle nodes according to the chain connection of the key cycle loop to strengthen the propagation of cycle constraints between cycle nodes; this implementation is suitable for scenarios with long key cycle loops.

[0053] The initial feature vectors of each type of node contain continuous and categorical components. This division is based on the fact that continuous components carry physical quantities with natural numerical magnitude relationships, while categorical components carry index values ​​with discrete categorical attributes; the two should not be confused numerically. If categorical components are directly fed into the graph attention mechanism as continuous values, adjacent values ​​may be misinterpreted as similar physical quantities, such as when numbering lanes. with lane number They are considered to be two adjacent lanes, but in reality, the lane numbers themselves do not have the physical meaning of such numerical proximity.

[0054] Continuous components are normalized dimension-wise based on the maximum value of each dimension. Dimension-wise normalization means that for each dimension of the continuous component, the maximum absolute value across all candidate locations in the current batch is taken as the normalization factor for that dimension. All values ​​in that dimension are then divided by the normalization factor, thus compressing the range of values ​​for each dimension to a minimum. Range. Following the aforementioned example of automated warehousing, assuming the current batch... The original maximum value of "number of aisle steps from the corresponding workstation in the task handover warehouse" for each candidate storage location is [value missing]. The smallest is Then the normalization factor for this dimension is taken as Original value Normalized to Original values Normalized to Dimensional normalization eliminates the interference of dimensional differences between different physical dimensions on graph attention calculation; batch normalization makes the model adaptive to changes in the scale of the automated warehouse.

[0055] The continuous component consists of the number of aisle steps and the cyclic average estimate of the distance to the workstation corresponding to the task handover warehouse. The physical meaning of the number of aisle steps is: the Manhattan distance from the grid of the workstation corresponding to the task handover warehouse to the grid of the candidate storage cell is the aisle step number when the aisles, layers, and columns of the automated warehouse are modeled as discrete grids. (Continuing from the previous...) Row List In a multi-level automated storage system implementation example, if the inlet is located at the first... Ranked The candidate storage cell is located in the first column. Ranked Liede For each floor, the number of steps in the alleyway is taken as... The cyclic average estimate is taken from the cyclic average estimate of the corresponding event node calculated in step 1 when the candidate storage cell is used as the target storage location. Introducing the cyclic average estimate as a continuous component of the storage location node allows the graph attention reinforcement learning decision module to see not only the static spatial attributes of the storage location when evaluating candidate storage locations, but also the potential impact of the storage location on the overall system rhythm.

[0056] The categorical components are expressed using one-hot encoding. One-hot encoding refers to mapping each category value to a... Only those at the index position in this category All other positions are A binary vector. One-hot encoding avoids incorrectly assigning numerical order relationships between category values. Category components include layer height color position, maximum weight color position, pallet weight color position, assigned aisle color position, batch arrival order color position, current event node number, location number, and associated cycle node number. Item. The one-hot encoding dimensions for each category component are, in order: layer height, color bit. Dimensions, weight limit, color position Dimensions, tray weight, color position Wei, the color position of the lanes under its jurisdiction Color position of batch arrival sequence Dimension, current event node sequence number Dimension, Position Number Dimension, sequence number of the beat node The location sequence number is used to enable the graph attention mechanism to distinguish candidate locations with the same attributes other than location when there is no natural order for similar nodes. The batch arrival order color is used... Unique-hot encoding rather than The original bit value is used because the model only needs to identify whether the tray is near the head of the queue in subsequent aggregation, and does not need to interpret the numerical value. and The precise difference between them.

[0057] Different types of nodes hold different subsets of category components. Location nodes hold the layer height color, weight limit color, assigned aisle color, location sequence number, and associated cycle node sequence number; task nodes hold the pallet weight color and batch arrival sequence color; equipment nodes hold the assigned aisle color and current event node sequence number; cycle nodes hold the associated cycle node sequence number. For category component locations not held by a node, all corresponding one-hot encoded dimensions are set. This sparse alignment method makes The initial feature vectors of class nodes have the same dimensional structure, which facilitates parallel processing on the systolic array unit with a unified data path; at the same time, the model can implicitly identify the type of node by whether the class component is non-zero, without the need for additional type embedding.

[0058] The number of rounds in multi-round neighbor aggregation is equal to the number of event nodes in the critical beat loop. Following the previous embodiment, this value is taken as... This means that information passes through heterogeneous graphs. Round propagation. The consideration for choosing this number of rounds is that the number of event nodes in the critical cycle loop characterizes the completion of the three-dimensional warehouse. The number of operation stages experienced in a complete handling cycle is used to align the aggregation wheel number with the number of stages, so that the storage location nodes can pass through... The aggregation rounds receive information from all the beat nodes on the critical beat loop, without the beat constraints being insufficiently transmitted due to too few aggregation rounds, or the feature vectors converging and degenerating due to too many aggregation rounds.

[0059] In each round of aggregation, perform the following operations on each node.

[0060] The current feature vector of this node is The fully connected mapping of paths based on shared parameters according to node categories yields query vector, key vector, and value vector, respectively. The parameter matrices of the fully connected path mapping are denoted as follows: , , subscript Indicates the node category index. These correspond to the location node, task node, equipment node, and cycle time node, respectively. Sharing parameters by node category means that nodes of the same category share the same set of parameters. , , Different types of nodes hold different parameters. Sharing parameters by node type balances type differentiation capability and sample efficiency. Let the nodes... The current feature vector is Its category index is Then the query vector, key vector, and value vector of this node are respectively ;in , , All indivual 3D column vector, For the hidden layer dimension, the optional implementation method is to take... . , , All A learnable parameter matrix of dimension 1, initialized with a mean of 1. Standard deviation is It follows a normal distribution.

[0061] The pulsating array unit takes the sum of the component-wise products of the query vector of this node and the key vector of each neighboring node as the similarity score of the corresponding neighboring node. Let node... The neighbor set is For each The similarity score is ;in For nodes Focus Nodes The original score, For query vector The Each component Key vector The Each component For the hidden layer dimension. The sum of component-wise products is equivalent to the dot product of the query vector and the key vector. The systolic array unit consists of... It consists of several multiplication and accumulation units, each of which is responsible for... The multiplication and summation of each component result in a delay of 1 / 2 for calculating the overall similarity score. It takes only one clock cycle, far less than the serial dot product on a general-purpose processor.

[0062] The normalized coefficients of all similarity scores for this node are obtained by applying softmax along the neighbor dimension. The normalized coefficients are: ;in For nodes For neighboring nodes The normalized attention coefficient, It is an exponential function with the natural constant as its base, and the denominator iterates through the nodes. All neighbors to ensure Performing softmax along the neighbor dimension rather than along the entire graph node dimension ensures that the attention distribution of each node is normalized only among its local neighbors, consistent with the local message passing semantics of heterogeneous graphs.

[0063] The value vectors of all neighboring nodes of this node are linearly combined according to their corresponding normalization coefficients. The result is then added component-by-component to the previous round's feature vector of this node, and finally passed through a linear rectified activation function to obtain the current round's feature vector of this node. The current round's feature vector is... ;in For nodes The feature vector after this round of aggregation For nodes Previous round feature vectors, For neighboring nodes The value vector, This is the linear rectified activation function acting on each component one by one. (Introduction) The direct addition terms are used to form residual connections and avoid gradient vanishing after multiple rounds of aggregation; the linear rectified activation function introduces nonlinearity into the model, enabling it to fit complex decision functions.

[0064] After multiple rounds of neighbor aggregation, mean pooling is performed on the final feature vectors of all storage location nodes to obtain graph-level state features. Let the final... After the aggregation of the wheel, the storage location node The feature vector is The size of the candidate storage location set is Then the graph-level state characteristics are ;in for The column vectors represent the current state of the heterogeneous graph globally. Pooling is performed only on the location nodes, not all nodes, because the final decision-making object is the candidate location. Information from other types of nodes has already been aggregated and integrated into the final feature vector of the location nodes through multiple rounds, and repeated pooling would introduce redundancy.

[0065] The strategy head processes the final feature vector of each storage location node in the candidate storage location set. A discrete probability distribution is obtained by cascading a fully connected layer and a softmax layer. The specific calculation is as follows: first, for each storage location node... The final feature vector through The fully connected layer obtains the action score for that storage location. ,in for dimensional column vector, scalar, superscript This represents the vector transpose; then the scores for all actions are calculated. The discrete probability distribution is obtained by feeding the entire data into the softmax layer. ;in For discrete probability distributions at the th The probability value for each candidate storage location. The size of the candidate storage location set. The softmax layer output satisfies... and This constitutes a valid discrete probability distribution.

[0066] Value head to graph-level state features The scalar value estimate of the current state of the fully connected layer is obtained. Specifically, the calculation is as follows: ,in For scalar value estimation, for dimensional column vector, For scalar, This represents graph-level state characteristics. Value estimation reflects the mathematical expectation of the cumulative reward that can be obtained by continuing to make decisions according to the current strategy under the current heterogeneous graph state, and is used as a benchmark for advantage estimation in proximal policy optimization training.

[0067] The acceleration kernel selects the location with the highest probability in the discrete probability distribution as the location allocation result for this batch of goods. The specific method is as follows: ,in This is the index value of the candidate storage locations selected in this batch. In execution mode, maximum probability selection is used to ensure decision determinism, while in the training phase, class sampling is used to preserve exploratory capabilities.

[0068] Accelerate nuclear general The corresponding physical coordinates of the storage compartments, including the lane number, column number, and layer number. Each field is encapsulated into a location allocation result write-back message, which is then written back to the WMS via a time-sensitive network. The specific method of time-sensitive network write-back utilizes its reserved time slots for time-sensitive transmission: the acceleration core reserves a fixed number of time slots within each time-sensitive network cycle. One high-priority time slot is used for write-back message transmission. The time slot length is configured according to the maximum message length. An optional implementation method is... Microseconds. By utilizing the high-priority time slots of time-sensitive networking, the end-to-end transmission delay of the location allocation results can be strictly controlled to the sub-millisecond level. This is closely coordinated with the start-up timing of stacker cranes, shuttle cars, and hoists to avoid equipment idleness caused by message delay jitter.

[0069] After receiving the location allocation result message, the WMS issues action instructions to the corresponding handling equipment according to the physical coordinates of the storage compartments carried in the message. The corresponding handling equipment then performs the inbound, relocation, or replenishment actions. The selection of specific equipment is completed by the WMS based on its scheduling strategy, while the acceleration core only undertakes the location decision-making task based on the heterogeneous map status.

[0070] In training mode, accelerated class sampling is performed on the discrete probability distribution to obtain training actions. The specific method of class sampling is as follows: Generate on interval A uniformly distributed random number The summation of discrete probability distributions yields , find Established index This serves as the index for the selected storage location during training. The agreement is as follows Class sampling allows training samples to cover actions with low probability but potentially better outcomes, preventing the policy from getting stuck in local optima.

[0071] The acceleration kernel will train the action and the old policy probability of the training action in a discrete probability distribution. Current graph-level state features And scalar value estimation of the value head Groups of data are written to the experience cache in the on-chip random access memory. The experience cache is implemented using a circular queue, and in an optional implementation, the depth of the circular queue is [details to be filled in]. , The preset trajectory length can be selected. Each slot is stored in a compact structure format. Group of empirical data, structure containing The fields correspond to the index value of the training action, the probability of the old policy, the graph-level state features, and the value estimate, respectively.

[0072] The acceleration core writes the training actions back to the corresponding transport transitions in the colored Petri net to trigger the corresponding transport transitions and complete the token transfer. After the token transfer, the acceleration core sets the state of the target storage location to a locked state and updates the edge length of the corresponding directed edge in the cycle time dependency graph. Setting the locked state prevents subsequent concurrent tasks from repeatedly occupying the same storage location; the edge length update is based on the timestamped sampling sequence actually collected from the fork pose sensor, track odometer, and floor encoder after the training action is executed, and is recalculated according to the actual action time calculation method described above.

[0073] The pulsating array unit re-executes the parsing on the updated clock dependency graph to obtain the actual clock cycle for this batch. The actual cycle time of this batch reflects the bottleneck cycle time of the automated warehousing after the training actions are executed, and is directly related to the quality of the storage locations selected in the training actions: selecting storage locations that require long-distance handling or conflict with cycle time constraints will increase the actual cycle time of this batch, while the opposite will keep it stable or even decrease it.

[0074] The acceleration unit uses the sum of the opposite of the actual cycle time of this batch and the target location lock-in success indication as the immediate feedback. The immediate feedback is... ;in For the first Instant returns for each batch For the first The actual cycle time of each batch Set the value for the indicator item indicating successful locking of the target storage location. or , The scaling factor for locking the successful term can be selected as follows: A smaller actual cycle time indicates higher system throughput; therefore, taking the opposite value aligns with the optimization direction. The lock success indicator provides a negative signal when an action fails due to contention or anomalies. Immediate rewards are appended to the experience cache for the slot corresponding to the training action. Field.

[0075] Proximity policy optimization coprocessor self-experience cache read continuation The probability of the old strategy, immediate return, value estimation and graph-level state characteristics of each batch. To preset the trajectory length, the aforementioned optional implementation method is used. .

[0076] Based on time difference residuals and preset discount factors The generalized dominance estimate is obtained by recursion over each batch. The specific recursion method is as follows: first, according to... ; Calculate the first The time difference residuals of each batch, among which For time difference residuals, For immediate returns, For the first Value estimation of each batch For the first Value estimation of each batch Preset discount factors can be selected. Press again. ; calculate the number in reverse order Generalized advantage estimation for each batch ,in The attenuation factor for generalized dominance estimation can be selected as follows: recursively from Let's move forward. The agreement is as follows Introducing a discount factor This is to reflect the present value of future returns through discounting, avoiding infinite accumulation; a decay factor is introduced. A trade-off is then made between bias and variance. Pick The variance of the time advantage estimate is large. Pick The time deviation is large, so take In practice, it can achieve a good balance in most scenarios.

[0077] Construct the policy ratio by comparing the old policy probability with the corresponding current policy probability. The current policy probability refers to the probability of the policy network that has been updated several steps in the current step. The probability value corresponding to the training action obtained by re-forward reasoning in each batch state. The strategy ratio is ;in For the first The strategy ratio for each batch, For the current strategy probability, This represents the old policy probability. The policy ratio reflects the relative change in the probability of the same action being selected before and after the policy update, and is a core quantity in the objective function of near-end policy optimization.

[0078] The truncated target is obtained by truncating the policy ratio and multiplying it by the generalized advantage estimate. The truncated target is... ;in For the first Each batch of cutoff targets, To be Limited to the range truncation function The preset cutoff range can be selected. Truncation of the policy ratio can suppress the destructive perturbation of the policy by a single update; limiting the ratio to a certain value... The interval allows the strategy to improve stably with small step sizes. Then take... The smaller value compared to the truncated version is protected from truncation when the advantage estimate is positive, and allows for a larger negative update when it is negative.

[0079] The truncated target is used to perform proximal policy optimization updates on all learnable parameters in the graph attention reinforcement learning decision module. All learnable parameters include... Each class node , , common Group parameter matrix, and strategy head , With value head , Update direction The gradient ascent direction of the mean of the batch cutoff target with respect to the parameters is determined, and the update step size is controlled by the learning rate. An optional implementation method uses a learning rate of [missing information]. The Adam optimizer is employed. In one optional implementation, the objective function may also include a mean squared error term for the value estimation. With policy entropy regularization term To simultaneously optimize the accuracy of value estimation and the exploratory nature of strategies, among which For the value objective, the scaling coefficients of the two additional terms can be taken as follows: and .

[0080] refer to Figure 4 , Figure 4 The horizontal axis represents the number of unfolded layers, with scales ranging from 1 to 20; the vertical axis represents the number of candidate storage locations, with scales ranging from 0 to 40. Figure 4Three curves are shown, each corresponding to a typical operating condition of automated warehousing: Curve 1 corresponds to peak hours with a vacancy rate of 5%, marked with square dots; Curve 2 corresponds to off-peak hours with a vacancy rate of 20%, marked with circular dots; and Curve 3 corresponds to low-peak hours with a vacancy rate of 45%, marked with triangular dots. Under these three curves, the number of candidate storage locations monotonically increases with the number of deployed layers and remains saturated after reaching a preset threshold: the low-peak condition reaches the threshold of 32 at layer 5 and saturates thereafter; the off-peak condition reaches the threshold at layer 8; and the peak condition increases slowly until reaching the threshold at layer 15. The differences between the three curves reflect the impact of the vacancy rate on the number of layers required for incremental accessibility deployment: the lower the vacancy rate, the more layers need to be deployed. Figure 4 The number of candidates is marked by a horizontal dashed line with a threshold of 32, and the initial number of unfolded layers is marked by a vertical dashed line with a threshold of 8.

[0081] This invention is not limited to the specific embodiments described above. The invention extends to any new feature or combination disclosed in this specification, as well as any new method or process step or combination disclosed herein.

Claims

1. A method for dynamic location allocation in a high-density automated warehouse (AS / RS) oriented towards a WMS (Warehouse Management System), wherein stacker cranes, shuttle cars, and hoists are arranged along the aisle direction in the AS / RS, and the edge nodes of the AS / RS are equipped with field-programmable gate array (FPGA) acceleration cores, which are interconnected with the WMS via a time-sensitive network. The method is characterized by... Includes the following steps: Step 1: The acceleration core constructs a beat dependency graph based on the current transport instructions issued by WMS, with transport motion as event nodes and action connection relationships as directed edges, and parses the beat cycle and key beat loops on the beat dependency graph. Step 2: Accelerate the construction of a colored Petri net. The colored Petri net includes storage locations, equipment-ready storage locations, batch queue storage locations, task handover storage locations, and handling transitions. The acceleration core takes the storage location where the token corresponding to the current WMS task is located as the root storage location, and performs incremental reachability expansion in a breadth-first manner according to preset enabling rules. During the expansion process, the storage locations reached and whose status color is empty are collected into a candidate storage location set. Step 3: Accelerate the construction of the graph attention reinforcement learning decision module and take the heterogeneous graph as input. Each grid in the candidate storage location set in the heterogeneous graph is used as a storage location node, the current task is used as a task node, each handling equipment is used as a device node, and each event node on the critical cycle loop is used as a cycle node. After performing multiple rounds of neighbor aggregation on the heterogeneous graph, the policy head outputs the discrete probability distribution on the candidate storage location set. The acceleration kernel selects the storage location with the highest probability in the discrete probability distribution as the storage location allocation result for this batch and writes it back to the WMS through the time-sensitive network. The WMS then drives the corresponding handling equipment to perform inbound, transfer or replenishment actions. In step 2, incremental reachability deployment is performed as follows: The acceleration core maintains a first-in-first-out queue carried by a lookup table unit and an access bitmap. All transport transitions connected to the root warehouse via the output arc are enqueued as transport transitions to be determined and their corresponding positions are set in the access bitmap. Transport transitions to be determined are retrieved one by one from the head of the queue. The lookup table unit performs a match on each retrieved transport transition according to the determination order of the enabling rules. For transport transitions to be determined that have passed all rules, the subsequent warehouses connected to them via the output arc are read. If the subsequent warehouse is a storage location and its status color is empty, it is registered as a candidate storage location. If the subsequent warehouse is a warehouse of other types, the next layer transport transition connected to this subsequent warehouse via the output arc is enqueued as a transport transition to be determined when the access bit is in the initial state and its corresponding position is set. The initial number of expansion layers for incremental reachability expansion is taken as the number of event nodes in the key cycle loop. When the number of candidate storage locations is less than the preset candidate number threshold and there are still pending handling changes in the first-in-first-out queue, the expansion continues at the preset expansion step size until any one of the following conditions is met: candidate number threshold, reachable boundary, or preset maximum search layer. The acceleration core writes the layer height color, weight limit color, aisle steps from the corresponding workstation of the task handover storage location, batch arrival order color of the first storage location in the batch queue, and the event node sequence number corresponding to the handling change triggered when this storage location is the target storage location as the cycle node sequence number of this storage location, and outputs them to step 3.

2. The method according to claim 1, characterized in that: The stacker crane is equipped with a fork position sensor, the shuttle is equipped with a track odometer, and the hoist is equipped with a floor encoder. The timestamped sampling sequences of the fork position sensor, track odometer, and floor encoder are uploaded to the acceleration core along the time-sensitive network to share a unified spatiotemporal reference. The accelerated core includes a systolic array unit, a lookup table unit, on-chip random access memory, and a near-end policy optimization coprocessor.

3. The method according to claim 1, characterized in that: In step 1, the event nodes include stacker crane picking fork extension, stacker crane carrying movement, stacker crane placing fork extension, hoist inter-level switching, shuttle car track movement, shuttle car picking fork extension, shuttle car placing fork extension, and task handover events. The task handover event consists of one type corresponding to the task type of the handling instruction in the inbound port reception, transfer handover, replenishment handover, or outbound port release. All event nodes are numbered sequentially according to the delivery order declared in the handling instruction. The action connection relationship includes the chain connection from stacker crane picking fork extension to stacker crane carrying movement, stacker crane carrying movement to stacker crane placing fork extension, stacker crane placing fork extension to hoist inter-level switching, hoist inter-level switching to shuttle car track movement, shuttle car track movement to shuttle car picking fork extension, shuttle car picking fork extension to shuttle car placing fork extension, shuttle car placing fork extension to task handover event, and task handover event returning to stacker crane picking fork extension.

4. The method according to claim 2, characterized in that: In step 1, the length of each directed edge is the greater of the corresponding action duration declared by the transport instruction and the corresponding actual action duration measured at the start and end times of the action based on the timestamped sampling sequence in the most recent complete action cycle. The adjacency list of the beat dependency graph is stored in on-chip random access memory.

5. The method according to claim 2, characterized in that: In step 1, the beat cycle and key beat loop are resolved by the pulsating array unit as follows: Let N be the total number of event nodes in the beat dependency graph; select the event node with the smallest number as the starting node; initialize the 0th round expansion value of each event node as follows: the 0th round expansion value of the starting node is 0, and the 0th round expansion value of the other event nodes is a preset minimum value; perform N rounds of layer-by-layer iterative updates starting from the 1st round. In each round of iteration, the current round expansion value of each event node is the maximum of the sum of the previous round expansion value and the corresponding directed edge length of all its predecessor event nodes. The predecessor event node pointer points to the predecessor event node that makes the current round expansion value the maximum. When there are multiple predecessors with the same maximum value, the predecessor event node pointer points to the predecessor event node with the smallest number; after N rounds of iteration, for each event node, traverse all values ​​of k from 0 to N minus 1, calculate the difference between the Nth round expansion value and the kth round expansion value, divide by N minus k, and retain the smallest quotient among all values ​​as the cyclic average estimate of this event node; The largest cyclic average estimate among all event nodes is retained as the beat period; the closed sequence of event nodes formed by tracing back level by level through the predecessor event node pointers corresponding to the beat period is marked as the critical beat loop; the beat period is equal to the maximum cyclic average time that limits the system throughput in the beat dependency graph.

6. The method according to claim 2, characterized in that: In step 2, each storage location corresponds one-to-one with a physical storage cell in the automated warehouse. Each storage location registers the layer height color, weight limit color, and status color of the corresponding physical storage cell. The status color takes one of the following: empty, occupied, or locked. The weight limit color corresponds to a preset weight level, and each weight level is mapped to a weight limit value. The equipment-ready storage location corresponds to the idle status of the stacker crane, shuttle, and elevator. Each equipment-ready storage location registers the color of the aisle it governs. The batch queue storage location records the pallets to be processed according to the order of pallet arrival. The task handover storage locations include the receiving storage location at the inbound port, the transfer handover storage location, the replenishment handover storage location, and the release storage location at the outbound port. The tokens circulating in the colored Petri net record the product identification color, batch arrival order color, pallet weight color, pallet layer height color, and target aisle color in a fixed-length field of no less than 32 bits. The delay of each handling change is assigned by the cycle time.

7. The method according to claim 6, characterized in that: In step 2, the enabling rules are determined in the following order:

1. All input locations for the transport change have tokens; 2. The target location's status color is empty; 3. The pallet height color of the input token belongs to the same height group as the registered height color of the target location; 4. The pallet weight color of the input token is less than or equal to the maximum weight value mapped by the maximum weight color of the target location; 5. When the transport instruction is an inbound task, the batch arrival order color of the input token is at the head of the batch queue location; when the transport instruction is a transfer task or a replenishment task... The batch arrival order color of the input token satisfies the batch priority constraint declared by WMS; sixth, the target aisle color of the input token is equal to the aisle color registered in the corresponding equipment ready warehouse; when WMS issues a new inbound, transfer, or replenishment task, the root warehouse is determined according to the following rules: when the task is an inbound task, the root warehouse is taken from the warehouse where the corresponding inbound pallet token is located in the batch queue warehouse; when the task is a transfer task, the root warehouse is taken from the warehouse where the corresponding pallet token is located in the original storage location warehouse; when the task is a replenishment task, the root warehouse is taken from the warehouse where the corresponding replenishment pallet token is located in the replenishment handover warehouse.

8. The method according to claim 7, characterized in that: In step 3, the edges of the heterogeneous graph are translated from the input-output relationship of the handling transition and supplemented with the following cross-type edges: Location nodes are connected to equipment nodes with the same color position in their assigned aisles; equipment nodes are connected to the cycle nodes corresponding to the current event node number on the key cycle loop; task nodes are connected to all location nodes in the candidate location set; the initial feature vectors of each type of node contain continuous and categorical components; continuous components are normalized dimension-wise according to the maximum value of each dimension; continuous components are the aisle steps and cycle average estimates of the distance to the workstation corresponding to the task handover warehouse; categorical components are expressed in one-hot encoding form; categorical components are the layer height color position, weight limit color position, pallet weight color position, assigned aisle color position, batch arrival order color position, current event node number, location number, and assigned cycle node number; multi-round neighbor aggregation. The number of rounds of aggregation is equal to the number of event nodes in the critical beat loop. In each round of neighbor aggregation, the following operations are performed on each node: The current feature vector of this node is mapped by a fully connected 3-way path according to the shared parameters of the node category to obtain the query vector, key vector, and value vector respectively; the pulsating array unit takes the sum of the component-wise products of the query vector of this node and the key vector of each neighbor node as the similarity score of the corresponding neighbor node; all similarity scores of this node are normalized by softmax along the neighbor dimension; the value vectors of all neighbor nodes of this node are linearly combined according to the corresponding normalization coefficients, and the result is added to the feature vector of this node in the previous round component by component and then passed through the linear rectified activation function to obtain the feature vector of this node in the current round; after multiple rounds of neighbor aggregation, the final feature vectors of all storage location nodes are averaged and pooled to obtain the graph-level state features; The graph attention reinforcement learning decision module has a policy head and a value head. The policy head concatenates the final feature vector of each storage location node in the candidate storage location set with a fully connected layer and a softmax layer to obtain a discrete probability distribution. The value head obtains a scalar value estimate of the current state by passing the graph-level state features through a fully connected layer.

Images