A double-instruction-cycle task real-time decision method for a cross-layer shuttle vehicle storage system

Abstract

The application provides a double-instruction cycle task real-time decision method of a cross-layer shuttle warehouse system, and belongs to the technical field of intelligent warehouse logistics scheduling. The method comprises the following steps: establishing a storage task queue and a retrieval task queue, and constructing a task pool; triggering decision when any shuttle is idle, generating a current system environment state; inputting the system environment state into a deep Q network to generate a decision action; generating a double-instruction cycle task and assigning an idle shuttle to execute; calculating a reward value according to a task execution result, updating deep Q network parameters by using an experience replay method until the model converges; and integrating the trained model into a warehouse management system to make real-time decisions on storage tasks and retrieval tasks. The method overcomes the defects of traditional heuristic rules, such as being easy to fall into local optimization or relying on static sorting, significantly reduces the average cycle time of tasks, realizes millisecond-level real-time decision, and effectively improves the scheduling capability and operation efficiency of the warehouse system.

Description

Technical Field

[0001] This invention belongs to the field of intelligent warehousing and logistics scheduling technology, and particularly relates to a real-time decision-making method for dual-instruction cycle tasks in a cross-level shuttle warehousing system. Background Technology

[0002] Automated storage and retrieval systems (AS / RS), as a core component of modern logistics systems, are gradually evolving towards higher density and greater flexibility. Among these, shuttle warehousing systems have attracted significant attention due to their high throughput and flexible scalability. To further improve space utilization and reduce equipment investment costs, a multi-level operation mode has been introduced into shuttle systems, forming a multi-level shuttle warehousing system. This system achieves multi-level storage and retrieval through the parallel collaboration of hoists and shuttles. To further improve operational efficiency, a "dual-command cycle" mode is often adopted in engineering to reduce shuttle idle time and improve system response speed.

[0003] However, this complex system architecture also presents serious challenges to task scheduling.

[0004] First, when expanding from a single-depth scenario to a multi-depth scenario, there may be a need for restocking, which increases the complexity and uncertainty of the operation process.

[0005] Secondly, cross-level operations result in a high degree of coupling between horizontal transportation and vertical lifting, which can easily create a bottleneck at the hoist.

[0006] Existing task scheduling methods mostly rely on traditional static heuristic rules, but these methods are often limited to local optimization or fixed order, lacking the ability to perceive the global state of the system and respond in real time. They are difficult to effectively deal with complex constraints such as multi-depth blocking and cross-layer job conflicts, resulting in limited system operating efficiency and throughput under dynamic high load environments.

[0007] Therefore, designing a real-time decision-making method that can perceive complex environmental conditions and has global optimization capabilities has become a key issue in improving the operational efficiency of cross-level shuttle warehouse systems. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of traditional static heuristic rules and provide a real-time decision-making method for dual-instruction cycle tasks in a cross-level shuttle warehouse system, comprising the following steps: S1: Establish a storage task queue With retrieval task queue And according to the principle of first-come, first-served (FCFS) from Extraction retrieval task construction scale is task pool ; S2: When any shuttle is detected to be idle, a decision is triggered to collect the spatiotemporal distribution characteristics of the hoist, shuttle, and candidate retrieval tasks to generate the current system environment state; S3: Input the system environment state into the Deep Q-Network (DQN) model, calculate the feasibility of the decision, generate a decision action, and update the system environment. ; S4: From Select the head-of-line storage task and the target retrieval task to generate a dual instruction cycle (DCC) task, assign an idle shuttle to execute it, and update the device instruction queue in real time. S5: Calculate the reward value based on the travel time of the two-instruction cycle, and update the parameters of the deep Q-network model using the experience replay method until the model converges; S6: Integrate the trained deep Q-network model into the warehouse management system (WMS) scheduling module to make event-driven real-time decisions on arriving storage and retrieval tasks.

[0009] Furthermore, in S1, a task pool is constructed. Specifically, this includes: setting the maximum capacity of the task pool to... During system initialization, the retrieval task queue is used. The first of the queues is extracted in sequence. Each task to fill in Task pool Once a task is removed or selected, service will be provided on a first-come, first-served basis. Supplemental tasks to maintain Capacity is Storage task queue Tasks are queued on a first-come, first-served basis and do not enter the queue. Participate in the selection.

[0010] Furthermore, in S2, the system environment status includes hoist characteristics, shuttle characteristics, and human characteristics; The elevator features include the number of tasks to be executed in the queues of the storage elevator, retrieval elevator, and shuttle elevator, as well as the dwell layer of the last task after the shuttle elevator has completed its queue; The characteristics of a shuttle include: whether the shuttle is idle, the level at which the shuttle resides, and the column of the storage location where the shuttle that triggered the decision to perform the storage task is located; Task characteristics include: for Each candidate retrieval task The task's target storage location depth, layer number, column number, layer difference between the task and the shuttle that triggered the decision, number of obstructed goods at the target storage location, number of inter-unit blocks required for unloading, and task details. Feasibility identification and In and the task Number of tasks at the same level.

[0011] Furthermore, S3 specifically includes the following steps: S31: Based on the algorithm of the deep Q-network model from Select a search task ; S32: Determine the retrieval task Whether the feasibility conditions are met includes: the goods corresponding to the retrieval task are in stock, and there is no cross-level conflict between the shuttle that triggers the decision and the floor where the retrieval task is located. S33: If this feasibility condition is met, then the retrieval task will be performed. If the task is identified as the target retrieval task, then proceed to step S34. S34: Retrieval task from Removed from the list and served on a first-come, first-served basis. Extract the next retrieval task and add it to the search. To maintain capacity ; S35: Return to S31, the agent is based on the updated... The search task is reselected until a search task that meets the feasibility criteria is selected. S36: Restore the infeasible retrieval tasks that were removed in S34. At the very front, for subsequent scheduling.

[0012] Furthermore, in S4, the real-time update of the device queue specifically includes the following steps: S41: WMS assigns a dual-instruction cycle task to the shuttle that triggers the decision. The shuttle is occupied until the cargo is unloaded into the buffer area, at which point the shuttle is released. S42: The shuttle requests to call the storage hoist, which is occupied until it returns to the roadway I / O point and is then released; S43: The shuttle car completes the storage task and begins to execute the retrieval task. If the retrieval task location is not on the same floor as the shuttle car, the shuttle car elevator is requested to be called. The shuttle car elevator is occupied until the shuttle car reaches the target floor. Then the shuttle car elevator is released and S44 is executed. Otherwise, S44 is executed directly. S44: The shuttle requests to use the retrieval elevator, which is then occupied until the goods are unloaded onto the outbound conveyor belt, at which point the retrieval elevator is released.

[0013] Furthermore, the formula for calculating the travel time in a two-instruction cycle is expressed as follows:

[0014] in, For the first One double instruction cycle; To retrieve the elevator in its first The end time of each cycle of the retrieval task. For storage booster in its first The start time of each cycle's execution of the storage task; The start and end times are calculated based on the travel time of the shuttle in the horizontal direction and the elevator in the vertical direction. The formulas for calculating these travel times are as follows:

[0015]

[0016] in, The time taken for the shuttle to travel horizontally; This refers to the number of unit intervals that the shuttle needs to move horizontally. Number of intervals per shelf level for each shelf; Width of the shelf unit; For the acceleration and deceleration of the shuttle; This represents the maximum speed of the shuttle vehicle. The vertical movement time of the hoist. This refers to the number of unit intervals that the hoist needs to move vertically. The number of shelves for each shelf; This refers to the height of the shelving unit. For the acceleration and deceleration of the hoist; This refers to the maximum moving speed of the elevator.

[0017] Furthermore, in S5, the formula for calculating the reward value is expressed as follows:

[0018] in, The action selected for the current moment; It is a positive scaling factor; It is a negative sensitivity coefficient; This represents the travel time for the current two-instruction cycle; This represents the minimum travel time for historical dual-instruction trips.

[0019] Furthermore, in S5, the specific mechanisms of the experience replay method include: Establish an experience cache to store the quadruple data obtained from interactions. ,in, For the present The state at any given moment; To perform the action The reward; When training a deep Q-algorithm model, a number of data points are randomly sampled from the experience buffer.

[0020] Furthermore, in S5, updating the parameters of the deep Q algorithm model includes evaluating the network. and target network Evaluate the network parameters The update is represented as:

[0021]

[0022]

[0023] in, For learning rate, ; The loss function; As a discount factor for future rewards, ; Parameters of the target network Updated to: Network parameters will be evaluated every fixed number of steps. Copy to target network parameters .

[0024] Furthermore, in S6, model integration and real-time decision-making include: freezing the parameters of the trained deep Q network, deploying them in the scheduling module of WMS, and cyclically executing S1-S4 during the decision-making phase.

[0025] Compared with the prior art, the beneficial effects of the present invention are mainly reflected in: 1. The present invention adopts a technical solution that constructs real-time decision input based on system environment status and inputs the system environment status into a deep Q-network to generate retrieval task selection results. This solution enables the scheduling process to make decisions by comprehensively utilizing the status of the elevator, the status of the shuttle, and the status of candidate retrieval tasks, thereby improving the real-time performance and environmental adaptability of task allocation.

[0026] 2. The present invention adopts a technical solution of determining the feasibility of candidate retrieval tasks and combining them with storage tasks to generate dual-instruction cycle tasks when the conditions are met. This solution avoids the incorrect assignment of tasks with no inventory or tasks with cross-layer conflicts, thereby improving the rationality of task allocation and reducing ineffective scheduling.

[0027] 3. This invention employs a technical solution that assigns shuttle cars to perform storage and retrieval tasks in a dual-instruction cycle manner under complex constraints such as single-depth / multi-depth, cross-layer, and reloading operations. This reduces the empty round trips of shuttle cars, coordinates cross-layer and reloading operations, and achieves the technical effects of reducing the average task cycle time and improving system operating efficiency.

[0028] 4. This invention adopts a technical solution that constructs reward values ​​based on task execution results, updates deep Q-network parameters using experience replay, and integrates the trained model into the WMS scheduling module for real-time scheduling. This solution enables the scheduling strategy to be continuously optimized through training and applied to dynamic task arrival scenarios, thereby improving the real-time scheduling capability and overall throughput of the warehousing system. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a multi-level shuttle warehouse system considering unloading operations in an embodiment of the present invention.

[0030] Figure 2 This is a flowchart illustrating the specific operation of the cross-level shuttle storage system in DCC mode in an embodiment of the present invention.

[0031] Figure 3 This is a flowchart of the WMS real-time decision-making process of the cross-level shuttle warehouse system in DCC mode in an embodiment of the present invention.

[0032] Figure 4 This is a comparison chart showing the variation of the average cycle time of the method of this invention and three classic task scheduling strategies with the number of simulation days.

[0033] Figure 5 This is a comparison chart showing the average cycle time of the method of this invention and the shortest job time priority strategy as a function of the number of simulation days.

[0034] Figure 6 This diagram shows the reduction in average cycle time under the near-idle priority restocking strategy of the present invention.

[0035] Figure 7 This is a graph showing the reduction in average cycle time under the maximum depth priority restocking strategy of the present invention. Detailed Implementation

[0036] The following will describe in more detail, with reference to the schematic diagram, a dual-instruction cycle task real-time decision-making method for a cross-level shuttle storage system according to the present invention, wherein preferred embodiments of the present invention are shown. It should be understood that those skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the present invention.

[0037] This invention provides a real-time decision-making method for dual-instruction cycle tasks in a cross-level shuttle warehouse system.

[0038] In this embodiment of the invention, a simulation and decision-making environment for a multi-level shuttle warehouse system was built to verify the feasibility and effectiveness of the method. Specifically, the embodiment uses Siemens' Tecnomatix PlantSimulation software as the simulation platform to construct a simulation environment as follows: Figure 1 The diagram shows a physical model of a multi-level shuttle storage system considering unloading operations. In the simulation platform, the geometric parameters, kinematic properties, and logical behaviors of components such as shelves, shuttles, elevators, and conveyors were defined using the SimTalk language. The shelf unit dimensions are set as follows: maximum shuttle speed acceleration Maximum moving speed of the hoist acceleration The speed of the shuttle's extension mechanism or the satellite vehicle's movement. The DQN algorithm was developed using the PyTorch framework within a Python environment. Real-time interaction between the simulation model and the DQN decision algorithm is achieved via a COM interface: the simulation model sends real-time system status data to the Python side, and the Python side returns decision instructions to the simulation model to drive the device execution after network inference. Based on the above environment, the method of this invention includes the following steps: Step 1: Establish a storage task queue With retrieval task queue And according to the first-come, first-served (FCFS) principle from Extraction retrieval task construction scale is task pool ; Specifically, such as Figure 3 As shown, the WMS system initializes the storage task queue. With retrieval task queue Task Pool Set the capacity to a fixed value During system operation, storage tasks only occur when... Queuing and waiting. For retrieval tasks, the system initializes from... The team leader extracts 5 tasks to fill the slots. In the decision-making process, once If a task is selected for execution or removed due to infeasibility, the system immediately follows the FCFS principle to select the next task from the list. Extract the next task to add. This ensures that the agent always has 5 candidate tasks to choose from.

[0039] Step 2: When any shuttle is detected to be idle, a decision is triggered to collect the spatiotemporal distribution characteristics of the hoist, shuttle, and candidate retrieval tasks to generate the current system environment status; Specifically, when the shuttle in the simulation environment completes the current instruction and enters the "idle" state, a decision request is triggered. The WMS collects the current data. System state vector For multi-depth, cross-level shuttle storage systems, considering the characteristics of unloading and cross-level operations, detailed information on the system environment is shown in Table 1.

[0040] Table 1 State-space characteristics

[0041] Among them, variables This indicates the identifier for the shuttle within the system. express The indicator of an idle shuttle that triggers a decision at any time. The number of shuttle cars; This indicates the tasks that exist in the task pool; when features 1, 2, and 3 are 0, it means that the service device is in an idle state; when feature 14 is 1, it means that the task has inventory and there is no cross-layer conflict, otherwise it is 0.

[0042] Step 3: Input the system environment state into a deep Q-network (DQN), generate decision actions through network calculation and feasibility assessment, and update the system environment state simultaneously. ; Specifically, this step includes a closed-loop process of "DQN inference" and "feasibility verification": (1) DQN inference: The WMS inputs the state vector collected in step S2 into the DQN network. The output layer contains Each node corresponds to middle The Q-values ​​of each candidate task are used during the model training phase. - Greedy strategy: based on probability Randomly select an action to explore the environment, with probability. Choose the action with the largest Q value to utilize experience. In the implementation decision-making phase, directly select the retrieval task with the largest Q value. (2) Feasibility verification: The system judges the selected task Feasibility criteria are met: The goods corresponding to the retrieval task are in stock, and there is no cross-level conflict between the shuttle that triggered the decision and other shuttles when it travels to the floor where the retrieval task is located. If these conditions are met, the task is designated as the target retrieval task; otherwise, the task is removed from the list. Remove from, and follow the FCFS principle from Extract the next retrieval task and add it to the search. Then, the state collection and DQN inference are repeated until a feasible task is selected. (3) Task rollback: In order to prevent task loss, all infeasible tasks that were temporarily removed during the above verification process will be put back into the retrieval task queue after the decision is made in this round. At the very front, waiting for the next scheduling.

[0043] Step 4: From Select the first storage task in the queue, combine it with the target retrieval task to generate a dual instruction cycle (DCC) task, and assign the idle shuttle to execute it, thereby updating the device instruction queue in real time. Specifically, such as Figure 2 As shown: WMS locked The storage task at the head of the queue is packaged with the retrieval task determined in step S3 into a DCC task and sent to the shuttle that triggers the decision. During the execution process, the occupancy and release status of each device is updated in real time: (1) Task allocation: WMS allocates the DCC task to an idle shuttle, and the shuttle status is updated to "occupied". (2) Storage process: The shuttle requests the storage lift, locks the storage lift, and sets its status to "occupied"; the storage lift transports the goods to the layer where the shuttle is located and unloads them into the buffer area; then the storage lift returns to the aisle I / O point, and its status is updated to "released", while the shuttle loads the goods and transports them to an idle storage location. (3) Cross-layer process (on demand): After the shuttle completes storage, it determines whether the target layer of the retrieval task is consistent with the current layer. If they are inconsistent, it requests the shuttle lift, locks the shuttle lift, and sets its status to "occupied"; the shuttle lift moves to the layer where the shuttle is located, loads the shuttle, and transports it to the target layer; after reaching the target layer, the shuttle drives out, and the shuttle lift status is updated to "released". (4) Retrieval process: The shuttle travels to the retrieval location on the target layer. If there is a blockage, it performs a cargo transfer operation. After obtaining the target cargo, the shuttle returns to the I / O point of that layer and unloads the cargo into the buffer area. The status is updated to "Release". The shuttle requests the retrieval elevator, locks the storage elevator, and sets its status to "Occupied". The retrieval elevator returns to the aisle I / O point and unloads the cargo onto the outbound conveyor belt. The status is updated to "Release".

[0044] Step 5: Calculate the reward value based on the trip time of the DCC, and update the DQN parameters using the experience replay method until the model converges; Specifically, the travel time of the DCC is calculated using formula (1).

[0045]

[0046] in, Indicates the first One double instruction cycle, To retrieve the elevator in its first The end time of each cycle of the retrieval task. For storage booster in its first The start time of each cycle's execution of the storage task.

[0047] The start and end times are calculated based on the travel time of the shuttle in the horizontal direction and the elevator in the vertical direction, which is calculated using equations (2) and (3).

[0048]

[0049]

[0050] In equation (2), The time taken for the shuttle to travel horizontally. This represents the number of unit intervals that the shuttle needs to move horizontally. The number of intervals per shelf level for each shelf. Width of the shelf unit For the acceleration and deceleration of the shuttle, This represents the maximum speed of the shuttle.

[0051] In equation (3), The vertical movement time of the hoist. This refers to the number of unit intervals that the hoist needs to move vertically. The number of shelves for each shelf, This refers to the height of the shelving unit. To improve the acceleration and deceleration of the elevator, This refers to the maximum moving speed of the elevator.

[0052] reward function Designed in a negative exponential form to increase sensitivity to action, as shown in equation (4).

[0053]

[0054] in, For the present The action chosen at any given moment; It is a positive scaling factor. It is a negative sensitivity coefficient; The current travel time of DCC is calculated using equation (1); This is the minimum travel time for historical DCCs.

[0055] During DQN training, an experience cache is established for storage. After each simulation step, a number of data points are randomly sampled from the experience buffer to update the DQN parameters. The evaluation network... parameters The update can be represented by equations (5), (6), and (7). The target network parameters Updated to: Network parameters will be evaluated every fixed number of steps. Copy to target network parameters .

[0056] (5) (6) (7) in, The learning rate; The loss function; This represents a discount factor for future rewards.

[0057] To verify the performance of the method of the present invention, nine sets of benchmark calculation examples were designed in this embodiment, as shown in Table 2. Different scales (number of layers) were considered. Column number ), different depths ( ), different storage location rates ( ) and the number of shuttles (S), set , , , The initial inventory distribution of each commodity follows an exponential distribution with an average inventory quantity of 3.

[0058] Table 2. Benchmark Examples

[0059] The DQN model is trained based on the above examples. Taking example 1, Figure 4 and Figure 5 The curve showing the change in average cycle time during the training phase of the DQN model and the comparison of the effects of different decision-making methods are presented.

[0060] in, Figure 4 The average cycle time variation curves are shown after running the classic task scheduling strategies (Random, FCFS, SPT) for the same number of simulation days as DQN training. It can be seen that the experimental results of the Random strategy and the First-Come, First-Served strategy (FCFS) fluctuate greatly throughout the operation and have poor performance; in contrast, the Shortest Job Time First strategy (SPT) and the method of this invention (DQN) perform better in terms of decision-making effectiveness and stability.

[0061] Figure 5The comparison curves of the average cycle time of DQN and SPT as a function of simulation days are further presented. It can be seen that the average cycle time of SPT fluctuates within a certain range, while DQN initially fluctuates more, but as the training time increases, it tends to stabilize on day 150 and remains at a level superior to SPT, thus verifying that DQN has better convergence performance and global optimization capability.

[0062] Step 6: Integrate the trained DQN model into the WMS scheduling module to make event-driven real-time decisions on arriving storage and retrieval tasks.

[0063] Specifically, the trained DQN network parameters are frozen and deployed in the intelligent scheduling module of WMS, replacing traditional heuristic rules. To further verify the performance of the method of this invention, in this embodiment, tests are conducted based on the nine sets of benchmark examples shown in Table 2 of step S5: two commonly used relocation strategies, namely the nearest idle priority relocation strategy and the maximum depth priority relocation strategy, are used to explore their impact on task scheduling performance and are compared with the SPT strategy. Figure 6 and Figure 7 The invention demonstrates the reduction in average cycle time compared to traditional decision-making methods under different restocking strategies. Among these, Figure 6 The invention demonstrates that when employing a near-idle priority restocking strategy, DQN can reduce the average cycle time by 16.6% to 32.7% compared to the SPT strategy. Figure 7 The results demonstrate that when employing a maximum depth-first repositioning strategy, the DQN method can reduce the average cycle time by 12.1% to 33.2% compared to the SPT strategy. The results show that the method of this invention can effectively adapt to complex environments involving single-depth / multi-depth, cross-layer, and repositioning operations. Under different system configurations and repositioning strategies, it can significantly reduce the average task cycle time and improve the overall throughput and operational efficiency of the warehousing system.

[0064] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A real-time decision-making method for dual-instruction cycle tasks in a multi-level shuttle warehouse system, characterized in that, Includes the following steps: S1: Establish a storage task queue With retrieval task queue And in accordance with the principle of first come, first served Extraction retrieval task construction scale is task pool ; S2: When any shuttle is detected to be idle, a decision is triggered to collect the spatiotemporal distribution characteristics of the hoist, shuttle, and candidate retrieval tasks to generate the current system environment state; S3: Input the system environment state into the deep Q-network model, calculate the feasibility of the decision, generate a decision action, and update the system environment. ; S4: From Select the head-of-line storage task and the target retrieval task to generate a dual-instruction cycle task, assign an idle shuttle to execute it, and update the device instruction queue in real time. S5: Calculate the reward value based on the travel time of the two-instruction cycle, and update the parameters of the deep Q-network model using the experience replay method until the model converges; S6: Integrate the trained deep Q-network model into the scheduling module of the warehouse management system to make event-driven real-time decisions on arriving storage and retrieval tasks.

2. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, In step S1, a task pool is constructed. Specifically, this includes: setting the maximum capacity of the task pool to... During system initialization, the retrieval task queue is used. The first of the queues is extracted in sequence. Each task to fill in Task pool Once a task is removed or selected, service will be provided on a first-come, first-served basis. Supplemental tasks to maintain Capacity is Storage task queue Tasks are queued on a first-come, first-served basis and do not enter the queue. Participate in the selection.

3. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 2, characterized in that, In S2, the system environment status includes hoist characteristics, shuttle characteristics, and human characteristics; The features of the elevator include the number of tasks to be executed in the queues of the storage elevator, the retrieval elevator, and the shuttle elevator, as well as the dwell layer of the last task after the shuttle elevator has completed its queue. The shuttle car features include: whether the shuttle car is idle, the shuttle car's dwell layer, and the column of the storage location where the shuttle car that triggered the decision to perform the storage task is located. The task characteristics include: for Each candidate retrieval task The task's target storage location depth, layer number, column number, layer difference between the task and the shuttle that triggered the decision, number of obstructed goods at the target storage location, number of inter-unit blocks required for unloading, and task details. Feasibility identification and In and the task Number of tasks at the same level.

4. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, S3 specifically includes the following steps: S31: Based on the algorithm of the deep Q-network model from Select a search task ; S32: Determine the retrieval task Whether the feasibility conditions are met includes: the goods corresponding to the retrieval task are in stock, and the shuttle that triggers the decision does not have cross-level conflicts with other shuttles when going to the floor where the retrieval task is located. S33: If this feasibility condition is met, then the retrieval task will be performed. If the task is identified as the target retrieval task, then proceed to step S34. S34: Retrieval task from Removed from the list and served on a first-come, first-served basis. Extract the next retrieval task and add it to the search. To maintain capacity ; S35: Return to S31, based on the updated... The search task is reselected until a search task that meets the feasibility criteria is selected. S36: Restore the infeasible retrieval tasks that were removed in S34. At the very front, for subsequent scheduling.

5. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, In step S4, the real-time update of the device queue specifically includes the following steps: S41: WMS assigns a dual-instruction cycle task to the shuttle that triggers the decision, and the shuttle is occupied until the goods are unloaded into the buffer area, at which point the shuttle is released. S42: The shuttle requests to call the storage hoist, which is occupied until it returns to the roadway I / O point and is then released; S43: The shuttle car completes the storage task and begins to execute the retrieval task. If the retrieval task location is not on the same floor as the shuttle car, the shuttle car elevator is requested to be called. The shuttle car elevator is occupied until the shuttle car reaches the target floor. Then the shuttle car elevator is released and S44 is executed. Otherwise, S44 is executed directly. S44: The shuttle car requests to call the retrieval elevator, so that the retrieval elevator is occupied until the goods are unloaded onto the outbound conveyor belt, at which point the retrieval elevator is released.

6. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, The formula for calculating the travel time in a two-instruction cycle is as follows: ； in, For the first One double instruction cycle; To retrieve the elevator in its first The end time of each cycle of the retrieval task. For storage booster in its first The start time of each cycle's execution of the storage task; The calculation of the start and end times is based on the travel time of the shuttle in the horizontal direction and the elevator in the vertical direction, and the formulas for calculating the travel time are expressed as follows: ； ； in, The time taken for the shuttle to travel horizontally; This refers to the number of unit intervals that the shuttle needs to move horizontally. Number of intervals per shelf level for each shelf; Width of the shelf unit; For the acceleration and deceleration of the shuttle; This represents the maximum speed of the shuttle vehicle. The vertical movement time of the hoist. This refers to the number of unit intervals that the hoist needs to move vertically. The number of shelves for each shelf; This refers to the height of the shelving unit. For the acceleration and deceleration of the hoist; This refers to the maximum moving speed of the elevator.

7. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, In step S5, the formula for calculating the reward value is expressed as follows: ； in, The action selected for the current moment; It is a positive scaling factor; It is a negative sensitivity coefficient; This represents the travel time for the current two-instruction cycle; This represents the minimum travel time for historical dual-instruction trips.

8. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, In S5, the specific mechanism of the experience playback method includes: Establish an experience cache to store the quadruple data obtained from interactions. ,in, For the present The state at any given moment; To perform the action The reward; When training a deep Q-algorithm model, a number of data points are randomly sampled from the experience buffer.

9. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, In S5, updating the parameters of the deep Q-algorithm model includes evaluating the network. and target network The parameters of the evaluation network The update is represented as: ； ； ； in, For learning rate, ; The loss function; As a discount factor for future rewards, ; The parameters of the target network Updated to: Network parameters will be evaluated every fixed number of steps. Copy to target network parameters .

10. The dual-instruction cycle task real-time decision-making method for a cross-level shuttle warehouse system according to claim 1, characterized in that, In step S6, the model integration and real-time decision-making includes: freezing the trained deep Q network parameters, deploying them in the scheduling module of WMS, and cyclically executing S1-S4 during the decision-making phase.

Images