Warehouse logistics dynamic scheduling optimization method and system based on deep learning

Abstract

The present application relates to warehouse logistics management and artificial intelligence technical field, specifically for warehouse logistics dynamic scheduling optimization method and system based on deep learning, including: collecting static map data and dynamic sensor data of warehouse environment; constructing dynamic space-time graph state of current time; in response to the business logic rules of warehouse management system, triggering scheduling optimization process, including: generating preliminary scheduling intent set by constraint reinforcement learning scheduler; through space-time graph attention network predictor, the future space-time congestion heat map is predicted; through the quantitative processing of the self-defined aggregation function, the future congestion cost of the scalar is determined; through the Lagrange solution framework of the constraint reinforcement learning scheduler, the preliminary scheduling intent set is corrected, and the final scheduling intent is generated; output the final scheduling intent to the warehouse control system for execution; the present application solves the problem that decision and prediction are disconnected in traditional scheduling, and realizes the change from lag response to active avoidance.

Description

Technical Field

[0001] This invention relates to the fields of warehousing and logistics management and artificial intelligence technology, specifically to a method and system for dynamic scheduling and optimization of warehousing and logistics based on deep learning. Background Technology

[0002] Dynamic scheduling of warehousing and logistics systems is the core element for achieving efficient material flow. Its scheduling efficiency directly affects the throughput, resource utilization, and operational stability of the entire warehousing system. This system involves multi-dimensional dynamic elements such as agile autonomous robots, shelves, and intersections, and needs to respond to orders and rules from the warehousing management system to perform path planning and task allocation. Balancing order throughput with future congestion costs under highly dynamic and complex constraints has become a technical challenge in this field.

[0003] Existing scheduling techniques often employ greedy strategies, such as pursuing immediate rewards or the fastest path. However, this leads to a disconnect between decision-making and future state prediction. Traditional methods struggle to anticipate potential future spatiotemporal congestion caused by scheduling decisions themselves, resulting in the system hitting a bottleneck while pursuing high throughput. Furthermore, the system state exhibits strong nonlinearity and dynamic coupling characteristics, making it difficult for conventional optimization methods to find the optimal balance between the conflicting objectives of immediate rewards and congestion constraints. Therefore, a scheduling scheme that can proactively predict the consequences of decisions and achieve intention-prediction-correction closed-loop optimization is urgently needed to address these issues. Summary of the Invention

[0004] The purpose of this invention is to provide a deep learning-based dynamic scheduling optimization method and system for warehouse logistics, which solves the problem of the disconnect between decision-making behavior and future state prediction in traditional scheduling. Furthermore, it can proactively avoid potential traffic bottlenecks and future spatiotemporal congestion while ensuring high order throughput. Specifically, the technical solution of this invention is as follows:

[0005] A deep learning-based dynamic scheduling optimization method for warehouse logistics includes:

[0006] Collect static map data and dynamic sensor data of the warehouse environment;

[0007] Based on static map data and dynamic sensor data, construct a dynamic spatiotemporal map of the current moment.

[0008] In response to the business logic rules of the warehouse management system, a scheduling optimization process is triggered, including:

[0009] Step 1: Based on the dynamic spatiotemporal graph state, a preliminary scheduling intent set is generated by a constraint reinforcement learning scheduler.

[0010] Step 2: Combining the dynamic spatiotemporal map state with the preliminary scheduling intention set, the spatiotemporal map attention network predictor is used to predict the future spatiotemporal congestion heat map.

[0011] Step 3: Quantify the future spatiotemporal congestion heatmap using a custom aggregation function to determine the scalar future congestion cost;

[0012] Step 4: Based on future congestion costs and immediate rewards, the initial scheduling intent set is modified and the final scheduling intent is generated by using the Lagrange solution framework of the constraint reinforcement learning scheduler.

[0013] Step 5: Output the final scheduling intent to the warehouse control system for execution.

[0014] Optional, dynamic spatiotemporal graph states include:

[0015] A graph node set, which includes agile autonomous robots, shelves, and intersections;

[0016] The set of physical connection edges between graph nodes;

[0017] The dynamic attribute feature matrix includes the position, power, speed, and load attributes of the agile autonomous robot, as well as the inventory attributes of the shelf.

[0018] Optionally, a spatiotemporal graph attention network predictor explicitly takes a preliminary scheduling intent set as input to learn the nonlinear coupling relationship between decision intent and future spatiotemporal state, in order to predict future congestion caused by the preliminary scheduling intent set.

[0019] Optionally, the future spatiotemporal congestion heatmap can be used to determine future congestion costs, including:

[0020] Based on preset node business weights and preset congestion activation functions;

[0021] We perform weighted aggregation and thresholding on the predicted congestion index of each node in the future spatiotemporal congestion heatmap.

[0022] Reduce the high-dimensional future spatiotemporal congestion heatmap to a single scalar of future congestion costs.

[0023] Optional Lagrange solution frameworks include:

[0024] Construct a constrained Markov decision process objective to maximize long-term discounted returns while satisfying the constraint that future congestion costs do not exceed a preset congestion cost threshold.

[0025] By introducing a cost-penalty hyperparameter as a Lagrange multiplier, the objective of the constrained Markov decision process is transformed into an unconstrained optimization objective.

[0026] The cost penalty hyperparameter is used to balance immediate rewards with future congestion costs.

[0027] Optionally, a congestion cost threshold can be preset, calibrated by the total discounted congestion cost of the simulation baseline strategy, and determined in conjunction with preset constraint factors.

[0028] Optionally, a closed-loop training step for model parameters may also be included:

[0029] Based on the actual congestion situation observed in the future and the prediction residual of the future spatiotemporal congestion heat map, gradient descent is used to update the network parameters of the spatiotemporal graph attention network predictor.

[0030] Based on the actual order rewards and actual congestion costs, a policy gradient algorithm is used to update the policy network of the constraint reinforcement learning scheduler.

[0031] A deep learning-based dynamic scheduling and optimization system for warehouse logistics includes:

[0032] The data acquisition module is used to collect static map data and dynamic sensor data of the warehouse environment;

[0033] The state construction module is used to construct the dynamic spatiotemporal map state at the current moment based on static map data and dynamic sensor data.

[0034] The scheduling optimization module is used to respond to the business logic rules of the warehouse management system and trigger the scheduling optimization process. The scheduling optimization module includes:

[0035] The intent generation unit is used to generate a preliminary set of scheduling intents based on the dynamic spatiotemporal graph state by a constraint reinforcement learning scheduler.

[0036] The congestion prediction unit is used to combine the dynamic spatiotemporal map status with the preliminary scheduling intention set, and predict the future spatiotemporal congestion heat map through the spatiotemporal map attention network predictor.

[0037] The cost quantification unit is used to quantify the future spatiotemporal congestion heatmap through a custom aggregation function to determine the scalar future congestion cost.

[0038] The strategy correction unit is used to correct the initial scheduling intention set based on future congestion costs and immediate rewards, and generate the final scheduling intention by using the Lagrange solution framework of the constraint reinforcement learning scheduler.

[0039] The scheduling execution module is used to output the final scheduling intent to the warehouse control system for execution.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. This invention constructs a closed-loop decision-making framework of intent-prediction-correction; by explicitly inputting the initial scheduling intent into the predictor, the system can predict the future congestion caused by the decision itself, solving the problem of the disconnect between decision and prediction in traditional scheduling, and realizing the transformation from delayed response to proactive avoidance.

[0042] 2. This invention employs a Lagrange solution framework, treating future congestion costs as a dynamic constraint. By introducing an automatically adjustable cost penalty parameter, it dynamically balances immediate rewards such as maximizing order throughput with satisfying preset congestion threshold constraints, thus achieving Pareto optimal decision-making under multi-objective conflict.

[0043] 3. This invention quantifies future congestion costs by introducing node business weights and congestion activation functions; this approach can reduce the dimensionality of high-dimensional congestion heatmaps to a single scalar and ensure that the system prioritizes severe congestion at key nodes such as main roads, thus aligning the optimization objective with the actual business logic.

[0044] 4. This invention also includes a closed-loop training step; the predictor is updated based on the actual observed congestion residuals, and the decision-maker is updated based on the actual order rewards and congestion costs. This allows the dual models to continuously approximate physical reality and iteratively optimize the strategy, ensuring the system maintains long-term robustness in dynamically changing environments. Attached Figure Description

[0045] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0046] Figure 1 This is a flowchart of the method of the present invention;

[0047] Figure 2 This is a structural diagram of the system of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0049] Example 1:

[0050] Please see Figure 1 A deep learning-based dynamic scheduling optimization method for warehouse logistics includes:

[0051] Collect static map data and dynamic sensor data of the warehouse environment;

[0052] Based on static map data and dynamic sensor data, construct a dynamic spatiotemporal map of the current moment.

[0053] In response to the business logic rules of the warehouse management system, a scheduling optimization process is triggered, including:

[0054] Step 1: Based on the dynamic spatiotemporal graph state, a preliminary scheduling intent set is generated by a constraint reinforcement learning scheduler.

[0055] Step 2: Combining the dynamic spatiotemporal map state with the preliminary scheduling intention set, the spatiotemporal map attention network predictor is used to predict the future spatiotemporal congestion heat map.

[0056] Step 3: Quantify the future spatiotemporal congestion heatmap using a custom aggregation function to determine the scalar future congestion cost;

[0057] Step 4: Based on future congestion costs and immediate rewards, the initial scheduling intent set is modified and the final scheduling intent is generated by using the Lagrange solution framework of the constraint reinforcement learning scheduler.

[0058] Step 5: Output the final scheduling intent to the warehouse control system for execution.

[0059] This embodiment provides a deep learning-based dynamic scheduling optimization method for warehouse logistics. The core of this method is to construct an intention-prediction-correction closed-loop decision framework to solve the problem of the disconnect between decision-making behavior and future state prediction in traditional scheduling.

[0060] This method collects static map data and dynamic sensor data of the warehouse environment. The static map data defines the physical topology of the warehouse; in this embodiment, it refers to the gridded layout, shelving, intersections, and fixed location information of charging stations. The dynamic sensor data captures the changing state of the environment in real time; in this embodiment, it refers to the dynamic attributes of the AGV, such as position, monitored in real time by IoT sensors, the AGV's own sensors, or the worker positioning system. Battery ,speed Load capacity Dynamic shelf attributes such as SKU and inventory and worker locations;

[0061] Based on static map data and dynamic sensor data, a dynamic spatiotemporal map state is constructed for the current moment; in this invention, the dynamic spatiotemporal map state specifically refers to a state in... Snapshot model of time Its function is to provide a unified, structured state input for subsequent prediction and decision-making, and its construction method is as follows:

[0062] ;

[0063] in It is a set of graph nodes, derived from static map data such as shelves and intersections, and dynamic sensor data such as AGVs;

[0064] This is a set of physically connected edges, derived from static map data, defining the nodes. The physical paths that are accessible between them;

[0065] for The dynamic attribute feature matrix at a given time moment is derived from dynamic sensor data and aggregates all nodes at that time moment. Dynamic attributes, such as ;

[0066] The system responds to the business logic rules of the Warehouse Management System (WMS) and triggers the scheduling optimization process. These business logic rules define the business boundaries of the scheduling; in this embodiment, they refer to order priorities issued by the WMS, such as expedited orders, or operational constraints, such as frozen goods must enter area A. These rules are encoded as hard constraints. The scheduling process must meet the following requirements. ;

[0067] After the scheduling optimization process is initiated, it specifically includes:

[0068] Based on dynamic spatiotemporal graph state Constraint reinforcement learning scheduler Generate a preliminary scheduling intent set Pi;

[0069] Constraint Reinforcement Learning Scheduler This is the core decision-making agent in this invention, whose purpose is to learn an optimal scheduling strategy; a preliminary scheduling intent set. Refers to the scheduler exist Time, based on the current state And maximize instant rewards For example, a greedy target for throughput, generating preliminary action plans without congestion prediction; In terms of data structure, it is a set representing the scheduler's intention for AGV_1 to execute task A, and to select the fastest path that appears to be currently available;

[0070] Combined with dynamic spatiotemporal diagram state With the initial scheduling intent set Predictors via spatiotemporal graph attention network A heat map of future spatiotemporal congestion can be predicted. ;

[0071] Spatiotemporal graph attention network predictor A deep learning model like STGAT aims to predict the future. Congestion conditions within a time slice; conventional prediction models are based solely on historical data. Making predictions has drawbacks in proactive decision-making systems because the decisions themselves can alter the future; therefore, this embodiment... Explicitly express the intention As the core input, to solve the problem of strong coupling between decision-making and prediction, the prediction model is as follows:

[0072] ;

[0073] in A heatmap of future spatiotemporal congestion, a vector or matrix, composed of... The calculated output represents the future. Each node Predicted congestion index ; The current spatiotemporal graph state is constructed from the preceding steps; The initial scheduling intent set is determined by the scheduler. generate; The predictor network parameters and model weights are optimized through subsequent closed-loop training.

[0074] This formula is achieved through fusion. Current road conditions and The impending influx of vehicles, Can predict At which points will this decision itself create new congestion?

[0075] Heat map of future spacetime congestion By using custom aggregate functions Quantification is performed to determine the scalar future congestion cost. ;

[0076] Custom aggregate functions It serves as a bridge connecting predictive and decision-making models, aiming to integrate high-dimensional congestion heatmaps. Reduce the dimensionality to a single scalar quantity that can be used to optimize the future congestion cost. The calculation method is as follows:

[0077] ;

[0078] in For the cost of future congestion, a scalar, by The calculated physical quantities can be either cost or dimensionless exponents, which are used as input for subsequent steps. For all nodes in the graph, from ; The predicted congestion index for node v comes from... ; The business weight for node v is a hyperparameter, preset based on business experience, such as the main road. high; For congestion activation function, a function, such as Used to set congestion thresholds , These are also preset hyperparameters;

[0079] This formula predicts congestion for all nodes. Weighting Summation, and use By filtering out tolerable low-level congestion, the complex spatiotemporal congestion risks are transformed into a single, measurable cost value. ;

[0080] Based on future congestion costs With instant rewards By using the Lagrangian solution framework of the constraint reinforcement learning scheduler, the initial scheduling intent set is corrected. Generate the final scheduling intent ;

[0081] The Lagrange solution framework is used in this embodiment to solve the constraint reinforcement learning scheduler. The core mathematical tool; the technical motivation lies in the fact that conventional RL only maximizes For example, throughput can cause AGVs to greedily take shortcuts, leading to congestion. This invention introduces As a dynamic constraint, it forces the scheduler To find the Pareto optimality between high throughput and low congestion, we define the constrained optimization objective CMDP:

[0082] ;

[0083] in The final scheduling strategy is the optimization objective of this step. for The instant reward at any given moment is a scalar, such as the number of orders completed at time t issued by WMS; for The cost of predicting future congestion at any given moment, a scalar, comes from ; As a discount factor, one The hyperparameters are used to balance immediate and future returns, such as setting them to 0.99; A hyperparameter, set according to the business SLA, representing the maximum acceptable total discount congestion cost threshold. In the scheduler Trajectory obtained by distribution downsampling The mathematical expectation; Indicates the time step in the decision-making process;

[0084] To solve this constrained problem, the Lagrange multiplier method is used to transform it into an unconstrained optimization objective. :

[0085] ;

[0086] in For cost-penalized hyperparameters or Lagrange multipliers, , is an adjustable parameter that is automatically updated during training. Its physical meaning is the exchange rate between congestion cost and order reward, ensuring that the two are consistent in terms of units.

[0087] Revise and generate the final scheduling intent The logical closed loop is: scheduler based on Target, generation For example, taking shortcuts; predictors Predict this This will lead to high like Scheduler Evaluate ,Discover The expected total value is very low; the scheduler Automatic correction strategy, generating an alternative. Such as detours; predictor Implicitly predicting This will lead to low like Scheduler Evaluate ,Discover The expected total value is higher; therefore, It was determined to be the final scheduling intention;

[0088] Output the final scheduling intent Executed by the Warehouse Control System (WCS);

[0089] WCS receives It breaks down these into specific instructions and issues them to the AGV for execution;

[0090] By constructing an intent-prediction-correction closed-loop scheduling framework, this invention solves the core problem of the disconnect between decision-making and prediction in traditional scheduling systems; it utilizes a spatiotemporal graph attention network. Explicitly predict scheduling intentions The resulting future congestion And quantify this congestion into costs. Then, through constraint reinforcement learning The Lagrange framework forces the scheduler Maximizing instant rewards and minimizing future congestion The system makes dynamic trade-offs between these factors; ultimately, it can generate a predictive final scheduling intention. While ensuring high order throughput, it proactively avoids potential traffic bottlenecks, significantly improving the overall operational efficiency and stability of the warehousing and logistics system.

[0091] Example 2:

[0092] Dynamic spatiotemporal graph states include:

[0093] A graph node set, which includes agile autonomous robots, shelves, and intersections;

[0094] The set of physical connection edges between graph nodes;

[0095] The dynamic attribute feature matrix includes the position, power, speed, and load attributes of the agile autonomous robot, as well as the inventory attributes of the shelf.

[0096] This embodiment, based on embodiment 1, focuses on the dynamic spatiotemporal diagram state. The composition of this state is specifically defined; include:

[0097] Graph node set Its purpose is to represent key entities in the warehouse; in this embodiment, the graph node set is specifically defined as including agile autonomous robots (AGVs), shelves, and intersections; AGVs are dynamic nodes, shelves are task target nodes, and intersections are critical path resource nodes; this partitioning method makes the graph model It can represent key interactive entities within the warehouse in a refined manner;

[0098] Physical connection edges between graph nodes Its purpose is to characterize physical topology; in this embodiment, Specifically defined as the physical path between the aforementioned nodes;

[0099] Dynamic attribute feature matrix Its purpose is to characterize the real-time state of the node; in this embodiment, Specifically defined as including the position of agile autonomous robots (AGVs) Battery ,speed With load capacity Attributes, and shelf inventory Attributes; incorporating these multi-dimensional real-time attributes , making the state The representation is richer;

[0100] By analyzing the dynamic spatiotemporal diagram state By finely defining the nodes and attributes, this invention constructs a high-information-density warehouse status representation; key dynamic information such as AGV power consumption, load capacity, and shelf inventory are integrated into the feature matrix. This enables subsequent predictors and decision-making machine It can not only sense spatial congestion, but also sense dynamic changes in the task and resource layers, such as low-battery AGVs and empty inventory shelves. This provides a sufficient state basis for achieving more complex and refined multi-objective collaborative scheduling, such as charging-task joint scheduling, and further improves the accuracy of scheduling and resource utilization.

[0101] Example 3:

[0102] The spatiotemporal graph attention network predictor explicitly takes a preliminary scheduling intent set as input to learn the nonlinear coupling relationship between decision intent and future spatiotemporal state, in order to predict future congestion caused by the preliminary scheduling intent set.

[0103] This embodiment, based on Embodiment 1, improves the spatiotemporal graph attention network predictor. The core technology was explained in detail;

[0104] As mentioned in the formula As shown, the core innovation of this predictor lies in explicitly setting the initial scheduling intent set. As input; It represents the decision-making process. This represents the current environment; in proactive decision-making scenarios like warehouse scheduling, future congestion... yes and The result of the combined effect; if Input only As with existing technology, it cannot distinguish between the drastically different future congestion caused by the two different intentions of AGV_1 about to go to Path_1 and AGV_1 about to go to Path_2;

[0105] Therefore, in this embodiment The design aims to learn decision-making intentions. With future spacetime state The nonlinear coupling relationship is used to predict the initial scheduling intention set. The resulting future congestion;

[0106] By intention By incorporating the input of the prediction model, this invention solves the problem of strong coupling between decision-making and prediction; thus enabling the predictor to... It is no longer a passive observer, but an active predictor capable of deducing "what-if" scenarios; it can accurately distinguish between different scheduling decisions. Different future congestion levels correspond to different This provides a high-precision, decision-correlated congestion cost for subsequent strategy adjustments. This is the technical prerequisite for proactively avoiding congestion rather than responding to it reactively.

[0107] Example 4:

[0108] Determine future congestion costs by using a future spatiotemporal congestion heatmap, including:

[0109] Based on preset node business weights and preset congestion activation functions;

[0110] We perform weighted aggregation and thresholding on the predicted congestion index of each node in the future spatiotemporal congestion heatmap.

[0111] Reduce the high-dimensional future spatiotemporal congestion heatmap to a single scalar of future congestion costs.

[0112] This embodiment, based on embodiment 1, determines the future congestion cost, i.e., the function. The internal implementation logic is specifically defined;

[0113] This quantization process is based on preset node business weights. and the preset congestion activation function ;

[0114] This refers to different nodes Different importance coefficients are assigned; these are pre-set based on business experience and simulation calibration, for example, at the main road intersections of the warehouse. Set the primary channel to 1.5, and the secondary channel to 0.8;

[0115] This refers to a nonlinear transformation function; its source is predefined, for example... ,in This is the congestion tolerance threshold, which can be calibrated based on acceptable congestion levels in historical data, such as the 80th percentile. ;

[0116] Heat map of future spatiotemporal congestion Predicted congestion index for each node Perform weighted aggregation and thresholding; this process corresponds to the formula ; Thresholding filters out values ​​below For example, a tolerable congestion level of 0.7 only focuses on... The portion exceeding the threshold; through Perform weighted aggregation to make the business weights Higher-than-threshold congestion on main roads incurs higher costs than congestion on secondary roads. ;

[0117] The ultimate goal of this step is to create a high-dimensional future spacetime congestion heatmap. A high-dimensional vector or matrix can be reduced to a single scalar of future congestion costs. This is to facilitate inputting the Lagrange solution framework;

[0118] By introducing business weights and activation function This invention achieves a business-oriented cost quantification; it ensures that the decision-maker... Optimized cost It refers to the truly effective costs that align with business logic, such as severe congestion on main roads, rather than the average cost across all nodes; this dimensionality reduction and quantification method enables high-dimensional congestion prediction maps to be more accurate. Scalar optimization objectives that can be reinforced by reinforcement learning Efficient utilization has built a crucial bridge between forecasting and decision-making.

[0119] Example 5:

[0120] The Lagrange solution framework includes:

[0121] Construct a constrained Markov decision process objective to maximize long-term discounted returns while satisfying the constraint that future congestion costs do not exceed a preset congestion cost threshold.

[0122] By introducing a cost-penalty hyperparameter as a Lagrange multiplier, the objective of the constrained Markov decision process is transformed into an unconstrained optimization objective.

[0123] The cost penalty hyperparameter is used to balance immediate rewards with future congestion costs.

[0124] This embodiment, based on embodiment 1, specifically defines the internal mechanism of the Lagrange solution framework;

[0125] The framework constructs a constrained Markov decision process (CMDP) objective; the objective is to maximize the long-run discounted return. Furthermore, it must ensure that future congestion costs do not exceed a preset congestion cost threshold. Constraints; this definition ensures the policy The primary goal is business throughput. But it must be within acceptable congestion limits. Within the boundary;

[0126] To solve this CMDP, this embodiment introduces a cost penalty hyperparameter. As a Lagrange multiplier, it transforms the constrained Markov decision process objective into an unconstrained optimization objective, i.e. ;

[0127] Among them, cost penalty hyperparameter Used to balance instant rewards and future congestion costs ; This is the core of the framework; its origin lies in the fact that it is not a preset hyperparameter, but an adjustable parameter that is automatically updated during training using gradient ascent or dual gradient descent; its function is... Updates and constraints Whether it is strongly correlated; if there is actual congestion Continue to exceed , The value will be automatically increased, thereby reaching the target. China and Canada The penalty weight forces the strategy More conservative sacrifice To reduce Conversely, if congestion is much lower , Automatically lowered, incentive strategy More radical pursuit ;

[0128] By introducing the Lagrange solution framework, this invention provides a principled approach to handling the multi-objective conflict between reward maximization and cost constraints. Its automatic adjustment mechanism enables it to adapt to cost-benefit exchange rates, avoiding the need for manual trial and error in weight adjustment; this makes the scheduler... Able to meet Under the premise of this rigid business SLA, adaptively find ways to maximize throughput. The Pareto optimal strategy achieves robust and efficient decision-making under complex constraints.

[0129] Example 6:

[0130] A pre-defined congestion cost threshold is set, calibrated using the total discounted congestion cost of the simulation baseline strategy, and determined in conjunction with pre-defined constraint factors.

[0131] This embodiment, based on embodiment 5, sets a preset congestion cost threshold. The method for determining it was specifically limited, which solved the problem. How to set it up;

[0132] Preset congestion cost threshold The total discounted congestion cost of the baseline strategy is simulated. Calibration is performed; the simulation baseline strategy refers to a simple, existing scheduling strategy, such as FIFO or the shortest path strategy; the calibration is implemented by running the baseline strategy, such as FIFO, in a simulation environment for a sufficient period of time, for example, covering 10,000 order tasks; the total discount congestion cost generated is then measured and calculated. , recorded as ; This represents the average congestion level under current technology;

[0133] Combined with preset constraint factors Determine; constraint factors It is a preset hyperparameter, such as This represents the expectations of business managers for congestion improvement; The final formula is:

[0134] ;

[0135] This embodiment provides a reproducible and evidence-based approach. Calibration method; It is no longer a number set out of thin air, but rather anchored to a baseline strategy. The actual performance; through setting For example, 0.8, the present invention is a constrained optimization objective. It provides a clear and quantifiable improvement target: congestion must be reduced by 20% compared to the FIFO strategy, which greatly enhances the feasibility, interpretability, and business alignment of the algorithm constraints.

[0136] Example 7:

[0137] This method also includes a closed-loop training step for model parameters:

[0138] Based on the actual congestion situation observed in the future and the prediction residual of the future spatiotemporal congestion heat map, gradient descent is used to update the network parameters of the spatiotemporal graph attention network predictor.

[0139] Based on the actual order rewards and actual congestion costs, a policy gradient algorithm is used to update the policy network of the constraint reinforcement learning scheduler.

[0140] This embodiment, based on embodiment 1, adds a closed-loop training step for model parameters to ensure... and Both models can continuously self-optimize; this training step is continuously performed during the actual operation of the system;

[0141] For spatiotemporal graph attention network predictors Training:

[0142] Based on future actual observations of congestion. Residual of the predicted future spatiotemporal congestion heatmap Gradient descent is used to update network parameters. ;

[0143] It refers to Make decisions at all times Afterwards, Nodes observed in real time through IoT sensors The actual congestion situation; the predicted residual is... The goal of gradient descent is to minimize Supervised training loss function :

[0144] ;

[0145] The purpose of this training is to enable The predictions are constantly getting closer to physical reality, narrowing the simulation-reality gap and ensuring Predicted It is accurate;

[0146] Constraint Reinforcement Learning Scheduler Training:

[0147] Rewards based on actual orders Compared to the actual cost of congestion The policy network is updated using the policy gradient algorithm;

[0148] It refers to Execute at all times The reward for the actual order throughput obtained by the system afterwards; It refers to according to Actual congestion at any given moment The calculated actual cost is Policy gradient algorithms, such as PPO and TRPO, utilize these... and Update the actual value The strategy network and value network, to maximize ;at the same time, Also based on and Gradient ascent update is performed based on comparisons;

[0149] This embodiment constructs a closed-loop self-optimizing system with two models and two objectives; predictor Through supervised learning Learning the laws of physics ensures accurate predictions; decision-making system Through reinforcement learning Learning scheduling strategies ensures optimal decision-making; both models use actual observation data. , , Training was conducted to enable the entire system to continuously adapt to dynamic changes in the warehouse environment, such as layout changes and the addition or removal of AGVs, thereby achieving long-term robustness of the model and continuous iterative improvement of performance.

[0150] Example 8:

[0151] Please see Figure 2 A deep learning-based dynamic scheduling and optimization system for warehousing and logistics includes:

[0152] The data acquisition module is used to collect static map data and dynamic sensor data of the warehouse environment;

[0153] The state construction module is used to construct the dynamic spatiotemporal map state at the current moment based on static map data and dynamic sensor data.

[0154] The scheduling optimization module is used to respond to the business logic rules of the warehouse management system and trigger the scheduling optimization process. The scheduling optimization module includes:

[0155] The intent generation unit is used to generate a preliminary set of scheduling intents based on the dynamic spatiotemporal graph state by a constraint reinforcement learning scheduler.

[0156] The congestion prediction unit is used to combine the dynamic spatiotemporal map status with the preliminary scheduling intention set, and predict the future spatiotemporal congestion heat map through the spatiotemporal map attention network predictor.

[0157] The cost quantification unit is used to quantify the future spatiotemporal congestion heatmap through a custom aggregation function to determine the scalar future congestion cost.

[0158] The strategy correction unit is used to correct the initial scheduling intention set based on future congestion costs and immediate rewards, and generate the final scheduling intention by using the Lagrange solution framework of the constraint reinforcement learning scheduler.

[0159] The scheduling execution module is used to output the final scheduling intent to the warehouse control system for execution.

[0160] This embodiment provides a deep learning-based dynamic scheduling and optimization system for warehouse logistics, which can be deployed on cloud servers, edge computing devices, or a combination thereof.

[0161] The system specifically includes:

[0162] The data acquisition module is designed to collect operational data. In this embodiment, it corresponds to the IoT sensors deployed in the warehouse, the AGV vehicle-mounted sensors, and the database interface with WMS / WCS, and is used to collect static map data and dynamic sensor data of the warehouse environment as described in Embodiment 1.

[0163] The state construction module aims to build a unified state view; in this embodiment, it is a data processing server or service that receives data from the data acquisition module to construct the dynamic spatiotemporal graph state at the current moment. ;

[0164] The scheduling optimization module is the core of the system's decision-making; it is deployed on an edge computing server or in the cloud to respond to the business logic rules of the warehouse management system and trigger the scheduling optimization process; this module is further decoupled into four collaborative units:

[0165] Intent generation unit, corresponding to constraint reinforcement learning scheduler A policy network is used to generate a preliminary set of scheduling intentions based on the dynamic spatiotemporal graph state. ;

[0166] Congestion prediction unit, corresponding to spatiotemporal graph attention network predictor Used to receive and A heat map of future spatiotemporal congestion can be predicted. ;

[0167] Cost quantification unit, corresponding to a custom aggregation function The computational logic is used to determine the scalar future congestion cost from the future spatiotemporal congestion heatmap. ;

[0168] Policy correction unit, corresponding to the constraint reinforcement learning scheduler The evaluation and optimization logic of the Lagrange solution framework is used to modify the final scheduling intention based on future congestion costs and immediate rewards. ;

[0169] The scheduling execution module is the system's execution interface; in this embodiment, it corresponds to the interface of the warehouse control system (WCS) and is used to receive the output from the scheduling optimization module. It then outputs the final scheduling intent to the warehouse control system for execution.

[0170] This system provides a complete system architecture for implementing the intent-prediction-correction closed-loop scheduling method described in Embodiment 1. Through the close coupling and collaborative work of the four units within the scheduling optimization module—intent generation, congestion prediction, cost quantification, and strategy correction—this system can efficiently execute the complete decision-making closed loop from state perception to intent generation, and then to future prediction, cost constraints, and strategy correction. The system architecture is clear and the functions are decoupled, enabling the generation of predictive final scheduling intents. While ensuring high order throughput, it proactively avoids potential traffic bottlenecks, significantly improving the overall operational efficiency and stability of the warehousing and logistics system.

[0171] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A deep learning-based dynamic scheduling optimization method for warehouse logistics, characterized in that, include: Collect static map data and dynamic sensor data of the warehouse environment; Based on static map data and dynamic sensor data, construct a dynamic spatiotemporal map of the current moment. In response to the business logic rules of the warehouse management system, a scheduling optimization process is triggered, including: Step 1: Based on the dynamic spatiotemporal graph state, a preliminary scheduling intent set is generated by a constraint reinforcement learning scheduler. Step 2: Combining the dynamic spatiotemporal map state with the preliminary scheduling intention set, the spatiotemporal map attention network predictor is used to predict the future spatiotemporal congestion heat map. The spatiotemporal map attention network predictor explicitly takes the preliminary scheduling intention set as input to learn the nonlinear coupling relationship between decision intention and future spatiotemporal state, so as to predict the future congestion caused by the preliminary scheduling intention set. Step 3: Quantify the future spatiotemporal congestion heatmap using a custom aggregation function to determine the scalar future congestion cost, including: Based on preset node business weights and preset congestion activation functions; We perform weighted aggregation and thresholding on the predicted congestion index of each node in the future spatiotemporal congestion heatmap. The high-dimensional future spatiotemporal congestion heatmap is reduced to a single scalar of future congestion cost; the calculation method of the custom aggregation function is as follows. ； In the formula, A scalar for the cost of future congestion; For user-defined aggregate functions; Heat map of future spatial and temporal congestion; For all nodes in the graph; For nodes The predicted congestion index; For nodes Business weight; Let be the congestion activation function, and be a nonlinear transformation function; Step 4: Based on future congestion costs and immediate rewards, the initial scheduling intent set is revised using the Lagrange multiplication table framework of the constraint reinforcement learning scheduler to generate the final scheduling intent. The Lagrange multiplication table framework includes: Construct a constrained Markov decision process objective to maximize long-term discounted returns while satisfying the constraint that future congestion costs do not exceed a preset congestion cost threshold. By introducing a cost-penalty hyperparameter as a Lagrange multiplier, the objective of the constrained Markov decision process is transformed into an unconstrained optimization objective. Among them, the cost penalty hyperparameter is used to balance immediate rewards with future congestion costs; Step 5: Output the final scheduling intent to the warehouse control system for execution.

2. The deep learning-based dynamic scheduling optimization method for warehouse logistics according to claim 1, characterized in that, Dynamic spatiotemporal graph states include: A graph node set, which includes agile autonomous robots, shelves, and intersections; The set of physical connection edges between graph nodes; The dynamic attribute feature matrix includes the position, power, speed, and load attributes of the agile autonomous robot, as well as the inventory attributes of the shelf.

3. The deep learning-based dynamic scheduling optimization method for warehousing and logistics according to claim 2, characterized in that, A pre-defined congestion cost threshold is set, calibrated using the total discounted congestion cost of the simulation baseline strategy, and determined in conjunction with pre-defined constraint factors.

4. The deep learning-based dynamic scheduling optimization method for warehouse logistics according to claim 1, characterized in that, It also includes the closed-loop training step for model parameters: Based on the actual congestion situation observed in the future and the prediction residual of the future spatiotemporal congestion heat map, gradient descent is used to update the network parameters of the spatiotemporal graph attention network predictor. Based on the actual order rewards and actual congestion costs, a policy gradient algorithm is used to update the policy network of the constraint reinforcement learning scheduler.

5. A deep learning-based dynamic scheduling and optimization system for warehousing and logistics, characterized in that, include: The data acquisition module is used to collect static map data and dynamic sensor data of the warehouse environment; The state construction module is used to construct the dynamic spatiotemporal map state at the current moment based on static map data and dynamic sensor data. The scheduling optimization module is used to respond to the business logic rules of the warehouse management system and trigger the scheduling optimization process. The scheduling optimization module includes: The intent generation unit is used to generate a preliminary set of scheduling intents based on the dynamic spatiotemporal graph state by a constraint reinforcement learning scheduler. The congestion prediction unit combines the dynamic spatiotemporal map state with the preliminary scheduling intention set and uses the spatiotemporal map attention network predictor to predict the future spatiotemporal congestion heat map. The spatiotemporal map attention network predictor explicitly takes the preliminary scheduling intention set as input and learns the nonlinear coupling relationship between decision intention and future spatiotemporal state in order to predict future congestion caused by the preliminary scheduling intention set. The cost quantification unit is used to quantify the future spatiotemporal congestion heatmap using a custom aggregation function to determine a scalar future congestion cost. This includes: based on preset node business weights and preset congestion activation functions; performing weighted aggregation and thresholding on the predicted congestion indices of each node in the future spatiotemporal congestion heatmap; and reducing the high-dimensional future spatiotemporal congestion heatmap to a single future congestion cost scalar. The custom aggregation function is calculated as follows: In the formula, A scalar for the cost of future congestion; For user-defined aggregate functions; Heat map of future spatial and temporal congestion; For all nodes in the graph; For nodes The predicted congestion index; For nodes Business weight; Let be the congestion activation function, and be a nonlinear transformation function; The strategy correction unit is used to correct the initial scheduling intention set and generate the final scheduling intention based on the future congestion cost and the immediate reward, using the Lagrange solution framework of the constrained reinforcement learning scheduler. The Lagrange solution framework includes: constructing a constrained Markov decision process objective to maximize long-term discounted returns while satisfying the constraint that the future congestion cost does not exceed a preset congestion cost threshold; introducing a cost penalty hyperparameter as a Lagrange multiplier to transform the constrained Markov decision process objective into an unconstrained optimization objective; wherein, the cost penalty hyperparameter is used to balance the immediate reward and the future congestion cost. The scheduling execution module is used to output the final scheduling intent to the warehouse control system for execution.

Images