A multi-modal path planning and dynamic obstacle avoidance control system based on intelligent warehousing

Abstract

The application provides a multi-modal path planning and dynamic obstacle avoidance control system based on intelligent warehousing, and belongs to the technical field of warehouse path control. The application solves the technical problems of single sensing dimension of the existing intelligent warehousing control system, and difficulty in path planning and dynamic obstacle avoidance in complex warehouse environment. The application comprises a multi-source data acquisition layer, a multi-modal data fusion layer, a digital twin and simulation layer, a two-stage multi-modal path planning core layer, a predictive dynamic obstacle avoidance control layer, a full device group intelligence collaborative scheduling layer, an edge cloud collaborative processing layer, a decision audit and safety compliance layer, an intelligent adaptation and self-evolution layer, a semantic SLAM and semantic map layer, and a modular plug-in layer. The application realizes full-process closed-loop path planning and dynamic obstacle avoidance control system through key technologies such as optimized multi-modal data fusion, high-fidelity twin simulation, two-stage path planning, dynamic obstacle avoidance prediction, full-device multi-agent collaboration, and double-loop evolution.

Description

Technical Field

[0001] This invention belongs to the field of warehouse path control technology, and relates to a multimodal path planning and dynamic obstacle avoidance control system, particularly a multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing. Background Technology

[0002] As the smart warehousing industry upgrades towards large-scale, automated, and multi-scenario applications, the collaborative operation of automated equipment such as AGVs, robotic arms, and stacker cranes has become a mainstream requirement. Path planning and dynamic obstacle avoidance capabilities directly determine the efficiency, safety, and compatibility of warehousing operations. Existing technologies present numerous technical bottlenecks in related path planning and obstacle avoidance solutions.

[0003] The existing system only targets a single mobile robot, has a limited environmental perception dimension, and is weak in resisting interference from complex warehouse environments (light changes, shelf obstructions, oil stains on the ground). The algorithm only uses DDPG (Depth Deterministic Gradient Strategy) to achieve passive action selection for a single robot, lacking a two-stage architecture of static global planning and dynamic local adjustment, and lacking the ability to predict obstacle movement trajectories, resulting in delayed obstacle avoidance response. It does not involve multi-device collaborative scheduling, and can only achieve simple path planning for a single robot, which cannot be adapted to large-scale warehouse scenarios with multiple AGVs and robotic arms. It has not built a complete control system, and only uses a single path planning method, without a closed-loop control system, lacking core modules such as data processing, decision auditing, and scene adaptation. Its performance is fixed after deployment, and it cannot adapt to the needs of multiple industry scenarios. It also lacks self-evolution capabilities and cannot continuously optimize performance from operational data. It has not introduced digital twin technology, and cannot achieve virtual-real linkage simulation and risk-free trial and error.

[0004] Therefore, we propose a multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems in existing technologies by proposing a multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing. The technical problem this invention aims to solve is: how to achieve closed-loop path planning and dynamic obstacle avoidance control throughout the entire process by optimizing key technologies such as multimodal data fusion and high-fidelity twin simulation, and thus constructing a complete control system.

[0006] The objective of this invention can be achieved through the following technical solutions: A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing is characterized by comprising a multi-source data acquisition layer, a multimodal data fusion layer, a digital twin and simulation layer, a two-stage multimodal path planning core layer, a predictive dynamic obstacle avoidance control layer, a full-equipment group intelligent collaborative scheduling layer, an edge-cloud collaborative processing layer, a decision audit and safety compliance layer, an intelligent adaptation and self-evolution layer, a semantic SLAM and semantic map layer, and a modular pluggable layer. The multi-source data acquisition layer collects real-time data on the warehousing environment, the status of all equipment, and operational tasks through various sensors and system interfaces. The multimodal data fusion layer performs spatiotemporal alignment, repair, and denoising on heterogeneous data, and extracts fusion features through the Transformer cross-modal attention mechanism to provide unified input for decision-making. The digital twin and simulation layer constructs a high-precision 1:1 virtual warehouse and full-equipment model, predicts risks and verifies solutions through simulation, and achieves bidirectional virtual-real drive and simulation-as-control. The two-stage multimodal path planning core layer adopts… The system employs a two-stage architecture combining GCN static global optimization and PPO dynamic local response to generate and dynamically adjust the optimal path. The predictive dynamic obstacle avoidance control layer predicts obstacle trajectories using an LSTM network, generates alternative paths in real time, and achieves multi-device collaborative obstacle avoidance through multi-agent game theory. The all-device group intelligent collaborative scheduling layer adopts a centralized-distributed hybrid decision-making architecture and multi-agent reinforcement learning to coordinate collaborative operations among various heterogeneous devices. The edge-cloud collaborative processing layer performs real-time control at the edge and global scheduling and digital twin simulation at the cloud. The decision audit and security compliance layer records the decision-making process and performs compliance review and data encryption. The intelligent adaptation and self-evolution layer achieves rapid system adaptation and continuous self-optimization based on transfer learning and a dual-loop evolution mechanism. The semantic SLAM and semantic map layer constructs an environmental map containing semantic information, combining human intent understanding to achieve human-machine collaboration and dynamic right-of-way allocation. The modular plug-and-play layer defines standardized interfaces and a hardware abstraction layer to enable plug-and-play functionality for third-party algorithms and heterogeneous devices.

[0007] The multi-source data acquisition layer includes an environmental perception module, a full device status module, and a task and system data module. The environmental perception module integrates LiDAR, a 3D vision camera, an ultrasonic sensor, and an IoT sensor. LiDAR provides high-precision spatial positioning and map building, the 3D vision camera performs object recognition and pose estimation, the ultrasonic sensor assists in blind spot detection, and the IoT sensor collects environmental temperature and humidity parameters. Through multi-source complementarity, it comprehensively perceives static structures, dynamic obstacles, and environmental conditions. The full device status module includes a real-time device data acquisition unit, a historical maintenance database, and a device parameter configuration library. The real-time device data acquisition unit collects the position and speed of all devices in real time. The system collects operational status data such as power consumption and load, and its historical maintenance database aggregates historical fault and maintenance records. The equipment parameter configuration library integrates the equipment's dynamic parameters and communication protocols, enabling comprehensive monitoring of equipment from real-time operation to long-term health. The task and system data module includes an enterprise information system interface, a digital twin data interface, and a CAD / BIM data interface. The enterprise information system interface obtains order, inventory, and task instruction operation information from the upper-level business system. The CAD / BIM data interface accesses the warehouse geometric model from CAD / BIM and the simulation data from the digital twin. The digital twin data interface provides the system with operation requirements and virtual simulation data sources, supporting task-driven planning and scheduling.

[0008] The multimodal data fusion layer includes a spatiotemporal alignment module, a data repair and noise suppression module, and a cross-modal feature extraction and fusion module. The spatiotemporal alignment module includes a sliding window aligner, a clock synchronization service, and a cross-device timing calibrator. The clock synchronization service establishes a unified timestamp reference for all data sources. The sliding window aligner handles the differences in acquisition frequencies between different sensors, performing data matching within a preset time-domain tolerance. The cross-device timing calibrator specifically synchronizes the action timing of AGVs and robotic arms, ensuring accurate alignment of collaborative operation data. The data repair and noise suppression module includes a linear interpolation compensator, an outlier filter, and a wavelet threshold denoising processor. For data breaks caused by temporary sensor malfunctions, linear interpolation is used for filling and repair. A filter based on the 3σ criterion identifies and removes outlier data deviating from the normal range. Wavelet transform... The signal is decomposed, and the high-frequency coefficients representing noise are thresholded before reconstruction, effectively suppressing various sensor noises and improving the signal-to-noise ratio and continuity of the data. The cross-modal feature extraction and fusion module includes a Transformer cross-modal attention mechanism unit, a spatiotemporal feature extractor, and a feature standardization output unit. The Transformer cross-modal attention mechanism unit calculates and strengthens the correlation weights between different modal data such as laser point clouds, visual images, IoT data, device status, and task information. Based on this, the spatiotemporal feature extractor analyzes the spatial distribution and temporal series dependencies of the data, extracting multi-dimensional deep features such as environmental topology, obstacle movement trends, device status, and task priority. The feature standardization output unit integrates and standardizes the extracted heterogeneous features, outputting a unified, high-quality fused feature vector for use by upper-level decision-making.

[0009] The digital twin and simulation layer includes a digital twin engine module, a warehouse 3D simulation module, a warehousing equipment 3D simulation module, a transportation equipment 3D simulation module, a work scene simulation package module, a map dynamic update module, a virtual-physical linkage simulation module, and a predictive maintenance module. The digital twin engine module includes a high-fidelity modeling core unit, a data synchronization service unit, and a reverse control interface unit. The high-fidelity modeling core unit constructs a high-fidelity virtual model corresponding to the physical warehouse at a 1:1 scale based on multimodal fusion data and original CAD / BIM data. The data synchronization service unit enables real-time bidirectional driving between physical data and the virtual model. The reverse control interface unit can map physical data to the virtual model. The simulated environment allows control commands validated in the virtual environment to be directly and indiscriminately sent to physical devices for execution, achieving "simulation as control." The warehouse 3D simulation module includes a CAD / BIM parser and reconstructor, a parametric model library, and a physical property mapper. The CAD / BIM parser and reconstructor imports and accurately parses the warehouse's CAD or BIM design data. The parametric model library automatically constructs a complete 3D model of the warehouse with millimeter-level precision, including all structures, not only restoring the geometric appearance. The physical property mapper maps physical properties to the model, providing a realistic physical environment foundation for simulation. The 3D simulation module for warehousing equipment includes a parametric model library for fixed equipment and a dynamic parameter mapping... The system includes a simulator for the device and communication protocol interface, and a parametric model library for fixed equipment. This library provides 1:1 3D models of various fixed storage equipment, including precise external appearance and internal joint structures. A dynamic parameter configurator allows configuration of dynamic parameters such as mass and speed thresholds. The communication protocol interface simulator simulates the actual communication control protocol, accurately replicating the equipment's motion trajectory, operation process, and fault status in a virtual environment, enabling visualized simulation and performance analysis of fixed equipment operation. The 3D simulation module for transportation equipment includes a parametric model library for mobile devices, a kinematics simulator, and a load state simulator. The parametric model library for mobile devices constructs 3D models of mobile transportation equipment with detailed kinematic models. By integrating kinematic equations and control interfaces through a kinematic simulator and a load state simulator, the system realistically simulates the dynamic characteristics of equipment's driving, steering, acceleration and deceleration, as well as the impact of load changes on motion. It supports three-dimensional simulation and evaluation of single-vehicle or multi-vehicle collaborative transportation scenarios. The work scenario simulation package module includes a library of normal and abnormal scenario models and an adversarial training scenario generator. The normal and abnormal scenario model library is pre-loaded with a rich digital scenario library, containing simulation models of emergency scenarios such as sensor failure, network latency, random motion of dynamic obstacles, equipment actuator failure, extreme environments, and even fire and water leakage. The adversarial training scenario generator is used for systematic adversarial training, stress testing, and emergency plan simulation.The map dynamic update module includes a real-time data listener, an automatic map element updater, and a manual calibration interface. The real-time data listener continuously monitors the multimodal fusion data stream. When it detects changes in obstacle positions, equipment status, or environmental parameters in the real environment, the automatic map element updater automatically modifies these changes synchronously in the digital twin map. The manual calibration interface provides a manual calibration interface to ensure that the virtual map remains consistent with the physical world, providing a reliable environmental foundation for planning decisions. The virtual-real linkage simulation module includes a parallel solution inferrer, a safety / efficiency evaluator, and an optimal solution provider. In the digital twin environment, the parallel solution inferrer performs advanced parallel simulations of various candidate strategies generated from path planning and obstacle avoidance control. The safety / efficiency evaluator calculates each optimal solution through high-fidelity simulation. The feasibility, safety, collision risk, and operational efficiency of each path are assessed in advance to predict and avoid conflicts and failures. The optimal solution issuer directly issues the optimal solution after comprehensive evaluation for execution, realizing "simulation verification before actual execution" and reducing the cost and risk of physical trial and error. The predictive maintenance module includes a critical component degradation model library, a real-time performance data analyzer, and an early warning trigger and report generator. The critical component degradation model library is based on real-time equipment operating data and historical fault databases to build mathematical models of lifespan and performance degradation for critical components. The real-time performance data analyzer continuously analyzes the deviation between real-time data and model prediction curves. The early warning trigger and report generator provides early warnings of potential failure risks before actual component failures occur and generates maintenance suggestions, realizing the transformation from "retroactive maintenance" to "proactive prevention."

[0010] Digital twin: The composite formula of simulation as control + virtual-real bidirectional drive is as follows: ; In the formula, Represents the virtual twin model at time t; Indicates the physical warehouse state at time t; F represents the physical-to-virtual mapping; This represents the optimal control command output by the simulation; G represents virtual-to-physical inverse control.

[0011] The core layer of the dual-stage multimodal path planning includes a GCN static global optimization module, a PPO dynamic local response module, and a multimodal planning mode selection module. The GCN static global optimization module includes a warehouse topology graph builder, a graph convolutional neural network node feature aggregator, and a multi-objective cost optimization function unit. The warehouse topology graph builder constructs a topology graph reflecting the connectivity of warehouse space based on digital twin maps and globally fused data. The graph convolutional neural network node feature aggregator uses a graph convolutional neural network to learn and aggregate the features of nodes in the graph, generating the globally optimal basic path. The multi-objective cost optimization function unit's optimization objective is a configurable multi-objective cost function that comprehensively considers time, energy consumption, equipment wear and tear, and energy recovery efficiency. The PO dynamic local response module includes a PPO policy network unit, an improved UKF motion parameter estimator, and a real-time feedback interface unit. The real-time feedback interface unit receives real-time sensor data, equipment status, and simulation feedback from the digital twin online. The PPO policy network unit's PPO reinforcement learning algorithm performs local fine-tuning and action decisions on the global basic path generated by GCN based on real-time environmental interaction experience to cope with sudden obstacles. The multimodal planning mode selection module includes a mode decision-maker and an algorithm fusion executor. The mode decision-maker adaptively selects a single optimal mode or triggers a fusion strategy of multiple modes based on contextual information such as current task attributes, equipment type, and real-time environmental congestion. The fusion executor coordinates the selected algorithms to work together. The improved UKF motion parameter estimator is an improved unscented Kalman filter algorithm. Based on the standard unscented Kalman filter, it introduces AGV load weight factor, ground friction coefficient adaptive correction term and sigma point weighted optimization strategy to estimate AGV kinematic parameters in real time at a high frequency of 200Hz, thereby improving the control accuracy under ground slippage and load change conditions. The state update formula for the improved UKF algorithm is as follows: ; Its gain correction formula is: ; In the formula: This represents the optimal state estimate of the AGV at time k after filtering correction, including kinematic parameters such as position, speed, attitude, and steering angle. This represents the predicted state value at time k, which is calculated based on the state at the previous time and the motion model. This represents the adaptive Kalman gain at time k, used to balance the weights of the "predicted value" and the "observed value". This represents the actual observation value of the sensor at time k, which comes from data collected by lidar, vision, odometer, IMU, etc. This represents the observed predicted value corresponding to the state at time k, which is obtained by mapping the state variables. The state prediction covariance at time k represents the confidence level / uncertainty of the predicted value. This represents the observation matrix, used to map state variables to observations; This represents the transpose of the observation matrix; This represents the observation noise covariance, which reflects the noise level of the sensor itself. This is the load adaptive coefficient, a newly added category used to adjust the filter gain according to the load strength of the AGV; This indicates the real-time load quality of the AGV, i.e., the current load weight, which will change dynamically from heavy load to light load to empty load. Ground friction correction coefficient is a new category used to adjust filtering based on the smoothness / roughness / slipperiness of the ground. This represents the real-time ground adhesion coefficient; a smaller value indicates a more slippery surface, while a larger value indicates a less slippery surface; -1 indicates matrix inversion. The improved UKF estimates the AGV's kinematic parameters in real time at a high frequency of 200Hz, and uses adaptive correction based on load and ground friction to ensure high-precision state estimation and path tracking stability under load changes and ground slippage conditions.

[0012] Global phase: The improved GCN spatiotemporal weighted path planning formula is as follows: ; In the formula, This represents the node feature matrix output by the (l+1)th layer graph convolution, which is the new feature obtained after convolution for each point in the warehouse map; This represents a non-linear activation function used to enhance the model's expressive power and extract features from complex warehousing environments. This represents the warehouse topology adjacency matrix with self-loops, i.e., the connectivity between various points in the warehouse; Indicates and The corresponding degree matrix shows how many paths each point is connected to. Represents the normalized inverse square root of the degree matrix, used to normalize graph convolutions to prevent numerical explosion / gradient vanishing; The node feature matrix represents the input of the l-th layer, which contains information including, but not limited to, point coordinates, passage status, device location, and obstacles. The weight matrix represents the convolution of the l-th layer graph, and the optimal parameters for path planning learned automatically by the neural network. represents the spatiotemporal weight coefficient, with a value range of 0.2 to 0.5, which controls the influence of "historical efficiency, current congestion, and future tasks" on the path; T represents the spatiotemporal feature matrix of the node, including three types of warehouse-specific features: historical traffic efficiency, current congestion degree, and future task prediction value; The spatiotemporal feature enhancements enable path planning to predict congestion and dynamically optimize; output: globally optimal basic path Pglobal. Local stage: The improved PPO dynamic path optimization formula is as follows: ; In the formula, This represents the loss function value of the improved PPO policy network; This represents the mathematical expectation based on time step t; Indicates the probability ratio of the strategies; Represents the dominance function; This indicates a truncation function to limit the policy update magnitude; This represents the clipping threshold, with a value ranging from 0.1 to 0.2. This represents the obstacle distance penalty coefficient, which is a positive hyperparameter. Indicates the distance between the AGV and the nearest obstacle; This represents the task priority reward coefficient, which is a positive hyperparameter. This represents the priority weight of the current task; the final output is the locally fine-tuned optimal path Plocal. The final composite formula for two-stage path planning is:

[0013] That is, the final optimal path = fusion of improved GCN spatiotemporal global planning + improved PPO penalty-reward local planning; In the formula, This represents the globally optimal and locally safe final path that the AGV ultimately executes. This represents a fusion function of global and local paths, combining macro-planning with dynamic adjustments; This represents an improved graph convolutional neural network that incorporates spatiotemporal weighting factors to generate globally optimal paths. This represents the global base path for warehousing output by the improved GCN; This indicates an improved proximal policy optimization algorithm that incorporates obstacle penalties and task rewards to generate locally dynamically corrected paths; This represents the local obstacle avoidance fine-tuning path output by the improved PPO; the formula clearly expresses the core architecture of the two-stage path planning of "global static planning + local dynamic correction". First, the global optimal path is obtained by improving GCN, and then real-time obstacle avoidance and priority adjustment are performed by improving PPO. Finally, a safe, efficient and stable final driving path is obtained by fusion.

[0014] The predictive dynamic obstacle avoidance control layer includes an interruption event detection and obstacle classification module, a trajectory prediction module, an alternative path solving and speed control module, and an obstacle avoidance coordination module. The interruption event detection and obstacle classification module includes a computer vision real-time processor, a multimodal data association analyzer, and an obstacle classifier. The computer vision real-time processor processes data streams fused from multimodal sensors such as cameras and LiDAR in real time. The multimodal data association analyzer quickly detects sudden obstacles in the environment with a response latency controlled within 0.3 seconds. Simultaneously, the obstacle classifier classifies the obstacles. The trajectory prediction module includes an LSTM network prediction model unit, a short-term trajectory extrapolator, and a collision risk predictor. The LSTM network prediction model unit learns the historical motion sequences of dynamic obstacles to predict their short-term future trajectories and speeds. The short-term trajectory extrapolator, combined with environmental information from the digital twin scenario, extrapolates the possible routes of obstacles in real time and pre-calculates the spatiotemporal intersection with the path of the local device. The collision risk predictor proactively predicts potential collisions. The collision risk and alternative path solution and speed control module includes a spatiotemporal alternative path fast solver, a speed adaptive controller, and a path constraint filter. When a collision risk is predicted, the spatiotemporal alternative path fast solver quickly generates one or more feasible collision-free alternative paths based on the current environment map and device status using a spatiotemporal alternative path algorithm. The path constraint filter automatically filters road segments that do not meet constraints such as height and weight limits. The speed adaptive controller dynamically adjusts the vehicle's speed according to the distance, movement trend, and path conditions of obstacles. The obstacle avoidance coordination module includes a multi-device communication coordinator, an avoidance priority arbitrator, and a multi-agent game decision-maker. The multi-device communication coordinator coordinates the obstacle avoidance intentions and plans of each device and arbitrates temporary right-of-way based on factors such as task priority and device status. The multi-agent game decision-maker introduces a multi-agent game mechanism, allowing device agents to conduct distributed negotiation in a virtual environment. The avoidance priority arbitrator quickly reaches the optimal group avoidance sequence and path adjustment scheme, avoiding new conflicts or deadlocks caused by individual obstacle avoidance.

[0015] The all-device group intelligent collaborative scheduling layer includes a task dynamic allocation module, a path conflict resolution module, and a group intelligent decision-making module. The task dynamic allocation module includes a centralized global optimizer, a multi-dimensional device status evaluator, and a real-time task queue and traffic balancer. The multi-dimensional device status evaluator continuously acquires the real-time status of all devices, the real-time task queue and traffic balancer continuously acquires the global order task queue, and the centralized global optimizer dynamically decomposes and allocates orders to the most suitable devices by comprehensively considering task urgency, device adaptability, and overall warehouse traffic through optimization algorithms. The path conflict resolution module includes a multi-device path intersection real-time monitor, a dynamic avoidance node optimizer, and a timing and restricted area adjuster. The multi-device path intersection real-time monitor monitors the planned paths and current locations of all mobile devices in real time, predicting potential spatiotemporal intersection conflict points. Once a conflict risk is detected, the dynamic avoidance node optimizer adjusts the passage order of devices and optimizes temporary waiting times through calculation. Nodes, or time-series and restricted zone adjusters, set instantaneous dynamic restricted zones on a digital map to guide devices to safely and efficiently avoid each other in a non-intrusive manner. The swarm intelligence decision-making module includes a centralized-distributed hybrid decision-making architecture unit, a multi-agent reinforcement learning cluster unit, a virtual parallel inference environment unit, and a fast negotiation and consensus protocol unit. In the digital twin environment, the centralized-distributed hybrid decision-making architecture unit creates a corresponding agent agent for each physical device. Under the guidance of a centralized global goal, each agent, based on distributed local perception, utilizes the multi-agent reinforcement learning and game theory model of the multi-agent reinforcement learning cluster unit. The virtual parallel inference environment unit infers multiple possible sequences of cooperative actions in parallel within the next few seconds in a virtual scene. When a spatiotemporal conflict is predicted, the fast negotiation and consensus protocol unit exchanges information among agents through a fast communication protocol to conduct distributed negotiation, in order to reach a locally optimal consensus and generate the final cooperative instruction.

[0016] The compound formula for predictive obstacle avoidance is: ; In the formula, This indicates the predicted position of the obstacle at time k in the future; This represents a bidirectional LSTM model with an attention mechanism. This indicates the historical trajectory of the obstacle over the past L frames; This represents the set of optimal control instructions for multi-agent cooperation. Indicates the distance between the AGV and the obstacle; This indicates the risk of collisions between AGVs; Indicates the task priority of the i-th AGV; This represents the obstacle avoidance weight coefficient; N represents the number of AGVs.

[0017] The edge-cloud collaborative processing layer includes an edge computing node module and a cloud scheduling center module. The edge computing node module includes a real-time data stream processor, a local intelligent agent policy executor, a high-frequency control command generator, and an edge storage. The real-time data stream processor directly connects to the sensor network and device controller, performing millisecond-level processing on the generated high-frequency real-time data stream. The local intelligent agent policy executor and high-frequency control command generator execute latency-sensitive core tasks. The edge storage stores and runs localized intelligent agent models, enabling rapid response to environmental changes and synchronizing processing results and summary data to the cloud. The cloud scheduling center module includes a global task and resource scheduler, a data... The system comprises a digital twin simulation cluster unit, a historical big data warehouse, and an algorithm model training and version management center. The global task and resource scheduler focuses on non-real-time, global, and computationally intensive tasks, and is responsible for integrating all warehouse information, performing macro-level task planning, cross-regional resource scheduling, and global path topology optimization. The digital twin simulation cluster unit interfaces with and drives the high-fidelity digital twin system to perform large-scale, multi-scheme virtual simulation and verification. The historical big data warehouse collects all historical and operational data. The algorithm model training and version management center utilizes powerful computing power to perform offline training, parameter iteration, and performance evaluation of algorithm models, and silently distributes the optimized models to edge nodes for updates.

[0018] The composite formula for a clear division of labor between edge and cloud is: ; In the formula, This indicates a real-time task at the edge. This indicates millisecond-level real-time processing; Indicates a global task in the cloud; This represents global scheduling, simulation, and training.

[0019] The decision audit and security compliance layer includes a decision tracing module, an AI compliance self-check module, and a data security protection module. The decision tracing module includes a decision process mirror recorder, an interactive decision tree builder, and a real-time decision deviation scanner and alarm. The decision process mirror recorder records all key decision-making processes in real time. Based on the above data, the interactive decision tree builder can reverse-engineer an interactive and visualized decision path tree. The real-time decision deviation scanner and alarm scans the deviation between the current decision and historical patterns or preset rules in real time, and immediately alarms if the deviation exceeds a safety threshold, ensuring that any abnormal or inefficient decision can be traced, analyzed, and reviewed. The AI ​​compliance self-check module includes a multi-industry compliance rule library, an automated verification engine unit, and a standardized audit report generator. The multi-industry compliance rule library embeds regulations and standards from different industries, and the automated verification engine unit… The system automatically compares and verifies the actual operations of the system with the selected compliance rules. After verification, the standardized audit report generator automatically generates a structured, regulatory-compliant standardized audit report. The data security protection module includes an AES-256 end-to-end encryption / decryption engine unit, an intrusion detection and behavior analysis unit, and a real-time alarm and blocking controller. The AES-256 end-to-end encryption / decryption engine unit uses the AES-256 high-strength encryption algorithm to encrypt control commands, warehouse data, etc., to prevent them from being stolen or tampered with. The intrusion detection and behavior analysis unit continuously monitors network traffic and access behavior, and identifies abnormal or attack behaviors such as "illegally modifying path parameters" and "unauthorized data acquisition" through behavior analysis models. Once detected, the real-time alarm and blocking controller immediately issues an alarm and initiates the blocking mechanism, building a solid data security defense for the system.

[0020] The intelligent adaptation and self-evolution layer includes a scene adaptation module, a self-evolutionary learning engine module, an API template extension module, a human-computer interaction and visualization platform, and a VR / AR human-computer interaction module. The scene adaptation module includes a transfer learning framework unit, a cross-scene parameter auto-adjuster, and a simulation scene rapid configurator. The transfer learning framework unit utilizes transfer learning technology to quickly adapt algorithm models and strategies trained in a certain type of warehouse to a new warehouse environment. The cross-scene parameter auto-adjuster automatically analyzes the characteristics of the new scene and adjusts key parameters such as obstacle avoidance distance and path weight. The simulation scene rapid configurator synchronously configures the simulation scene in the digital twin, achieving rapid cross-scene adaptation. Rapid deployment and performance maintenance significantly reduce retraining and manual configuration costs. The self-evolutionary learning engine module includes an instant decision loop, an offline evolution loop, and an evolutionary driving unit. The instant decision loop runs online, performing parallel deductions in the digital twin and selecting the optimal solution for execution. The offline evolution loop starts during system idle time, comparing actual and simulation results and automatically generating various extreme and fault scenarios using simulation packages. The evolutionary driving unit uses historical and simulation data to perform large-scale, safe batch retraining and evaluation of the core algorithm model, silently updating the entire network after performance improvements, driving continuous self-optimization of the system. The API template extension module includes a standardized API template library and automatic interface adapters, standardizing… The API template library provides a rich set of proven, standardized API connection templates. When needing to interface with external systems, there's no need for development from scratch. Simply use the automatic adapter to call the corresponding template and complete simple configuration to achieve fast and seamless data integration and command communication with mainstream cloud services and enterprise management systems, reducing system integration difficulty and deployment costs. The human-computer interaction and visualization platform includes a 3D panoramic visualization engine, a historical data backtracking analyzer, and a simulation scene editing and injection interface unit. The 3D panoramic visualization engine is deeply connected to the digital twin engine, displaying the entire warehouse's operational status, equipment location, and task progress in real-time and intuitively in 3D form, and enabling historical data backtracking analysis. The system can replay and analyze operations at any point in history. Administrators can manually intervene in the simulation scene editing and injection interface unit to adjust plans, modify parameters, or directly inject new test scenarios in the simulation environment. The VR / AR human-computer interaction module includes AR glasses, a VR immersive simulation environment unit, and a gesture and voice recognition interactive device. Through the AR glasses, administrators can overlay virtual operating data, planned paths, and status alarms of the equipment in their real field of vision. The gesture and voice recognition interactive device can directly perform remote scheduling and intervention through gestures or voice commands. The VR immersive simulation environment unit is used to provide immersive employee training, emergency plan drills, and system maintenance guidance.

[0021] The composite formula for dual-loop self-evolution—combining an online decision-making loop and an offline evolutionary loop—is as follows:

[0022] ; In the formula, This indicates the optimal online strategy in real time. This represents the online decision function, which calculates the optimal strategy in real time based on the old strategy, the physical environment, and the simulation environment. Indicates the old strategy; Indicates the physical environment status; Indicates the state of the simulation environment; This indicates the new strategy after offline evolution; This represents the offline evolutionary training function, which automatically optimizes the algorithm using historical data and adversarial scenario data. This indicates the real-time strategy being used during the job execution, serving as the basis for offline evolution; Represents the historical running dataset; This indicates the generation of a dataset for adversarial scenarios.

[0023] The semantic SLAM and semantic map layer includes a semantic map module, a visual SLAM module, and a human-machine collaboration and intent recognition module. The semantic map module includes a semantic map construction and storage engine unit, a multi-level semantic tag library, and a dynamic semantic element updater. The semantic map construction and storage engine unit integrates high-precision geometric information with rich semantic information, recording spatial structure during map construction. The multi-level semantic tag library also labels the functional attributes of each region or object. The dynamic semantic element updater provides rich context for upper-level planning, enabling the device to understand the environment and make decisions. The visual SLAM module includes a visual odometry, a feature point extractor and matcher, and a dynamic object semantic recognition and tracker. The visual odometry utilizes cameras... The first-class vision sensor performs real-time synchronous positioning and map building during equipment movement. The feature point extractor and matcher are deeply integrated with semantic information: it can locate itself and build a geometric map. The dynamic object semantic recognition and tracker can identify and semantically classify and track dynamic objects in real time. The human-machine collaboration and intention recognition module includes a visual gesture recognition unit, a human posture and intention parser, and a human-machine shared space dynamic right-of-way allocator. The visual gesture recognition unit uses vision equipment to identify the gestures, postures, and movement trends of workers in the warehouse in real time. The human posture and intention parser interprets their operation intentions. Based on this intention recognition and combined with semantic map information, the human-machine shared space dynamic right-of-way allocator dynamically adjusts the passage rights in the human-machine shared space.

[0024] The composite formula for semantic SLAM, human intent understanding, and dynamic right-of-way allocation is: ; In the formula, Represents a semantic map; This represents the semantic SLAM building blocks; Represents visual images; Represents laser point clouds; Indicates the device pose; Indicates the optimal dynamic right-of-way at any given time; Indicates the intention of human manipulation; Indicates AGV safety constraints; Indicate task priority; This indicates the weight of right-of-way allocation.

[0025] The modular pluggable layer includes a standardized interface module and a hardware abstraction module. The standardized interface module includes an algorithm interface protocol library, a service registration and discovery center unit, and a compatibility testing unit. The algorithm interface protocol library defines a unified and clear application programming interface specification and communication protocol for the core functions of the system. Third-party algorithm providers in the service registration and discovery center unit only need to develop algorithm services according to this standard and register them, and the system can automatically identify and call them. The compatibility testing unit, in conjunction with compatibility testing, ensures that algorithms from different sources are plug-and-play and seamlessly replaced in the system, realizing an open and flexible algorithm ecosystem. The hardware abstraction module includes a device driver manager, a general instruction-protocol converter, and a hardware configuration template library. The device driver manager builds a unified "translation" and "adaptation" layer between specific hardware devices and the upper-layer software system. The general instruction-protocol converter converts the control instructions and communication protocols specific to various devices into general instructions that the system can understand. The hardware configuration template library allows the replacement or upgrade of underlying hardware and upper-layer applications to operate through this unified interface, shielding hardware differences.

[0026] Compared with existing technologies, this multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing has the following advantages: This invention utilizes a two-stage planning architecture combining multimodal perception fusion and dynamic / static integration to achieve intelligent decision-making from global optimization to proactive prediction, thereby improving the robustness and efficiency of path planning. Based on high-fidelity digital twin technology, the system constructs a "simulation-as-control" virtual-real linkage and multi-agent collaborative scheduling mechanism, ensuring the safety and overall efficiency of large-scale cluster operations. Employing edge-cloud collaborative processing and a modular plug-and-play design, it achieves optimized computing power and an open ecosystem, guaranteeing system real-time performance, scalability, and ease of integration. Simultaneously, the system possesses full-process decision auditing, security and compliance protection, and self-optimization capabilities based on transfer learning and dual-loop evolution, enabling rapid cross-scenario adaptation and continuous performance improvement. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the system framework of the present invention.

[0028] Figure 2 This is a system framework diagram of the multi-source data acquisition layer in this invention.

[0029] Figure 3This is a system framework diagram of the multimodal data fusion layer in this invention.

[0030] Figure 4 This is a system framework diagram of the digital twin and simulation layer in this invention.

[0031] Figure 5 This is a system framework diagram of the core layer of the two-stage multimodal path planning in this invention.

[0032] Figure 6 This is a system framework diagram of the predictive dynamic obstacle avoidance control layer in this invention.

[0033] Figure 7 This is a system framework diagram of the intelligent collaborative scheduling layer for all equipment in this invention.

[0034] Figure 8 This is a system framework diagram of the edge-cloud collaborative processing layer in this invention.

[0035] Figure 9 This is a system framework diagram of the decision audit and security compliance layer in this invention.

[0036] Figure 10 This is a system framework diagram of the intelligent adaptation and self-evolution layer in this invention.

[0037] Figure 11 This is a system framework diagram of semantic SLAM and semantic map layer in this invention.

[0038] Figure 12 This is a system framework diagram of the modular pluggable layer in this invention. Detailed Implementation

[0039] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings to further illustrate the technical solutions of the present invention. However, the present invention does not include these embodiments.

[0040] like Figures 1-12As shown, this multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing includes a multi-source data acquisition layer, a multimodal data fusion layer, a digital twin and simulation layer, a two-stage multimodal path planning core layer, a predictive dynamic obstacle avoidance control layer, a full-equipment group intelligent collaborative scheduling layer, an edge-cloud collaborative processing layer, a decision audit and safety compliance layer, an intelligent adaptation and self-evolution layer, a semantic SLAM and semantic map layer, and a modular plug-and-play layer. The multi-source data acquisition layer collects real-time data on the warehousing environment, the status of all equipment, and operational tasks through various sensors and system interfaces. The multimodal data fusion layer performs spatiotemporal alignment, repair, and denoising on heterogeneous data, and extracts fusion features through the Transformer cross-modal attention mechanism to provide unified input for decision-making. The digital twin and simulation layer constructs a high-precision 1:1 virtual warehouse and full-equipment model, predicts risks and verifies solutions through simulation, and achieves bidirectional virtual-real drive and simulation-as-control. The two-stage multimodal path planning core layer adopts GCN. A two-stage architecture combining static global optimization and PPO dynamic local response generates and dynamically adjusts the optimal path; the predictive dynamic obstacle avoidance control layer predicts obstacle trajectories based on LSTM networks, generates alternative paths in real time, and achieves multi-device collaborative obstacle avoidance through multi-agent game theory; the all-device group intelligent collaborative scheduling layer adopts a centralized-distributed hybrid decision-making architecture and multi-agent reinforcement learning to coordinate the collaborative operation of multiple heterogeneous devices; the edge-cloud collaborative processing layer performs real-time control at the edge and global scheduling and digital twin simulation at the cloud; the decision audit and security compliance layer records the decision-making process and performs compliance review and data encryption; the intelligent adaptation and self-evolution layer achieves rapid system adaptation and continuous self-optimization based on transfer learning and a dual closed-loop evolution mechanism; the semantic SLAM and semantic map layer constructs an environmental map containing semantic information, and combines human intention understanding to achieve human-machine collaboration and dynamic right-of-way allocation; the modular plug-and-play layer achieves plug-and-play functionality for third-party algorithms and heterogeneous devices by defining standardized interfaces and a hardware abstraction layer.

[0041] The multi-source data acquisition layer includes an environmental perception module, a full equipment status module, and a task and system data module. The environmental perception module integrates LiDAR, 3D vision cameras, ultrasonic sensors, and IoT sensors, comprehensively utilizing multiple sensors. LiDAR provides high-precision spatial positioning and map building, 3D vision cameras perform object (e.g., tray) recognition and pose estimation, ultrasonic sensors assist in blind spot detection, and IoT sensors collect environmental temperature and humidity parameters. Through multi-source complementarity, it comprehensively perceives static structures, dynamic obstacles, and environmental conditions. The full equipment status module includes a real-time equipment data acquisition unit, a historical maintenance database, and an equipment parameter configuration library. The real-time equipment data acquisition unit collects the position data of all equipment, including AGVs and robotic arms, in real time. The system collects operational status data such as position, speed, power consumption, and load. The historical maintenance database aggregates historical fault and maintenance records. The equipment parameter configuration library integrates the equipment's dynamic parameters and communication protocols, enabling comprehensive monitoring of equipment from real-time operation to long-term health. The task and system data module includes enterprise information system interfaces (WMS / ERP / MES), digital twin data interfaces, and CAD / BIM data interfaces. The enterprise information system interface obtains order, inventory, and task instruction operation information from the upper-level business system. The CAD / BIM data interface accesses the warehouse geometric model from CAD / BIM and the simulation data from the digital twin. The digital twin data interface provides the system with operation requirements and virtual simulation data sources, supporting task-driven planning and scheduling.

[0042] The multimodal data fusion layer includes a spatiotemporal alignment module, a data repair and noise suppression module, and a cross-modal feature extraction and fusion module. The spatiotemporal alignment module includes a sliding window aligner, a clock synchronization service, and a cross-device timing calibrator. The clock synchronization service establishes a unified timestamp benchmark for all data sources. The sliding window aligner handles the differences in acquisition frequencies between different sensors (such as a 200Hz LiDAR and a 30Hz camera), performing data matching within a preset time-domain tolerance (≤30 seconds). The cross-device timing calibrator specifically synchronizes the action timing of linked devices such as AGVs and robotic arms, ensuring accurate alignment of collaborative operation data. The data repair and noise suppression module includes a linear interpolation compensator, an outlier filter (based on the 3σ criterion), and a wavelet threshold denoising processor. For data breaks caused by temporary sensor malfunctions, linear interpolation is used for filling and repair. A filter based on the 3σ criterion identifies and removes outlier data that deviates from the normal range, and wavelet transform is used to further denoise the data. The signal is decomposed, and the high-frequency coefficients representing noise are thresholded before reconstruction, effectively suppressing various sensor noises and improving the signal-to-noise ratio and continuity of the data. The cross-modal feature extraction and fusion module includes a Transformer cross-modal attention mechanism unit, a spatiotemporal feature extractor (generating Euclidean distance matrix and autoregressive coefficients), and a feature standardization output unit. Using the Transformer cross-modal attention mechanism unit, the correlation weights between different modal data such as laser point clouds, visual images, IoT data, device status, and task information are calculated and strengthened. Based on this, the spatiotemporal feature extractor analyzes the spatial distribution (Euclidean distance) and temporal series dependencies (autoregression) of the data, extracting multi-dimensional deep features such as environmental topology, obstacle movement trends, device status, and task priority. Finally, the feature standardization output unit integrates and standardizes the extracted heterogeneous features, outputting a unified, high-quality fused feature vector for use by upper-level decision-making.

[0043] The digital twin and simulation layer includes a digital twin engine module, a warehouse 3D simulation module, a warehousing equipment 3D simulation module, a transportation equipment 3D simulation module, a work scene simulation package module, a map dynamic update module, a virtual-physical linkage simulation module, and a predictive maintenance module. The digital twin engine module includes a high-fidelity modeling core unit, a data synchronization service unit, and a reverse control interface unit. The high-fidelity modeling core unit constructs a high-fidelity virtual model corresponding to the physical warehouse at a 1:1 scale based on multimodal fusion data and original CAD / BIM data. Through the data synchronization service unit, real-time bidirectional driving between physical data and the virtual model is achieved. The reverse control interface unit can map physical data to the virtual environment and also implement verified control methods within the virtual environment. Commands are directly and indiscriminately sent to physical equipment for execution, achieving "simulation as control." The warehouse 3D simulation module includes a CAD / BIM parser and reconstructor, a parametric model library, and a physical property mapper. The CAD / BIM parser and reconstructor imports and accurately parses the warehouse's CAD or BIM design data. The parametric model library automatically constructs a complete 3D model of the warehouse with millimeter-level precision, including all structures such as shelves, aisles, partitions, and platforms, not only restoring the geometric appearance, but also mapping physical properties (such as the ground friction coefficient and shelf load-bearing capacity) to the model, providing a realistic physical environment foundation for simulation. The 3D simulation module for warehousing equipment includes a parametric model library for fixed equipment (stacking cranes, intelligent shelves, and robotic arms). The system includes a dynamic parameter configurator and a communication protocol interface simulator. The fixed equipment parameterized model library provides 1:1 3D models of various fixed storage equipment, including precise external appearance and internal joint structures. The dynamic parameter configurator allows configuration of dynamic parameters such as mass and speed thresholds, and the communication protocol interface simulator simulates the actual communication control protocol, accurately replicating the equipment's motion trajectory, operation process, and fault status in a virtual environment, achieving visualized simulation and performance analysis of fixed equipment operation. The transportation equipment 3D simulation module includes a mobile device (AGV, forklift) parameterized model library, a kinematics simulator, and a load state simulator. The mobile device parameterized model library constructs 3D models of mobile transportation equipment with detailed kinematic models. By integrating kinematic equations and control interfaces through a kinematic simulator and a load state simulator, the system realistically simulates the driving, steering, acceleration and deceleration dynamics of AGVs and other equipment, as well as the impact of load changes on motion. It supports three-dimensional simulation and evaluation of single-vehicle or multi-vehicle collaborative transportation scenarios. The work scenario simulation package module includes a library of normal and abnormal scenario models and an adversarial training scenario generator. The normal and abnormal scenario model library is pre-loaded with a rich digital "scenario library" containing simulation models of emergency scenarios such as sensor failure, network latency, random motion of dynamic obstacles, equipment actuator failure, extreme environments, and even fire and water leakage. The adversarial training scenario generator conducts systematic adversarial training, stress testing, and emergency plan simulation to expose potential risks in advance.The map dynamic update module includes a real-time data listener, an automatic map element updater, and a manual calibration interface. The real-time data listener continuously monitors the multimodal fusion data stream. When it detects changes in obstacle positions, equipment status, or environmental parameters in the real environment, the automatic map element updater automatically modifies these changes synchronously in the digital twin map. Simultaneously, the manual calibration interface provides a manual calibration interface to ensure the virtual map remains consistent with the physical world, providing a reliable environmental foundation for planning decisions. The virtual-real linkage simulation module includes a parallel solution inferrer, a safety / efficiency evaluator, and an optimal solution provider. In the digital twin environment, the parallel solution inferrer performs advanced parallel simulations of various candidate strategies generated from path planning and obstacle avoidance control. The safety / efficiency evaluator calculates the feasibility of each path through high-fidelity simulation. Indicators such as safety collision risk and operational efficiency are assessed in advance to predict and avoid conflicts and failures. The optimal solution issuer directly issues the optimal solution after comprehensive evaluation for execution, realizing "simulation verification before actual execution" and reducing the cost and risk of physical trial and error. The predictive maintenance module includes a degradation model library for key components (motors, bearings), a real-time performance data analyzer, and an early warning trigger and report generator. The key component degradation model library is based on real-time equipment operating data (such as vibration, temperature, current) and a historical fault database to build mathematical models of life and performance degradation for key components. The real-time performance data analyzer continuously analyzes the deviation between real-time data and model prediction curves. The early warning trigger and report generator provides early warnings of potential failure risks before actual component failures occur and generates maintenance suggestions, realizing the transformation from "retroactive maintenance" to "proactive prevention."

[0044] Digital twin: The composite formula of simulation as control + virtual-real bidirectional drive is as follows: ; In the formula, Represents the virtual twin model at time t; Indicates the physical warehouse state at time t; F represents the physical → virtual mapping (real-time synchronization); This represents the optimal control command output by the simulation; G represents virtual → physical inverse control (simulation is execution).

[0045] The plaintext algorithm for digital twins driven by both virtual and real elements: 1. Real-time acquisition of the physical world: devices, environment, and tasks; 2. Synchronous driving of the virtual model, 1:1 high-fidelity mapping; 3. Parallel simulation of multiple path schemes in the virtual world; 4. Selection of the optimal scheme and direct control of physical devices; 5. Closed loop: physical execution results are then fed back to the virtual model for correction.

[0046] The core layer of the two-stage multimodal path planning system includes a GCN static global optimization module, a PPO dynamic local response module, and a multimodal planning mode selection module. The GCN static global optimization module comprises a warehouse topology map builder, a graph convolutional neural network node feature aggregator, and a multi-objective cost optimization function unit. The warehouse topology map builder constructs a topology map reflecting the connectivity of warehouse space based on digital twin maps and globally fused data. The graph convolutional neural network node feature aggregator uses a graph convolutional neural network to learn and aggregate the features of nodes (such as aisle intersections and shelf locations) in the graph, generating the globally optimal basic path. The multi-objective cost optimization function unit's optimization objective is a configurable multi-objective cost function, comprehensively considering time, energy consumption, equipment wear and tear, and energy recovery efficiency to achieve globally optimal planning in a static environment. The PPO dynamic local response module includes a PPO (Proximal Policy Optimization) policy network unit, an improved UKF (Unscented Kalman Filter) motion parameter estimator, and a real-time feedback interface unit. The real-time feedback interface unit... The system receives real-time sensor data, equipment status, and simulation feedback from digital twins. The PPO (Proximal Policy Optimization) strategy network unit's PPO reinforcement learning algorithm, based on real-time environmental interaction experience, performs local fine-tuning and action decisions on the global basic path generated by the GCN to cope with sudden obstacles. An improved UKF (Unscented Kalman Filter) motion parameter estimator integrates an improved UKF algorithm to estimate the AGV's accurate kinematic parameters in real-time at a high frequency of 200Hz, ensuring control accuracy and path tracking stability under dynamic disturbances such as load changes and ground slippage. The multimodal planning mode selection module includes a mode decision-maker and an algorithm fusion executor. The mode decision-maker adaptively selects a single optimal mode or triggers a fusion strategy of multiple modes based on contextual information such as current task attributes (e.g., urgency), equipment type (e.g., AGV or robotic arm), and real-time environmental congestion. The fusion executor coordinates the selected algorithms to work together, thereby solving the problem of insufficient adaptability of a single algorithm and improving the system's planning robustness and efficiency under different working conditions.

[0047] The improved UKF motion parameter estimator is an improved unscented Kalman filter algorithm. Based on the standard unscented Kalman filter, it introduces AGV load weight factor, ground friction coefficient adaptive correction term and sigma point weighted optimization strategy to estimate AGV kinematic parameters in real time at a high frequency of 200Hz, thereby improving the control accuracy under ground slippage and load change conditions. The state update formula for the improved UKF algorithm is as follows: ; Its gain correction formula is: ; In the formula: This represents the optimal state estimate of the AGV at time k after filtering correction, including kinematic parameters such as position, speed, attitude, and steering angle. This represents the predicted state value at time k, which is calculated based on the state at the previous time and the motion model. This represents the adaptive Kalman gain at time k, used to balance the weights of the "predicted value" and the "observed value". This represents the actual observation value of the sensor at time k, which comes from data collected by lidar, vision, odometer, IMU, etc. This represents the observed predicted value corresponding to the state at time k, which is obtained by mapping the state variables. The state prediction covariance at time k represents the confidence level / uncertainty of the predicted value. This represents the observation matrix, used to map state variables (position, velocity) to observations; This represents the transpose of the observation matrix; This represents the observation noise covariance, which reflects the noise level of the sensor itself. This is the load adaptive coefficient, a newly added category used to adjust the filter gain according to the load strength of the AGV; This indicates the real-time load quality of the AGV, i.e., the current load weight, which will change dynamically from heavy load to light load to empty load. Ground friction correction coefficient is a new category used to adjust filtering based on the smoothness / roughness / slipperiness of the ground. The value represents the real-time ground adhesion coefficient. The smaller the value, the more slippery the ground (such as oil stains or water stains), and the larger the value, the less slippery the ground. -1 represents the matrix inversion operation. The improved UKF estimates the AGV kinematic parameters in real time at a high frequency of 200Hz. Through adaptive correction of load and ground friction, it ensures high-precision state estimation and path tracking stability under load changes and ground slippage conditions.

[0048] Global phase: The improved GCN spatiotemporal weighted path planning formula is as follows: ; In the formula, This represents the node feature matrix output by the (l+1)th layer graph convolution, which is the new feature obtained after convolution for each point (shelf, aisle, intersection) in the warehouse map; This represents a non-linear activation function used to enhance the model's expressive power and extract features from complex warehousing environments. This represents the warehouse topology adjacency matrix with self-loops, i.e., the connectivity between various points in the warehouse; Indicates and The corresponding degree matrix shows how many paths each point is connected to. Represents the normalized inverse square root of the degree matrix, used to normalize graph convolutions to prevent numerical explosion / gradient vanishing; The node feature matrix represents the input of the l-th layer, which contains information including, but not limited to, point coordinates, passage status, device location, and obstacles. The weight matrix represents the convolution of the l-th layer graph, and the optimal parameters for path planning learned automatically by the neural network. represents the spatiotemporal weight coefficient, with a value range of 0.2 to 0.5, which controls the influence of "historical efficiency, current congestion, and future tasks" on the path; T represents the spatiotemporal feature matrix of the node, including three types of warehouse-specific features: historical traffic efficiency, current congestion degree, and future task prediction value; The spatiotemporal feature enhancements enable path planning to predict congestion and dynamically optimize; output: globally optimal basic path Pglobal. The improvement of the spatiotemporal weighted path planning in GCN over the standard GCN algorithm lies in the addition of a spatiotemporal feature enhancement term to the standard graph convolution. The historical traffic efficiency, current congestion level, and future task predictions specific to the warehousing scenario are incorporated as a spatiotemporal feature matrix T into the graph convolution process; adjustable spatiotemporal weight coefficients are used to... (0.2~0.5) Dynamically control the influence of spatiotemporal features on path planning to enable the planning results to have dynamic predictive capabilities; retain the ability of standard graph convolution to model the static topology of the warehouse, and at the same time superimpose dynamic features to form a joint planning mechanism of static structure + dynamic trend.

[0049] The improvements bring the following benefits: overcoming the limitations of traditional GCNs that can only plan routes based on static maps, it can predict congested areas in advance and avoid hotspots, greatly improving the rationality of the overall route; it integrates future task prediction features, making the route planning forward-looking and avoiding subsequent task conflicts; the efficiency of global route planning is improved by 22%, the empty running rate of AGVs is reduced, and the overall circulation efficiency is significantly improved; the weight coefficient α can be adaptively adjusted, making it suitable for different scenarios such as pharmaceutical warehouses, e-commerce warehouses, and cold chain warehouses.

[0050] Local stage: The improved PPO dynamic path optimization formula is as follows: ; In the formula, This represents the loss function value of the improved PPO policy network; This represents the mathematical expectation based on time step t; Indicates the probability ratio of the strategies; Represents the dominance function; This indicates a truncation function to limit the policy update magnitude; This represents the clipping threshold, with a value ranging from 0.1 to 0.2. This represents the obstacle distance penalty coefficient, which is a positive hyperparameter. Indicates the distance between the AGV and the nearest obstacle; This represents the task priority reward coefficient, which is a positive hyperparameter. This represents the priority weight of the current task; the final output is the locally fine-tuned optimal path Plocal. The specific improvement of the improved PPO dynamic path optimization over the standard PPO algorithm lies in the addition of an obstacle distance penalty term to the standard PPO pruning loss function. This allows AGVs to be automatically penalized when they approach obstacles, thus proactively moving them away from dangerous areas; a new task priority reward item has been added. This allows high-priority tasks to receive greater weight in path adjustments, ensuring that critical tasks are executed first; and it also reduces obstacle distance. With task priority Simultaneously, strategy optimization is incorporated to achieve simultaneous optimization of safe obstacle avoidance and task order preservation; the advantages of stable training and smooth updates of the PPO algorithm are maintained, and path jitter caused by excessive local adjustments is avoided.

[0051] The benefits of this improvement are as follows: it enables active obstacle avoidance instead of passive obstacle avoidance, allowing the AGV to move away from obstacles in advance and reducing the risk of collision to near zero; the response time for local path adjustment is shortened to less than 50ms, making control more sensitive and stable; high-priority tasks can pass first, avoiding task delays caused by excessive obstacle avoidance and improving the overall operational efficiency of the system; and it can maintain path stability under complex working conditions such as slippery ground, load changes, and sudden obstacles, significantly enhancing robustness.

[0052] The final composite formula for two-stage path planning is:

[0053] That is, the final optimal path = fusion of improved GCN spatiotemporal global planning + improved PPO penalty-reward local planning; In the formula, This represents the globally optimal and locally safe final path that the AGV ultimately executes. This represents a fusion function of global and local paths, used to combine macro-planning with dynamic adjustments; This represents an improved graph convolutional neural network that incorporates spatiotemporal weighting factors to generate globally optimal paths. This represents the global base path for warehousing output by the improved GCN; This represents an improved proximal policy optimization algorithm that incorporates obstacle penalties and task rewards to generate locally dynamically corrected paths. This represents the local obstacle avoidance fine-tuning path output by the improved PPO. The formula clearly expresses the core architecture of the two-stage path planning: "global static planning + local dynamic correction". First, the globally optimal path is obtained by improving GCN, and then real-time obstacle avoidance and priority adjustment are performed by improving PPO. Finally, the paths are merged to obtain a safe, efficient and stable final driving path.

[0054] The final optimal path wind guiding Python algorithm code is as follows: import numpy as np import torch # ====================== # Phase 1: Improve GCN Global Path Planning # ====================== class ImprovedGCN(torch.nn.Module): def __init__(self,input_dim,hidden_dim,alpha=0.3): super().__init__() self.W=torch.nn.Linear(input_dim,hidden_dim) self.alpha=alpha # Spatiotemporal weighting coefficient self.relu = torch.nn.ReLU() def forward(self, A, D, H, T): # Standard graph convolution conv=D.pow(-0.5)@A@D.pow(-0.5)@H@self.W.weight # New addition: Spatiotemporal weighting factor spacetime=self.alpha*T@H # Improve GCN output return self.relu(conv+spacetime) def gcn_global_planning(A,D,node_feature,spacetime_feature): gcn=ImprovedGCN(input_dim=6, hidden_dim=16) H=gcn(A,D,node_feature,spacetime_feature) # Generate the globally optimal path P_global=generate_path_from_graph(H) return P_global # ====================== # Phase 2: Improve PPO Local Path Planning # ====================== ImprovedPPO class: def __init__(self, beta=0.8, gamma=1.2): self.beta=beta # Obstacle distance penalty coefficient self.gamma=gamma # Task priority reward coefficient def compute_loss(self,r_t,A_t,d_t,p_t,eps=0.2): # Standard PPO trimming loss clip_loss=torch.min(r_t*A_t,torch.clamp(r_t,1-eps,1+eps)*A_t) # New Additions: Obstacle Penalties + Mission Rewards total_loss=clip_loss-self.beta*d_t+self.gamma*p_t return total_loss.mean() def ppo_local_adjust(P_global,obs_dist,task_priority): ppo=ImprovedPPO() # High-frequency local path correction P_local=ppo.update_policy(P_global,obs_dist,task_priority) return P_local # ====================== # Two-stage path planning main entry point # ====================== def dual_stage_path_planning(A,D,H,T,d_t,p_t): # 1. Global Planning P_global=gcn_global_planning(A,D,H,T) # 2. Local Correction P_local=ppo_local_adjust(P_global,d_t,p_t) # 3. Final Path of Fusion P_final=merge_global_local(P_global,P_local) return P_final.

[0055] The predictive dynamic obstacle avoidance control layer includes an interruption event detection and obstacle classification module, a trajectory prediction module, an alternative path solution and speed control module, and an obstacle avoidance coordination module. The interruption event detection and obstacle classification module includes a real-time computer vision processor, a multimodal data association analyzer, and an obstacle classifier. The real-time computer vision processor processes data streams fused from multimodal sensors such as cameras and LiDAR in real time. The multimodal data association analyzer quickly detects sudden obstacles in the environment (such as moving people, fallen objects, and malfunctioning equipment), with a response latency controlled within 0.3 seconds. Simultaneously, the obstacle classifier categorizes obstacles. The system categorizes obstacles (e.g., personnel, other AGVs, stationary objects) to provide a basis for subsequent differentiated obstacle avoidance strategies. The trajectory prediction module includes an LSTM (Long Short-Term Memory) network prediction model unit, a short-term trajectory extrapolator, and a collision risk predictor. The LSTM network prediction model unit learns the historical motion sequences of dynamic obstacles to predict their future short-term (e.g., 3 seconds) trajectories and speeds. The short-term trajectory extrapolator combines environmental information from the digital twin scenario to extrapolate the possible routes of obstacles in real time and pre-calculates the spatiotemporal intersection with the path of the local equipment. The collision risk predictor proactively predicts obstacles. Potential collision risks are identified, enabling early obstacle avoidance decisions. The alternative path solution and speed control module includes a spatiotemporal alternative path fast solver, a speed adaptive controller, and a path constraint filter. When a collision risk is predicted, the spatiotemporal alternative path fast solver quickly generates one or more feasible collision-free alternative paths within a short time (e.g., 3 seconds) based on the current environment map and equipment status, using a spatiotemporal alternative path algorithm. The path constraint filter automatically filters out road segments that do not meet constraints such as height and weight limits. The speed adaptive controller dynamically adjusts the vehicle's speed based on the distance to obstacles, their movement trends, and path conditions (such as slope). To achieve safe and smooth obstacle avoidance and rapid recovery of efficient operation after the danger has passed, the obstacle avoidance coordination module includes a multi-device communication coordinator, an obstacle avoidance priority arbitrator, and a multi-agent game decision-maker. The multi-device communication coordinator coordinates the obstacle avoidance intentions and plans of each device and arbitrates temporary right-of-way based on factors such as task priority and device status. The multi-agent game decision-maker introduces a multi-agent game mechanism, allowing device agents to conduct distributed negotiation in a virtual environment. The obstacle avoidance priority arbitrator quickly reaches the optimal group obstacle avoidance sequence and path adjustment scheme, avoiding new conflicts or deadlocks caused by individual obstacle avoidance and improving the overall collaborative obstacle avoidance efficiency of the system.

[0056] The all-equipment group intelligent collaborative scheduling layer includes a task dynamic allocation module, a path conflict resolution module, and a group intelligent decision-making module. The task dynamic allocation module includes a centralized global optimizer, a multi-dimensional equipment status evaluator, and a real-time task queue and traffic balancer. The multi-dimensional equipment status evaluator continuously acquires the real-time status (battery level, location, load, health status) of all AGVs, robotic arms, and other equipment. The real-time task queue and traffic balancer continuously acquires the global order task queue. The centralized global optimizer, through optimization algorithms, comprehensively considers task urgency, equipment adaptability, and overall warehouse traffic, dynamically decomposes orders and allocates them to the most suitable equipment, achieving macro-scheduling and optimal resource allocation for cross-equipment full-process collaborative operations such as "AGV transfer - robotic arm picking and placing - stacker crane lifting." The path conflict resolution module includes a multi-equipment path intersection real-time monitor, a dynamic avoidance node optimizer, and a timing and restricted area adjuster. The multi-equipment path intersection real-time monitor monitors the planned paths and current positions of all mobile devices (such as hundreds of AGVs) in real time, predicting potential spatiotemporal intersection conflict points. Once a conflict risk is detected, the dynamic avoidance node optimizer adjusts the passage order of equipment and optimizes temporary... Waiting nodes (such as virtual traffic lights) or time-series and restricted area adjusters set instantaneous dynamic restricted areas on digital maps to guide devices to safely and efficiently avoid each other in a non-intrusive manner, ensuring smooth operation of large-scale cluster operations. The swarm intelligence decision-making module includes a centralized-distributed hybrid decision-making architecture unit, a multi-agent reinforcement learning cluster unit, a virtual parallel inference environment unit, and a fast negotiation and consensus protocol unit. In the digital twin environment, the centralized-distributed hybrid decision-making architecture unit creates a corresponding agent agent for each physical device. Under the guidance of a centralized global goal, each agent, based on distributed local perception (its own state, surrounding environment), utilizes the multi-agent reinforcement learning and game model of the multi-agent reinforcement learning cluster unit. The virtual parallel inference environment unit infers multiple possible sequences of collaborative actions in parallel in the virtual scene within the next few seconds. When a spatiotemporal conflict is predicted, the fast negotiation and consensus protocol unit exchanges information (such as task priority, estimated arrival time) between agents through a fast communication protocol to conduct distributed negotiation, in order to reach a locally optimal consensus and generate the final collaborative instruction. The entire process continuously learns and evolves, constantly improving the efficiency of group collaboration.

[0057] The compound formula for predictive obstacle avoidance is: ; In the formula, This indicates the predicted position of the obstacle at time k in the future; This represents a bidirectional LSTM model with an attention mechanism. This indicates the historical trajectory of the obstacle over the past L frames; This represents the set of optimal control instructions for multi-agent cooperation. Indicates the distance between the AGV and the obstacle; This indicates the risk of collisions between AGVs; Indicates the task priority of the i-th AGV; This represents the obstacle avoidance weight coefficient; N represents the number of AGVs.

[0058] Plaintext algorithm for predictive dynamic obstacle avoidance (LSTM trajectory prediction + multi-agent cooperation): Input: historical trajectory, real-time obstacles, AGV status, task priority.

[0059] Output: Safe and conflict-free local control instructions.

[0060] 1.1. Predict the 3-second trajectory O_hat of obstacles using an attention-enabled LSTM; 2. Calculate the distance and collision risk between each AGV and the predicted obstacle; 3. Multi-agent game negotiation: priority, passage order, and avoidance strategy; 4. Generate cooperative obstacle avoidance control commands. 5. Update paths in real time to ensure no collisions and prioritize high-priority paths.

[0061] The edge-cloud collaborative processing layer includes an edge computing node module and a cloud scheduling center module. The edge computing node module comprises a real-time data stream processor, a local intelligent agent policy executor, a high-frequency control command generator, and an edge storage. The real-time data stream processor directly connects to the sensor network and device controllers, performing millisecond-level processing on high-frequency real-time data streams generated by LiDAR, cameras, etc. The local intelligent agent policy executor and high-frequency control command generator execute latency-sensitive core tasks, such as path dynamic fine-tuning based on PPO policies and emergency obstacle avoidance control based on LSTM predictions. The edge storage stores and runs localized intelligent agent models, enabling rapid response to environmental changes and synchronizing processing results and summary data to the cloud, reducing network transmission latency. Bandwidth pressure; The cloud scheduling center module includes a global task and resource scheduler, a digital twin simulation cluster unit, a historical big data warehouse, and an algorithm model training and version management center. The global task and resource scheduler focuses on non-real-time, global, and computationally intensive tasks, and is responsible for integrating all warehouse information, performing macro-task planning, cross-regional resource scheduling, and global path topology optimization. The digital twin simulation cluster unit connects to and drives the high-fidelity digital twin system to perform large-scale, multi-scheme virtual simulation and verification. The historical big data warehouse collects all historical and operational data. The algorithm model training and version management center uses powerful computing power to perform offline training, parameter iteration, and performance evaluation of algorithm models, and silently distributes the optimized models to edge nodes for updates.

[0062] The composite formula for a clear division of labor between the edge and the cloud is: ; In the formula, This indicates a real-time task at the edge. This indicates millisecond-level real-time processing; Indicates a global task in the cloud; This refers to global scheduling, simulation, and training.

[0063] Clear division of labor, edge computing: real-time perception, PPO fine-tuning, LSTM obstacle avoidance, and high-frequency control; Cloud-based: Digital twin simulation, global path planning, model training, and version distribution; Plaintext algorithm for edge-cloud collaborative architecture: Edge-end: 1. High-frequency sensor data processing; 2. PPO local path correction; 3. LSTM emergency obstacle avoidance; 4. Real-time control command output.

[0064] Cloud-based: 1. Global task scheduling; 2. Large-scale digital twin simulation; 3. Offline training of algorithm models; 4. Silent deployment of optimal models to the edge.

[0065] The decision audit and security compliance layer includes a decision tracing module, an AI compliance self-inspection module, and a data security protection module. The decision tracing module includes a decision process mirror recorder, an interactive decision tree builder, and a real-time decision deviation scanner and alarm. The decision process mirror recorder records all key decision processes (such as the strategy distribution of the PPO algorithm, the probability of action selection in multi-agent games, and specific path planning logic). Based on this data, the interactive decision tree builder can reverse-engineer an interactive and visualized decision path tree. The real-time decision deviation scanner and alarm scans the deviation between the current decision and historical patterns or preset rules in real time, and immediately alarms if the deviation exceeds a safety threshold, ensuring that any abnormal or inefficient decision can be traced, analyzed, and reviewed. The AI ​​compliance self-inspection module includes a multi-industry compliance rule library (such as GMP, GDPR), an automated verification engine unit, and a standardized audit report generator. The multi-industry compliance rule library embeds regulations and standards from different industries, and the automated verification engine unit automatically... The system's actual operations (such as whether the path avoids restricted areas, data collection range, and temperature control records) are compared and verified with the selected compliance rules. After verification, the standardized audit report generator automatically generates a structured, regulatory-compliant standardized audit report, reducing the burden of manual auditing and ensuring that the workflow always complies with industry regulatory requirements. The data security protection module includes an AES-256 end-to-end encryption / decryption engine unit, an intrusion detection and behavior analysis unit, and a real-time alarm and blocking controller. The AES-256 end-to-end encryption / decryption engine unit uses the AES-256 high-strength encryption algorithm to encrypt control commands, warehouse data, etc., to prevent theft or tampering. The intrusion detection and behavior analysis unit continuously monitors network traffic and access behavior, and identifies abnormal or attack behaviors such as "illegally modifying path parameters" and "unauthorized data acquisition" through behavior analysis models. Once detected, the real-time alarm and blocking controller immediately alarms and initiates the blocking mechanism, building a solid data security defense for the system.

[0066] The intelligent adaptation and self-evolution layer includes a scene adaptation module, a self-evolutionary learning engine module, an API template extension module, a human-computer interaction and visualization platform, and a VR / AR human-computer interaction module. The scene adaptation module includes a transfer learning framework unit, a cross-scene parameter auto-adjuster, and a simulation scene rapid configurator. The transfer learning framework unit utilizes transfer learning technology to quickly transfer and adapt algorithm models and strategies trained in a certain type of warehouse (such as an e-commerce warehouse) to a new warehousing environment (such as a pharmaceutical warehouse). The cross-scene parameter auto-adjuster automatically analyzes the characteristics of the new scene and adjusts key parameters such as obstacle avoidance distance and path weight. The simulation scene rapid configurator synchronizes... Configure simulation scenarios in the digital twin to achieve rapid deployment and performance maintenance across scenarios, significantly reducing retraining and manual configuration costs. The self-evolutionary learning engine module includes an instant decision loop (virtual simulation and verification), an offline evolution loop (batch training and evaluation), and an evolutionary driving unit (adversarial scenario generation). The instant decision loop and the offline evolution loop construct a unique "dual closed-loop" evolutionary mechanism. When the instant decision loop runs online, it simulates in parallel within the digital twin and selects the optimal solution for execution. The offline evolution loop starts during system idle time, compares actual and simulation results, and automatically generates various extreme and fault scenarios by calling simulation packages. The evolutionary driving unit utilizes historical and simulation data... Real data is used for large-scale, secure batch retraining and evaluation of the core algorithm model. After performance improvement, the entire network is silently updated, driving continuous self-optimization of the system. The API template extension module includes a standardized API template library and automatic interface adapters for mainstream cloud platforms and business systems (such as Huawei Cloud, Alibaba Cloud IoT, WMS / ERP). The standardized API template library has a rich set of verified standardized API connection templates. When it is necessary to interface with external systems, there is no need to develop from scratch. Simply call the corresponding template through the automatic adapter and complete a simple configuration to achieve fast and seamless data integration with mainstream cloud services and enterprise management systems. Interoperability with commands reduces system integration difficulty and deployment costs; the human-computer interaction and visualization platform includes a 3D panoramic visualization engine, a historical data backtracking analyzer, and a simulation scene editing and injection interface unit. The 3D panoramic visualization engine is deeply connected with the digital twin engine, displaying the entire warehouse's operating status, equipment location, and task progress in real time and intuitively in 3D form. The historical data backtracking analyzer replays and analyzes operations at any point in history. Administrators can manually intervene in the simulation scene editing and injection interface unit to adjust plans, modify parameters, or directly "inject" new test scenarios in the simulation environment, achieving highly flexible human-computer collaborative decision-making.The VR / AR human-computer interaction module includes AR glasses, a VR immersive simulation environment unit, and a gesture and voice recognition interactive device. Through the AR glasses, administrators can overlay virtual operating data, planned paths, and status alarms onto their real-world view. The gesture and voice recognition interactive device allows for direct remote scheduling and intervention via gestures or voice commands. The VR immersive simulation environment unit provides immersive employee training, emergency response drills, and system maintenance guidance, enhancing the intuitiveness and efficiency of human-computer collaboration.

[0067] The composite formula for dual-loop self-evolution—combining an online decision-making loop and an offline evolutionary loop—is as follows: ; In the formula, This indicates the optimal online strategy in real time. This represents the online decision function, which calculates the optimal strategy in real time based on the old strategy, the physical environment, and the simulation environment. Indicates the old strategy; Indicates the physical environment status; Indicates the state of the simulation environment; This indicates the new strategy after offline evolution; This represents the offline evolutionary training function, which automatically optimizes the algorithm using historical data and adversarial scenario data. This indicates the real-time strategy being used during the job execution, serving as the basis for offline evolution; Represents the historical running dataset; This indicates the generation of a dataset for adversarial scenarios.

[0068] The double-loop self-evolving plaintext algorithm: [Online Real-Time Decision-Making Loop] 1. Real-time environmental awareness; 2. Virtual parallel simulation of N strategies; 3. Selection and execution of the optimal strategy; Offline Evolution Ring 1. Collect historical and simulation data; 2. Automatically generate adversarial / extreme scenarios; 3. Batch train and optimize models; 4. After simulation verification, silently and hot-update the entire network.

[0069] The semantic SLAM and semantic map layer includes a semantic map module, a visual SLAM module, and a human-machine collaboration and intent recognition module. The semantic map module comprises a semantic map construction and storage engine unit, a multi-level semantic tag library (shelf type, aisle attributes, no-parking zones, charging areas, etc.), and a dynamic semantic element updater. The semantic map construction and storage engine unit integrates high-precision geometric information with rich semantic information. When building the map, it not only records the spatial structure, but the multi-level semantic tag library also labels each area or object with its functional attributes (e.g., "shelf - electronics area," "aisle - main aisle," "area - no-parking zone"). The dynamic semantic element updater provides rich "context" for upper-level planning, enabling devices to understand the environment and make decisions (e.g., only park in the charging area, avoid entering no-parking zones). The visual SLAM module includes a visual odometry, a feature point extractor and matcher, and a dynamic object semantic recognition and tracker. The visual odometry utilizes visual sensors such as cameras to perform real-time synchronous positioning and mapping during device movement. The system integrates feature point extraction and matching with semantic information: Beyond locating itself and building a geometric map, the dynamic object semantic recognition and tracker can identify and semantically classify dynamic objects in real time, such as distinguishing between "slowly walking workers" and "high-speed, out-of-control AGVs." This distinction is crucial for risk assessment and implementing different levels of avoidance strategies. The human-machine collaboration and intent recognition module includes a visual gesture recognition unit, a human posture and intent parser, and a dynamic right-of-way allocation device for the human-machine shared space. The visual gesture recognition unit uses visual equipment to identify the gestures, postures, and movement trends of workers in the warehouse in real time. The human posture and intent parser interprets their operational intentions (such as "indicating to follow," "requesting to give way," or "maintenance in progress"). Based on this intent recognition and combined with semantic map information, the dynamic right-of-way allocation device for the human-machine shared space dynamically adjusts the right-of-way, achieving flexible and efficient collaboration rather than rigid, mechanical avoidance. This maximizes operational efficiency while ensuring absolute personnel safety.

[0070] The composite formula for semantic SLAM, human intent understanding, and dynamic right-of-way allocation is: ; In the formula, Represents a semantic map (geometry + function + attributes); This represents the semantic SLAM building blocks; Represents visual images; Represents laser point clouds; Indicates the device pose; Indicates the optimal dynamic right-of-way at any given time; Indicates the intention of human manipulation; Indicates AGV safety constraints; Indicate task priority; This indicates the weight of right-of-way allocation.

[0071] The plaintext algorithm combining semantic SLAM, human intent, and dynamic right-of-way is as follows: 1. Construct an environment map Msem with semantic labels; Visual recognition: 2. Human posture, gestures, and movement trends; 3. Understand human intentions: give way, pass, maintain, emergency, etc.; 4. Dynamically allocate right-of-way based on semantic map + intention + safety + task; 5. Output real-time right-of-way scheme 0*(t) to achieve safe human-machine collaboration.

[0072] The modular pluggable layer includes a standardized interface module and a hardware abstraction module. The standardized interface module includes an algorithm interface protocol library, a service registration and discovery center unit, and a compatibility testing unit. The algorithm interface protocol library defines a unified and clear application programming interface (API) specification and communication protocol for the system's core functions (such as path planning and obstacle avoidance). Third-party algorithm providers in the service registration and discovery center unit only need to develop algorithm services according to this standard and register them, and the system can automatically identify and call them. The compatibility testing unit, in conjunction with compatibility testing, ensures that algorithms from different sources are plug-and-play and seamlessly replaceable in the system, realizing an open and flexible algorithm ecosystem. The hardware abstraction module includes a device driver manager, a general instruction-protocol converter, and a hardware configuration template library. The device driver manager builds a unified "translation" and "adaptation" layer between specific hardware devices and the upper-layer software system. The general instruction-protocol converter converts the control instructions and communication protocols specific to various devices into general instructions that the system can understand. The hardware configuration template library allows the replacement or upgrade of underlying hardware and upper-layer applications to operate through this unified interface, shielding hardware differences and reducing the complexity and cost of integrating devices of different brands and models. Example 1

[0073] This embodiment is applied to a pharmaceutical temperature-controlled intelligent warehousing center. The warehousing center is equipped with 60 AGVs, 20 robotic arms, 50 sets of intelligent shelves, and 15 stacker cranes, covering tasks such as drug warehousing, temperature-controlled storage, sorting and outbound, and cold chain transportation. It needs to meet GMP compliance requirements. The warehousing environment includes personnel passages, temporary replenishment areas, and steep slope transfer passages. It has extremely high requirements for real-time path planning, obstacle avoidance safety, temperature and humidity control, compliance traceability, and system stability. At the same time, it needs to be different from the single-robot path planning method of patent CN110032189A and adapt to the needs of multi-device collaborative operation and continuous evolution.

[0074] The specific implementation steps of the control system of this invention are as follows: 1. System Deployment: The system integrates with the existing WMS, ERP, and digital twin platforms of the warehousing center through an intelligent adaptation layer. This includes importing basic data such as warehouse CAD / BIM drawings, shelf locations, drug storage temperature and humidity thresholds, and GMP compliance rules; importing 3D CAD models of AGVs, robotic arms, and stacker cranes; configuring their dynamic parameters such as mass, maximum speed, and acceleration; and configuring the GCN static optimizer weights. =0.5 (time weight) =0.3 (energy consumption weight) =0.1 (equipment loss weight) =0.1 (energy recovery weight); set obstacle avoidance safety distance threshold of 1.0 meter, AGV speed adaptive range of 0.2-1.8m / s; quickly adapt to medical constant temperature scenario parameters through transfer learning algorithm, without the need for manual retraining of model and agent strategy.

[0075] 2. Data Acquisition and Fusion: The multi-source data acquisition layer collects environmental and obstacle data through LiDAR and 3D vision cameras, while IoT sensors monitor warehouse temperature and humidity (controlled at 28℃) and shelf vibration values ​​in real time. It also collects the operating status, historical maintenance data, and dynamic parameters of AGVs, robotic arms, and stacker cranes simultaneously. The multimodal data fusion layer completes data synchronization through a sliding window aligner. After wavelet threshold denoising, feature fusion is completed through the Transformer cross-modal attention mechanism to output standardized fused data. Edge computing nodes complete high-frequency data preprocessing.

[0076] 3. Digital Twin Construction and Simulation: Based on fused data and CAD / BIM data, the digital twin and simulation layer constructs a high-fidelity virtual warehouse model, updating temperature and humidity distribution, obstacle locations, and equipment status (including dynamic parameters) in real time. For pharmaceutical cold chain transportation tasks, it calls upon a work scenario simulation package (including random personnel movement and sensor noise models) to simulate multiple path schemes in parallel in the virtual scenario. It conducts advanced simulation and safety verification of candidate schemes, predicts obstacle avoidance risks in steep slopes and densely populated areas, and selects the optimal path and collaborative operation scheme that is "shortest in time + lowest in energy consumption + compliant with temperature and humidity."

[0077] 4. Path Planning and Dynamic Obstacle Avoidance: The core layer of the two-stage multimodal path planning uses the GCN static global optimizer to generate a global basic path (e.g., from the receiving cold chain area through the constant temperature sorting area to the target shelf). The PPO dynamic local responder combines real-time temperature and humidity data with simulation feedback to fine-tune the path. The dynamic obstacle avoidance control layer uses the LSTM module to predict personnel movement trajectories and adjust the AGV path in advance. When an AGV travels to a corner of the sorting area and detects a temporarily stacked medicine box (sudden obstacle), the alternative path solver is triggered within 0.2 seconds to generate a detour path. The speed adaptive controller reduces the AGV speed from 1.5m / s to 0.4m / s, and quickly restores the optimal speed after obstacle avoidance. The obstacle avoidance coordination unit coordinates with surrounding equipment to avoid new conflicts.

[0078] 5. Full Equipment Collaboration and Task Linkage: The centralized task optimizer of the full equipment group intelligent collaborative scheduling layer dynamically assigns tasks to AGVs and robotic arms based on the priority of drug outbound operations; the distributed intelligent agent cluster assigns an agent to each device, and conducts game negotiation in the digital twin engine. When the AGV and the stacker crane paths intersect, the intelligent agents reach a consensus on avoidance by exchanging priorities and estimated arrival times, and determine the sequence of coordinated actions such as the arrival time of the AGV and the timing of the robotic arm picking up goods; after the AGV arrives at the target shelf, the cross-device linkage module automatically triggers the robotic arm to pick up and put down goods. After the robotic arm completes its operation, it sends a feedback signal to the AGV to execute the next transfer task.

[0079] 6. Decision Audit, Safety Protection, and Dual Closed-Loop Evolution: The decision audit and safety compliance layer records the PPO algorithm, the decision-making process of multi-agent game, AGV driving trajectory, and robotic arm operation logs. The AI ​​compliance self-check module verifies whether the work process complies with GMP standards. Data transmission uses AES256 encryption to prevent drug information leakage. The self-evolutionary learning engine detects that the actual passage efficiency of AGVs in a certain area is 15% lower than the simulation prediction. At night, an offline evolution loop is triggered, calling the work scenario simulation package to inject denser dynamic obstacles and simulate network latency in that area, and to perform large-scale batch training on the agent strategy and path planning parameters. After training, the new strategy passes the simulation test and is silently updated to the edge computing node and cloud scheduling center to improve the accuracy of the next day's operation.

[0080] After testing, the AGV collision accident rate in the warehouse center was reduced to 0 after the system was deployed, the daily drug transfer volume increased by 50%, the overall energy consumption decreased by 26%, the operation process fully complied with GMP compliance requirements, and the system operated stably and reliably. Compared with the single robot path planning method of patent CN110032189A, the operation efficiency was improved by more than 60%, and it has advantages in anti-interference ability, multi-device collaboration ability, scene adaptability, and continuous evolution ability. Compared with a single patent, the integration of high-fidelity simulation and multi-agent collaboration reduced the trial and error cost by 40% and the collaboration conflict rate by 70%, fully demonstrating the inventiveness and practicality of the invention.

[0081] In summary, this invention achieves comprehensive and interference-resistant perception of the warehouse environment through multi-source sensor fusion and Transformer cross-modal attention mechanism. It also adopts a two-stage architecture combining GCN global planning and PPO local response, as well as LSTM trajectory prediction, to form a complete intelligent decision-making capability from global optimization and dynamic obstacle avoidance to proactive prediction, thereby improving the robustness and efficiency of path planning.

[0082] The system constructs a high-fidelity digital twin model to achieve virtual-real linkage control of "simulation verification first, then actual execution". Based on a centralized-distributed hybrid intelligent mechanism, it performs task allocation, conflict resolution and game negotiation for various types of equipment such as AGVs and robotic arms in a virtual environment, ensuring the safety and overall collaborative efficiency of large-scale cluster operations.

[0083] The system adopts an edge-cloud collaborative processing architecture, which rationally allocates millisecond-level real-time response tasks with computationally intensive tasks such as global scheduling and model training. Combined with a modular plug-and-play design, it supports plug-and-play third-party algorithms and heterogeneous devices through standardized interfaces and hardware abstraction layers, thereby enhancing the system's real-time performance, scalability, and ease of integration.

[0084] Built-in decision traceability, AI compliance self-checking, and high-strength data encryption protection mechanisms ensure that the entire process is auditable, compliant with regulations, and secure and reliable. At the same time, based on transfer learning and a unique dual-closed-loop evolution mechanism, the system can quickly adapt to different warehousing scenarios and continuously optimize itself using operational data, achieving intelligent evolution from "deployment-appropriate" to "better with use".

[0085] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing, characterized in that, The system comprises a multi-source data acquisition layer, a multimodal data fusion layer, a digital twin and simulation layer, a two-stage multimodal path planning core layer, a predictive dynamic obstacle avoidance control layer, a full-equipment intelligent collaborative scheduling layer, an edge-cloud collaborative processing layer, a decision auditing and security compliance layer, an intelligent adaptation and self-evolution layer, a semantic SLAM and semantic map layer, and a modular plug-and-play layer. The multi-source data acquisition layer collects real-time data on the warehouse environment, the status of all equipment, and operational tasks through various sensors and system interfaces. The multimodal data fusion layer performs spatiotemporal alignment, repair, and denoising on heterogeneous data, and extracts fusion features through the Transformer cross-modal attention mechanism to provide a unified input for decision-making. The digital twin and simulation layer constructs a high-precision 1:1 virtual warehouse and full-equipment model, predicting risks and verifying solutions through simulation, and achieving bidirectional virtual-real driving and simulation-as-control. The dual-stage multimodal path planning core layer adopts a dual-stage architecture combining GCN static global optimization and PPO dynamic local response to generate and dynamically adjust the optimal path. The predictive dynamic obstacle avoidance control layer predicts obstacle trajectories based on LSTM networks, generates alternative paths in real time, and achieves multi-equipment collaborative obstacle avoidance through multi-agent game theory. The full-equipment group intelligent collaborative scheduling layer adopts a centralized-distributed hybrid decision-making architecture and multi-agent reinforcement learning to coordinate the collaborative operation of multiple heterogeneous devices. The edge-cloud collaborative processing layer performs real-time control at the edge and global scheduling and digital twin simulation at the cloud. The decision audit and security compliance layer records the decision-making process and performs compliance review and data encryption. The intelligent adaptation and self-evolution layer achieves rapid system adaptation and continuous self-optimization based on transfer learning and a dual closed-loop evolution mechanism. Semantic SLAM and the semantic map layer construct an environmental map containing semantic information, and combine human intention understanding to realize human-machine collaboration and dynamic right-of-way allocation; the modular plug-and-play layer enables plug-and-play use of third-party algorithms and heterogeneous devices by defining standardized interfaces and hardware abstraction layers.

2. The multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 1, characterized in that, The multi-source data acquisition layer includes an environmental perception module, a full equipment status module, and a task and system data module. The environmental perception module integrates LiDAR, 3D vision cameras, ultrasonic sensors, and IoT sensors. The full equipment status module includes a real-time equipment data acquisition unit, a historical operation and maintenance database, and an equipment parameter configuration library. The task and system data module includes an enterprise information system interface, a digital twin data interface, and a CAD / BIM data interface.

3. The multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 2, characterized in that, The multimodal data fusion layer includes a spatiotemporal alignment module, a data repair and noise suppression module, and a cross-modal feature extraction and fusion module. The spatiotemporal alignment module includes a sliding window aligner, a clock synchronization service, and a cross-device timing calibrator. The clock synchronization service performs a unified timestamp reference construction step to establish a unified time reference for all data sources. The sliding window aligner performs a frequency difference matching step to handle differences in the acquisition frequencies of different sensors. The cross-device timing calibrator performs a linkage device timing synchronization step to synchronize the action timing of the AGV and the robotic arm. The data repair and noise suppression module includes a linear interpolation compensator, an outlier filter, and a wavelet threshold denoising processor. The linear interpolation compensator performs a data breakpoint filling step to repair data loss caused by sensor failures. The outlier filter performs an abnormal data removal step to identify and remove data that deviates from the normal range. The wavelet threshold denoising processor... The system performs a signal denoising step, decomposing the signal and suppressing sensor noise. The cross-modal feature extraction and fusion module includes a Transformer cross-modal attention mechanism unit, a spatiotemporal feature extractor, and a feature normalization output unit. The Transformer cross-modal attention mechanism unit is used to perform the modal association weight calculation step, strengthening the association relationship between multi-source data. The spatiotemporal feature extractor is used to perform the deep feature extraction step, analyzing the spatial distribution and temporal series dependency relationship of the data. The feature normalization output unit is used to perform the feature integration step, outputting a unified fused feature vector. The multimodal data fusion layer handles the acquisition frequency differences of different sensors through a sliding window aligner, uses wavelet thresholding to suppress sensor noise, and then calculates and strengthens the association weights between laser point clouds, visual images, IoT data, device status, and task information through the Transformer cross-modal attention mechanism.

4. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 3, characterized in that, The digital twin and simulation layer includes a digital twin engine module, a warehouse 3D simulation module, a warehousing equipment 3D simulation module, a transportation equipment 3D simulation module, a work scene simulation package module, a map dynamic update module, a virtual-real linkage simulation module, and a predictive maintenance module. The digital twin engine module includes a high-fidelity modeling core unit, a data synchronization service unit, and a reverse control interface unit. The high-fidelity modeling core unit executes the 1:1 virtual model construction step, building a virtual warehouse model based on multimodal data. The data synchronization service unit executes the bidirectional synchronization step between virtual and physical data, achieving real-time linkage between the physical and virtual models. The reverse control interface unit executes the simulation command issuance step, directly transmitting the virtual-verified commands to the physical devices. The warehouse 3D simulation module includes a CAD / BIM parser and reconstructor, a parametric model library, and a physical attribute mapper. The CAD / BIM parser and reconstructor executes the simulation command issuance step, directly transmitting the virtual-verified commands to the physical devices. The paper-based analysis process imports and parses warehouse design data; the parametric model library is used to perform automated 3D model building, constructing millimeter-level warehouse 3D models; the physical property mapper is used to perform physical property binding, mapping properties including ground friction and load-bearing capacity to the model; the 3D simulation module for warehousing equipment includes a fixed equipment parametric model library, a dynamic parameter configurator, and a communication protocol interface simulator; the fixed equipment parametric model library is used to perform 1:1 modeling of fixed equipment, constructing 3D models of stacker cranes and robotic arms; the dynamic parameter configurator is used to perform equipment parameter configuration, setting dynamic parameters, including mass and velocity; The communication protocol interface simulator is used to perform protocol simulation steps and replicate the actual communication logic of the device. The 3D simulation module for transportation equipment includes a parametric model library for mobile devices, a kinematics simulator, and a load state simulator. The parametric model library for mobile devices is used to perform the AGV modeling step and build a model of the mobile transportation equipment. The kinematics simulator is used to perform the motion characteristic simulation step and simulate the dynamic characteristics of the equipment's driving and steering. The load state simulator is used to perform the load effect simulation step and simulate the impact of load changes on the equipment's motion. The work scenario simulation package module includes a library of normal and abnormal scenario models and an adversarial training scenario generator. The normal and abnormal scenario model library is used to execute scenario preset steps and store normal, fault, and emergency simulation scenarios; The adversarial training scenario generator is used to execute the adversarial scenario generation step, providing the system with stress test and contingency plan simulation scenarios; the map dynamic update module includes a real-time data listener, a map element automatic updater, and a manual calibration interface; the real-time data listener is used to execute the environmental data monitoring step, detecting changes in the warehouse environment in real time; the map element automatic updater is used to execute the virtual map synchronization step, automatically updating the digital twin map; the manual calibration interface is used to execute the manual calibration step, providing an interface for manually correcting the virtual map; the virtual-real linkage simulation module includes a parallel scheme inferrer, a safety / efficiency evaluator, and an optimal scheme issuer; the parallel scheme inferrer is used to execute the multi-strategy parallel simulation step, simulating multiple path schemes in advance; the safety / efficiency evaluator is used to execute the scheme evaluation step, calculating path feasibility, safety, and operational efficiency. The optimal solution issuer is used to execute the optimal instruction issuance steps and transmit the simulated optimal solution to the physical device; the predictive maintenance module includes a critical component degradation model library, a real-time performance data analyzer, and an early warning trigger and report generator; The critical component degradation model library is used to perform the life modeling step and build mathematical models of critical component degradation; the real-time performance data analyzer is used to perform the performance deviation analysis step and compare real-time data with model prediction curves; the early warning trigger and report generator is used to perform the fault early warning step and generate component failure early warnings and maintenance suggestions in advance. The digital twin and simulation layer directly sends control commands verified in the virtual environment to the physical device for execution through the reverse control interface unit. It performs advanced parallel simulation of multiple candidate strategies in the digital twin environment and sends the optimal solution after safety and efficiency evaluation.

5. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 4, characterized in that, The core layer of the dual-stage multimodal path planning includes a GCN static global optimization module, a PPO dynamic local response module, and a multimodal planning mode selection module. The GCN static global optimization module includes a warehouse topology graph builder, a graph convolutional neural network node feature aggregator, and a multi-objective cost optimization function unit. The PPO dynamic local response module includes a PPO policy network unit, an improved UKF motion parameter estimator, and a real-time feedback interface unit. The multimodal planning mode selection module includes a mode decision-maker and an algorithm fusion executor. In the core layer of the dual-stage multimodal path planning, the GCN static global optimization module... The global optimal basic path is generated based on the warehouse topology map. The PPO dynamic local response module receives real-time feedback at a high frequency of 200Hz and performs local fine-tuning on the global path. The improved UKF algorithm ensures control accuracy under load changes and ground slippage. The improved UKF motion parameter estimator is an improved unscented Kalman filter algorithm. Based on the standard unscented Kalman filter, it introduces AGV load weight factor, ground friction coefficient adaptive correction term and sigma point weighted optimization strategy to estimate AGV kinematic parameters in real time at a high frequency of 200Hz, thereby improving control accuracy under ground slippage and load change conditions. The state update formula for the improved UKF algorithm is as follows: ; Its gain correction formula is: ; In the formula: This represents the optimal state estimate of the AGV at time k after filtering correction, including but not limited to kinematic parameters such as position, velocity, attitude, and steering angle; This represents the predicted state value at time k, which is calculated based on the state at the previous time and the motion model. This represents the adaptive Kalman gain at time k, used to balance the weights of "predicted values" and "observed values". This represents the actual observation value of the sensor at time k, and the data comes from, but is not limited to, lidar, vision, odometer, and IMU. This represents the observed predicted value corresponding to the state at time k, which is obtained by mapping the state variables. The state prediction covariance at time k represents the confidence level / uncertainty of the predicted value. This represents the observation matrix, used to map state variables to observations; This represents the transpose of the observation matrix; This represents the observation noise covariance, which reflects the noise level of the sensor itself. This is the load adaptive coefficient, used to adjust the filter gain according to the load strength of the AGV. This indicates the real-time load quality of the AGV, i.e., the current load weight, which will change dynamically from heavy load to light load to empty load. Ground friction correction coefficient, used to adjust filtering according to the smoothness / roughness / slipperiness of the ground; The value represents the real-time ground adhesion coefficient. The smaller the value, the more slippery the ground, and the larger the value, the less slippery the ground. -1 represents the matrix inversion operation. The improved UKF estimates the AGV kinematic parameters in real time at a high frequency of 200Hz. Through adaptive correction of load and ground friction, it ensures high-precision state estimation and path tracking stability under load changes and ground slippage conditions. Global phase: The improved GCN spatiotemporal weighted path planning formula is as follows: ； In the formula, This represents the node feature matrix output by the (l+1)th layer graph convolution, which is the new feature obtained after convolution for each point in the warehouse map; This represents a non-linear activation function used to enhance the model's expressive power and extract features from complex warehousing environments. This represents the warehouse topology adjacency matrix with self-loops, i.e., the connectivity between various points in the warehouse; Indicates and The corresponding degree matrix shows how many paths each point is connected to. Represents the normalized inverse square root of the degree matrix, used to normalize graph convolutions to prevent numerical explosion / gradient vanishing; The node feature matrix represents the input of the l-th layer, which contains information including, but not limited to, point coordinates, passage status, device location, and obstacles. The weight matrix represents the convolution of the l-th layer graph, and the optimal parameters for path planning learned automatically by the neural network. represents the spatiotemporal weight coefficient, with a value range of 0.2 to 0.5, which controls the influence of "historical efficiency, current congestion, and future tasks" on the path; T represents the node spatiotemporal feature matrix, which includes three types of warehouse-specific features: historical traffic efficiency, current congestion degree, and future task prediction value; The spatiotemporal feature enhancements enable path planning to predict congestion and dynamically optimize; output: globally optimal basic path Pglobal. Local stage: The improved PPO dynamic path optimization formula is as follows: ； In the formula, This represents the loss function value of the improved PPO policy network; This represents the mathematical expectation based on time step t; Indicates the probability ratio of the strategies; Represents the dominance function; This indicates a truncation function to limit the policy update magnitude; This represents the clipping threshold, with a value ranging from 0.1 to 0.

2. This represents the obstacle distance penalty coefficient, which is a positive hyperparameter. Indicates the distance between the AGV and the nearest obstacle; This represents the task priority reward coefficient, which is a positive hyperparameter. This represents the priority weight of the current task; the final output is the locally fine-tuned optimal path Plocal. The final composite formula for two-stage path planning is: ； That is, the final optimal path = fusion of improved GCN spatiotemporal global planning + improved PPO penalty-reward local planning; In the formula, This represents the globally optimal and locally safe final path that the AGV ultimately executes. This represents a fusion function of global and local paths, used to combine macro-planning with dynamic adjustments; This represents an improved graph convolutional neural network that incorporates spatiotemporal weighting factors to generate globally optimal paths. This represents the global base path of the warehouse output by the improved GCN; This represents an improved proximal policy optimization algorithm that incorporates obstacle penalties and task rewards to generate locally dynamically corrected paths. This represents the local obstacle avoidance fine-tuning path output by the improved PPO; the formula clearly expresses the core architecture of the two-stage path planning of "global static planning + local dynamic correction". First, the global optimal path is obtained by improving GCN, and then real-time obstacle avoidance and priority adjustment are performed by improving PPO. Finally, a safe, efficient and stable final driving path is obtained by fusion.

6. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 5, characterized in that, The predictive dynamic obstacle avoidance control layer includes an interruption event detection and obstacle classification module, a trajectory prediction module, an alternative path solution and speed control module, and an obstacle avoidance coordination module; the interruption event detection and obstacle classification module includes a computer vision real-time processor, a multimodal data association analyzer, and an obstacle classifier; The computer vision real-time processor performs real-time visual data processing, handling data streams from cameras and LiDAR. The multimodal data association analyzer performs sudden obstacle detection, quickly identifying dynamic obstacles in the warehouse environment. The obstacle classifier performs obstacle type differentiation, classifying obstacle types including personnel, AGVs, and debris. The trajectory prediction module includes an LSTM network prediction model unit, a short-term trajectory extrapolator, and a collision risk predictor. The LSTM network prediction model unit performs obstacle trajectory prediction, learning historical trajectories and predicting future obstacle paths. The short-term trajectory extrapolator performs path intersection calculation, extrapolating the spatiotemporal intersection of obstacles and AGV paths. The collision risk predictor performs collision risk assessment, proactively predicting potential collisions. In the collision risk management module, the alternative path solution and speed control module includes a spatiotemporal alternative path fast solver, a speed adaptive controller, and a path constraint filter. The spatiotemporal alternative path fast solver is used to perform the collision-free path generation step, quickly generating feasible alternative paths. The path constraint filter is used to perform the constraint screening step, eliminating road sections that do not meet height and weight limits. The speed adaptive controller is used to perform the speed dynamic adjustment step, adjusting the AGV's travel speed according to the obstacle distance. The obstacle avoidance coordination module includes a multi-device communication coordinator, an obstacle avoidance priority arbitrator, and a multi-agent game decision-maker. The multi-device communication coordinator is used to perform the obstacle avoidance intention synchronization step, coordinating the obstacle avoidance plans of surrounding devices. The obstacle avoidance priority arbitrator is used to perform the right-of-way allocation step, determining the obstacle avoidance order based on task priority. A multi-agent game decision-maker is used to execute collaborative negotiation steps, and distributed negotiation generates the optimal group avoidance scheme; the predictive dynamic obstacle avoidance control layer completes interruption event detection and obstacle classification within 0.3 seconds, predicts the movement trajectory of obstacles in the next 3 seconds through an LSTM network, and the spatiotemporal alternative path fast solver generates feasible collision-free alternative paths within 3 seconds.

7. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 6, characterized in that, The full-equipment group intelligent collaborative scheduling layer includes a task dynamic allocation module, a path conflict resolution module, and a group intelligent decision-making module; the task dynamic allocation module includes a centralized global optimizer, a multi-dimensional equipment status evaluator, and a real-time task queue and traffic balancer; the multi-dimensional equipment status evaluator is used to perform equipment status acquisition steps to obtain the real-time operating status of AGVs and robotic arms; the real-time task queue and traffic balancer is used to perform task queue synchronization steps to obtain global order tasks; The centralized global optimizer executes the task allocation step, allocating tasks based on task urgency and equipment compatibility. The path conflict resolution module includes a multi-device path intersection real-time monitor, a dynamic avoidance node optimizer, and a timing and restricted area adjuster. The multi-device path intersection real-time monitor executes the conflict prediction step, monitoring device paths and predicting spatiotemporal intersection conflicts. The dynamic avoidance node optimizer executes the passage order adjustment step, optimizing device avoidance waiting nodes. The timing and restricted area adjuster executes the dynamic restricted area setting step, setting instantaneous restricted areas on the digital map to guide avoidance. The swarm intelligence decision-making module includes a centralized-distributed hybrid decision architecture unit, a multi-agent reinforcement learning cluster unit, a virtual parallel inference environment unit, and a rapid negotiation mechanism. The system comprises the following components: a consensus protocol unit; a centralized-distributed hybrid decision-making architecture unit for executing agent creation steps, creating virtual agents for physical devices; a multi-agent reinforcement learning cluster unit for executing collaborative decision-making learning steps, optimizing group collaboration strategies through reinforcement learning; a virtual parallel inference environment unit for executing collaborative action simulation steps, inferring future collaborative action sequences in parallel; a fast negotiation and consensus protocol unit for executing distributed negotiation steps, exchanging information among agents to reach a consensus; and a full-device group intelligent collaborative scheduling layer for creating corresponding agent agents for each physical device. Under the guidance of a centralized global goal, each agent engages in game negotiation based on distributed local perception, exchanging information through a fast communication protocol to reach a locally optimal consensus.

8. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 7, characterized in that, The edge-cloud collaborative processing layer includes an edge computing node module and a cloud scheduling center module. The edge computing node module includes a real-time data stream processor, a local intelligent agent policy executor, a high-frequency control command generator, and an edge storage. The cloud scheduling center module includes a global task and resource scheduler, a digital twin simulation cluster unit, a historical big data warehouse, and an algorithm model training and version management center. In the edge-cloud collaborative processing layer, the edge computing nodes process high-frequency real-time data streams generated by LiDAR and cameras, and execute PPO path fine-tuning and LSTM emergency obstacle avoidance tasks. The cloud scheduling center performs macro-task planning, digital twin simulation cluster computation, and offline training and version management of algorithm models. The decision audit and security compliance layer includes a decision tracing module, an AI compliance self-check module, and a data security protection module. The decision tracing module includes a decision process mirror recorder, an interactive decision tree builder, and a real-time decision deviation scanner and alarm. The AI ​​compliance self-check module includes a multi-industry compliance rule library, an automated verification engine unit, and a standardized audit report generator. The data security protection module includes an AES-256 end-to-end encryption / decryption engine unit, an intrusion detection and behavior analysis unit, and a real-time alarm and blocking controller.

9. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 8, characterized in that, The intelligent adaptation and self-evolution layer includes a scene adaptation module, a self-evolutionary learning engine module, an API template extension module, a human-computer interaction and visualization platform, and a VR / AR human-computer interaction module; the scene adaptation module includes a transfer learning framework unit, a cross-scene parameter automatic adjuster, and a simulation scene rapid configurator; the transfer learning framework unit is used to perform the model transfer step to transfer mature algorithms to new warehouse scenarios. The cross-scene parameter automatic adjuster is used to perform parameter adaptation steps and automatically adjust obstacle avoidance and path weight parameters; the simulation scene rapid configurator is used to perform virtual scene configuration steps and quickly adapt to the new scene digital twin simulation environment; the self-evolutionary learning engine module includes an instant decision loop, an offline evolution loop and an evolution driving unit. The instant decision loop is used to execute online virtual simulation steps, simulating in parallel and selecting the optimal solution for execution; the offline evolution loop is used to execute batch retraining steps, optimizing the algorithm model using historical and simulation data; the evolutionary driving unit is used to execute adversarial scenario generation steps, generating extreme scenario reinforcement model training; the API template extension module includes a standardized API template library and automatic interface adapters. The standardized API template library is used to execute the pre-set template storage steps, storing interface templates for mainstream cloud platforms and business systems; the interface automatic adapter is used to execute the rapid integration steps, calling templates to complete seamless integration with external systems; the human-computer interaction and visualization platform includes a 3D panoramic visualization engine, a historical data backtracking analyzer, and a simulation scene editing and injection interface unit. The 3D panoramic visualization engine is used to perform the 3D display steps, presenting the real-time operational status of the entire warehouse; the historical data backtracking analyzer is used to perform the historical playback steps, backtracking and analyzing operational data at any point in time. The simulation scene editing and injection interface unit is used to execute manual intervention steps, supporting administrators to modify simulation parameters and inject test scenes; the VR / AR human-computer interaction module includes AR glasses, a VR immersive simulation environment unit, and a gesture and voice recognition interactive device; the AR glasses are used to execute the virtual-real overlay display step, overlaying virtual data of the device onto the real field of vision; the VR immersive simulation environment unit is used to execute the immersive training step, providing an environment for employee training and contingency plan drills; the gesture and voice recognition interactive device is used to execute the command interaction step, realizing remote scheduling intervention through gestures and voice; the dual closed-loop evolution mechanism of intelligent adaptation and self-evolution layer includes an instant decision-making loop and an offline evolution loop. When the instant decision-making loop runs online, it deduces the optimal solution in parallel in the digital twin, while the offline evolution loop uses historical and simulation data to batch retrain the algorithm model during system idle time.

10. A multimodal path planning and dynamic obstacle avoidance control system based on intelligent warehousing according to claim 9, characterized in that, The semantic SLAM and semantic map layer includes a semantic map module, a visual SLAM module, and a human-computer collaboration and intent recognition module. The semantic map module includes a semantic map construction and storage engine unit, a multi-level semantic tag library, and a dynamic semantic element updater. The semantic map construction and storage engine unit performs the semantic map fusion step, fusing geometric and semantic information to construct an environment map. The multi-level semantic tag library performs the semantic annotation step, labeling regions and objects with functional attributes. The dynamic semantic element updater performs the semantic information update step, updating map semantic elements in real time. The visual SLAM module includes a visual odometry, a feature point extractor and matcher, and a dynamic object semantic recognition and tracker. The visual odometry is used to perform real-time positioning... The device is positioned and mapped synchronously during movement. A feature point extractor and matcher performs feature matching, extracting visual features and completing the matching process. A dynamic object semantic recognition and tracker performs dynamic object recognition, classifying and tracking dynamic objects. The human-machine collaboration and intent recognition module includes a visual gesture recognition unit, a human posture and intent parser, and a dynamic right-of-way allocation unit for the human-machine shared space. The visual gesture recognition unit performs human posture recognition, recognizing worker gestures and movement trends. The human posture and intent parser performs operation intent parsing, analyzing the operator's intent. The dynamic right-of-way allocation unit for the human-machine shared space performs dynamic right-of-way adjustment, adjusting the right-of-way based on intent. The semantic SLAM and semantic map layer annotates functional attributes for each region or object, identifies and distinguishes different types of dynamic objects in real time, understands human operation intentions through visual gesture recognition and human posture analysis, and dynamically adjusts access rights in the human-machine shared space. The modular pluggable layer includes a standardized interface definition module and a hardware abstraction module. The standardized interface definition module includes an algorithm interface protocol library, a service registration and discovery center unit, and a compatibility testing unit. The algorithm interface protocol library is used to execute interface specification formulation steps and establish a unified algorithm API and communication protocol. The service registration and discovery center unit is used to execute third-party algorithm registration steps and automatically identify and call compliant third-party algorithms. The compatibility testing unit is used to execute... The system performs compatibility checks to ensure seamless algorithm replacement. The hardware abstraction module includes a device driver manager, a generic instruction-to-protocol converter, and a hardware configuration template library. The device driver manager performs hardware adaptation steps, building a unified hardware and software adaptation layer. The generic instruction-to-protocol converter performs instruction conversion steps, converting device-specific instructions into system-wide generic instructions. The hardware configuration template library performs hardware configuration steps, shielding hardware differences between different brands of devices. The modular pluggable layer's hardware abstraction module uses the generic instruction-to-protocol converter to convert various device-specific control instructions into system-wide generic instructions, shielding hardware differences between different brands and models of devices and achieving seamless integration of heterogeneous devices.

Images