A reinforcement learning unmanned forklift obstacle avoidance scheduling method and system for dynamic obstacles

Abstract

The present application discloses a reinforcement learning unmanned forklift obstacle avoidance scheduling method and system for dynamic obstacles, which relates to the field of unmanned driving technology. The disclosed reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles generates a dynamic obstacle state matrix through multimodal sensor data fusion, and combines the reinforcement learning network to perform global path planning and local obstacle avoidance decision-making, thereby solving the problems of obstacle avoidance response delay and unreasonable path planning in dynamic environments, improving the obstacle avoidance response speed, path planning rationality and multimodal data fusion accuracy in dynamic environments, and thus improving the safety and efficiency of unmanned forklift scheduling.

Description

Technical Field

[0001] The present application relates to the field of unmanned driving technology, and in particular to a reinforcement learning unmanned forklift obstacle avoidance scheduling method and system for dynamic obstacles. Background Art

[0002] Obstacle avoidance and scheduling is the core technology for unmanned forklifts to achieve safe and efficient operation in dynamic industrial environments. Existing technologies mainly use rule-based or static path planning methods. These methods often have problems such as untimely response and unreasonable path planning when facing dynamic obstacles, which limits the operating efficiency and safety of unmanned forklifts in complex environments. Specifically, rule-based methods usually rely on preset obstacle recognition rules and obstacle avoidance strategies, which are difficult to adapt to complex and changing dynamic environments. Static path planning methods focus on finding the optimal path in a static environment and lack the ability to perceive and respond to dynamic obstacles in real time. Therefore, in industrial environments where dynamic obstacles frequently appear, these methods often cannot guarantee the safe and efficient operation of unmanned forklifts. In addition, existing technologies often lack an effective fusion mechanism when processing multimodal sensor data, resulting in insufficient perception accuracy of dynamic obstacles, further affecting the effect of obstacle avoidance and scheduling.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a reinforcement learning unmanned forklift obstacle avoidance scheduling method and system for dynamic obstacles, aiming to improve the safety and efficiency of unmanned forklift scheduling.

[0005] To achieve the above objectives, this application proposes a reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles, the method comprising:

[0006] Acquire raw perception data through a multimodal sensor; the raw perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data;

[0007] Preprocessing the original perception data to generate corresponding preprocessed data;

[0008] Performing spatiotemporal alignment and feature fusion on the pre-processed data to generate a dynamic obstacle state matrix including obstacle position, velocity, and category;

[0009] Inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence;

[0010] The dynamic obstacle state matrix and global path data are input into a pre-trained local obstacle avoidance decision network to generate a driving control instruction including a steering angle and acceleration to dispatch the unmanned forklift.

[0011] In one embodiment, the step of preprocessing the original perception data to generate corresponding preprocessed data includes:

[0012] Performing voxel filtering on the laser radar point cloud data to remove noise to generate filtered point cloud data, and performing dynamic obstacle feature extraction on the filtered point cloud data to generate dynamic grid map data including obstacle positions and velocities;

[0013] Performing semantic segmentation on the image data to generate obstacle category label data and depth mask data;

[0014] The UWB positioning data is combined with the IMU data to perform Kalman filtering to generate corresponding posture data.

[0015] In one embodiment, the step of extracting dynamic obstacle features from the filtered point cloud data to generate dynamic grid map data containing obstacle positions and speeds includes:

[0016] Obstacle clustering is performed on the filtered point cloud data based on the Euclidean clustering algorithm to generate cluster bounding box data containing the three-dimensional coordinates of the obstacles;

[0017] The displacement of each cluster bounding box is calculated by matching the continuous frame point cloud, and the moving speed is calculated according to the preset time interval to generate the obstacle motion vector data containing the position and speed;

[0018] The obstacle motion vector data is mapped to a grid map coordinate system to generate dynamic grid map data including the obstacle position and speed.

[0019] In one embodiment, the step of performing spatiotemporal alignment and feature fusion on the pre-processed data to generate a dynamic obstacle state matrix including obstacle position, speed, and category includes:

[0020] Fusing the dynamic raster map data with the depth mask data to generate semantic raster data with velocity vectors;

[0021] Encode the obstacle category label data into a one-hot vector to generate obstacle type feature data;

[0022] The semantic grid data is spliced with obstacle type feature data to form a dynamic obstacle state matrix.

[0023] In one embodiment, the step of inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence includes:

[0024] Input the current task target position and dynamic obstacle state matrix into the pre-trained global path planning network and output a sequence of candidate path nodes;

[0025] The candidate path node sequence is smoothly optimized to generate global path data containing Bezier curve parameters.

[0026] In one embodiment, the step of inputting the current task target position and the dynamic obstacle state matrix into a pre-trained global path planning network and outputting a candidate path node sequence includes:

[0027] Obtain multiple candidate paths;

[0028] Calculate the actual length, number of turns, and minimum safe distance between the current candidate path and obstacles for each candidate path;

[0029] Substitute the actual length of the candidate path, the number of turns, and the minimum safe distance between the current candidate path and the obstacle into the preset reward formula to calculate the reward value of each path;

[0030] A preset number of candidate paths are selected from high to low reward values as the candidate node sequence.

[0031] In one embodiment,

[0032] The preset reward formula is:

[0033] ;

[0034] in, is the reward value; is the path efficiency weight coefficient; is the theoretical shortest path length from the starting point to the end point; is the actual length of the candidate path; is the steering penalty weight coefficient; is the number of turns in the candidate path; is the maximum number of turns allowed; is the safety priority weight coefficient; is the minimum safe distance between the candidate path and the obstacle; The preset safety distance threshold.

[0035] In one embodiment, the step of inputting the dynamic obstacle state matrix and global path data into a local obstacle avoidance decision network and outputting a drive control instruction including a steering angle and acceleration in real time to dispatch the unmanned forklift includes:

[0036] Obstacles with speeds greater than a preset speed in the dynamic obstacle state matrix are extracted as key obstacles;

[0037] Calculate the influence weight of each key obstacle on the current decision through the attention mechanism;

[0038] Adjusting the hidden state of the local obstacle avoidance decision network according to the influence weight to generate an action instruction that takes into account the intention of the dynamic obstacle;

[0039] Based on the path node sequence, the artificial potential field method is used to calculate the current environmental potential and perform trajectory correction on the preliminary action instructions;

[0040] Collision risk assessment is used to filter high-risk commands and generate final drive control commands.

[0041] In one embodiment, the method further comprises:

[0042] When an unrecognized obstacle type is detected, storing an event data packet including a motion pattern of the obstacle;

[0043] Constructing the motion prediction model of the obstacle through contrastive learning algorithm;

[0044] The motion prediction model is integrated into the local obstacle avoidance decision network to update the weight parameters of the local obstacle avoidance decision network.

[0045] In addition, to achieve the above objectives, this application also proposes a reinforcement learning unmanned forklift obstacle avoidance and scheduling system for dynamic obstacles, the system comprising:

[0046] A data acquisition module is used to acquire raw perception data through a multimodal sensor; the raw perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data;

[0047] A preprocessing module, configured to preprocess the original perception data to generate corresponding preprocessed data;

[0048] A dynamic obstacle matrix generation module is used to perform spatiotemporal alignment and feature fusion on the pre-processed data to generate a dynamic obstacle state matrix including obstacle position, speed and category;

[0049] A global path data generation module, configured to input the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence;

[0050] The drive control instruction generation module is used to input the dynamic obstacle state matrix and global path data into the pre-trained local obstacle avoidance decision network to generate a drive control instruction including a steering angle and acceleration to dispatch the unmanned forklift.

[0051] This application proposes a reinforcement learning-based obstacle avoidance and scheduling method for unmanned forklifts with dynamic obstacles. The method uses multimodal sensor data to acquire raw sensor data, including lidar point cloud data, image data, UWB positioning data, and IMU data. The raw sensor data is then preprocessed to generate corresponding preprocessed data. The preprocessed data is then subjected to spatiotemporal alignment and feature fusion to generate a dynamic obstacle state matrix containing obstacle positions, velocities, and categories. The dynamic obstacle state matrix is then input into a pretrained global path planning network to generate global path data containing a sequence of path nodes. Finally, the dynamic obstacle state matrix and global path data are input into a pretrained local obstacle avoidance decision network to generate drive control instructions containing steering angles and accelerations for scheduling the unmanned forklift. This method, by fusing multimodal sensor data to generate a dynamic obstacle state matrix and combining it with a reinforcement learning network for global path planning and local obstacle avoidance decision-making, addresses the issues of delayed obstacle avoidance response and irrational path planning in dynamic environments. It improves obstacle avoidance response speed, path planning rationality, and multimodal data fusion accuracy in dynamic environments, thereby enhancing the safety and efficiency of unmanned forklift scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A flowchart illustrating an embodiment of a reinforcement learning unmanned forklift obstacle avoidance and scheduling method for dynamic obstacles provided by this application;

[0055] Figure 2 For this application Figure 1 Detailed flow diagram of step S200;

[0056] Figure 3 For this application Figure 2 Detailed flowchart of step S210;

[0057] Figure 4 For this application Figure 1 Detailed flow diagram of step S300;

[0058] Figure 5 For this application Figure 1 Detailed flowchart of step S400;

[0059] Figure 6 For this application Figure 5 Detailed flowchart of step S410;

[0060] Figure 7 For this application Figure 1 Detailed flowchart of step S500;

[0061] Figure 8 A flowchart illustrating another embodiment of the present invention's reinforcement learning unmanned forklift obstacle avoidance and scheduling method for dynamic obstacles;

[0062] Figure 9 This is a structural diagram of an embodiment of a reinforcement learning unmanned forklift obstacle avoidance and scheduling system for dynamic obstacles provided by this application.

[0063] Explanation of the accompanying figures: 10. Reinforcement learning unmanned forklift obstacle avoidance and dispatching system for dynamic obstacles; 100. Data acquisition module; 200. Preprocessing module; 300. Dynamic obstacle matrix generation module; 400. Global path data generation module; 500. Drive control instruction generation module.

[0064] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0065] The technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. The components of this application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents selected embodiments of the application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0066] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0067] Unmanned forklift obstacle avoidance and scheduling technology has long faced challenges with perception and decision-making in dynamic environments. Traditional rule-based obstacle avoidance methods rely on fixed thresholds to determine the threat level of obstacles. When encountering pallet trucks or walking workers with varying speeds, the unpredictable trajectory often leads to sudden stops or detours. While static path planning algorithms can generate an initial optimal path, in warehouse environments where forklifts and AGVs frequently intersect, fixed paths cannot adapt to the ever-changing distribution of obstacles, making path blockage and repeated planning more likely.

[0068] To address these issues, we first considered how to overcome the limitations of single-sensor perception. Experiments revealed that lidar point clouds are sparse in rainy and foggy conditions, while visual sensors are prone to failure in low-light environments. This led us to propose a multimodal sensor complementarity solution. Secondly, to address the challenge of predicting dynamic obstacle trajectories, we observed the correlation between historical motion data and the current velocity vector, and subsequently designed a spatiotemporal alignment mechanism to capture motion trends. Finally, to address the coordination between global planning and local obstacle avoidance, we constructed a hierarchical decision-making model using a reinforcement learning framework. The upper-layer network is responsible for macroscopic path generation, while the lower-layer network focuses on real-time obstacle avoidance, achieving an organic connection between the decision-making levels.

[0069] Therefore, this application proposes a reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles, referring to Figure 1 The method includes steps S100 to S500, wherein:

[0070] Step S100: acquiring raw perception data through a multimodal sensor; the raw perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data;

[0071] Step S200, preprocessing the original perception data to generate corresponding preprocessed data;

[0072] Step S300 , performing spatiotemporal alignment and feature fusion on the pre-processed data to generate a dynamic obstacle state matrix including obstacle position, speed, and category;

[0073] Step S400: inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence;

[0074] Step S500: Input the dynamic obstacle state matrix and global path data into a pre-trained local obstacle avoidance decision network to generate a drive control instruction including a steering angle and acceleration to dispatch the unmanned forklift.

[0075] In this embodiment, a multimodal sensor refers to a device combination that integrates a lidar, camera, UWB module, and inertial measurement unit. Specifically, a 16-line lidar and an RGB-D camera can be used in a spatially complementary arrangement to achieve simultaneous acquisition of three-dimensional spatial perception and two-dimensional visual information. Spatiotemporal alignment refers to coordinate system conversion and timestamp calibration of multi-source data. Specifically, this can be achieved through an extrinsic calibration matrix to unify the sensor coordinate system, while hardware synchronization triggering can be used to eliminate data acquisition latency. The dynamic obstacle state matrix refers to structured data containing the spatial coordinates, motion vectors, and semantic categories of obstacles. Specifically, it can be stored as a three-dimensional tensor, with each grid cell recording the velocity vector and category code of the obstacle. The global path planning network refers to a path generation model based on deep reinforcement learning. Specifically, a double-delayed deep deterministic policy gradient algorithm can be used. A reward function can be used to guide the network to learn a planning strategy that balances path length and safe distance. The local obstacle avoidance decision network refers to a real-time motion control model. Specifically, a proximal policy optimization algorithm can be used to generate smooth control commands based on the current velocity constraint and path tracking deviation.

[0076] In this embodiment, specifically, after the lidar point cloud is voxel filtered, the obstacle displacement vector is calculated through continuous frame matching to form a dynamic raster map. After semantic segmentation, the visual data is extracted from category labels such as pallets and pedestrians, and fused with the point cloud velocity information to form a motion vector with semantic annotations. After Kalman filtering, the UWB and IMU data provide centimeter-level positioning accuracy. After these pre-processed data are aligned in time and space, a three-dimensional state matrix is formed through feature splicing, in which the velocity vector helps predict the movement trend of the obstacle, and the category information distinguishes the avoidance priority. After receiving the state matrix, the global path planning network outputs the path node parameterized by the Bezier curve, and eliminates sudden turns through path smoothing. The local obstacle avoidance decision network combines the real-time obstacle status and the global path node, uses the attention mechanism to identify high-risk obstacles, dynamically adjusts the steering angle and acceleration, and realizes progressive avoidance during the path tracking process.

[0077] Compared with the existing technology, the multimodal sensor fusion solution of this embodiment overcomes the perception blind spots of a single sensor in complex working conditions. For example, UWB signals are used to supplement positioning information in areas blocked by shelves. The dynamic obstacle state matrix breaks through the limitations of traditional two-dimensional grid maps in representing motion trends and realizes motion trajectory prediction through the velocity vector field. The hierarchical reinforcement learning architecture decouples global path optimization from local obstacle avoidance, avoiding the problem of strategy oscillation of a single decision model in long-term planning. The spatiotemporal alignment mechanism ensures that multi-source data has a consistent spatial reference when making decisions, eliminating decision errors caused by differences in sensor sampling frequencies.

[0078] Through the above technical solutions, the present application realizes real-time trajectory prediction and hierarchical avoidance of dynamic obstacles, effectively reducing the frequency of emergency braking in warehousing and logistics scenarios. Multi-source data fusion improves the robustness of environmental perception under complex lighting conditions. For example, in areas with strong light, the overexposed areas of the visual sensor are compensated by the lidar point cloud. The hierarchical decision-making mechanism takes into account both the global optimality of the path and the real-time performance of local obstacle avoidance, ensuring a safe distance from moving obstacles while maintaining transportation efficiency. The construction of the dynamic obstacle state matrix enables the forklift to predict the worker's walking path and slow down and give way in advance instead of emergency avoidance. That is, the reinforcement learning unmanned forklift obstacle avoidance scheduling method and system for dynamic obstacles described in this embodiment can improve the safety and efficiency of unmanned forklift scheduling.

[0079] In one possible implementation, reference Figure 2 , step S200 includes steps S210 to S230, wherein:

[0080] Step S210: voxel filtering is performed on the lidar point cloud data to remove noise to generate filtered point cloud data, and dynamic obstacle features are extracted from the filtered point cloud data to generate dynamic grid map data containing obstacle positions and velocities;

[0081] Step S220: performing semantic segmentation on the image data to generate obstacle category label data and depth mask data;

[0082] Step S230 combines the UWB positioning data with the IMU data to perform Kalman filtering to generate corresponding posture data.

[0083] In this embodiment, voxel filtering refers to dividing the three-dimensional point cloud into cubic units of equal volume for downsampling processing, which can be implemented by using a voxel grid downsampling algorithm, eliminating outlier noise by merging adjacent point clouds, while retaining the obstacle contour features. Semantic segmentation refers to semantic category labeling of image pixels, which can be implemented by using a semantic segmentation model based on a convolutional neural network, identifying obstacle types through pixel-by-pixel classification and generating mask areas. Kalman filtering refers to optimal estimation of sensor data through a state-space model, which can be implemented by using a linear Kalman filter, suppressing measurement noise by fusing the absolute positioning data of the UWB with the relative motion data of the IMU.

[0084] Specifically, in this embodiment, the LiDAR point cloud is voxel filtered to form a downsampled point cloud, effectively eliminating the interference of environmental noise on obstacle detection. A clustering algorithm is then used to extract the three-dimensional coordinates of the obstacles, and the motion speed is calculated by combining continuous frame displacement analysis to generate a dynamic grid map. Image data undergoes semantic segmentation to obtain obstacle category labels, which are then combined with depth information to generate a three-dimensional spatial mask, complementing the shortcomings of LiDAR in object recognition. UWB and IMU data are time-series fused using Kalman filtering, using the IMU's high-frequency characteristics to compensate for UWB signal delays. The UWB absolute position is then used to correct IMU accumulated errors, ultimately outputting high-precision pose data. These three preprocessing processes are performed simultaneously to form complementary obstacle perception information.

[0085] Compared with existing technologies, traditional methods typically perform direct cluster analysis on laser point clouds, which can easily lead to false detections in noisy environments; using a single sensor for positioning is susceptible to signal interference; and image processing is limited to two-dimensional detection, lacking three-dimensional coordinate correlation. This solution reduces the noise sensitivity of point clouds through voxel filtering and improves the accuracy of dynamic obstacle detection through multi-frame motion analysis. It achieves three-dimensional obstacle classification through semantic segmentation and depth information fusion; and improves positioning robustness through multi-sensor temporal fusion, providing a more reliable perception foundation for subsequent decision-making.

[0086] Through the above technical solutions, this application effectively eliminates noise interference in the raw sensor data, accurately extracts the spatial position, motion speed, and category information of dynamic obstacles, and improves the integrity and accuracy of environmental perception. Laser point cloud preprocessing avoids false detections and missed detections caused by noise, image semantic segmentation enhances obstacle type recognition, and multi-source positioning data fusion ensures the stability of pose estimation, providing high-quality input data for path planning and obstacle avoidance decisions, reducing safety risks caused by perception errors.

[0087] In one possible implementation, reference Figure 3 Step S210 includes steps S211 to S213, wherein:

[0088] Step S211: performing obstacle clustering on the filtered point cloud data based on the Euclidean clustering algorithm to generate cluster bounding box data containing the three-dimensional coordinates of the obstacles;

[0089] Step S212: Calculate the displacement of each cluster bounding box by matching consecutive frame point clouds, and calculate the movement speed according to a preset time interval to generate obstacle motion vector data including position and speed;

[0090] Step S213 : Mapping the obstacle motion vector data to a grid map coordinate system to generate dynamic grid map data including the obstacle position and speed.

[0091] In this embodiment, the Euclidean clustering algorithm refers to a point cloud segmentation algorithm based on a spatial distance threshold. Specifically, it can be implemented using a Euclidean clusterer constructed based on a kd-tree in the point cloud library. By setting the minimum number of cluster points and the maximum cluster radius parameters, the discrete point cloud is segmented into independent obstacle objects. Continuous frame point cloud matching refers to the registration of adjacent time-series point clouds using an iterative nearest point algorithm or a feature matching algorithm. Specifically, a feature descriptor based on normal vector feature extraction can be used to associate corresponding points between frames, thereby calculating the displacement change of the obstacle in three-dimensional space. The preset time interval can be the sensor acquisition period, such as a fixed time difference in the range of 0.1 seconds to 0.5 seconds, which is used to convert the displacement into an instantaneous velocity vector. The dynamic grid map coordinate system refers to a two-dimensional grid coordinate system with the current position of the unmanned forklift as the origin. Specifically, a grid division method with a grid resolution between 0.1 meters and 0.5 meters can be used to project the three-dimensional motion vector onto a two-dimensional plane to achieve spatial mapping.

[0092] Specifically, in this embodiment, the solution first spatially segments the filtered point cloud data using a Euclidean clustering algorithm to eliminate noise interference and distinguish individual obstacles, forming independent bounding boxes with three-dimensional coordinates. Displacement is then calculated using the corresponding obstacle bounding boxes in two consecutive frames of point cloud data. This displacement is converted into an instantaneous velocity vector using a time interval parameter to create a dynamic description of the obstacle's motion trajectory. Finally, the motion vectors in three-dimensional space are projected onto a two-dimensional grid map, generating real-time position and velocity distribution data for the obstacle within each grid cell. This phased process decouples obstacle spatial positioning from motion state analysis, reducing computational complexity while ensuring the accuracy of dynamic information extraction.

[0093] Compared with existing technologies, traditional methods typically use a single frame of point cloud data to directly calculate obstacle speed, resulting in a lag in motion state estimation. Some solutions use fixed thresholds to determine the movement state of obstacles, which cannot adapt to dynamic environments with different motion patterns. This solution accurately captures the instantaneous motion characteristics of obstacles by combining continuous frame matching with time parameters. While existing technologies often use fixed coordinate system transformations to map obstacle information, this solution dynamically projects motion vectors onto a real-time updated grid map, solving the timeliness issue of dynamic environment representation.

[0094] Through the above technical solution, this application effectively solves the technical problem of inaccurate extraction of dynamic obstacle motion information, providing accurate real-time obstacle position and velocity data for path planning. The velocity calculation method based on continuous frame matching eliminates the error accumulation of single-frame data estimation, improving the reliability of dynamic obstacle trajectory prediction. The dynamic update mechanism of the grid map enables obstacle status information to reflect environmental changes in real time, avoiding obstacle avoidance decision errors caused by information lag.

[0095] In one possible implementation, reference Figure 4 , the step S300 includes steps S310 to S330, wherein:

[0096] Step S310, fusing the dynamic raster map data with the depth mask data to generate semantic raster data with velocity vectors;

[0097] Step S320: Encode the obstacle category label data into a one-hot vector to generate obstacle type feature data;

[0098] Step S330: splicing the semantic raster data and the type feature data to form a dynamic obstacle state matrix.

[0099] In this embodiment, dynamic raster map data refers to a two-dimensional rasterized data structure containing the position and speed of obstacles. Specifically, it can be generated by lidar point cloud clustering and inter-frame displacement matching algorithm, and is used to characterize the movement trend of obstacles in space. Depth mask data refers to pixel-level obstacle area identification generated by semantic segmentation and depth estimation. Specifically, it can be implemented by stereo vision or monocular depth estimation model to enhance the accuracy of the spatial position of obstacles. One-hot vector encoding refers to the conversion of discrete category labels into binary vector representations. Specifically, it can be implemented by a dimension allocation method based on the number of categories to avoid interference of category features on model training. Semantic raster data refers to dynamic raster data that integrates depth information. Specifically, it can be generated by spatial coordinate system alignment and channel superposition to unify the movement characteristics and spatial distribution characteristics of obstacles. Splicing operation refers to combining multidimensional features along a specific dimension. Specifically, it can be implemented by matrix splicing or channel merging to construct a multimodal fusion obstacle state representation.

[0100] In this embodiment, specifically, the dynamic raster map data obtains the obstacle velocity vector through continuous frame point cloud matching, combines the three-dimensional spatial information provided by the depth mask data, and generates semantic raster data with the velocity vector after coordinate system alignment. The obstacle category label is encoded by a one-hot vector to form a type feature matrix that matches the channel dimension of the semantic raster data. By expanding the semantic raster data in the channel dimension and superimposing the type feature matrix, a dynamic obstacle state matrix containing three-dimensional information of position, speed, and category is formed. This matrix uses multimodal feature fusion to enable the subsequent path planning network to simultaneously perceive the obstacle movement trend, spatial distribution, and category attributes, effectively avoiding the problem of feature loss caused by a single data source.

[0101] Compared with existing technologies, traditional methods typically process obstacle motion trajectories and category information independently, resulting in the decision model's inability to associate motion features with semantic attributes. This solution uses multimodal data fusion to uniformly encode velocity vectors, depth information, and category labels into a matrix structure, achieving a joint representation of different feature dimensions. For example, in existing technologies, obstacle velocity and category data belong to different processing flows, requiring secondary correlation calculations during the obstacle avoidance decision-making stage. However, this solution completes information integration during the feature fusion stage, reducing computational latency and improving decision accuracy.

[0102] Through the above technical solution, this application solves the problem of obstacle avoidance decision bias caused by insufficient fusion of dynamic obstacle state information. By constructing a state matrix containing multi-dimensional features such as position, speed, and category, the path planning network can accurately assess the motion risks of different types of obstacles. For example, fast-moving humanoid obstacles and slow-moving cargo stacks have different treatment bases in the obstacle avoidance strategy, thereby improving the accuracy of obstacle avoidance decisions and the real-time response in dynamic environments.

[0103] In one possible implementation, reference Figure 5 , the step S400 includes steps S410 to S420, wherein:

[0104] Step S410 , inputting the current mission target position and the dynamic obstacle state matrix into a pre-trained global path planning network, and outputting a candidate path node sequence;

[0105] Step S420 , performing smoothing optimization processing on the candidate path node sequence to generate global path data including Bezier curve parameters.

[0106] In this embodiment, the global path planning network refers to a neural network trained by reinforcement learning, which can be implemented by a deep Q-network (DQN) or a proximal policy optimization (PPO) algorithm, and is used to generate multiple path candidates based on the dynamic obstacle state and task objectives. Bezier curve parameters refer to parameterized expressions formed by fitting path nodes using cubic Bezier curves, which can be implemented by a control point interpolation algorithm to eliminate angle mutations at path turning points. The dynamic obstacle state matrix refers to a multidimensional data set containing obstacle positions, speeds, and categories, which can be implemented by a raster encoding method to characterize the obstacle distribution and movement trends in the current environment. The candidate path node sequence refers to a discrete path point set generated by the global path planning network, which can be generated by a graph search algorithm combined with a neural network prediction to provide a selectable initial path solution.

[0107] Specifically, in this embodiment, after receiving the task target position and the dynamic obstacle state matrix, the global path planning network extracts spatial features through a multi-layer convolutional neural network and combines it with a fully connected layer to predict path nodes. The candidate path node sequence output by the network is filtered through non-maximum suppression, and candidate paths that meet the safety distance constraint are retained. Subsequently, the Bezier curve parameterization method is used to convert the discrete path nodes into a continuous and differentiable curved path, and a smooth transition of the path curvature is achieved by adjusting the coordinates of the control points. This process ensures that there are no sudden changes in speed or acceleration at the connection points of the path through the second-order continuity characteristics of the cubic Bezier curve.

[0108] Compared to existing technologies, traditional global path planning methods typically use the A* or RRT algorithms to generate broken-line paths, resulting in sharp angles at path transitions, which can easily cause vibration in the mechanical transmission system. While improved solutions based on the dynamic windowing algorithm (DWA) can generate smooth paths, they cannot effectively integrate dynamic obstacle states for multi-path optimization. This solution, through the synergy of neural networks and Bezier curves, can adapt to dynamic obstacle environments to generate reasonable path nodes and eliminate path irregularities through parameterized curves.

[0109] Through the above technical solution, this application can generate a smooth global path that conforms to kinematic constraints, effectively balancing path safety and mechanical losses in dynamic industrial environments. The neural network generation mechanism of the path node sequence can respond to changes in obstacle motion in real time, and Bezier curve parameterization ensures a continuous and directional path, avoiding the mechanical impact caused by traditional broken line paths and extending the service life of the transmission system.

[0110] In one possible implementation, reference Figure 6 , the step S410 includes steps S411 to S414, wherein:

[0111] Step S411, obtaining multiple candidate paths;

[0112] Step S412, calculating the actual length of each candidate path, the number of turns, and the minimum safe distance between the current candidate path and obstacles;

[0113] Step S413: Substituting the actual length of the candidate path, the number of turns, and the minimum safe distance between the current candidate path and the obstacle into a preset reward formula to calculate the reward value for each path;

[0114] Step S414 , selecting a preset number of candidate paths from high to low reward values as a candidate node sequence.

[0115] In this embodiment, the actual length refers to the physical extension distance of the candidate path, which can be specifically achieved by accumulating the Euclidean distance of the path node coordinate sequence, and is used to quantify the spatial efficiency of the path. The number of turns refers to the number of direction changes in the path, which can be specifically achieved by using the angle threshold of adjacent path segment vectors to reflect the energy loss and time cost caused by the turning operation. The minimum safe distance refers to the shortest interval distance between the candidate path and the dynamic obstacle, which can be specifically achieved by predicting the obstacle motion trajectory and calculating the path geometry relationship, and is used to evaluate the path safety risk. The preset reward formula refers to a multi-objective evaluation function that integrates path efficiency, turning penalty and safety priority factors. Specifically, the weight coefficient can be adjusted to adapt to the needs of different scenarios to achieve a dynamic balance of multi-dimensional indicators.

[0116] In this embodiment, specifically, a plurality of candidate paths are first generated by a path generation algorithm, for example, a differentiated path set is generated based on the A* algorithm or the RRT algorithm. For each path, the actual measured length, the number of direction changes, and the minimum distance data from obstacles are calculated respectively. Furthermore, the path efficiency weight coefficient, the steering penalty weight coefficient, and the safety priority weight coefficient are substituted into the preset linear reward function to perform a normalized score on each path. For example, the path efficiency weight coefficient can be set to 0.4, the steering penalty coefficient to 0.3, and the safety priority coefficient to 0.3, forming an evaluation system that takes into account multi-objective optimization. Finally, the candidate paths are sorted according to the scoring results, and the top N high-scoring paths are selected as candidate node sequences, for example, the top 3 paths are selected for subsequent path optimization processing.

[0117] Compared to existing technologies, traditional methods typically screen routes based solely on path length or a single safety threshold, such as simply selecting the shortest path or enforcing a fixed safety distance. This solution establishes a multi-dimensional evaluation system that simultaneously considers efficiency, energy consumption, and dynamic safety factors during the path planning phase. Specifically, it dynamically adapts to different operational scenarios through adjustable weighting coefficients. For example, during peak cargo handling periods, the weighting of path efficiency can be increased, while in crowded areas, the priority factor for safety can be prioritized, thereby achieving adaptive optimization of the planning strategy.

[0118] Through the above technical solution, this application effectively solves the technical problem of a single path screening standard in a dynamic obstacle environment. By introducing multi-dimensional quantitative evaluation indicators, it is possible to avoid path planning defects caused by a single standard, such as ignoring the risk of collision with dynamic obstacles due to excessive pursuit of the shortest path. At the same time, the reward mechanism based on the dynamic combination of weight coefficients can automatically adjust the evaluation focus according to the real-time environmental characteristics, maintaining the rationality and reliability of the path planning results under complex working conditions.

[0119] In a feasible implementation manner, the preset reward formula is:

[0120] ;

[0121] in, is the reward value; is the path efficiency weight coefficient; is the theoretical shortest path length from the starting point to the end point; is the actual length of the candidate path; is the steering penalty weight coefficient; is the number of turns in the candidate path; is the maximum number of turns allowed; is the safety priority weight coefficient; is the minimum safe distance between the candidate path and the obstacle; The preset safety distance threshold.

[0122] In this embodiment, the path efficiency weight coefficient refers to a parameter used to adjust the impact of the deviation of the path length from the optimal solution on the reward value. Specifically, it can be implemented using an empirical value or a value dynamically adjusted during the reinforcement learning training process to suppress the generation of redundant paths. The steering penalty weight coefficient refers to a parameter used to quantify the impact of the frequency of path curvature changes on the reward value. Specifically, it can be implemented by presetting a fixed value or dynamically calculating in combination with the forklift kinematic model to reduce control instability caused by frequent steering. The safety priority weight coefficient refers to a parameter used to characterize the impact of the spatial distribution characteristics of obstacles on path safety. Specifically, it can be implemented using an algorithm based on dynamic adjustment of obstacle density to enhance the safe distance between the path and dynamic obstacles.

[0123] Specifically, in a dynamic obstacle environment, the actual length of the candidate path is quantitatively evaluated by the ratio of the path efficiency weight coefficient to the theoretical shortest path, thereby screening out candidate solutions that deviate less from the optimal path. The number of turns in the candidate path is dynamically constrained by the ratio of the turning penalty weight coefficient to the preset maximum allowable value to avoid frequent reversing of the forklift actuator due to sudden changes in path curvature. The minimum safe distance between the candidate path and the obstacle is calculated by integrating the ratio of the safety priority weight coefficient to the safety threshold, and the spatial characteristics of the obstacle motion trajectory are mapped to the path evaluation system. The above three evaluation indicators are linearly superimposed to form a comprehensive reward value, and finally the global path node sequence is selected based on the reward value ranking mechanism to achieve multi-objective optimization of path economy, motion smoothness and environmental safety.

[0124] Compared to existing technologies, traditional path planning methods typically only consider a single evaluation dimension, such as optimizing only path length or a safety distance threshold. This solution builds a multi-factor coupled reward function model that incorporates the spatial distribution characteristics of dynamic obstacles and the motion control characteristics of forklifts into the path evaluation system. This enables the path selection process to respond to dynamic environmental changes in real time, effectively overcoming the limitations of static evaluation metrics in complex industrial scenarios.

[0125] Through the above technical solution, this application solves the technical problem of the difficulty in balancing efficiency and safety during path planning in a dynamic obstacle environment. Specifically, it suppresses the generation of redundant paths through the path efficiency weight coefficient to reduce energy consumption; constrains the frequency of path curvature changes through the steering penalty coefficient to reduce actuator wear; and enhances obstacle avoidance capabilities through the safety priority coefficient to avoid collisions with dynamic obstacles.

[0126] In one possible implementation, reference Figure 7 , step S500 includes steps S510 to S550, wherein:

[0127] Step S510: extracting obstacles with speeds greater than a preset speed from the dynamic obstacle state matrix as key obstacles;

[0128] Step S520, calculating the influence weight of each key obstacle on the current decision through the attention mechanism;

[0129] Step S530, adjusting the hidden state of the local obstacle avoidance decision network according to the influence weight to generate an action instruction;

[0130] Step S540, calculating the environmental potential energy correction trajectory based on the path node sequence using the artificial potential field method;

[0131] Step S550 : filtering high-risk instructions through collision risk assessment to generate a final driving control instruction.

[0132] In this embodiment, the preset speed refers to an obstacle screening threshold set based on the dynamic characteristics of the scene. This can be implemented using historical motion data analysis or empirical value setting. Its purpose is to distinguish high-speed obstacles from low-speed interference targets, reducing computational complexity. The attention mechanism refers to a method for weighting the impact of different obstacle characteristics on decision-making through a neural network. This can be implemented using a multi-head self-attention module and is used to establish an association assessment between dynamic obstacles and path nodes. The hidden state refers to the feature vector that transmits temporal information within the decision network. This can be implemented using the state of a long-short-term memory network unit. Weight adjustment can be used to integrate obstacle motion trends with path tracking requirements. The artificial potential field method refers to a path optimization method that maps the repulsive force of an obstacle and the attractive force of a target point into a potential field. This can be implemented using an exponential decay function to calculate the potential gradient and correct for trajectory deviation. Collision risk assessment refers to calculating the risk level based on the spatiotemporal overlap between the obstacle's motion trajectory and the control instructions of the unmanned forklift. This can be implemented using a Monte Carlo simulation or a safety margin detection algorithm.

[0133] Specifically, in this embodiment, during real-time dynamic obstacle avoidance, a set of key obstacles with a high collision risk is first screened based on a preset speed threshold. For example, obstacles exceeding 1 meter per second are included in the decision-making process. Subsequently, an attention mechanism is used to calculate the positional correlation between each key obstacle and the current path node. For example, cosine similarity is used to calculate the angle between the obstacle's motion direction and the path tangent vector, generating an influence weight coefficient matrix. The decision network's hidden state dynamically adjusts the information fusion ratio based on the weight coefficients. For example, the velocity vector of a high-weight obstacle is weightedly concatenated with the path deviation vector to generate preliminary steering instructions that take into account interaction intent. An artificial potential field is constructed based on the global path node sequence. For example, the gravitational field coefficient is set to 0.8 in the path tracking direction and the obstacle repulsion field coefficient is set to 1.2. The acceleration value of the preliminary instructions is corrected using the potential energy gradient calculation. Finally, the corrected instructions are safety verified by a collision risk assessment module. For example, the trajectory within the next three seconds is predicted to intersect with the obstacle trajectory, and instructions that pose a collision risk are truncated.

[0134] Compared to existing technologies, traditional methods typically use fixed priority rules to handle obstacle avoidance, which cannot quantitatively assess the dynamic threat level of different obstacles and lacks the ability to predict obstacle movement trends. This solution uses an attention mechanism to construct a dynamic weight allocation model, which can adaptively adjust the focus of the decision network. It also combines potential field correction and risk assessment to form a multi-layered safety verification mechanism, effectively solving the problem of local optimal traps during trajectory correction.

[0135] Through the above technical solution, this application realizes the quantitative evaluation of the interaction intention of dynamic obstacles and multi-level optimization of control instructions, significantly reducing the collision risk of sudden obstacles while ensuring path tracking accuracy, solving the response delay and trajectory oscillation problems caused by the single decision logic of traditional methods, and improving the obstacle avoidance success rate and movement smoothness of unmanned forklifts in complex dynamic environments.

[0136] In one possible implementation, reference Figure 8 The method further includes steps S610 to S630, wherein:

[0137] Step S610 , when an unidentified obstacle type is detected, storing an event data packet including a motion pattern of the obstacle;

[0138] Step S620, constructing a motion prediction model of the obstacle through a contrast learning algorithm;

[0139] Step S630 : Integrate the motion prediction model into the local obstacle avoidance decision network to update the weight parameters of the local obstacle avoidance decision network.

[0140] In this embodiment, the event data packet refers to a structured data set containing the obstacle's motion trajectory, acceleration changes, and spatial position information. It can be implemented using a time series database to record the dynamic behavior characteristics of unknown obstacles and provide data support for subsequent model training. The contrastive learning algorithm refers to a machine learning method that learns the intrinsic characteristics of data by constructing positive and negative sample pairs. It can be implemented using a neural network based on a triplet loss function to extract the motion patterns of unknown obstacles from historical motion data. The motion prediction model refers to a mathematical model that describes the future motion trend of an obstacle. It can be implemented using a gated recurrent unit network to predict the motion trajectory of an unknown obstacle. Integration into the local obstacle avoidance decision network refers to fusing the newly constructed model parameters with the original network. It can be implemented using the fine-tuning method in transfer learning to enhance the decision network's response capability to unknown obstacles.

[0141] In this embodiment, when the multimodal sensor detects an obstacle not identified by the classifier, its motion trajectory, velocity change, and other data are encapsulated into a timestamped event data packet and stored in a buffer. A contrastive learning algorithm compares the motion pattern in the event data packet with the typical behavior of known obstacles in feature space, generating a vector representation that reflects the unique motion patterns of the obstacle. Based on this vector representation, a motion prediction model for the unknown obstacle is trained. Subsequently, the parameters of this new model are updated to the local obstacle avoidance decision network via gradient backpropagation, enabling the network to adjust control instructions based on the newly learned motion patterns during subsequent decision-making.

[0142] In some specific embodiments, the storage period of event data packets can be dynamically adjusted based on memory capacity. For example, when storage space is insufficient, data with a time interval exceeding a set threshold can be preferentially deleted. Contrastive learning algorithms can utilize a feature comparison module based on an attention mechanism, generating a contrastive loss by calculating the similarity differences between motion trajectories. The integration of motion prediction models can utilize knowledge distillation techniques, using the output of the new model as soft labels to guide parameter updates in the existing network.

[0143] Compared to existing technologies, traditional methods rely on a predefined library of obstacle types. When encountering unknown obstacles, their motion prediction modules fail due to a lack of training data. This solution, through online data collection and comparative learning, can autonomously construct motion models for new obstacles during operation. It also dynamically evolves the decision network through parameter updates, significantly improving the system's adaptability to sudden, unknown obstacles.

[0144] Through the above technical solution, this application achieves online learning and prediction of the motion patterns of unknown obstacle types, resolving the path planning failure problem caused by the inability of traditional obstacle avoidance systems to recognize new obstacle types. By updating the decision network parameters in real time, unmanned forklifts can still generate safe obstacle avoidance instructions when faced with unregistered equipment or moving objects that suddenly appear in the warehouse environment, avoiding collision risks and maintaining operational continuity.

[0145] In this embodiment, the reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles proposed in this application obtains original perception data through multimodal sensors; the original perception data includes lidar point cloud data, image data, UWB positioning data and IMU data; the original perception data is then preprocessed to generate corresponding preprocessed data; and the preprocessed data is subjected to spatiotemporal alignment and feature fusion to generate a dynamic obstacle state matrix containing obstacle position, speed and category; the dynamic obstacle state matrix is then input into a pre-trained global path planning network to generate global path data containing a path node sequence; finally, the dynamic obstacle state matrix and the global path data are input into a pre-trained local obstacle avoidance decision network to generate drive control instructions containing steering angle and acceleration to schedule the unmanned forklift. In this way, a dynamic obstacle state matrix is generated by fusing multimodal sensor data, and global path planning and local obstacle avoidance decisions are performed in combination with a reinforcement learning network. This solves the problems of obstacle avoidance response delay and unreasonable path planning in dynamic environments, improves the obstacle avoidance response speed, path planning rationality and multimodal data fusion accuracy in dynamic environments, and thus improves the safety and efficiency of unmanned forklift scheduling.

[0146] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the reinforcement learning unmanned forklift obstacle avoidance and scheduling method for dynamic obstacles in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0147] This application also provides a reinforcement learning unmanned forklift obstacle avoidance and dispatching system 10 for dynamic obstacles, Figure 9 , the system comprising:

[0148] The data acquisition module 100 is used to acquire raw sensory data through multimodal sensors; this raw sensory data includes lidar point cloud data, image data, UWB positioning data, and IMU data. In this embodiment, the data acquisition module 100 acquires raw sensory data through multimodal sensors, specifically using a combination of lidar, cameras, UWB positioning modules, and inertial measurement units. This module is used to capture three-dimensional point clouds, image information, and high-precision position and motion state data in the environment, addressing the comprehensiveness of dynamic obstacle detection.

[0149] Preprocessing module 200, connected to data acquisition module 100, is used to preprocess the raw sensory data and generate corresponding preprocessed data. In this embodiment, preprocessing module 200 performs differential processing on the raw sensory data. Specifically, it can be implemented using voxel filtering, semantic segmentation algorithms, and Kalman filtering to eliminate noise, extract obstacle category labels, and fuse positioning data to provide high signal-to-noise ratio input for subsequent modules.

[0150] The dynamic obstacle matrix generation module 300, connected to the preprocessing module 200, is configured to perform spatiotemporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix containing obstacle positions, velocities, and categories. In this embodiment, the dynamic obstacle matrix generation module 300 generates a structured matrix through spatiotemporal alignment and feature fusion. Specifically, this is achieved by using multi-source data coordinate system conversion and feature splicing to unify obstacle position, velocity, and category information to form a state representation of the dynamic environment.

[0151] Global path data generation module 400, connected to dynamic obstacle matrix generation module 300, is configured to input the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data comprising a path node sequence. In this embodiment, the global path planning network generates the path node sequence based on a pre-trained reinforcement learning model. Specifically, this can be implemented using an offline-trained policy network combined with Bezier curve optimization to balance path efficiency and obstacle avoidance requirements.

[0152] The drive control instruction generation module 500, connected to the global path data generation module 400, is configured to input the dynamic obstacle state matrix and global path data into a pre-trained local obstacle avoidance decision network to generate drive control instructions including steering angle and acceleration for dispatching the unmanned forklift. In this embodiment, the local obstacle avoidance decision network combines an attention mechanism with an artificial potential field method to generate control instructions. Specifically, this can be implemented using an online reinforcement learning framework combined with a collision risk assessment module to adjust the steering angle and acceleration in real time to ensure the feasibility of dynamic obstacle avoidance.

[0153] Specifically, the data acquisition module 100 acquires lidar point cloud, image, positioning, and inertial data through a multimodal sensor combination, covering three-dimensional spatial perception and dynamic motion capture. The preprocessing module 200 performs filtering and denoising, semantic segmentation, and positioning fusion for different data types, generating a dynamic grid map, obstacle category labels, and high-precision pose data. The dynamic obstacle matrix generation module 300 aligns this data in time and space, using coordinate mapping and feature concatenation to form a matrix containing obstacle positions, velocities, and categories, which serves as input for subsequent path planning and decision-making. The global path data generation module 400 generates a sequence of candidate path nodes based on a reinforcement learning model. It calculates rewards based on path length, number of turns, and safety distance, selects the optimal global path, and performs smoothing. The drive control command generation module 500 then receives the global path and real-time obstacle state matrix through a local obstacle avoidance decision network. It uses an attention mechanism to screen critical obstacles, incorporates an artificial potential field method to correct the trajectory, and filters high-risk commands, ultimately outputting steering angle and acceleration control signals.

[0154] Compared with existing technologies, traditional methods rely on preset rules or static path planning, making them difficult to adapt to real-time changes in dynamic obstacles. This solution achieves active perception and intention prediction of dynamic obstacles through the collaboration of multimodal sensor fusion and reinforcement learning networks. The global path planning network combines offline training with online optimization to effectively resolve the conflict between path efficiency and safety. The local obstacle avoidance decision network, through a layered control architecture, inherits global path guidance while adjusting control instructions in real time, overcoming the problems of response delay and path conflicts in dynamic environments.

[0155] Through the above technical solution, the present application can perceive the motion state of dynamic obstacles in real time and predict their trajectory, generate a global path that takes into account both efficiency and safety, and at the same time adjust control instructions in real time through a hierarchical decision-making mechanism to avoid collision risks caused by rigid path planning or response delays, thereby improving the operation smoothness and safety of unmanned forklifts in dynamic industrial environments.

[0156] The reinforcement learning unmanned forklift obstacle avoidance and scheduling system 10 for dynamic obstacles provided in this application utilizes the reinforcement learning unmanned forklift obstacle avoidance and scheduling method for dynamic obstacles in the above-mentioned embodiments, thereby improving the safety and efficiency of unmanned forklift scheduling. Compared with the prior art, the reinforcement learning unmanned forklift obstacle avoidance and scheduling system 10 for dynamic obstacles provided in this application has the same beneficial effects as the reinforcement learning unmanned forklift obstacle avoidance and scheduling method for dynamic obstacles provided in the above-mentioned embodiments. The other technical features of the reinforcement learning unmanned forklift obstacle avoidance and scheduling system 10 for dynamic obstacles are the same as those disclosed in the above-mentioned embodiments and are not further described here.

[0157] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles, characterized by: The method includes: Acquire raw perception data through a multimodal sensor; the raw perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data; Preprocessing the original perception data to generate corresponding preprocessed data; Performing spatiotemporal alignment and feature fusion on the pre-processed data to generate a dynamic obstacle state matrix including obstacle position, velocity, and category; Inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence; Inputting the dynamic obstacle state matrix and global path data into a pre-trained local obstacle avoidance decision network to generate drive control instructions including steering angle and acceleration to dispatch the unmanned forklift; The step of preprocessing the original perception data to generate corresponding preprocessed data includes: Performing voxel filtering on the laser radar point cloud data to remove noise to generate filtered point cloud data, and performing dynamic obstacle feature extraction on the filtered point cloud data to generate dynamic grid map data including obstacle positions and velocities; Performing semantic segmentation on the image data to generate obstacle category label data and depth mask data; Perform Kalman filtering on the UWB positioning data in combination with the IMU data to generate corresponding posture data; The step of extracting dynamic obstacle features from the filtered point cloud data to generate dynamic grid map data containing obstacle positions and speeds includes: Obstacle clustering is performed on the filtered point cloud data based on the Euclidean clustering algorithm to generate cluster bounding box data containing the three-dimensional coordinates of the obstacles; The displacement of each cluster bounding box is calculated by matching the continuous frame point cloud, and the moving speed is calculated according to the preset time interval to generate the obstacle motion vector data containing the position and speed; Mapping the obstacle motion vector data to a grid map coordinate system to generate dynamic grid map data containing the obstacle position and speed; The step of performing spatiotemporal alignment and feature fusion on the pre-processed data to generate a dynamic obstacle state matrix including obstacle position, speed, and category includes: Fusing the dynamic raster map data with the depth mask data to generate semantic raster data with velocity vectors; Encode the obstacle category label data into a one-hot vector to generate obstacle type feature data; The semantic grid data is spliced with obstacle type feature data to form a dynamic obstacle state matrix.

2. The reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles according to claim 1, characterized in that: The step of inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence includes: Input the current task target position and dynamic obstacle state matrix into the pre-trained global path planning network and output a sequence of candidate path nodes; The candidate path node sequence is smoothly optimized to generate global path data containing Bezier curve parameters.

3. The reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles according to claim 2, characterized in that: The step of inputting the current task target position and the dynamic obstacle state matrix into the pre-trained global path planning network and outputting a candidate path node sequence includes: Obtain multiple candidate paths; Calculate the actual length, number of turns, and minimum safe distance between the current candidate path and obstacles for each candidate path; Substitute the actual length of the candidate path, the number of turns, and the minimum safe distance between the current candidate path and the obstacle into the preset reward formula to calculate the reward value of each path; A preset number of candidate paths are selected from high to low reward values as the candidate node sequence.

4. The reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles according to claim 3, characterized in that: The preset reward formula is: ； in, is the reward value; is the path efficiency weight coefficient; is the theoretical shortest path length from the starting point to the end point; is the actual length of the candidate path; is the steering penalty weight coefficient; is the number of turns in the candidate path; is the maximum number of turns allowed; is the safety priority weight coefficient; is the minimum safe distance between the candidate path and the obstacle; The preset safety distance threshold.

5. The reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles according to claim 1, characterized in that: The step of inputting the dynamic obstacle state matrix and global path data into a local obstacle avoidance decision network and outputting a driving control instruction including a steering angle and acceleration in real time to dispatch the unmanned forklift includes: Obstacles with speeds greater than a preset speed in the dynamic obstacle state matrix are extracted as key obstacles; Calculate the influence weight of each key obstacle on the current decision through the attention mechanism; Adjusting the hidden state of the local obstacle avoidance decision network according to the influence weight to generate an action instruction that takes into account the intention of the dynamic obstacle; Based on the path node sequence, the artificial potential field method is used to calculate the current environmental potential and perform trajectory correction on the preliminary action instructions; Collision risk assessment is used to filter high-risk commands and generate final drive control commands.

6. The reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles according to claim 1, characterized in that: The method further comprises: When an unrecognized obstacle type is detected, storing an event data packet including a motion pattern of the obstacle; Constructing the motion prediction model of the obstacle through contrastive learning algorithm; The motion prediction model is integrated into the local obstacle avoidance decision network to update the weight parameters of the local obstacle avoidance decision network.

7. A reinforcement learning unmanned forklift obstacle avoidance and dispatching system for dynamic obstacles, characterized by: The system comprises: A data acquisition module is used to acquire raw perception data through a multimodal sensor; the raw perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data; The preprocessing module is used to preprocess the original sensory data to generate corresponding preprocessed data, including: Performing voxel filtering on the laser radar point cloud data to remove noise to generate filtered point cloud data, and performing dynamic obstacle feature extraction on the filtered point cloud data to generate dynamic grid map data including obstacle positions and velocities; Performing semantic segmentation on the image data to generate obstacle category label data and depth mask data; Perform Kalman filtering on the UWB positioning data in combination with the IMU data to generate corresponding posture data; The step of extracting dynamic obstacle features from the filtered point cloud data to generate dynamic grid map data containing obstacle positions and speeds includes: Obstacle clustering is performed on the filtered point cloud data based on the Euclidean clustering algorithm to generate cluster bounding box data containing the three-dimensional coordinates of the obstacles; The displacement of each cluster bounding box is calculated by matching the continuous frame point cloud, and the moving speed is calculated according to the preset time interval to generate the obstacle motion vector data containing the position and speed; Mapping the obstacle motion vector data to a grid map coordinate system to generate dynamic grid map data containing the obstacle position and speed; The dynamic obstacle matrix generation module is used to perform spatiotemporal alignment and feature fusion on the pre-processed data to generate a dynamic obstacle state matrix containing obstacle positions, speeds, and categories, including: Fusing the dynamic raster map data with the depth mask data to generate semantic raster data with velocity vectors; Encode the obstacle category label data into a one-hot vector to generate obstacle type feature data; splicing the semantic raster data with obstacle type feature data to form a dynamic obstacle state matrix; A global path data generation module, configured to input the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence; The drive control instruction generation module is used to input the dynamic obstacle state matrix and global path data into the pre-trained local obstacle avoidance decision network to generate a drive control instruction including a steering angle and acceleration to dispatch the unmanned forklift.

Images