Dynamic obstacle-oriented reinforcement learning unmanned forklift obstacle avoidance scheduling method and system

Abstract

The invention discloses a reinforcement learning unmanned forklift truck obstacle avoidance scheduling method and system for a dynamic obstacle, relates to the technical field of unmanned driving, and discloses the reinforcement learning unmanned forklift truck obstacle avoidance scheduling method for the dynamic obstacle. According to the method, the system and the system, the system and the system, global path planning and local obstacle avoidance decision making are carried out in combination with the reinforcement learning network, the problems of obstacle avoidance response delay and unreasonable path planning in a dynamic environment are solved, the obstacle avoidance response speed, the path planning rationality and the multi-modal data fusion precision in the dynamic environment are improved, and then the safety and efficiency of unmanned forklift dispatching are improved.

Description

Technical Field

[0001] This application relates to the field of driverless technology, and particularly to a reinforcement learning-based obstacle avoidance and scheduling method and system for driverless forklifts facing dynamic obstacles. Background Art

[0002] Obstacle avoidance and scheduling is the core technology for driverless forklifts to achieve safe and efficient operation in a dynamic industrial environment. In the prior art, methods based on rules or static path planning are mainly adopted. These methods often have problems such as untimely response and unreasonable path planning when facing dynamic obstacles, which limit the operation efficiency and safety of driverless forklifts in complex environments. Specifically, rule-based methods usually rely on preset obstacle recognition rules and obstacle avoidance strategies, and it is difficult to adapt to complex and changeable dynamic environments. Static path planning methods, on the other hand, focus on finding the optimal path in a static environment and lack the ability to perceive and respond to dynamic obstacles in real time. Therefore, in an industrial environment where dynamic obstacles frequently appear, these methods often cannot ensure the safe and efficient operation of driverless forklifts. In addition, in the prior art, when dealing with multi-modal sensor data, there is often a lack of an effective fusion mechanism, resulting in insufficient perception accuracy of dynamic obstacles, which further affects the effect of obstacle avoidance and scheduling.

[0003] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a reinforcement learning-based obstacle avoidance and scheduling method and system for driverless forklifts facing dynamic obstacles, aiming to improve the safety and efficiency of driverless forklift scheduling.

[0005] To achieve the above purpose, this application proposes a reinforcement learning-based obstacle avoidance and scheduling method for driverless forklifts facing dynamic obstacles. The method includes:

[0006] Obtaining original perception data through multi-modal sensors; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data;

[0007] Preprocessing the original perception data to generate corresponding preprocessed data;

[0008] Performing spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including the position, speed, and category of obstacles;

[0009] Inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a sequence of path nodes;

[0010] Input the dynamic obstacle state matrix and the global path data into the pre-trained local obstacle avoidance decision network to generate drive control instructions including steering angle and acceleration for scheduling the automated forklift.

[0011] In one embodiment, the step of preprocessing the original perception data to generate corresponding preprocessed data includes:

[0012] Perform voxel filtering on the lidar point cloud data to remove noise points to generate filtered point cloud data, and perform dynamic obstacle feature extraction on the filtered point cloud data to generate dynamic grid map data including obstacle positions and speeds;

[0013] Perform semantic segmentation on the image data to generate obstacle category label data and depth mask data;

[0014] Perform Kalman filtering on the UWB positioning data combined with the IMU data to generate corresponding pose data.

[0015] In one embodiment, the step of performing dynamic obstacle feature extraction on the filtered point cloud data to generate dynamic grid map data including obstacle positions and speeds includes:

[0016] Perform obstacle clustering on the filtered point cloud data based on the Euclidean clustering algorithm to generate clustering bounding box data including the three-dimensional coordinates of the obstacles;

[0017] Calculate the displacement of each clustering bounding box through continuous frame point cloud matching and calculate the moving speed according to a preset time interval to generate obstacle motion vector data including positions and speeds;

[0018] Map the obstacle motion vector data to the grid map coordinate system to generate dynamic grid map data including obstacle positions and speeds.

[0019] In one embodiment, the step of performing spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including obstacle positions, speeds, and categories includes:

[0020] Fuse the dynamic grid map data and the depth mask data to generate semantic grid data with velocity vectors;

[0021] Encode the obstacle category label data into one-hot vectors to generate obstacle type feature data;

[0022] Stitch the semantic grid data and the obstacle type feature data to form a dynamic obstacle state matrix.

[0023] In one embodiment, the step of inputting the dynamic obstacle state matrix into the pre-trained global path planning network to generate global path data including a sequence of path nodes includes:

[0024] Input the current task target position and the dynamic obstacle state matrix into the pre-trained global path planning network, and output a candidate sequence of path nodes;

[0025] Perform a smoothing optimization process on the candidate sequence of path nodes to generate global path data including Bessel curve parameters.

[0026] In one embodiment, the step of inputting the current task target position and the dynamic obstacle state matrix into the pre-trained global path planning network to output a candidate sequence of path nodes includes:

[0027] Obtain multiple candidate paths;

[0028] Calculate the actual length, number of turns, and minimum safety distance between the current candidate path and obstacles for each candidate path;

[0029] Substitute the actual length, number of turns, and minimum safety distance between the candidate path and obstacles into a preset reward formula to calculate the reward value for each path;

[0030] Select a preset number of candidate paths with the highest to lowest reward values as the candidate node sequence.

[0031] In one embodiment,

[0032] The preset reward formula is:

[0033] ;

[0034] Wherein, is the reward value; is the path efficiency weight coefficient; is the theoretical shortest path length from the starting point to the ending point; is the actual length of the candidate path; is the steering penalty weight coefficient; is the number of turns of the candidate path; is the maximum allowed number of turns; is the safety priority weight coefficient; is the minimum safety distance between the candidate path and obstacles; is the preset safety distance threshold.

[0035] In one embodiment, the step of inputting the dynamic obstacle state matrix and the global path data into the local obstacle avoidance decision network to real-time output drive control instructions including steering angle and acceleration to schedule the automated forklift includes:

[0036] Extract the obstacles with speeds greater than the preset speed in the dynamic obstacle state matrix as key obstacles;

[0037] Calculate the influence weights of each key obstacle on the current decision through the attention mechanism;

[0038] Adjust the hidden state of the local obstacle avoidance decision network according to the influence weights to generate an action instruction considering the intentions of dynamic obstacles;

[0039] Based on the path node sequence, calculate the potential energy of the current environment using the artificial potential field method to correct the trajectory of the preliminary action instruction;

[0040] Filter high-risk instructions through collision risk assessment to generate the final drive control instruction.

[0041] In one embodiment, the method further includes:

[0042] When an unrecognized obstacle type is detected, store an event data packet containing the movement pattern of the obstacle;

[0043] Construct a motion prediction model for the obstacle through a contrastive learning algorithm;

[0044] Integrate the motion prediction model into the local obstacle avoidance decision network to update the weight parameters of the local obstacle avoidance decision network.

[0045] In addition, to achieve the above object, the present application also proposes a reinforcement learning unmanned forklift obstacle avoidance scheduling system for dynamic obstacles, and the system includes:

[0046] A data acquisition module for obtaining original perception data through multi-modal sensors; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data;

[0047] A preprocessing module for preprocessing the original perception data to generate corresponding preprocessed data;

[0048] A dynamic obstacle matrix generation module for performing spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including obstacle positions, speeds, and categories;

[0049] A global path data generation module for inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence;

[0050] A drive control instruction generation module for inputting the dynamic obstacle state matrix and the global path data into a pre-trained local obstacle avoidance decision network to generate a drive control instruction including a steering angle and an acceleration to schedule the unmanned forklift.

[0051] The reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles proposed in this application obtains original perception data through multi-modal sensors; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data; then preprocesses the original perception data to generate corresponding preprocessed data; and performs spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including the position, speed, and category of the obstacle; then inputs the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a sequence of path nodes; finally, inputs the dynamic obstacle state matrix and the global path data into a pre-trained local obstacle avoidance decision network to generate drive control instructions including steering angle and acceleration to schedule the unmanned forklift. In this way, by fusing multi-modal sensor data to generate a dynamic obstacle state matrix, and combining reinforcement learning networks for global path planning and local obstacle avoidance decision-making, the problems of obstacle avoidance response delay and unreasonable path planning in a dynamic environment are solved, the obstacle avoidance response speed, path planning rationality, and multi-modal data fusion accuracy in a dynamic environment are improved, and thus the safety and efficiency of unmanned forklift scheduling are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0054] Figure 1 It is a schematic flowchart of an embodiment of a reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles provided by this application;

[0055] Figure 2 For this application Figure 1 It is a detailed schematic flowchart of step S200;

[0056] Figure 3 For this application Figure 2 It is a detailed schematic flowchart of step S210;

[0057] Figure 4 For this application Figure 1 It is a detailed schematic flowchart of step S300;

[0058] Figure 5 For this applicationFigure 1 Schematic diagram of the detailed process of step S400 in

[0059] Figure 6 This application Figure 5 Schematic diagram of the detailed process of step S410 in

[0060] Figure 7 This application Figure 1 Schematic diagram of the detailed process of step S500 in

[0061] Figure 8 Schematic diagram of the process provided by another embodiment of a reinforcement learning-based obstacle avoidance scheduling method for an unmanned forklift facing dynamic obstacles in this application

[0062] Figure 9 Schematic diagram of the structure provided by an embodiment of a reinforcement learning-based obstacle avoidance scheduling system for an unmanned forklift facing dynamic obstacles in this application

[0063] Explanation of the reference numerals in the drawings: 10. Reinforcement learning-based obstacle avoidance scheduling system for an unmanned forklift facing dynamic obstacles; 100. Data acquisition module; 200. Preprocessing module; 300. Dynamic obstacle matrix generation module; 400. Global path data generation module; 500. Driving control instruction generation module.

[0064] The implementation, functional features, and advantages of this application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Detailed implementation manners

[0065] Next, the technical solutions in this application will be clearly and completely described in conjunction with the drawings in this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. The components of this application usually described and illustrated in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of this application required to be protected, but only represents the selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative efforts fall within the scope of protection of this application.

[0066] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0067] In the existing technology, the obstacle avoidance and scheduling technology for driverless forklifts has long faced the problems of perception and decision-making in dynamic environments. Traditional rule-based obstacle avoidance methods rely on fixed thresholds to judge the threat level of obstacles. When encountering a pallet truck or a walking worker with a changing moving speed, due to the inability to predict their movement trajectories, sudden stops or circuitous paths often occur. Although static path planning algorithms can generate an initial optimal path, in a scenario where forklifts and AGVs frequently cross-operate in a warehousing environment, fixed paths cannot adapt to the real-time changing obstacle distribution, and path blockages or repeated planning are likely to occur.

[0068] To solve the above problems, first, consider how to break through the limitations of single-sensor perception. Through experiments, it is found that the point cloud is sparse for lidar in rainy and foggy weather, and vision sensors are prone to failure in low-illumination environments. Therefore, a multi-modal sensor complementary scheme is proposed. Secondly, for the problem of predicting the trajectories of dynamic obstacles, the correlation between historical motion data and the current velocity vector is observed, and then a spatio-temporal alignment mechanism is designed to capture the motion trend. Finally, to solve the coordination problem between global planning and local obstacle avoidance, a hierarchical decision-making model is constructed through a reinforcement learning framework. The upper network is responsible for generating macroscopic paths, and the lower network focuses on real-time obstacle avoidance, realizing the organic connection of decision-making levels.

[0069] Therefore, this application proposes a reinforcement learning-based obstacle avoidance and scheduling method for driverless forklifts facing dynamic obstacles. Referring to Figure 1 , the method includes steps S100 to S500, where:

[0070] Step S100, obtaining original perception data through multi-modal sensors; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data;

[0071] Step S200, preprocessing the original perception data to generate corresponding preprocessed data;

[0072] Step S300, performing spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including the positions, speeds, and categories of obstacles;

[0073] Step S400, inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a sequence of path nodes;

[0074] Step S500, inputting the dynamic obstacle state matrix and the global path data into a pre-trained local obstacle avoidance decision network to generate drive control instructions including steering angles and accelerations for scheduling the driverless forklift.

[0075] In this embodiment, the multi-modal sensor refers to a device combination integrating lidar, camera, UWB module and inertial measurement unit. Specifically, a spatial complementary layout of a 16-line lidar and an RGB-D camera can be adopted to realize the synchronous acquisition of three-dimensional space perception and two-dimensional visual information. Spatiotemporal alignment refers to the coordinate transformation and timestamp calibration of multi-source data. Specifically, the unified sensor coordinate system can be realized through the external parameter calibration matrix, and the data acquisition delay can be eliminated by hardware synchronous triggering. The dynamic obstacle state matrix refers to the structured data containing the spatial coordinates, motion vectors and semantic categories of obstacles. Specifically, it can be stored in the form of a three-dimensional tensor, and each grid cell records the velocity vector and category encoding of the obstacle. The global path planning network refers to a path generation model based on deep reinforcement learning. Specifically, the double-delayed deep deterministic policy gradient algorithm can be adopted, and the reward function is used to guide the network to learn a planning strategy that takes into account both the path length and the safety distance. The local obstacle avoidance decision network refers to a real-time motion control model. Specifically, the proximal policy optimization algorithm can be adopted, and the smooth control instruction is generated by combining the current speed constraint and the path tracking deviation.

[0076] In this embodiment, specifically, after the lidar point cloud is filtered by voxels, the displacement vector of the obstacle is calculated through continuous frame matching to form a dynamic grid map. After the visual data is semantically segmented, the category labels such as pallets and pedestrians are extracted and fused with the point cloud velocity information into a motion vector with semantic annotation. After the UWB and IMU data are filtered by Kalman filter, centimeter-level positioning accuracy is provided. After these preprocessed data are spatiotemporally aligned, a three-dimensional state matrix is formed through feature stitching, where the velocity vector helps predict the motion trend of the obstacle, and the category information distinguishes the avoidance priority. After receiving the state matrix, the global path planning network outputs the path nodes parameterized by the Bezier curve, and eliminates the sudden turn through path smoothing. The local obstacle avoidance decision network combines the real-time obstacle state and the global path nodes, adopts the attention mechanism to identify high-risk obstacles, and dynamically adjusts the steering angle and acceleration to achieve progressive avoidance during the path tracking process.

[0077] Compared with the prior art, the multi-modal sensor fusion scheme of this embodiment overcomes the perception blind area of a single sensor under complex working conditions. For example, in the area blocked by the shelf, the UWB signal is used to supplement the positioning information. The dynamic obstacle state matrix breaks through the limitation of the traditional two-dimensional grid map in representing the motion trend, and realizes the motion trajectory prediction through the velocity vector field. The hierarchical reinforcement learning architecture decouples the global path optimization and local obstacle avoidance, avoiding the policy oscillation problem of a single decision-making model in long-term planning. The spatiotemporal alignment mechanism ensures that the multi-source data has a consistent spatial reference during decision-making, and eliminates the decision error caused by the difference in the sensor sampling frequency.

[0078] Through the above technical solutions, the present application realizes real-time trajectory prediction and hierarchical avoidance of dynamic obstacles, effectively reducing the frequency of emergency braking in the warehousing and logistics scenarios. The multi-source data fusion improves the environmental perception robustness under complex lighting conditions. For example, in the area with strong light irradiation, the lidar point cloud compensates for the overexposed area of the vision sensor. The hierarchical decision-making mechanism takes into account both the global optimality of the path and the real-time obstacle avoidance, ensuring a safe distance from moving obstacles while maintaining the transportation efficiency. The construction of the dynamic obstacle state matrix enables the forklift to predict the walking path of the worker in advance and decelerate to give way instead of making an emergency avoidance. That is, the reinforcement learning-based obstacle avoidance scheduling method and system for unmanned forklifts facing dynamic obstacles described in this embodiment can improve the safety and efficiency of unmanned forklift scheduling.

[0079] In a feasible implementation manner, referring to Figure 2 , step S200 includes steps S210 to S230, where:

[0080] Step S210, performing voxel filtering on the lidar point cloud data to remove noise points, generating filtered point cloud data, and extracting dynamic obstacle features from the filtered point cloud data to generate dynamic grid map data including the positions and speeds of obstacles;

[0081] Step S220, performing semantic segmentation on the image data to generate obstacle category label data and depth mask data;

[0082] Step S230 combines the UWB positioning data with the IMU data for Kalman filtering to generate corresponding pose data.

[0083] In this embodiment, voxel filtering refers to dividing the three-dimensional point cloud into cube units of equal volume for downsampling processing, which can be specifically implemented by using the voxel grid downsampling algorithm. By merging adjacent point clouds, outlier noise points are eliminated while the contour features of obstacles are retained. Semantic segmentation refers to performing semantic category annotation on image pixels, which can be specifically implemented by using a semantic segmentation model based on a convolutional neural network. By classifying each pixel, the obstacle type is identified and the mask area is generated. Kalman filtering refers to performing optimal estimation on sensor data through a state space model, which can be specifically implemented by using a linear Kalman filter. By fusing the absolute positioning data of UWB and the relative motion data of IMU, the measurement noise is suppressed.

[0084] In this embodiment, specifically, the lidar point cloud forms a downsampled point cloud after voxel filtering, effectively eliminating the interference of environmental noise on obstacle detection. Then, the three-dimensional coordinates of obstacles are extracted through a clustering algorithm, and the motion speed is calculated by combining the displacement analysis of consecutive frames to generate a dynamic grid map. The image data obtains obstacle category labels after semantic segmentation, and a three-dimensional space mask is generated in combination with depth information to make up for the deficiencies of lidar in object recognition. The UWB and IMU data are fused in time series through Kalman filtering. The high-frequency characteristics of the IMU are used to compensate for the UWB signal delay, and at the same time, the absolute position of the UWB is used to correct the cumulative error of the IMU, and finally, high-precision pose data is output. The three preprocessing processes are carried out synchronously to form complementary obstacle perception information.

[0085] Compared with the prior art, traditional methods usually directly perform clustering analysis on lidar point clouds, which are prone to false detections in a noisy environment; using a single sensor for positioning is vulnerable to signal interference; image processing is only used for two-dimensional detection and lacks three-dimensional coordinate association. This solution reduces the noise sensitivity of the point cloud through voxel filtering, combines multi-frame motion analysis to improve the dynamic obstacle detection accuracy; realizes three-dimensional space obstacle classification through the fusion of semantic segmentation and depth information; improves the positioning robustness through multi-sensor time series fusion, providing a more reliable perception basis for subsequent decision-making.

[0086] Through the above technical solutions, this application effectively eliminates the noise interference in the original sensor data, accurately extracts the spatial position, motion speed and category information of dynamic obstacles, and improves the integrity and accuracy of environmental perception. The lidar point cloud preprocessing avoids false detections and missed detections caused by noise, the image semantic segmentation enhances the ability to identify obstacle types, and the multi-source positioning data fusion ensures the stability of pose estimation, providing high-quality input data for path planning and obstacle avoidance decision-making, and reducing the safety risks caused by perception errors.

[0087] In a feasible implementation, referring to Figure 3 , step S210 includes steps S211 to S213, where:

[0088] Step S211, based on the Euclidean clustering algorithm, performs obstacle clustering on the filtered point cloud data to generate clustering bounding box data containing the three-dimensional coordinates of obstacles;

[0089] Step S212, calculates the displacement of each clustering bounding box through consecutive frame point cloud matching, and calculates the moving speed according to a preset time interval to generate obstacle motion vector data containing position and speed;

[0090] Step S213, maps the obstacle motion vector data to the grid map coordinate system to generate dynamic grid map data containing the position and speed of obstacles.

[0091] In this embodiment, the Euclidean clustering algorithm refers to a point cloud segmentation algorithm based on a spatial distance threshold. Specifically, it can be implemented using an Euclidean clusterer constructed based on a kd-tree in the point cloud library. By setting parameters such as the minimum number of cluster points and the maximum cluster radius, the discrete point cloud is segmented into independent obstacle objects. The continuous frame point cloud matching refers to registering adjacent temporal point clouds through the iterative closest point algorithm or the feature matching algorithm. Specifically, a feature descriptor based on normal vector feature extraction can be used to associate corresponding points between frames, thereby calculating the displacement change of the obstacle in three-dimensional space. The preset time interval can be the sensor acquisition period, such as a fixed time difference within the range of 0.1 second to 0.5 seconds, which is used to convert the displacement amount into an instantaneous velocity vector. The dynamic grid map coordinate system refers to a two-dimensional grid coordinate system with the current position of the automated forklift as the origin. Specifically, a grid resolution between 0.1 meter and 0.5 meters can be used for grid division, and the three-dimensional motion vector is projected onto the two-dimensional plane to achieve spatial mapping.

[0092] In this embodiment, specifically, the solution first performs spatial segmentation on the filtered point cloud data through the Euclidean clustering algorithm, eliminates noise interference and distinguishes different obstacle individuals, forming independent bounding boxes with three-dimensional coordinates. Then, the displacement amount is calculated using the corresponding obstacle bounding boxes in two consecutive frames of point cloud data, and the displacement is converted into an instantaneous velocity vector through the time interval parameter to establish a dynamic description of the obstacle movement trajectory. Finally, the motion vector in three-dimensional space is projected onto the two-dimensional grid map to form the real-time position and velocity distribution data of the obstacle in each grid cell. This process realizes the decoupling of obstacle spatial positioning and motion state analysis through staged processing, reducing the computational complexity while ensuring the accuracy of dynamic information extraction.

[0093] Compared with the prior art, traditional methods usually directly calculate the obstacle speed using single-frame point cloud data, resulting in a lag in motion state estimation. Some solutions use fixed thresholds to judge the moving state of obstacles and cannot adapt to dynamic environments with different motion patterns. This solution can accurately capture the instantaneous motion characteristics of obstacles through the combination of continuous frame matching and time parameters. In the prior art, the mapping of obstacle information mostly uses fixed coordinate system conversion, while this solution dynamically projects the motion vector onto a grid map that is updated in real time, solving the timeliness problem of dynamic environment representation.

[0094] Through the above technical solution, this application effectively solves the technical problem of inaccurate extraction of dynamic obstacle motion information, providing accurate real-time position and speed data of obstacles for path planning. The speed calculation method based on continuous frame matching can eliminate the error accumulation of single-frame data estimation and improve the reliability of dynamic obstacle trajectory prediction. The dynamic update mechanism of the grid map enables the obstacle state information to reflect environmental changes in real time, avoiding obstacle avoidance decision-making errors caused by information lag.

[0095] In a feasible implementation manner, refer to Figure 4 , step S300 includes steps S310 to S330, where:

[0096] Step S310: Fuse the dynamic grid map data and the depth mask data to generate semantic grid data with velocity vectors;

[0097] Step S320: Encode the obstacle category label data into one-hot vectors to generate obstacle type feature data;

[0098] Step S330: Concatenate the semantic grid data and the type feature data to form a dynamic obstacle state matrix.

[0099] In this embodiment, the dynamic grid map data refers to a two-dimensional rasterized data structure containing obstacle positions and velocities, which can be specifically generated by a lidar point cloud clustering and inter-frame displacement matching algorithm and is used to characterize the movement trend of obstacles in space. The depth mask data refers to a pixel-level obstacle area identifier generated through semantic segmentation and depth estimation, which can be specifically implemented by a stereo vision or monocular depth estimation model and is used to enhance the accuracy of obstacle spatial positions. One-hot vector encoding refers to converting discrete category labels into binary vector representations, which can be specifically implemented by a dimension allocation method based on the number of categories and is used to avoid interference of category features on model training. The semantic grid data refers to dynamic grid data fused with depth information, which can be specifically generated by aligning the spatial coordinate system and superimposing channels and is used to unify the movement features and spatial distribution features of obstacles. The concatenation operation refers to combining multi-dimensional features along a specific dimension, which can be specifically implemented by matrix concatenation or channel merging and is used to construct a multi-modal fusion obstacle state representation.

[0100] In this embodiment, specifically, the dynamic grid map data obtains the obstacle velocity vectors through continuous frame point cloud matching, combines the stereo spatial information provided by the depth mask data, and generates semantic grid data with velocity vectors after coordinate system alignment. The obstacle category labels form a type feature matrix that matches the channel dimension of the semantic grid data after one-hot vector encoding. By expanding the semantic grid data in the channel dimension and superimposing the type feature matrix, a dynamic obstacle state matrix containing three-dimensional information of position, velocity, and category is formed. Through multi-modal feature fusion, the subsequent path planning network can simultaneously perceive the movement trend, spatial distribution, and category attributes of obstacles, effectively avoiding the problem of feature loss caused by a single data source.

[0101] Compared with the prior art, traditional methods usually process obstacle motion trajectories and category information independently, resulting in the inability of the decision-making model to associate motion features with semantic attributes. In this solution, through multimodal data fusion, velocity vectors, depth information, and category labels are uniformly encoded into a matrix structure to achieve joint representation of different feature dimensions. For example, in the prior art, obstacle velocity and category data belong to different processing flows and need to be secondarily associated and calculated during the obstacle avoidance decision-making stage. However, in this solution, information integration is completed during the feature fusion stage, reducing computational latency and improving decision-making accuracy.

[0102] Through the above technical solution, this application solves the problem of obstacle avoidance decision-making deviation caused by insufficient fusion of dynamic obstacle state information. By constructing a state matrix containing multi-dimensional features such as position, velocity, and category, the path planning network can accurately evaluate the motion risks of different categories of obstacles. For example, a fast-moving humanoid obstacle and a slow-moving cargo stack have different processing bases in the obstacle avoidance strategy, thereby improving the accuracy of obstacle avoidance decision-making and the real-time response in a dynamic environment.

[0103] In a feasible implementation manner, referring to Figure 5 , step S400 includes steps S410 to S420, where:

[0104] Step S410, input the current task target position and the dynamic obstacle state matrix into a pre-trained global path planning network, and output a sequence of candidate path nodes;

[0105] Step S420, perform a smoothing optimization process on the sequence of candidate path nodes to generate global path data containing Bezier curve parameters.

[0106] In this embodiment, the global path planning network refers to a neural network trained through reinforcement learning, and specifically can be implemented using a deep Q-network (DQN) or proximal policy optimization (PPO) algorithm, which is used to generate multiple path candidates according to the dynamic obstacle state and task target. The Bezier curve parameters refer to the parametric expressions formed by fitting path nodes using cubic Bezier curves, and can be specifically implemented through a control point interpolation algorithm, which is used to eliminate the angular mutation at the path turning points. The dynamic obstacle state matrix refers to a multi-dimensional data set containing obstacle position, velocity, and category, and can be specifically implemented using a rasterized encoding method, which is used to represent the obstacle distribution and motion trend in the current environment. The sequence of candidate path nodes refers to a set of discrete path points generated by the global path planning network, and can be specifically generated by combining a graph search algorithm with neural network prediction, which is used to provide an optional initial path plan.

[0107] In this embodiment, specifically, after receiving the task target position and the dynamic obstacle state matrix, the global path planning network extracts spatial features through a multi-layer convolutional neural network and combines a fully connected layer to predict path nodes. The candidate path node sequence output by the network is screened through non-maximum suppression to retain the candidate paths that satisfy the safety distance constraint. Subsequently, the Bezier curve parameterization method is used to convert the discrete path nodes into a continuous and differentiable curve path, and the path curvature is smoothly transitioned by adjusting the control point coordinates. This process ensures that there are no sudden changes in speed or acceleration at the connection points of the path through the second-order continuity feature of the cubic Bezier curve.

[0108] Compared with the prior art, traditional global path planning methods usually use the A* algorithm or the RRT algorithm to generate a polyline path, resulting in sharp angles at the path turning points, which are likely to cause vibrations in the mechanical transmission system. Although the improved scheme based on the dynamic window approach (DWA) can generate a smooth path, it cannot effectively combine the dynamic obstacle state for multi-path optimization selection. This solution can not only generate reasonable path nodes adaptable to the dynamic obstacle environment through the synergistic effect of the neural network and the Bezier curve, but also eliminate the path non-smoothness phenomenon through the parametric curve.

[0109] Through the above technical solutions, this application can generate a smooth global path that meets the kinematic constraints, effectively balancing path safety and mechanical loss in a dynamic industrial environment. The neural network generation mechanism of the path node sequence can respond to the movement changes of obstacles in real time, and the Bezier curve parameterization process ensures that the path is continuous and differentiable, avoiding mechanical shocks caused by traditional polyline paths and extending the service life of the transmission system.

[0110] In a feasible implementation manner, referring to Figure 6 , step S410 includes steps S411 to S414, where:

[0111] Step S411, obtaining multiple candidate paths;

[0112] Step S412, calculating the actual length, the number of turns of each candidate path, and the minimum safety distance between the current candidate path and the obstacle;

[0113] Step S413, substituting the actual length, the number of turns of the candidate path, and the minimum safety distance between the current candidate path and the obstacle into a preset reward formula to calculate the reward value of each path;

[0114] Step S414, selecting a preset number of candidate paths with the highest to lowest reward values as the candidate node sequence.

[0115] In this embodiment, the actual length refers to the physical extension distance of the candidate path, which can be specifically calculated by accumulating the Euclidean distances of the path node coordinate sequence, and is used to quantify the spatial efficiency of the path. The number of turns refers to the number of direction changes in the path, which can be specifically determined by the threshold of the included angle between adjacent path segment vectors, and reflects the energy loss and time cost brought by the turning operation. The minimum safety distance refers to the shortest interval distance between the candidate path and the dynamic obstacle, which can be specifically calculated by predicting the movement trajectory of the obstacle and the geometric relationship of the path, and is used to evaluate the path safety risk. The preset reward formula refers to a multi-objective evaluation function that comprehensively considers path efficiency, turning penalty, and safety priority factors, and can specifically adapt to different scenario requirements by adjusting the weight coefficients to achieve the dynamic balance of multi-dimensional indicators.

[0116] In this embodiment, specifically, first, multiple candidate paths are generated by a path generation algorithm, such as generating a differentiated path set based on the A* algorithm or the RRT algorithm. For each path, the measured length, the number of direction changes, and the minimum distance from the obstacle are calculated respectively. Further, the path efficiency weight coefficient, the turning penalty weight coefficient, and the safety priority weight coefficient are substituted into the preset linear reward function to normalize the score of each path. For example, the path efficiency weight coefficient can be set to 0.4, the turning penalty coefficient can be set to 0.3, and the safety priority coefficient can be set to 0.3 to form an evaluation system that takes into account multi-objective optimization. Finally, the candidate paths are sorted according to the scoring results, and the top N high-scoring paths are selected as the candidate node sequences. For example, the top 3 paths are selected for subsequent path optimization processing.

[0117] Compared with the prior art, traditional methods usually only screen based on the path length or a single safety threshold. For example, simply select the shortest path or force a fixed safety distance to be maintained. This solution establishes a multi-dimensional evaluation system, synchronously considering efficiency, energy consumption, and dynamic safety factors during the path planning stage, especially dynamically adapting to different operation scenarios through adjustable weight coefficients. For example, the path efficiency weight can be increased during the peak period of goods handling, and the safety priority coefficient can be emphasized in crowded areas, so as to achieve the adaptive optimization of the planning strategy.

[0118] Through the above technical solution, this application effectively solves the technical problem of the single path screening standard in the dynamic obstacle environment. By introducing multi-dimensional quantitative evaluation indicators, it can avoid the path planning defects caused by a single standard. For example, the situation of ignoring the collision risk of dynamic obstacles due to excessive pursuit of the shortest path. At the same time, based on the reward mechanism of dynamic combination of weight coefficients, the evaluation focus can be automatically adjusted according to the real-time environmental characteristics, and the rationality and reliability of the path planning results can be maintained under complex working conditions.

[0119] In a feasible implementation manner, the preset reward formula is:

[0120] ;

[0121] Wherein, is the reward value; is the path efficiency weight coefficient; is the theoretical shortest path length from the starting point to the ending point; is the actual length of the candidate path; is the turning penalty weight coefficient; is the number of turns of the candidate path; is the maximum allowed number of turns; is the safety priority weight coefficient; is the minimum safety distance between the candidate path and the obstacle; is the preset safety distance threshold.

[0122] In this embodiment, the path efficiency weight coefficient refers to a parameter used to adjust the influence of the deviation of the path length from the optimal solution on the reward value, which can be specifically implemented by using empirical values or values dynamically adjusted during the reinforcement learning training process, and is used to suppress the generation of redundant paths. The turning penalty weight coefficient refers to a parameter used to quantify the influence of the path curvature change frequency on the reward value, which can be specifically implemented by presetting fixed values or dynamically calculating in combination with the forklift kinematic model, and is used to reduce the control instability caused by frequent turning. The safety priority weight coefficient refers to a parameter used to characterize the influence of the obstacle spatial distribution characteristics on the path safety, which can be specifically implemented by using an algorithm dynamically adjusted based on the obstacle density, and is used to strengthen the safety distance between the path and the dynamic obstacle.

[0123] Specifically, in a dynamic obstacle environment, the actual length of the candidate path is quantitatively evaluated by the ratio of the path efficiency weight coefficient to the theoretical shortest path, so as to screen out candidate solutions with a small deviation from the optimal path. The number of turns of the candidate path is dynamically constrained by the ratio of the turning penalty weight coefficient to the preset maximum allowable value, avoiding frequent commutation of the forklift actuator due to sudden changes in path curvature. The minimum safety distance between the candidate path and the obstacle is calculated by fusing the ratio of the safety priority weight coefficient to the safety threshold, mapping the spatial characteristics of the obstacle motion trajectory to the path evaluation system. The above three evaluation indicators form a comprehensive reward value through linear superposition, and finally select the global path node sequence based on the reward value sorting mechanism to achieve multi-objective optimization of path economy, motion smoothness and environmental safety.

[0124] Compared with the prior art, traditional path planning methods usually only consider a single evaluation dimension, such as only optimizing the path length or the safety distance threshold. This solution constructs a reward function model with multi-factor coupling, integrates the spatial distribution characteristics of dynamic obstacles and the forklift motion control characteristics into the path evaluation system, enables the path selection process to respond to environmental dynamic changes in real time, and effectively overcomes the limitations of static evaluation indicators in complex industrial scenarios.

[0125] Through the above technical solution, this application solves the technical problem that it is difficult to balance efficiency and safety in path planning in a dynamic obstacle environment, which is specifically manifested as follows: redundant path generation is suppressed by the path efficiency weight coefficient, reducing energy consumption; the frequency of path curvature change is restricted by the steering penalty coefficient, reducing the wear of the actuator; the obstacle avoidance ability is strengthened by the safety priority coefficient, avoiding collisions with dynamic obstacles.

[0126] In a feasible implementation, referring to Figure 7 , step S500 includes steps S510 to S550, where:

[0127] Step S510, extract the obstacles with a speed greater than the preset speed in the dynamic obstacle state matrix as key obstacles;

[0128] Step S520, calculate the influence weight of each key obstacle on the current decision through the attention mechanism;

[0129] Step S530, adjust the hidden state of the local obstacle avoidance decision network according to the influence weight to generate an action instruction;

[0130] Step S540, based on the path node sequence, use the artificial potential field method to calculate the environmental potential energy to correct the trajectory;

[0131] Step S550, filter high-risk instructions through collision risk assessment to generate the final drive control instruction.

[0132] In this embodiment, the preset speed refers to the obstacle screening threshold set based on the dynamic characteristics of the scene, which can be specifically implemented by analyzing historical motion data or setting empirical values. Its function is to distinguish high-speed moving obstacles from low-speed interference targets and reduce the computational complexity. The attention mechanism refers to the method of weight distribution for calculating the influence of different obstacle features on the decision through a neural network, which can be specifically implemented by a multi-head self-attention module and is used to establish the correlation degree evaluation between dynamic obstacles and path nodes. The hidden state refers to the feature vector that transmits temporal information inside the decision network, which can be specifically implemented by the state of a long short-term memory network unit. Through weight adjustment, the motion trend of the obstacle and the path tracking requirement can be fused. The artificial potential field method refers to the path optimization method that maps the repulsive force of the obstacle and the attractive force of the target point into a potential energy field, and the potential energy gradient can be specifically calculated by an exponential decay function to correct the trajectory deviation. The collision risk assessment refers to calculating the risk level based on the spatio-temporal overlap degree between the obstacle motion trajectory and the unmanned forklift control instruction, which can be specifically implemented by Monte Carlo simulation or a safety boundary detection algorithm.

[0133] In this embodiment, specifically, during the real-time obstacle avoidance of dynamic obstacles, first, a set of key obstacles with high collision risks is screened based on a preset speed threshold. For example, obstacles with a speed exceeding 1 meter per second are included in the decision-making scope. Subsequently, the position correlation degree between each key obstacle and the current path node is calculated through an attention mechanism. For example, the cosine similarity is used to calculate the angle between the movement direction of the obstacle and the path tangent vector to generate an influence weight coefficient matrix. The hidden state of the decision-making network dynamically adjusts the information fusion ratio according to the weight coefficients. For example, the speed vector of the high-weight obstacle and the path deviation vector are weighted and spliced to generate a preliminary steering instruction considering the interaction intention. When constructing an artificial potential field based on the global path node sequence, for example, the gravitational field coefficient is set to 0.8 and the obstacle repulsion field coefficient is set to 1.2 in the path tracking direction, and the acceleration value of the preliminary instruction is calculated and corrected through the potential energy gradient. Finally, the collision risk assessment module conducts a safety verification on the corrected instruction. For example, it predicts whether there is an intersection between the movement trajectory in the next 3 seconds and the obstacle trajectory, and truncates the instruction with a collision risk.

[0134] Compared with the prior art, traditional methods usually adopt fixed priority rules to handle obstacle avoidance, unable to quantitatively evaluate the dynamic threat levels of different obstacles, and lacking the ability to predict the movement trends of obstacles. This solution constructs a dynamic weight allocation model through an attention mechanism, which can adaptively adjust the focus of the decision-making network. At the same time, it combines potential field correction and risk assessment to form a multi-layer safety verification mechanism, effectively solving the local optimal trap problem in the trajectory correction process.

[0135] Through the above technical solutions, this application realizes the quantitative evaluation of the interaction intention of dynamic obstacles and the multi-level optimization of control instructions, significantly reducing the collision risk of sudden obstacles on the premise of ensuring path tracking accuracy, solving the response delay and trajectory oscillation problems caused by the single decision-making logic of traditional methods, and improving the obstacle avoidance success rate and motion smoothness of unmanned forklifts in complex dynamic environments.

[0136] In a feasible implementation manner, referring to Figure 8 , the method further includes steps S610 to S630, where:

[0137] Step S610, when an unrecognized obstacle type is detected, store an event data packet containing the movement pattern of the obstacle;

[0138] Step S620, construct a motion prediction model of the obstacle through a contrastive learning algorithm;

[0139] Step S630, integrate the motion prediction model into the local obstacle avoidance decision-making network to update the weight parameters of the local obstacle avoidance decision-making network.

[0140] In this embodiment, the event data packet refers to a structured data set containing the motion trajectory, acceleration change, and spatial position information of the obstacle. Specifically, it can be implemented using a time-series database, which is used to record the dynamic behavior characteristics of unknown obstacles and provide data support for subsequent model training. The contrastive learning algorithm refers to a machine learning method that learns the internal characteristics of data by constructing positive and negative sample pairs. Specifically, it can be implemented using a neural network based on the triplet loss function, which is used to extract the motion laws of unknown obstacles from historical motion data. The motion prediction model refers to a mathematical model that describes the future motion trend of the obstacle. Specifically, it can be implemented using a gated recurrent unit network, which is used to predict the motion trajectory of unknown obstacles. Integrating it into the local obstacle avoidance decision network means fusing the newly constructed model parameters with the original network. Specifically, it can be implemented using the fine-tuning method in transfer learning, which is used to enhance the response ability of the decision network to unknown obstacles.

[0141] In this embodiment, when the multi-modal sensor detects an obstacle that cannot be recognized by the classifier, data such as its motion trajectory and speed change are encapsulated into an event data packet with a timestamp and stored in the buffer. Through the contrastive learning algorithm, the motion pattern in the event data packet is compared with the typical behaviors of known obstacles in the feature space to generate a vector representation reflecting the unique motion law of the obstacle. Based on this vector representation, a motion prediction model for the unknown obstacle is trained. Subsequently, the parameters of the new model are updated to the local obstacle avoidance decision network through the gradient backpropagation method, enabling it to adjust the control instructions by combining the newly learned motion laws in subsequent decision-making processes.

[0142] In some specific embodiments, the storage period of the event data packet can be dynamically adjusted according to the memory capacity. For example, when the storage space is insufficient, data with a time interval exceeding the set threshold is preferentially deleted. The contrastive learning algorithm can adopt a feature contrast module based on the attention mechanism to generate a contrastive loss by calculating the similarity difference between motion trajectories. The integration of the motion prediction model can adopt the knowledge distillation technique, using the output of the new model as a soft label to guide the parameter update of the original network.

[0143] Compared with the prior art, traditional methods rely on a predefined obstacle type library. When encountering an obstacle of an unknown type, their motion prediction module fails due to the lack of training data. This solution can autonomously construct a motion model for new obstacles during operation through online data collection and contrastive learning mechanisms, and achieve the dynamic evolution of the decision network through parameter updates, significantly improving the system's adaptability to sudden unknown obstacles.

[0144] Through the above technical solutions, the present application realizes the online learning and prediction of the motion patterns of unknown types of obstacles, and solves the problem of path planning failure in traditional obstacle avoidance systems caused by the inability to identify new obstacle types. By updating the decision network parameters in real time, when the unmanned forklift encounters unregistered equipment or moving objects that suddenly appear in the warehouse environment, it can still generate safe obstacle avoidance instructions, avoid collision risks and maintain operation continuity.

[0145] In this embodiment, the reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles proposed by the present application obtains original perception data through multi-modal sensors; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data; then preprocesses the original perception data to generate corresponding preprocessed data; and performs spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including the position, speed, and category of the obstacle; then inputs the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a sequence of path nodes; finally, inputs the dynamic obstacle state matrix and the global path data into a pre-trained local obstacle avoidance decision network to generate drive control instructions including steering angle and acceleration to schedule the unmanned forklift. In this way, by fusing multi-modal sensor data to generate a dynamic obstacle state matrix, and combining a reinforcement learning network for global path planning and local obstacle avoidance decision-making, the problems of obstacle avoidance response delay and unreasonable path planning in a dynamic environment are solved, the obstacle avoidance response speed, path planning rationality, and multi-modal data fusion accuracy in a dynamic environment are improved, and thus the safety and efficiency of unmanned forklift scheduling are improved.

[0146] It should be noted that the above examples are only for understanding the present application and do not constitute a limitation on the reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles of the present application. Any simple transformation in more forms based on this technical concept is within the protection scope of the present application.

[0147] The present application also provides a reinforcement learning unmanned forklift obstacle avoidance scheduling system 10 for dynamic obstacles. Refer to Figure 9 , the system includes:

[0148] A data acquisition module 100, configured to obtain original perception data through multi-modal sensors; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data. In this embodiment, the data acquisition module 100 obtains original perception data through multi-modal sensors, and can specifically be implemented by combining a lidar, a camera, a UWB positioning module, and an inertial measurement unit, and is used to capture three-dimensional point clouds, image information, high-precision position, and motion state data in the environment to solve the comprehensiveness problem of dynamic obstacle detection.

[0149] The preprocessing module 200, connected to the data acquisition module 100, is used to preprocess the original perception data to generate corresponding preprocessed data. In this embodiment, the preprocessing module 200 performs differential processing on the original perception data, which can be specifically implemented by voxel filtering, semantic segmentation algorithms, and Kalman filtering, to eliminate noise, extract obstacle category labels, and fuse positioning data, providing a high signal-to-noise ratio input for subsequent modules.

[0150] The dynamic obstacle matrix generation module 300, connected to the preprocessing module 200, is used to perform spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix containing obstacle positions, speeds, and categories. In this embodiment, the dynamic obstacle matrix generation module 300 generates a structured matrix through spatio-temporal alignment and feature fusion, which can be specifically implemented by multi-source data coordinate system conversion and feature stitching, to unify obstacle position, speed, and category information and form a state representation of the dynamic environment.

[0151] The global path data generation module 400, connected to the dynamic obstacle matrix generation module 300, is used to input the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data containing a sequence of path nodes. In this embodiment, the global path planning network generates a sequence of path nodes based on a pre-trained reinforcement learning model, which can be specifically implemented by combining an offline-trained policy network with Bezier curve optimization, to balance path efficiency and obstacle avoidance requirements.

[0152] The drive control instruction generation module 500, connected to the global path data generation module 400, is used to input the dynamic obstacle state matrix and the global path data into a pre-trained local obstacle avoidance decision network to generate drive control instructions containing steering angles and accelerations to schedule the automated forklift. In this embodiment, the local obstacle avoidance decision network combines the attention mechanism with the artificial potential field method to generate control instructions, which can be specifically implemented by combining an online reinforcement learning framework with a collision risk assessment module, to adjust the steering angle and acceleration in real time to ensure the feasibility of dynamic obstacle avoidance.

[0153] Specifically, the data acquisition module 100 obtains lidar point clouds, images, positioning, and inertial data through a multi-modal sensor combination, covering three-dimensional space perception and dynamic motion capture. The preprocessing module 200 performs filtering and denoising, semantic segmentation, and positioning fusion on different data types respectively, generating a dynamic grid map, obstacle category labels, and high-precision pose data. The dynamic obstacle matrix generation module 300 aligns the above data in space and time, and forms a matrix containing obstacle positions, velocities, and categories through coordinate mapping and feature splicing, serving as the input for subsequent path planning and decision-making. The global path data generation module 400 generates a candidate path node sequence based on a reinforcement learning model, calculates rewards by combining path length, number of turns, and safety distance, selects the optimal global path, and performs smoothing processing. Then, the drive control instruction generation module 500 receives the global path and the real-time obstacle state matrix through a local obstacle avoidance decision network, selects key obstacles through an attention mechanism, corrects the trajectory by combining the artificial potential field method, and filters high-risk instructions, finally outputting steering angle and acceleration control signals.

[0154] Compared with the prior art, traditional methods rely on preset rules or static path planning and are difficult to adapt to the real-time changes of dynamic obstacles. This solution realizes the active perception and intention prediction of dynamic obstacles through the collaboration of multi-modal sensor fusion and reinforcement learning networks. The global path planning network combines offline training and online optimization, effectively solving the contradiction between path efficiency and safety. The local obstacle avoidance decision network overcomes the problems of response delay and path conflict in a dynamic environment through a hierarchical control architecture, while inheriting the guidance of the global path and adjusting control instructions in real time.

[0155] Through the above technical solution, this application can perceive the motion state of dynamic obstacles in real time, predict their trajectories, generate a global path that takes into account both efficiency and safety, and at the same time adjust control instructions in real time through a hierarchical decision-making mechanism, avoiding the collision risk caused by rigid path planning or response delay, and improving the operation fluency and safety of the automated forklift in a dynamic industrial environment.

[0156] The reinforcement learning automated forklift obstacle avoidance scheduling system 10 for dynamic obstacles provided by this application adopts the reinforcement learning automated forklift obstacle avoidance scheduling method in the above embodiment, which can improve the safety and efficiency of automated forklift scheduling. Compared with the prior art, the beneficial effects of the reinforcement learning automated forklift obstacle avoidance scheduling system 10 for dynamic obstacles provided by this application are the same as those of the reinforcement learning automated forklift obstacle avoidance scheduling method for dynamic obstacles provided by the above embodiment, and other technical features in the reinforcement learning automated forklift obstacle avoidance scheduling system 10 for dynamic obstacles are the same as the features disclosed in the above embodiment method, and will not be elaborated here.

[0157] The above are only some embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A reinforcement learning-based obstacle avoidance scheduling method for unmanned forklifts facing dynamic obstacles, characterized in that, The described method includes: Obtaining original perception data through a multi-modal sensor; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data; Preprocessing the original perception data to generate corresponding preprocessed data; Performing spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including obstacle position, speed, and category; Inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a sequence of path nodes; Inputting the dynamic obstacle state matrix and the global path data into a pre-trained local obstacle avoidance decision network to generate drive control instructions including steering angle and acceleration for scheduling an automated forklift.

2. The obstacle avoidance scheduling method for an enhanced learning unmanned forklift facing dynamic obstacles according to claim 1, characterized in that, The step of preprocessing the original perception data to generate corresponding preprocessed data includes: Performing voxel filtering on the lidar point cloud data to remove noise to generate filtered point cloud data, and performing dynamic obstacle feature extraction on the filtered point cloud data to generate dynamic grid map data including obstacle position and speed; Performing semantic segmentation on the image data to generate obstacle category label data and depth mask data; Performing Kalman filtering on the UWB positioning data combined with the IMU data to generate corresponding pose data.

3. The obstacle avoidance scheduling method for an enhanced learning unmanned forklift facing dynamic obstacles according to claim 2, wherein The step of performing dynamic obstacle feature extraction on the filtered point cloud data to generate dynamic grid map data including obstacle position and speed includes: Performing obstacle clustering on the filtered point cloud data based on the Euclidean clustering algorithm to generate clustering bounding box data including the three-dimensional coordinates of obstacles; Calculating the displacement of each clustering bounding box through continuous frame point cloud matching and calculating the moving speed according to a preset time interval to generate obstacle motion vector data including position and speed; Mapping the obstacle motion vector data to the grid map coordinate system to generate dynamic grid map data including obstacle position and speed.

4. The obstacle avoidance scheduling method for a reinforcement learning unmanned forklift facing dynamic obstacles according to claim 3, characterized in that, The step of performing spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including obstacle position, speed, and category includes: Fusing the dynamic grid map data with the depth mask data to generate semantic grid data with velocity vectors; Encoding the obstacle category label data into one-hot vectors to generate obstacle type feature data; Concatenating the semantic grid data with the obstacle type feature data to form a dynamic obstacle state matrix.

5. The obstacle avoidance scheduling method for a reinforcement learning unmanned forklift facing dynamic obstacles according to claim 1, wherein, The step of inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a sequence of path nodes includes: Inputting the current task target position and the dynamic obstacle state matrix into a pre-trained global path planning network to output a candidate sequence of path nodes; Performing smoothing optimization processing on the candidate sequence of path nodes to generate global path data including B-spline curve parameters.

6. The obstacle avoidance scheduling method for an RL unmanned forklift facing dynamic obstacles according to claim 5, wherein, The step of inputting the current task target position and the dynamic obstacle state matrix into a pre-trained global path planning network to output a candidate sequence of path nodes includes: Obtaining multiple candidate paths; Calculate the actual length, number of turns, and minimum safety distance between the current candidate path and obstacles for each candidate path; Substitute the actual length, number of turns, and minimum safety distance between the current candidate path and obstacles of the candidate path into a preset reward formula to calculate the reward value of each path; Select a preset number of candidate paths with the highest to lowest reward values as the candidate node sequence.

7. The reinforcement learning unmanned forklift obstacle avoidance scheduling method for dynamic obstacles according to claim 6, wherein The preset reward formula is: ； Among them, is the reward value; is the path efficiency weight coefficient; is the theoretical shortest path length from the starting point to the ending point; is the actual length of the candidate path; is the turning penalty weight coefficient; is the number of turns of the candidate path; is the maximum allowed number of turns; is the safety priority weight coefficient; is the minimum safety distance between the candidate path and the obstacle; is the preset safety distance threshold.

8. The obstacle avoidance scheduling method for a reinforcement learning unmanned forklift facing dynamic obstacles according to claim 1, wherein, The step of inputting the dynamic obstacle state matrix and global path data into the local obstacle avoidance decision network to output in real time a driving control instruction including a steering angle and an acceleration to schedule the unmanned forklift includes: Extract the obstacles with a speed greater than a preset speed in the dynamic obstacle state matrix as key obstacles; Calculate the influence weight of each key obstacle on the current decision through an attention mechanism; Adjust the hidden state of the local obstacle avoidance decision network according to the influence weight to generate an action instruction considering the intention of the dynamic obstacle; Based on the path node sequence, use the artificial potential field method to calculate the current environmental potential energy and correct the trajectory of the preliminary action instruction; Filter high-risk instructions through collision risk assessment to generate the final driving control instruction.

9. The obstacle avoidance scheduling method for an enhanced learning unmanned forklift facing dynamic obstacles according to claim 1, characterized in that The method further includes: When an unrecognized obstacle type is detected, store an event data packet including the movement pattern of the obstacle; Construct a motion prediction model of the obstacle through a contrast learning algorithm; Integrate the motion prediction model into the local obstacle avoidance decision network to update the weight parameters of the local obstacle avoidance decision network.

10. A reinforcement learning-based obstacle avoidance and scheduling system for unmanned forklifts facing dynamic obstacles, characterized in that, The system includes: A data acquisition module for obtaining original perception data through a multi-modal sensor; the original perception data includes lidar point cloud data, image data, UWB positioning data, and IMU data; A preprocessing module for preprocessing the original perception data to generate corresponding preprocessed data; A dynamic obstacle matrix generation module for performing spatio-temporal alignment and feature fusion on the preprocessed data to generate a dynamic obstacle state matrix including the position, speed, and category of the obstacle; A global path data generation module for inputting the dynamic obstacle state matrix into a pre-trained global path planning network to generate global path data including a path node sequence; A driving control instruction generation module for inputting the dynamic obstacle state matrix and global path data into a pre-trained local obstacle avoidance decision network to generate a driving control instruction including a steering angle and an acceleration to schedule the unmanned forklift.

Images