Method and system for intelligent planning of a loading robot trajectory

Abstract

This invention relates to the field of machine vision technology, specifically to a method and system for intelligent trajectory planning of loading robots. The method first performs multi-source sensor data calibration and fusion on LiDAR data, high-definition camera data, and ultrasonic data from the loading area, and maps and constructs a lightweight, dynamic 3D loading scene. Then, based on scene information, it calculates environmental complexity, adaptively adjusts the scene update cycle, and predicts the movement trajectory and behavioral intent of high-risk obstacles. Next, relying on a grid map with intent markers, it outputs trajectory parameters adapted to the complex environment through a deep reinforcement learning model and generates a preliminary path. Finally, it smooths and adjusts the preliminary trajectory with energy consumption feedback, making the robot's loading movement more continuous and stable, reducing start-stop and jamming losses, and ultimately significantly improving the rationality of path planning and loading efficiency in complex dynamic environments such as changes in cargo position, personnel movement, and equipment movement.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a method and system for intelligent trajectory planning of loading robots. Background Technology

[0002] Loading scenarios typically involve complex conditions such as a wide variety of obstacles, disorderly movement of dynamic obstacles, irregular stacking of goods, narrow working spaces, and real-time dynamic changes in the environment. Autonomous loading by robots faces significant technical challenges, including: blind spots, noise interference, and measurement deviations of single sensors in complex environments; time-series misalignment of multi-source data leading to distorted environmental perception; and difficulty in constructing accurate dynamic 3D scenes. At the same time, the unpredictable behavior of dynamic obstacles and the lack of differentiated adaptation to environmental complexity can easily cause trajectory planning conflicts and motion stuttering, resulting in poor rationality of loading path planning, low work efficiency, and insufficient operational safety and stability.

[0003] Currently, existing robot loading path planning methods lack sufficient environmental perception accuracy, fail to judge the movement of dynamic obstacles, lack foresight in trajectory planning, and have a high risk of collision. At the same time, trajectory planning relies heavily on traditional static algorithms, which are weakly adaptable to complex dynamic environments and suffer from problems such as discontinuous motion, high energy consumption, and low loading efficiency. These methods are difficult to meet the actual needs of safe, efficient, and stable robot loading in complex scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for intelligent trajectory planning of loading robots.

[0005] The technical solution of this invention is as follows: A method for intelligent trajectory planning of a loading robot includes the following operations: S1. Acquire LiDAR data, HD camera data, and ultrasonic data from the loading operation area. After time-series misalignment calibration and measurement deviation calibration, fuse them to obtain multimodal fusion data. Through a coordinate transformation matrix, map the multimodal fusion data onto a three-dimensional grid matrix, retaining only the data within the robot's motion trajectory range to obtain a dynamic three-dimensional loading scene. S2. Obtain the obstacle density, movement speed, and cargo stacking height of the dynamic loading 3D scene, calculate the loading environment complexity, match the corresponding loading environment complexity level, adjust the update cycle of the dynamic loading 3D scene according to the loading environment complexity level, and predict the movement trajectory, speed change trend and corresponding behavioral intention of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future. After marking, obtain a dynamic environment raster map with intent marking. S3. Process the dynamic environment grid map with intent labels, target goods, and target loading position using a deep reinforcement learning model to obtain robot loading trajectory parameters; generate a path based on the robot loading trajectory parameters to obtain a preliminary robot loading trajectory; perform smoothing and energy consumption feedback adjustment on the preliminary robot loading trajectory to obtain the final robot loading trajectory.

[0006] In S2, the method for predicting the movement trajectory is as follows: Based on the grid state in the dynamic loading 3D scene, high-risk dynamic obstacles are screened out; based on the speed change, coordinate change, and distance to the robot's working area boundary of the high-risk dynamic obstacles in the first consecutive numerical frames, temporal attention weight and spatial attention weight are calculated, and a comprehensive attention weight is obtained by weighted summation; based on the comprehensive attention weight, the trajectory coordinates of the high-risk dynamic obstacles in the first time frame of the future are extrapolated and the trajectory is smoothed to obtain the movement trajectory of the high-risk dynamic obstacles in the first time frame of the future.

[0007] In S3, the method for obtaining the robot loading trajectory parameters is as follows: the dynamic environment grid map with intent markers is processed through a feature encoding layer to obtain the initial environment feature tensor; the boundary coordinates of the loading area are converted into a constraint mask matrix and multiplied element-wise with the initial environment feature tensor to obtain the environment feature tensor; the environment feature tensor is input into the Actor network to obtain the initial robot loading trajectory parameters; the initial robot loading trajectory parameters and the environment feature tensor are input into the Critic network for scoring, and the Actor network parameters are updated based on the scoring objective function to iteratively optimize and obtain the robot loading trajectory parameters.

[0008] In the Actor network, the environmental feature tensor is extracted by spatial convolution to obtain spatial convolution features; the spatial convolution features are processed by temporal attention to obtain temporal attention features; the temporal attention features are processed by a lightweight fully connected layer to obtain the initial robot loading trajectory parameters.

[0009] The scoring objective function is constructed by weighted summation of loading efficiency rewards, energy consumption control rewards, obstacle avoidance and safety rewards, and trajectory smoothness rewards. Different loading environment complexity levels in the dynamic environment grid map correspond to different scoring objective functions. When the loading environment complexity level is high, the weight of the obstacle avoidance and safety reward is at its maximum value and greater than the first weight threshold, used to strengthen obstacle avoidance. When the loading environment complexity level is medium, the difference between the weights of the loading efficiency reward, energy consumption control reward, obstacle avoidance and safety reward, and trajectory smoothness reward is less than the difference threshold, used to balance all rewards. When the loading environment complexity level is low, the weights of the loading efficiency reward and energy consumption control reward are both greater than the first weight threshold, used to enhance efficiency and pursue high-efficiency, low-consumption operation; the first weight threshold is 0.3.

[0010] In S3, the smoothing process is as follows: the initial trajectory is divided into multiple continuous segments, each segment containing only the starting node and motion transition nodes whose joint velocity change rate is less than the joint velocity change rate threshold, thus obtaining the trajectory segmentation result; the trajectory segmentation result is weighted based on the node weighting coefficient to obtain the weighting coefficient vector of each segment node; based on the trajectory segmentation result and the node weighting coefficient vector, interpolation calculation is performed to obtain the initial smoothed trajectory.

[0011] In S3, the energy consumption feedback adjustment process is as follows: if the instantaneous energy consumption of the initial smooth trajectory is greater than the allowable unit energy consumption threshold, then speed adjustment is initiated; if the instantaneous energy consumption of the initial smooth trajectory is not greater than the unit energy consumption threshold, then the current speed parameters are maintained.

[0012] A loading robot trajectory intelligent planning system, used to implement the above-mentioned loading robot trajectory intelligent planning method, includes: The dynamic loading 3D scene generation module is used to acquire LiDAR data, high-definition camera data, and ultrasonic data in the loading operation area. After time-series misalignment calibration and measurement deviation calibration, the data is fused to obtain multimodal fusion data. Through a coordinate transformation matrix, the multimodal fusion data is mapped to a 3D grid matrix, retaining only the data within the robot's motion trajectory range to obtain the dynamic loading 3D scene. The dynamic environment grid generation module with intent labeling is used to obtain the obstacle density, movement speed, and cargo stacking height of the dynamic loading 3D scene, calculate the loading environment complexity, match the corresponding loading environment complexity level, adjust the update cycle of the dynamic loading 3D scene according to the loading environment complexity level, and predict the movement trajectory, speed change trend and corresponding behavioral intent of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future. After labeling, the dynamic environment grid with intent labeling is obtained. The robot loading trajectory generation module is used to process the dynamic environment grid map with intent tags, the target cargo, and the target loading position through a deep reinforcement learning model to obtain robot loading trajectory parameters; based on the robot loading trajectory parameters, a path is generated to obtain a preliminary robot loading trajectory; the preliminary robot loading trajectory is smoothed and energy consumption feedback is adjusted to obtain the final robot loading trajectory.

[0013] A vehicle loading robot trajectory intelligent planning device includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-mentioned vehicle loading robot trajectory intelligent planning method.

[0014] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described intelligent trajectory planning method for a loading robot.

[0015] The beneficial effects of this invention are as follows: This invention provides an intelligent trajectory planning method for a loading robot. First, it performs multi-source sensor data calibration and fusion, combining LiDAR data, high-definition camera data, and ultrasonic data from the loading area, and maps this data to construct a lightweight, dynamic 3D loading scene. This allows for accurate perception of the entire environmental information even in complex environments with numerous obstacles. Then, based on the scene information, it calculates the environmental complexity, adaptively adjusts the scene update cycle, and predicts the movement trajectory and behavioral intent of high-risk obstacles. This enables early identification of safety hazards in complex environments, providing reliable pre-judgment basis for path planning. Next, relying on a grid map with intent markers, a deep reinforcement learning model outputs trajectory parameters adapted to the complex environment and generates a preliminary path. This effectively avoids dense obstacles, solving the problems of conflict-prone and difficult-to-adapt path planning in complex scenarios. Finally, it smooths and adjusts the preliminary trajectory with energy consumption feedback, making the robot's loading movement more continuous and stable, reducing start-stop and jamming losses. Ultimately, this significantly improves the rationality of path planning and loading efficiency in complex dynamic environments such as changes in cargo position, personnel movement, and equipment movement. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application are described clearly and completely below. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0017] This embodiment provides a method for intelligent trajectory planning of a loading robot, including the following operations: S1. Acquire LiDAR data, HD camera data, and ultrasonic data from the loading operation area. After time-series misalignment calibration and measurement deviation calibration, fuse them to obtain multimodal fusion data. Through a coordinate transformation matrix, map the multimodal fusion data onto a three-dimensional grid matrix, retaining only the data within the robot's motion trajectory range to obtain a dynamic three-dimensional loading scene. S2. Obtain the obstacle density, movement speed, and cargo stacking height of the dynamic loading 3D scene, calculate the loading environment complexity, match the corresponding loading environment complexity level, adjust the update cycle of the dynamic loading 3D scene according to the loading environment complexity level, and predict the movement trajectory, speed change trend and corresponding behavioral intention of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future. After marking, obtain a dynamic environment raster map with intent marking. S3. Process the dynamic environment grid map with intent labels, target goods, and target loading position using a deep reinforcement learning model to obtain robot loading trajectory parameters; generate a path based on the robot loading trajectory parameters to obtain a preliminary robot loading trajectory; and perform smoothing and energy consumption feedback adjustment on the preliminary robot loading trajectory to obtain the final robot loading trajectory. The specific steps are detailed below.

[0018] S1. Acquire LiDAR data, high-definition camera data, and ultrasonic data from the loading operation area. After time-series misalignment calibration and measurement deviation calibration, fuse them to obtain multimodal fusion data. Through a coordinate transformation matrix, map the multimodal fusion data onto a three-dimensional grid matrix, retaining only the data within the robot's motion trajectory range to obtain a dynamic three-dimensional loading scene.

[0019] First, a 3D LiDAR is deployed at the center of the top of the loading robot for long-range obstacle ranging and spatial positioning. Two industrial high-definition cameras are installed on either side of the robot's robotic arm, facing the loading area, for obstacle category identification and contour extraction. Six ultrasonic sensors are evenly distributed around the robot's chassis, corresponding to the robot's forward, backward, and lateral directions, for near-range blind spot obstacle detection, compensating for the short-range perception limitations of the LiDAR and cameras. All sensors are connected to the robot's main control unit for real-time data transmission.

[0020] Then, using a 10Hz periodic scanning trigger mode, a 360° horizontal and vertical scan is performed on the loading operation area to collect real-time 3D point cloud data of targets within the operation area. The output data includes the spatial 3D coordinates of obstacles, ranging distance values, and point cloud reflection intensity. Each frame of point cloud data is accompanied by a unified reference timestamp, resulting in the original LiDAR data frame. Simultaneously, using a 15Hz frame-by-frame acquisition mode, color images of the loading operation area are acquired in real-time. The contour features, texture features, and preliminary category determination information of targets (goods, vehicles, personnel) in the images are extracted synchronously. Each image frame is bound with a unified timestamp consistent with the LiDAR data, forming the original camera data frame. In addition, using a 20Hz polling trigger acquisition mode, 6 ultrasonic sensors sequentially complete ranging, collecting real-time ranging values, echo intensity, and binary judgment signals for the presence or absence of obstacles at close range. 1 indicates the presence of an obstacle, and 0 indicates the absence of an obstacle. The ranging result of each sensor is marked with the corresponding sampling timestamp, generating the original ultrasonic sensing data. The original LiDAR data frames, original camera data frames, and original ultrasonic sensor data with timestamps are cached separately. At the same time, the validity of each data frame is initially judged, and invalid frames with missing data or abnormal values ​​are removed.

[0021] Next, after time-series misalignment calibration and measurement deviation calibration of the lidar data, high-definition camera data, and ultrasonic data, they are fused. This can be achieved through weighted fusion to obtain multimodal fused data.

[0022] The above-mentioned timing misalignment calibration operation is as follows: using the timestamp of the lidar data as a reference, by calculating the time difference between the high-definition camera data, ultrasonic data and lidar data, the abnormal data that exceeds the time window is linearly interpolated to correct the timing consistency of the three types of sensor data.

[0023] The above-mentioned measurement deviation calibration is performed by combining the attitude deviation benchmark model to calibrate the lidar data, high-definition camera data, and ultrasonic data respectively.

[0024] The method for constructing the attitude deviation benchmark model is as follows: The robot's motion attitude data, including pitch, roll, and yaw angles, is collected in real time using the robot's onboard IMU (Inertial Measurement Unit). The sampling frequency is synchronized with the sensor data. The attitude data collected by the IMU is used as the calibration benchmark to establish a correlation model between sensor measurement deviation and robot attitude changes. The model expression is as follows: , For sensor measurement deviation, , Here, the first attitude deviation correction coefficient and the second attitude deviation correction coefficient are empirical values. To compensate for sensor temperature drift deviation, , These are pitch angle and roll angle, respectively.

[0025] LiDAR data calibration is achieved using the following formula: , , , , , For lidar data calibration values, , , These are the deviation values ​​in the x, y, and z directions of the lidar point cloud data. They are calculated by substituting the pitch and roll angles collected by the IMU into the attitude deviation benchmark model. At the same time, point cloud data that exceeds the loading operation area after correction is removed to avoid interference from invalid data.

[0026] The calibration operation for high-definition camera data is as follows: Based on the robot's posture deviation, the imaging angle parameters of the camera are adjusted to correct the image distortion caused by the robot's pitch and roll (supplementing posture distortion not covered by conventional distortion correction). At the same time, based on the point cloud data after LiDAR calibration, the pixel coordinates and spatial coordinates of the camera image are mapped and calibrated to ensure that the contour position of the target in the image is consistent with the actual spatial position. After calibration, a standardized image frame is output.

[0027] The calibration procedure for ultrasonic data is as follows: The ultrasonic ranging value is corrected by combining temperature drift deviation and sensor measurement deviation. The correction formula is: , This is the raw ultrasound data. This is the distance measurement value after calibration.

[0028] Finally, the multimodal fusion data is mapped onto a 3D grid matrix using a coordinate transformation matrix, clarifying the grid position corresponding to each target. Only data within the robot's motion trajectory is retained, resulting in a dynamic loading 3D scene. The data in the dynamic loading 3D scene includes: a dynamic environment grid map, grid status markers (idle, static obstacles, dynamic obstacles, cargo), 3D spatial coordinates, and time-series timestamps.

[0029] In this embodiment, S1 performs time-series misalignment and measurement deviation calibration and fusion on multi-source sensing data from LiDAR, high-definition cameras, and ultrasonic sensors. This can compensate for the perception blind spots, noise interference, and accuracy limitations of a single sensor, significantly improving the completeness and robustness of the vehicle loading environment perception. After coordinate transformation and mapping to a three-dimensional grid matrix, only valid data within the robot's motion trajectory range is retained. This can eliminate redundant and invalid information, reduce subsequent computing power consumption, and improve the efficiency and relevance of constructing dynamic vehicle loading three-dimensional scenes. This provides a precise and lightweight scene foundation for subsequent obstacle detection, environmental assessment, and trajectory planning.

[0030] S2. Obtain the obstacle density, movement speed, and cargo stacking height of the dynamic loading 3D scene, calculate the loading environment complexity, match the corresponding loading environment complexity level, adjust the update cycle of the dynamic loading 3D scene according to the loading environment complexity level, and predict the movement trajectory, speed change trend, and corresponding behavioral intention of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future, along with the target loading cargo. After marking, obtain a dynamic environment raster map with intent marking.

[0031] First, based on the multimodal fusion data, obstacle density, movement speed, and cargo stacking height are extracted. The movement speed is the average of the instantaneous movement speeds of the dynamic obstacles.

[0032] Then, based on obstacle density, moving speed, and cargo stacking height, the loading environment complexity is calculated using the following formula: , Due to the complexity of the loading environment, For movement speed, For obstacle density, Here, is the density-velocity coupling coefficient, and is an empirical value. In an exponential fashion, the greater the obstacle density and the faster the movement speed, the more exponentially the complexity increases. For the stacking height of goods, This is the cargo height gain coefficient, which is an empirical value. To the maximum allowed stacking height, As a higher-order gain term, the higher the goods are stacked, the narrower the working space becomes, and the complexity increases non-linearly. This is the correction factor for the safety boundary.

[0033] Finally, based on the complexity of the loading environment, a corresponding loading environment complexity level is matched, and the update cycle of the dynamic loading 3D scene is adjusted according to the loading environment complexity level; the greater the loading environment complexity and the higher the loading environment complexity level, the shorter the dynamic scene update cycle.

[0034] Meanwhile, an improved attention mechanism is adopted to predict the movement trajectory, speed change trend and behavioral intention of dynamic obstacles in the first moment of the future. The specific steps are detailed below.

[0035] Step 1: Based on the grid state (static obstacles, dynamic obstacles, idle areas, and loaded goods) in the dynamic loading 3D scene, construct an attention mask matrix. Mask static obstacle grids, idle grids, and grids with confidence scores < confidence thresholds, retaining only the grid data corresponding to dynamic obstacles (personnel, moving vehicles, and swaying goods). Combine the distance between dynamic obstacles and the robot's working area to filter out dynamic obstacles with a distance ≤ the effective distance threshold as high-risk dynamic obstacles, forming a high-risk dynamic obstacle set. This avoids invalid targets occupying computing resources and ensures that the attention mechanism focuses on the core monitoring objects.

[0036] Step 2: Based on the velocity changes, coordinate changes, and distances to the robot's work area boundary of the high-risk dynamic obstacles in the first consecutive numerical frames, calculate the temporal attention weight and spatial attention weight, and sum them by weighting to obtain the comprehensive attention weight of the high-risk dynamic obstacles.

[0037] Spatial attention weights It is obtained through the following calculation formula: , The shortest distance between high-risk dynamic obstacles and the boundary of the robot's operating area. The smaller the value, the higher the spatial attention weight. The larger the size, the higher the level of focus.

[0038] Temporal attention weights The calculation formula (based on the target time-series dynamic change rate) is as follows: , The velocity change rate of the high-risk dynamic obstacle in the first consecutive numerical frame. The rate of change of coordinates for the first consecutive numerical frame. To ensure the preset maximum dynamic change rate Normalized to [0.3, 1], the more drastic the dynamic changes of the target, the greater the temporal attention weight. The larger.

[0039] Step 3: Based on the comprehensive attention weight, the trajectory coordinates of the high-risk dynamic obstacle are extrapolated in the first time interval (preferably 300ms) in the future, and the trajectory is smoothed by using B-spline curves to obtain the movement trajectory of the high-risk dynamic obstacle in the first time interval in the future.

[0040] Trajectory coordinate derivation is achieved through the following calculation formula: , , , , , For the future k time steps ( k =1,2,3, for example, corresponding to 100ms, 200ms, 300ms) are the three-dimensional coordinates of the obstacle. , , for t Real-time coordinates of high-risk dynamic obstacles i The number of time-series frames (1-5). This represents the total number of timing frames. k Using this formula to accurately predict the trajectory for future time steps, and then using B-spline curves to smooth the predicted coordinates, a continuous and smooth future movement trajectory is obtained.

[0041] Simultaneously, based on the instantaneous velocity of high-risk dynamic obstacles in the first numerical frame and the comprehensive attention weight, the velocity change is calculated to predict the velocity change and velocity trend in the next time interval. The specific calculation formula is as follows: , , For the future k Prediction speed at each time step For the present t Speed ​​at any moment For the speed change in the first time in the future, according to Determine the trend of speed change: At that time, in order to accelerate the movement, For uniform motion, It is a decelerating motion.

[0042] Furthermore, based on the movement trajectory and speed change trend in the first time of the future, the intention judgment coefficient of high-risk dynamic obstacles is calculated. Combined with the category of high-risk dynamic obstacles and the preset behavior intention matching rules, the corresponding behavior intention is obtained.

[0043] The formula for calculating the intent determination coefficient is as follows: , The coefficient for determining behavioral intent. For the future k The distance between high-risk dynamic obstacles and the boundary of the work area at each time step. For the present t Distance in time The speed influence coefficient is: personnel speed influence coefficient > moving vehicle speed influence coefficient > shaking cargo speed influence coefficient.

[0044] When the high-risk dynamic obstacle is a person. I A value < -0.1 is considered "close to the work area," and a value > 0.1 is considered "far from the work area." I |≤0.1 is considered "stationary".

[0045] When the high-risk dynamic obstacle is a moving vehicle And if I < 0, it is determined as "about to stop". Furthermore, if I < 0, it is determined as "approaching the work area and moving at a constant speed", and if I > 0, it is determined as "moving away from the work area".

[0046] When the high-risk dynamic obstacle is swaying cargo. (i.e., shaking amplitude > 5mm) and I < 0 are judged as "severe shaking and close to the work area". It was determined to be "slight shaking".

[0047] Finally, the movement trajectory, speed change trend and corresponding behavioral intention of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future, as well as the target loading cargo, are tagged and fused to obtain a dynamic environment raster map with intent tags.

[0048] The specific processing procedure is as follows: the future movement trajectory of high-risk dynamic obstacles marked with dashed coordinate points, the speed change trend of text labels indicating acceleration, constant speed, or deceleration, and the behavioral intent of exclusive icons or text labels are marked onto the corresponding grid cells in the dynamic environment grid map, thus generating a dynamic environment grid map with intent labels.

[0049] In this embodiment, S2 extracts obstacle density, movement speed, and cargo stacking height to quantitatively assess the complexity of the loading environment and dynamically adjusts the scene update cycle. This ensures real-time environmental perception in high-complexity environments and effectively reduces computing power consumption in low-complexity scenarios, achieving a balanced adaptation between modeling efficiency and system resources. Simultaneously, an improved attention mechanism is used to predict and label the future movement trajectory, speed changes, and behavioral intentions of dynamic obstacles. This enables the environmental grid map to have forward-looking dynamic perception capabilities, predict collision risks in advance, and significantly improve the safety, response speed, and scene adaptability of subsequent trajectory planning, ensuring the loading robot operates smoothly and efficiently in dynamic and changing working environments.

[0050] S3. Process the dynamic environment grid map with intent labels, target goods, and target loading position using a deep reinforcement learning model to obtain robot loading trajectory parameters; generate a path based on the robot loading trajectory parameters to obtain a preliminary robot loading trajectory; perform smoothing and energy consumption feedback adjustment on the preliminary robot loading trajectory to obtain the final robot loading trajectory.

[0051] First, the dynamic environment grid map with intent labels, the target goods, and the target loading position are processed by a deep reinforcement learning model to obtain the robot loading trajectory parameters. The specific steps are detailed below.

[0052] Step 1: Process the dynamic environment grid map with intent markers through the feature encoding layer to obtain the initial environment feature tensor; convert the boundary coordinates of the loading area into a constraint mask matrix, multiply it element-wise with the initial environment feature tensor, remove invalid grid features that exceed the loading area, limit the robot's movement range, and obtain the environment feature tensor.

[0053] The specific operation of the feature encoding layer is as follows: one-hot encoding is performed on the dynamic environment grid map with intent labels, and different types of grid states, such as idle, obstacles, target loading goods, personnel, etc., and corresponding intent labels are converted into multi-dimensional feature vectors to obtain the initial environment feature tensor, which contains grid state, intent label and coordinate information.

[0054] Step 2: The environmental feature tensor is processed by the Actor network to obtain the initial robot loading trajectory parameters.

[0055] This embodiment of the Actor network abandons the conventional fully connected layer design and adopts spatial convolution extraction and temporal attention. It focuses on extracting spatial obstacle distribution and temporal intention risk features from the dynamic grid image. In the improved Actor network, the environmental feature tensor undergoes spatial convolution extraction processing. Convolution operations are performed on the environmental feature tensor to extract spatial features such as obstacle distribution and cargo stacking height in local grid regions, resulting in spatial convolution features. These spatial convolution features are then processed by temporal attention, highlighting the feature regions corresponding to dynamic obstacles (personnel approach, vehicle start / stop), resulting in temporal attention features. Finally, the temporal attention features are processed by a lightweight fully connected layer to obtain the initial robot loading trajectory parameters. Temporal attention processing is existing technology and will not be described again here for cost savings.

[0056] The spatial convolution extraction operation described above is implemented using the following calculation formula: , For position is (i,j) The s Each channel spatial convolution feature The weights are the convolution kernel weights, and the weights are the first... m line, number n Column, No. p The channel weights are weights learned by the network through training, and are used to measure the importance of environmental information at different locations and in different channels within a local grid area; For the dynamic raster map environment features after embedding constraints, for location (i+m,j+n) First k+p The feature values ​​of the passage correspond to information such as the location of obstacles, the stacking height of goods, and the dynamic state of obstacles in the local grid area of ​​the original environment. These are the bias parameters for the convolutional layer. For activation function, M This represents the total height of the convolution kernel along the raster row direction. N The total width in the column direction. P The above calculation formula, which represents the total number of channels in the convolution kernel, slides a convolution kernel of fixed size (M×N×P) on the environmental feature tensor to weight and aggregate information such as obstacle distribution and cargo height in local grid areas. After nonlinear transformation by the activation function, more abstract and discriminative spatial convolution features are obtained, providing core environmental perception features for subsequent temporal attention processing and robot motion trajectory generation.

[0057] Step 3: Input the initial robot loading trajectory parameters and environmental feature tensors into the Critic network for scoring, and update the Actor network parameters based on the scoring objective function to iteratively optimize and obtain the robot loading trajectory parameters. The robot loading trajectory parameters include: 3D coordinates of trajectory nodes, robot end effector attitude angles, linear velocity and acceleration along the trajectory, angles and angular velocities of each joint of the robotic arm, and indices and positions of key nodes in the loading operation (grabbing point, placement point, transition point).

[0058] In Critic network processing, the scoring objective function is constructed by weighted summation based on loading efficiency reward, energy consumption control reward, obstacle avoidance safety reward, and trajectory smoothness reward; the scoring objective function varies depending on the complexity level of the loading environment in the dynamic environment grid.

[0059] When the vehicle environment complexity level is high complexity, the weight of the obstacle avoidance safety reward is the maximum value, which is greater than the first weight threshold (preferably 0.3), and is used to enhance obstacle avoidance.

[0060] When the loading environment complexity level is medium complex, the difference between the weights of loading efficiency reward, energy consumption control reward, obstacle avoidance safety reward, and trajectory smoothness reward is less than the difference threshold. The four indicators are evenly distributed to balance all rewards, taking into account work efficiency, motion smoothness, obstacle avoidance safety, and energy consumption control.

[0061] When the loading environment complexity level is low, the weights of loading efficiency rewards and energy consumption control rewards are both greater than the first weight threshold, which is used to enhance efficiency and pursue high-efficiency and low-consumption operations.

[0062] The objective function calculation formulas for low-complexity environment level, medium-complexity environment level, and high-complexity environment level are as follows: , , , , , These represent the objective function values ​​for low, medium, and high complexity environment levels, respectively. , , , These are respectively: loading efficiency bonus, energy consumption control bonus, obstacle avoidance safety bonus, and trajectory smoothness bonus. , These are the first parameter and the second parameter, respectively. , .

[0063] Loading efficiency bonus It is obtained through the following calculation formula: , To process the trajectory parameters using the A* algorithm to obtain the trajectory length, The maximum allowed trajectory length, For trajectory execution time, The shorter the trajectory and the less time it takes, the higher the reward.

[0064] Energy consumption control rewards It is obtained through the following calculation formula: , for Acceleration at all times This is the robot's maximum acceleration. The total time is the time. The smaller the acceleration and the smoother the motion, the lower the energy consumption and the higher the reward.

[0065] Obstacle Avoidance Safety Rewards It is obtained through the following calculation formula: , For trajectory and dynamic obstacles e The minimum distance, For safe distance threshold, The total number of dynamic obstacles; the farther away from the dynamic obstacles, the lower the risk and the higher the reward.

[0066] Trajectory smoothness reward It is obtained through the following calculation formula: , , They are respectively time, The fewer the inflection points and the smaller the change in angular velocity at any given moment, the higher the smoothness.

[0067] Then, based on the robot loading trajectory parameters, a path is generated to obtain a preliminary robot trajectory. The process of generating a path from the robot loading trajectory parameters can be implemented using existing trajectory generation methods, such as linear interpolation, polynomial interpolation, and A* path generation methods, to generate a continuous path based on the trajectory parameters, thereby obtaining the preliminary robot loading trajectory.

[0068] Finally, the initial robot trajectory is smoothed and energy consumption is adjusted to obtain a smooth, efficient, safe robot loading trajectory that is suitable for loading scenarios.

[0069] Specifically, the initial trajectory is divided into multiple continuous segments, each containing only the starting node and motion transition nodes whose joint velocity change rate is less than the joint velocity change rate threshold. This ensures the smooth motion of the robotic arm within each segment and avoids insufficient local smoothness caused by global interpolation, resulting in a trajectory segmentation result. The trajectory segmentation result is then weighted based on node weighting coefficients to obtain a weighting coefficient vector for each segment node. Based on the trajectory segmentation result and the node weighting coefficient vector, an initial smoothed trajectory is obtained through interpolation calculation. Finally, the initial smoothed trajectory is subjected to energy consumption feedback adjustment to obtain an energy-optimized smoothed trajectory.

[0070] The above node weighting coefficients are obtained using the following formula: , For the first The first segment The weighting coefficients of each node, For the first The first segment The joint stiffness coefficient of the robotic arm at each node. For the first The first segment Joint load force at each node, This is the maximum allowable load threshold for the joint.

[0071] The operation of energy consumption feedback adjustment is as follows: if the instantaneous energy consumption of the initial smoothed trajectory is greater than the allowable unit energy consumption threshold, then speed adjustment is initiated; if the instantaneous energy consumption of the initial smoothed trajectory is not greater than the unit energy consumption threshold, then the current speed parameters are maintained.

[0072] Speed ​​adjustment is achieved through the following calculation formula: , The corrected speed, To provide energy feedback gain, ensure a smooth speed correction and avoid affecting trajectory smoothness. For instantaneous energy consumption, , For energy consumption feedback gain, control the correction magnitude. The allowable unit energy consumption threshold, For the present t Speed ​​at any moment For the present t Acceleration at all times.

[0073] In this embodiment, S3 uses a deep reinforcement learning model to output robot trajectory parameters that fit the dynamic loading scenario from a dynamic environment grid map with intent markers, ensuring that trajectory planning accurately matches environmental risks and obstacle intents. Then, based on the trajectory parameters, a preliminary loading path is generated to build a basic operation path. Finally, through smoothing processing and energy consumption feedback adjustment, the energy consumption of the robotic arm is reduced while ensuring continuous and smooth trajectory and safe and stable operation. Overall, the safety, smoothness and economy of trajectory planning in dynamic loading scenarios are optimized in a coordinated manner, effectively improving loading operation efficiency and operational reliability.

[0074] This embodiment also provides an intelligent trajectory planning system for a loading robot, used to implement the above-mentioned intelligent trajectory planning method for a loading robot, including: The dynamic loading 3D scene generation module is used to acquire LiDAR data, high-definition camera data, and ultrasonic data in the loading operation area. After time-series misalignment calibration and measurement deviation calibration, the data is fused to obtain multimodal fusion data. Through a coordinate transformation matrix, the multimodal fusion data is mapped to a 3D grid matrix, retaining only the data within the robot's motion trajectory range to obtain the dynamic loading 3D scene. The dynamic environment grid generation module with intent labeling is used to obtain the obstacle density, movement speed, and cargo stacking height of the dynamic loading 3D scene, calculate the loading environment complexity, match the corresponding loading environment complexity level, adjust the update cycle of the dynamic loading 3D scene according to the loading environment complexity level, and predict the movement trajectory, speed change trend and corresponding behavioral intent of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future. After labeling, the dynamic environment grid with intent labeling is obtained. The robot loading trajectory generation module is used to process the dynamic environment grid map with intent tags, the target cargo, and the target loading position through a deep reinforcement learning model to obtain robot loading trajectory parameters; based on the robot loading trajectory parameters, a path is generated to obtain a preliminary robot loading trajectory; the preliminary robot loading trajectory is smoothed and energy consumption feedback is adjusted to obtain the final robot loading trajectory.

[0075] This embodiment also provides a vehicle loading robot trajectory intelligent planning device, including a processor and a memory, wherein the processor executes the computer program stored in the memory to implement the above-described vehicle loading robot trajectory intelligent planning method.

[0076] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described intelligent trajectory planning method for loading robots.

[0077] This embodiment provides an intelligent trajectory planning method for a loading robot. First, it performs multi-source sensor data calibration and fusion on LiDAR data, high-definition camera data, and ultrasonic data from the loading operation area, and maps and constructs a lightweight dynamic 3D loading scene. This allows for accurate perception of the entire environmental information even in complex environments with numerous obstacles. Then, based on the scene information, it calculates the environmental complexity, adaptively adjusts the scene update cycle, and predicts the movement trajectory and behavioral intentions of high-risk obstacles. This enables early identification of safety hazards in complex environments, providing a reliable pre-judgment basis for path planning. Next, relying on a grid map with intent markers, a deep reinforcement learning model outputs trajectory parameters adapted to the complex environment and generates a preliminary path. This effectively avoids dense obstacles and solves the problems of path planning conflicts and difficulty in adaptation in complex scenarios. Finally, it smooths and adjusts the preliminary trajectory with energy consumption feedback, making the robot's loading movement more continuous and stable, reducing start-stop and jamming losses. Ultimately, this significantly improves the rationality of path planning and the efficiency of loading operations in complex dynamic environments such as changes in cargo position, personnel movement, and equipment movement.

[0078] While exemplary embodiments of the invention have been described herein, many other variations or modifications conforming to the principles of the invention can be directly determined or derived from the disclosure of this invention without departing from its spirit and scope. Therefore, the scope of the invention should be understood and recognized to cover all such other variations or modifications.

Claims

1. A method for intelligent trajectory planning of a loading robot, characterized in that, This includes the following operations: S1. Acquire LiDAR data, HD camera data, and ultrasonic data from the loading operation area. After time-series misalignment calibration and measurement deviation calibration, fuse them to obtain multimodal fusion data. Through a coordinate transformation matrix, map the multimodal fusion data onto a three-dimensional grid matrix, retaining only the data within the robot's motion trajectory range to obtain a dynamic three-dimensional loading scene. S2. Obtain the obstacle density, movement speed, and cargo stacking height of the dynamic loading 3D scene, calculate the loading environment complexity, match the corresponding loading environment complexity level, adjust the update cycle of the dynamic loading 3D scene according to the loading environment complexity level, and predict the movement trajectory, speed change trend and corresponding behavioral intention of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future. After marking, obtain a dynamic environment raster map with intent marking. S3. Process the dynamic environment grid map with intent labels, target goods, and target loading position using a deep reinforcement learning model to obtain robot loading trajectory parameters; generate a path based on the robot loading trajectory parameters to obtain a preliminary robot loading trajectory; perform smoothing and energy consumption feedback adjustment on the preliminary robot loading trajectory to obtain the final robot loading trajectory.

2. The intelligent trajectory planning method for loading robots according to claim 1, characterized in that, In S2, the method for predicting the movement trajectory is as follows: High-risk dynamic obstacles are identified based on the grid state in the dynamic loading 3D scene. Based on the velocity changes, coordinate changes, and distances to the robot's work area boundary of high-risk dynamic obstacles in the first consecutive numerical frames, the temporal attention weight and spatial attention weight are calculated and then summed to obtain the comprehensive attention weight. Based on comprehensive attention weights, the trajectory coordinates of high-risk dynamic obstacles are extrapolated and smoothed in the first time interval of the future to obtain the movement trajectory of the high-risk dynamic obstacles in the first time interval of the future.

3. The intelligent trajectory planning method for loading robots according to claim 1, characterized in that, In S3, the method for obtaining the robot loading trajectory parameters is as follows: The dynamic environment grid map with intent markers is processed through a feature encoding layer to obtain the initial environment feature tensor. The boundary coordinates of the loading area are converted into a constraint mask matrix and multiplied element-wise with the initial environment feature tensor to obtain the environment feature tensor. The environment feature tensor is input into the Actor network to obtain the initial robot loading trajectory parameters. The initial robot loading trajectory parameters and the environment feature tensor are input into the Critic network for scoring, and the Actor network parameters are updated based on the scoring objective function to iteratively optimize and obtain the robot loading trajectory parameters.

4. The intelligent trajectory planning method for loading robots according to claim 3, characterized in that, In the Actor network, the environmental feature tensor is extracted by spatial convolution to obtain spatial convolution features; the spatial convolution features are processed by temporal attention to obtain temporal attention features; the temporal attention features are processed by a lightweight fully connected layer to obtain the initial robot loading trajectory parameters.

5. The intelligent trajectory planning method for loading robots according to claim 3, characterized in that, The scoring objective function is constructed by weighted summation based on loading efficiency reward, energy consumption control reward, obstacle avoidance safety reward, and trajectory smoothness reward. The loading environment complexity level of the dynamic environment grid map is different, and the corresponding scoring objective function is different; When the loading environment complexity level is high, the weight of the obstacle avoidance safety reward is at its maximum value and is greater than the first weight threshold, which is used to enhance obstacle avoidance; when the loading environment complexity level is medium, the difference between the weights of the loading efficiency reward, energy consumption control reward, obstacle avoidance safety reward, and trajectory smoothness reward is less than the difference threshold, which is used to balance all rewards; when the loading environment complexity level is low, the weights of the loading efficiency reward and energy consumption control reward are both greater than the first weight threshold, which is used to enhance efficiency and pursue high-efficiency and low-consumption operation. The first weight threshold is 0.

3.

6. The intelligent trajectory planning method for loading robots according to claim 1, characterized in that, In S3, the smoothing operation is as follows: The initial trajectory is divided into multiple continuous segments, each segment containing only the starting node and motion transition nodes whose joint velocity change rate is less than the joint velocity change rate threshold, thus obtaining the trajectory segmentation result. The trajectory segmentation result is then weighted based on the node weighting coefficient to obtain the weighting coefficient vector of each segment node. Based on the trajectory segmentation result and the node weighting coefficient vector, interpolation calculation is performed to obtain the initial smooth trajectory.

7. The intelligent trajectory planning method for loading robots according to claim 1, characterized in that, In S3, the energy consumption feedback adjustment process is as follows: if the instantaneous energy consumption of the initial smooth trajectory is greater than the allowable unit energy consumption threshold, then speed adjustment is initiated; if the instantaneous energy consumption of the initial smooth trajectory is not greater than the unit energy consumption threshold, then the current speed parameters are maintained.

8. A trajectory intelligent planning system for a loading robot, used to implement the trajectory intelligent planning method for a loading robot as described in claim 1, characterized in that, include: The dynamic loading 3D scene generation module is used to acquire LiDAR data, high-definition camera data, and ultrasonic data in the loading operation area. After time-series misalignment calibration and measurement deviation calibration, the data is fused to obtain multimodal fusion data. Through a coordinate transformation matrix, the multimodal fusion data is mapped to a 3D grid matrix, retaining only the data within the robot's motion trajectory range to obtain the dynamic loading 3D scene. The dynamic environment grid generation module with intent labeling is used to obtain the obstacle density, movement speed, and cargo stacking height of the dynamic loading 3D scene, calculate the loading environment complexity, match the corresponding loading environment complexity level, adjust the update cycle of the dynamic loading 3D scene according to the loading environment complexity level, and predict the movement trajectory, speed change trend and corresponding behavioral intent of high-risk dynamic obstacles in the dynamic loading 3D scene in the first time in the future. After labeling, the dynamic environment grid with intent labeling is obtained. The robot loading trajectory generation module is used to process the dynamic environment grid map with intent tags, the target cargo, and the target loading position through a deep reinforcement learning model to obtain robot loading trajectory parameters; based on the robot loading trajectory parameters, a path is generated to obtain a preliminary robot loading trajectory; the preliminary robot loading trajectory is smoothed and energy consumption feedback is adjusted to obtain the final robot loading trajectory.

9. A vehicle loading robot trajectory intelligent planning device, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the intelligent trajectory planning method for loading robots as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the intelligent trajectory planning method for loading robots as described in any one of claims 1-7.