A special end-to-end planning control method and system for unmanned logistics vehicles

Abstract

The application relates to the technical field of automatic driving, in particular to a special end-to-end planning control method and system for unmanned logistics vehicles. The application adopts an end-to-end design, integrates functions such as perception, decision-making, planning and control into a unified model, simplifies the system architecture, reduces the development complexity, and improves the real-time performance and consistency of the system. The application innovatively combines two methods of imitation learning and reinforcement learning. Imitation learning is used for model initialization, so that the model quickly has a safe and gentle driving style; reinforcement learning is used for strategy optimization, so that the model can surpass the level of human drivers and find better driving strategies. This collaborative optimization method takes into account the training efficiency and performance. The application is specially designed for unmanned logistics vehicles, fully considering the particularity of the vehicles in the urban distribution scene, so that the vehicles can better adapt to the application requirements of unmanned logistics vehicles.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically a dedicated end-to-end planning and control method and system for unmanned logistics vehicles. Background Technology

[0002] Traditional autonomous driving systems generally employ a rule-based modular architecture, which decomposes complex driving tasks into multiple independent sub-modules such as perception, localization, prediction, decision-making, planning, and control. Each module is responsible for a specific function and interacts with data through well-defined interfaces. For example, the perception module is responsible for identifying and tracking surrounding obstacles, lane lines, traffic signs, etc.; the prediction module is responsible for predicting the future trajectories of other road users; the decision-making module makes macroscopic driving behavior decisions (such as following, changing lanes, overtaking, etc.) based on the perception and prediction results; the planning module generates a specific, collision-free driving trajectory that meets vehicle dynamics constraints based on the decision results; and finally, the control module is responsible for translating the planned trajectory into accelerator, brake, and steering commands. The advantage of this architecture is that the module responsibilities are clear, facilitating independent development and debugging, and allowing for rapid identification of specific modules when system problems occur. However, its disadvantages are also quite obvious. First, there is loss and delay in information transmission between modules; the output error of upstream modules can be propagated and amplified at each level, affecting the final control effect. Second, the performance of the entire system is limited by the weakest link, the "barrel effect"—the performance bottleneck of any module will limit the overall system performance. More importantly, such systems based on human design and rules have limited complexity and coverage, making it difficult to cope with the endless long-tail scenarios and emergencies in urban traffic. The system's generalization ability and robustness are also poor. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a dedicated end-to-end planning and control method and system for unmanned logistics vehicles to solve the problems in the background art.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] This application discloses a dedicated end-to-end planning and control method for unmanned logistics vehicles, comprising the following steps:

[0006] Acquire driving sample data, wherein the driving sample data includes perception data samples and control data samples. The perception data samples include image sequences from multiple high-definition cameras, radar data, and the vehicle's own motion state information; the control data samples include steering wheel angle and acceleration / deceleration commands.

[0007] Training data is constructed using the perceived data samples as labels, and a pre-constructed neural network model is trained based on the training data to obtain a preliminary control model that represents vehicle perception information and control commands. The neural network model includes an input layer for inputting the perceived data, a backbone network for extracting features from the perceived data, a fusion layer for fusing the extracted features to obtain fused features, a time-series processing layer for constructing time-series feature vectors based on the fused features, and an output layer for outputting control data based on the time-series feature vectors.

[0008] A simulation environment for unmanned logistics vehicles is constructed, and multiple target working conditions for unmanned logistics vehicles are generated in the simulation environment, and target perception data is extracted from the target working conditions.

[0009] The target perception data is input into the preliminary control model to obtain the control data of the target operating condition; the preliminary control model is then subjected to reinforcement learning based on the pre-constructed evaluation function and the control data of the target operating condition to obtain the final control model.

[0010] The unmanned logistics vehicle is controlled end-to-end based on the final control model.

[0011] In one embodiment of this application, the backbone network includes a ResNet-50 network for extracting visual features of image sequences and a PointNet network for extracting 3D geometric features of LiDAR point cloud data, wherein both the ResNet-50 network and the PointNet network are obtained through pre-training.

[0012] In one embodiment of this application, a pre-constructed neural network model is trained based on training data to obtain a preliminary control model representing vehicle perception information and control commands, including:

[0013] S1, input the sensor data samples in the training data into the input layer, and obtain the corresponding prediction control data from the output layer;

[0014] S2, calculate the loss between the predicted control data and the control data samples in the training data based on the pre-built loss function, and adjust the parameters of the neural network model based on the loss and the gradient descent method;

[0015] S3. Repeat steps S1-S2 until training is complete, and a preliminary control model representing vehicle perception information and control commands is obtained.

[0016] In one embodiment of this application, the mathematical expression of the loss function is:

[0017]

[0018] In the formula, Indicates loss, This represents the number of sample frames in each training round. Indicates the frame number of the training sample. Indicates the weighting coefficient. This represents the steering wheel angle in the predictive control data. This indicates acceleration / deceleration commands in the predictive control data. This indicates the steering wheel angle in the control data sample. This indicates acceleration / deceleration commands in the control data sample.

[0019] In one embodiment of this application, a simulation environment for unmanned logistics vehicles is constructed, and multiple target operating conditions for unmanned logistics vehicles are generated in the simulation environment, including:

[0020] A simulation environment model is built based on a physics engine, wherein the simulation environment model includes a road network, traffic elements, and a weather system;

[0021] Input multiple frames of real driving data and calculate the probability of occurrence of various operating conditions in the real driving data, wherein the real driving data includes sensor logs and control commands;

[0022] High-risk operating conditions with a probability of occurrence below a preset probability threshold are selected, and rules for the high-risk operating conditions are extracted; random operating conditions are generated based on the simulation environment model, and random operating conditions that conform to the rules of the high-risk operating conditions are used as target operating conditions.

[0023] In one embodiment of this application, the preliminary control model is subjected to reinforcement learning based on a pre-constructed evaluation function and control data of the target operating condition to obtain a final control model, including:

[0024] The operation of the unmanned logistics vehicle is simulated in the simulation environment model according to the control data of the target working condition, and various parameters and target events during the operation are extracted.

[0025] The evaluation score of the unmanned logistics vehicle's operation process is calculated based on a pre-constructed evaluation function, penalty function, multiple parameters, and target events. The parameters of the preliminary control model are then adjusted based on the evaluation score to obtain the final control model.

[0026] In one embodiment of this application, the mathematical expression of the evaluation function is:

[0027]

[0028]

[0029]

[0030] In the formula, Indicates the evaluation score. Indicates the reward score. Indicates the penalty score. Represents time frame The corresponding state, Represents time frame The corresponding action, This indicates a reward for achieving the goal. This represents the indicator function for achieving the goal. Indicates an efficiency reward. Represents time frame The corresponding speed, Indicates a comfort reward. Indicates the comfort threshold. This represents the step size penalty coefficient. Indicates the intensity of the collision penalty. This represents the collision indication function. Indicates the severity of penalties for violations. This indicates a violation indicator function.

[0031] This application also provides a dedicated end-to-end planning and control system for unmanned logistics vehicles, including:

[0032] The acquisition module is used to acquire driving sample data, wherein the driving sample data includes perception data and control data. The perception data includes image sequences from multiple high-definition cameras, radar data, and the vehicle's own motion state information; the control data includes steering wheel angle and acceleration / deceleration commands.

[0033] The imitation learning module is used to construct training data using the steering wheel angle and the acceleration / deceleration command as labels, and to train a pre-constructed neural network model based on the training data to obtain a preliminary control model that represents vehicle perception information and control commands. The neural network model includes an input layer for inputting perception data, a backbone network for extracting features from the perception data, a fusion layer for fusing the extracted features to obtain fused features, a time-series processing layer for constructing time-series feature vectors based on the fused features, and an output layer for outputting control data based on the time-series feature vectors.

[0034] The environment simulation module is used to construct a simulation environment for unmanned logistics vehicles, generate multiple target working conditions for unmanned logistics vehicles in the simulation environment, and extract target perception data from the target working conditions.

[0035] The reinforcement learning module is used to input the target perception data into the preliminary control model to obtain the control data of the target operating condition; and to perform reinforcement learning on the preliminary control model based on the pre-constructed evaluation function and the control data of the target operating condition to obtain the final control model.

[0036] The control module is used to perform end-to-end control of the unmanned logistics vehicle based on the final control model.

[0037] This application also provides an electronic device, including: a processor and a memory;

[0038] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to cause the electronic device to perform the methods described above.

[0039] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0040] The beneficial effects of this application are as follows: This application provides a dedicated end-to-end planning and control method and system for unmanned logistics vehicles. This application adopts an end-to-end design, integrating perception, decision-making, planning, and control functions into a unified model, simplifying the system architecture, reducing development complexity, and improving the system's real-time performance and consistency. This application innovatively combines imitation learning and reinforcement learning methods. Imitation learning is used for model initialization, enabling the model to quickly acquire a safe and gentle driving style; reinforcement learning is used for policy optimization, enabling the model to surpass the level of human drivers and discover better driving strategies. This collaborative optimization approach balances training efficiency and performance. This application is specifically designed for unmanned logistics vehicles, fully considering their unique characteristics in urban delivery scenarios. Targeted optimizations have been made in model design, training data selection, and reward function design to better adapt to the application requirements of unmanned logistics vehicles. Attached Figure Description

[0041] The present application will be further described below with reference to the accompanying drawings and embodiments:

[0042] Figure 1 This is a schematic diagram of the structure of a neural network model in one embodiment of this application;

[0043] Figure 2 This is a flowchart illustrating a dedicated end-to-end planning and control method for unmanned logistics vehicles in one embodiment of this application;

[0044] Figure 3 This is a structural diagram of a dedicated end-to-end planning and control system for unmanned logistics vehicles, shown in one embodiment of this application. Detailed Implementation

[0045] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0046] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the layers related to this application and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0047] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of this application; however, it will be apparent to those skilled in the art that embodiments of this application may be practiced without these specific details.

[0048] In urban open roads, especially in urban delivery scenarios involving unmanned logistics vehicles, the traffic environment is extremely complex and dynamic. Traditional planning and control algorithms fall short in handling these scenarios. For example, in congested areas, vehicles need to frequently accelerate, decelerate, and maintain lanes at close range. Rule-based following models often struggle to achieve smooth and efficient control, easily leading to "phantom traffic jams." In narrow roads or construction zones, vehicles need to make precise detours and obstacle avoidances. Traditional path planning algorithms may not be able to generate sufficiently smooth and feasible trajectories. In complex intersections with unprotected left turns or mixed pedestrian and vehicle traffic, the decision-making module needs to comprehensively consider the intentions and interactions of multiple dynamic traffic participants. Decision-making methods based on finite state machines (FSMs) or simple optimization models struggle to make optimal judgments that align with human driving habits. These scenarios share the common characteristics of significant uncertainty and interactivity. Due to their modular and rule-based nature, traditional algorithms lack a deep understanding of the overall traffic environment and the ability to optimize globally. This results in overly conservative or inflexible decisions when facing these complex scenarios, impacting driving safety, efficiency, and comfort.

[0049] In recent years, with the development of deep learning technology, the end-to-end concept has been widely applied in the perception module of autonomous driving. In particular, the bird's-eye-view (BEV) based perception solution fuses data from multiple cameras, LiDAR, and other sensors, directly performing target detection, segmentation, and tracking within a unified BEV space, achieving end-to-end optimization of the perception module. This approach utilizes powerful neural network architectures such as Transformer to effectively process multimodal information and capture the spatiotemporal relationships between sensor data, significantly improving the accuracy and robustness of perception. For example, Neolix's autonomous vehicle pioneered a visual BEV algorithm based on the Transformer architecture, achieving multimodal pre-fusion perception, enabling its autonomous vehicle to have a 360-degree surround-view perception capability of 120 meters. However, this end-to-end application mainly focuses on the perception layer, and its output is still based on abstract results understood by humans, such as the position, speed, and category of obstacles, not fully connecting the entire link from perception to control.

[0050] Extending the end-to-end architecture from the perception module to the decision-making and planning module is an inevitable trend in the development of autonomous driving technology, but it also faces enormous challenges. End-to-end implementation of the decision-making and planning module means building a single neural network model that directly processes sensor data into vehicle control commands. This "One Model" architecture eliminates the traditional division of functional modules, theoretically enabling global optimization, avoiding information loss between modules, and possessing stronger generalization capabilities. However, the challenges are also severe. First, there's the data issue: training such a complex model requires massive amounts of high-quality data covering various extreme scenarios, and acquiring and labeling this data is extremely costly. Second, there are the issues of model interpretability and safety. Deep learning models are often considered "black boxes," and their decision-making processes are difficult to understand and verify, which is unacceptable for autonomous driving systems with extremely high safety requirements. Furthermore, designing effective training strategies that enable the model to learn safe, efficient, and compliant driving strategies is also a core challenge. Currently, although there has been some research and exploration, such as Waymo's Driver 5 trial version eliminating the traditional planning module and Tesla's FSD v12 also adopting end-to-end technology, this technical approach is still in the early stages of development and has not yet formed a mature, large-scale deployable solution.

[0051] Unmanned logistics vehicles are primarily used in urban last-mile delivery scenarios, and their operating environment differs significantly from that of highways or structured industrial parks. First, their speeds are typically low, generally below 50 km / h, or even lower. Second, their routes are relatively fixed, usually conducting point-to-point deliveries within specific communities, commercial districts, or industrial parks. Third, as service vehicles, unmanned logistics vehicles have limited right-of-way requirements and need to proactively yield to pedestrians, non-motorized vehicles, and other motorized vehicles to ensure traffic harmony and safety. Finally, the types of traffic participants in urban delivery scenarios are complex, with high behavioral uncertainty. There are numerous non-motorized vehicles, pedestrians, temporarily parked vehicles, and shuttling delivery electric vehicles, resulting in highly dynamic and random traffic flow. These characteristics dictate that the planning and control algorithms for unmanned logistics vehicles cannot simply be applied to algorithms designed for general passenger vehicles; instead, they require targeted optimization.

[0052] Based on the characteristics of the aforementioned scenarios, unmanned logistics vehicles have specific requirements for driving style. Unlike passenger cars that prioritize maximum traffic efficiency, unmanned logistics vehicles emphasize a driving style of "safety first, right-hand traffic, low interference, and high adaptability." This means that the vehicle should maintain a high level of vigilance at all times, prioritizing safety and actively avoiding other road users. In route selection, it should prioritize right-hand traffic, avoiding frequent lane changes and cutting in to minimize interference with other vehicles. In control strategies, it should strive for smoothness, avoiding sudden acceleration, sudden braking, and sharp turns to ensure the safety of the cargo and provide a comfortable traffic environment for surrounding road users. This gentle, stable, and courteous driving style needs to be solidified and reflected through algorithms. While traditional rule-based algorithms can achieve similar effects by setting conservative parameters, they often sacrifice traffic efficiency and are difficult to adapt to all scenarios. Therefore, how to learn and solidify this specific driving style through data-driven methods, especially imitation learning, is a major challenge facing unmanned logistics vehicle planning and control technology.

[0053] To address the aforementioned issues, the main objective of this application is to resolve numerous problems inherent in existing autonomous driving planning and control technologies, particularly traditional modular architectures and general end-to-end algorithms, when applied to urban unmanned logistics vehicle scenarios. To address the shortcomings of traditional modular architectures, such as information loss between modules, latency, and difficulty in covering long-tail scenarios, this invention proposes an end-to-end solution. By constructing a single neural network model, it directly maps raw sensor data to vehicle control commands, thereby achieving the integration of perception, decision-making, planning, and control, reducing intermediate steps, and improving the overall system performance and response speed. Addressing the challenges of general end-to-end algorithms in terms of data requirements, model interpretability, and safety, this invention introduces a training strategy combining imitation learning and reinforcement learning. It rapidly initializes the model using real operational data and solidifies a safe and gentle driving style. Then, it conducts large-scale training in a simulated environment using reinforcement learning to improve the model's decision-making ability and robustness in rare and complex scenarios, thus achieving continuous optimization and evolution of the algorithm while ensuring safety.

[0054] The core of this application is to construct an end-to-end deep neural network model, which acts as the "brain" of the unmanned logistics vehicle, responsible for directly converting perceived environmental information into vehicle control commands. Unlike traditional modular architectures, this invention eliminates the explicit division between intermediate stages such as perception, prediction, decision-making, and planning, implicitly integrating these functions into a unified network structure. This design minimizes information transmission losses and delays between modules, achieving global optimization and thus improving the overall system performance and robustness. The model's input receives raw data or pre-processed multimodal features from vehicle sensors (such as LiDAR, cameras, millimeter-wave radar, IMU, etc.). The model's output directly generates the vehicle's low-level control signals, such as steering wheel angle, throttle opening, and brake pressure. To achieve this, the model typically employs complex network structures, such as combining convolutional neural networks (CNNs) to process spatial information like images and point clouds, and recurrent neural networks (RNNs) or Transformer structures to process temporal information, thereby capturing the dynamic changes in the traffic environment.

[0055] Figure 1 This is a schematic diagram of the structure of a neural network model in one embodiment of this application, as shown below. Figure 1 As shown, the neural network model in this application includes:

[0056] The input layer 110 is used to input perception data, which includes multimodal sensor data (such as image sequences from multiple high-definition cameras for identifying lane lines, traffic signs, traffic lights, pedestrians, vehicles, etc.; point cloud data from LiDAR for accurately measuring the three-dimensional position and shape of obstacles; and data from millimeter-wave radar for providing reliable target detection and speed measurement in adverse weather conditions) and vehicle state data (including vehicle speed, acceleration, heading angle, steering wheel angle, gear position, etc., which are crucial for the model to understand the current dynamic characteristics of the vehicle).

[0057] The backbone network 120 is used to extract features from the perceived data. Specifically, the backbone network includes a ResNet-50 network for extracting visual features from image sequences and a PointNet network for extracting 3D geometric features from LiDAR point cloud data. Both the ResNet-50 network and the PointNet network are pre-trained. Data from the IMU and the vehicle's CAN bus can be directly used as state features.

[0058] The fusion layer 130 is used to fuse the extracted features to obtain fused features; feature vectors from different sources are fused in the fusion layer of the network (e.g., through splicing or attention mechanisms) to form a comprehensive environmental representation.

[0059] Temporal processing layer 140 is used to construct temporal feature vectors based on fused features; the fused feature vectors are then fed into a temporal processing module, such as a Long Short-Term Memory (LSTM) network or a Transformer, to model the dynamic evolution of the traffic environment.

[0060] The output layer (150) generates continuous control commands, such as steering wheel angle and speed, based on the output of the timing module. The entire network is trained end-to-end through imitation learning and reinforcement learning to optimize its weight parameters and minimize the gap between the predicted control commands and the actions of real experts.

[0061] Figure 2 This is a flowchart illustrating the practical application of a dedicated end-to-end planning and control method for unmanned logistics vehicles, as described in one embodiment of this application. Figure 2 As shown, the specific process includes:

[0062] S210, acquire driving sample data, wherein the driving sample data includes perception data samples and control data samples, the perception data samples include image sequences from multiple high-definition cameras, radar data, and the vehicle's own motion state information; the control data samples include steering wheel angle and acceleration / deceleration commands;

[0063] S220, using the perceived data samples as labels to construct training data, and training the pre-constructed neural network model based on the training data to obtain a preliminary control model representing vehicle perception information and control commands;

[0064] The specific training process includes:

[0065] S1, input the sensor data samples in the training data into the input layer, and obtain the corresponding prediction control data from the output layer;

[0066] S2, calculate the loss between the predicted control data and the control data samples in the training data based on the pre-built loss function, and adjust the parameters of the neural network model based on the loss and the gradient descent method;

[0067] S3. Repeat steps S1-S2 until training is complete, and a preliminary control model representing vehicle perception information and control commands is obtained.

[0068] In one embodiment of this application, the mathematical expression of the loss function is:

[0069]

[0070] In the formula, Indicates loss, This represents the number of sample frames in each training round. Indicates the frame number of the training sample. Indicates the weighting coefficient. This represents the steering wheel angle in the predictive control data. This indicates acceleration / deceleration commands in the predictive control data. This indicates the steering wheel angle in the control data sample. This indicates acceleration / deceleration commands in the control data sample.

[0071] To ensure that unmanned logistics vehicles possess safe and gentle driving capabilities from the outset, this invention employs imitation learning to initialize and train the end-to-end model.

[0072] The core idea of ​​the aforementioned imitation learning is to allow the model to learn driving strategies by observing and imitating the behavior of experts (i.e., excellent human drivers). In this invention, we first collect a large amount of real-world operational data in urban delivery scenarios by deploying a data acquisition system on unmanned logistics vehicles. This data includes environmental information perceived by sensors under various complex traffic conditions, as well as the corresponding human driver operations (such as steering wheel angle, accelerator and brake pedal positions). These "state-action" pairs are then used as training samples to train an end-to-end model through supervised learning. The model's goal is to make its output control commands as consistent as possible with the operations of an expert driver. In this way, the model can quickly learn the driving habits, safety awareness, and strategies for handling complex situations of human drivers, thereby solidifying a stable, safe, and gentle driving style. This provides a high-quality initial strategy for subsequent reinforcement learning optimization, significantly shortens training time, and improves the safety of learning.

[0073] S230, Construct a simulation environment for unmanned logistics vehicles, generate multiple target working conditions for unmanned logistics vehicles in the simulation environment, and extract target perception data from the target working conditions;

[0074] After obtaining a good initial policy through imitation learning, this invention introduces Deep Reinforcement Learning (DRL) for policy optimization to further improve the model's decision-making ability and robustness in unknown and rare scenarios. The core of reinforcement learning is to allow the agent (i.e., the unmanned logistics vehicle) to learn the optimal policy through interaction and trial and error in an environment. The foundation for implementing reinforcement learning is to construct a simulated environment and generate rare operating conditions. The model is then continuously adjusted within these rare conditions. Specifically, the process of constructing the simulated environment includes:

[0075] S231, Construct a simulation environment model based on a physics engine, wherein the simulation environment model includes a road network, traffic elements, and a weather system;

[0076] This application uses the open-source autonomous driving simulator CARLA0.9.14+ (or NVIDIA DRIVESim, LGSVL), which supports: high-precision vehicle dynamics models (Ackermann steering, tire friction, inertial forces); multimodal sensor simulation (camera, LiDAR, IMU, GNSS); programmable traffic flow and pedestrian behavior; real-time weather and lighting rendering.

[0077] Simultaneously, a road network is constructed, and real city map data (such as OpenStreetMap + Cityscapes annotations) is imported to build typical logistics scenarios, including urban main roads (including traffic lights, zebra crossings, and bus stops), internal roads of logistics parks (height restriction poles, loading and unloading areas, and narrow alleys), and highway ramps and roundabouts (to test lane change decisions).

[0078] Then configure traffic elements, including vehicles, pedestrians, and bicycles / electric bikes. Their behavior patterns include following other vehicles, changing lanes, and running red lights.

[0079] The weather system supports combinations of extreme conditions, such as heavy rain, nighttime, and slippery roads, which greatly reduces the coefficient of friction.

[0080] S232, Input multiple frames of real driving data and calculate the probability of occurrence of various operating conditions in the real driving data, wherein the real driving data includes sensor logs and control commands;

[0081] Then, over 2 million frames of real-world driving data were collected from the unmanned logistics fleet across 50+ cities. This data included images from 11 cameras, 2 LiDAR point clouds, an IMU (accelerometer / angular velocity sensor), and a CAN bus (vehicle speed, steering wheel angle). Corresponding control commands included human driver input (e.g., steering wheel angle). Accelerator / Brake .

[0082] Define 20+ types of operating conditions and extract key features for each type. Then, use kernel density estimation (KDE) or histogram method to calculate the frequency of occurrence of each operating condition.

[0083] S233, filter high-risk working conditions with a probability of occurrence lower than a preset probability threshold, and extract the rules of the high-risk working conditions; and generate random working conditions based on the simulation environment model, and take the random working conditions that conform to the rules of the high-risk working conditions as the target working conditions.

[0084] Set threshold: The system identifies 1,600 anomalous samples from 2 million frames. Each high-risk sample is then clustered, and structured rules are extracted. Finally, a simulated environment model is used to generate random operating conditions, and the features of these random conditions are matched against the structured rules; the conditions that match the rules are the target operating conditions.

[0085] For example, for each frame, the following 4-dimensional features are extracted:

[0086] Table 1. Schematic diagram of four-dimensional features

[0087]

[0088] In the identification results, 1,600 frames belonged to the low-probability event category of "sudden pedestrian". Then, clustering was performed to extract the subclass rule of "sudden pedestrian in truck blind spot". Within this cluster, 520 frames were sampled, and the statistical boundaries of each feature were calculated as follows:

[0089] Table 2. Schematic diagram of the statistical boundary of four-dimensional vectors

[0090]

[0091] After obtaining the rule data, 2000 random working conditions are generated → multiple target working conditions are matched.

[0092] Each target operating condition includes:

[0093] (1) Multi-camera image sequence (1920×1080@30fps).

[0094] (2) LiDAR point cloud (including rain and fog attenuation).

[0095] (3) Expert control commands (generated by human drivers in the simulation).

[0096] (4) Scene metadata (conforming to the above JSON rules).

[0097] The target perception data required for reinforcement training can be extracted from the multiple matched target conditions.

[0098] S240, The target perception data is input into the preliminary control model to obtain the control data of the target operating condition; the preliminary control model is subjected to reinforcement learning based on the pre-constructed evaluation function and the control data of the target operating condition to obtain the final control model;

[0099] After obtaining the target perception data, reinforcement learning is required. Its core idea is that at each time step, the agent selects an action (control command) based on the current state (sensor input). The environment then rewards (or punishes) the agent based on this action and transitions to the next state. The agent's goal is to learn a policy that maximizes the cumulative reward gained over long-term operation. The specific process includes:

[0100] S241, simulate the operation of the unmanned logistics vehicle in the simulation environment model according to the control data of the target working condition, and extract various parameters and target events during the operation.

[0101] S242, calculate the evaluation score of the unmanned logistics vehicle's operation process based on the pre-constructed evaluation function, penalty function, multiple parameters and target events, and adjust the parameters of the preliminary control model based on the evaluation score to obtain the final control model.

[0102] Specifically, the process of adjusting the parameters of the preliminary control model through evaluation scores is similar to the model imitation learning stage described above. However, in order to further adapt to extreme scenarios, the specific adjustment process will not be repeated here.

[0103] The mathematical expression for the evaluation function is:

[0104]

[0105]

[0106]

[0107] In the formula, Indicates the evaluation score. Indicates the reward score. Indicates the penalty score. Represents time frame The corresponding state, Represents time frame The corresponding action, This indicates a reward for achieving the goal. This represents the target achievement indicator function (value 1 when the target location is reached, otherwise 0). Indicates an efficiency reward. Represents time frame The corresponding speed, Indicates a comfort reward. Indicates the comfort threshold. This represents the step size penalty coefficient. Indicates the intensity of the collision penalty. This represents the collision indicator function (1 when a collision event occurs, 0 otherwise). Indicates the severity of penalties for violations. This indicates the violation indicator function (1 if a violation occurs, 0 otherwise).

[0108] This application designs a multi-objective reward function that comprehensively considers multiple aspects such as safety, efficiency, and comfort. Through large-scale reinforcement learning training in a highly realistic simulation environment, the model can explore better driving strategies than in the imitation learning stage, learn to handle various extreme situations and long-tail scenarios, thereby achieving higher traffic efficiency and better driving smoothness while ensuring safety.

[0109] S250, perform end-to-end control of the unmanned logistics vehicle based on the final control model.

[0110] Finally, the resulting final control model is deployed in the unmanned logistics vehicle, transferring this knowledge to the vehicle's driving tasks. For example, the model can help the system better understand the semantics of traffic signs and markings, predict the complex intentions of pedestrians and other vehicles, and even make reasonable decisions based on its powerful common-sense reasoning ability when faced with unprecedented scenarios. The application of large-scale models will significantly improve the cognitive intelligence level of end-to-end models, enabling them to better cope with various challenges in urban traffic.

[0111] The beneficial effects of this application are as follows:

[0112] (1) This application adopts an end-to-end design, which integrates functions such as perception, decision-making, planning and control into a unified model, which simplifies the system architecture, reduces development complexity, and improves the real-time performance and consistency of the system.

[0113] (2) This application innovatively combines two methods: imitation learning and reinforcement learning. Imitation learning is used for model initialization, enabling the model to quickly acquire a safe and gentle driving style; reinforcement learning is used for policy optimization, enabling the model to surpass the level of human drivers and discover better driving strategies. This collaborative optimization approach balances training efficiency and performance.

[0114] (3) This application is specifically designed for unmanned logistics vehicles and fully considers their special characteristics in urban delivery scenarios. Targeted optimizations have been made in model design, training data selection, and reward function design to better adapt to the application needs of unmanned logistics vehicles.

[0115] (4) This application supports continuous learning and evolution of the model. By collecting real operational data, we can continuously fine-tune and optimize the model, enabling its driving style to evolve and better adapt to different traffic environments and delivery tasks. This continuous learning capability makes this application highly adaptable and scalable.

[0116] like Figure 3 As shown, this application also provides a dedicated end-to-end planning and control system for unmanned logistics vehicles, including:

[0117] The acquisition module is used to acquire driving sample data, wherein the driving sample data includes perception data and control data. The perception data includes image sequences from multiple high-definition cameras, radar data, and the vehicle's own motion state information; the control data includes steering wheel angle and acceleration / deceleration commands.

[0118] The imitation learning module is used to construct training data using the steering wheel angle and the acceleration / deceleration command as labels, and to train a pre-constructed neural network model based on the training data to obtain a preliminary control model that represents vehicle perception information and control commands. The neural network model includes an input layer for inputting perception data, a backbone network for extracting features from the perception data, a fusion layer for fusing the extracted features to obtain fused features, a time-series processing layer for constructing time-series feature vectors based on the fused features, and an output layer for outputting control data based on the time-series feature vectors.

[0119] The environment simulation module is used to construct a simulation environment for unmanned logistics vehicles, generate multiple target working conditions for unmanned logistics vehicles in the simulation environment, and extract target perception data from the target working conditions.

[0120] The reinforcement learning module is used to input the target perception data into the preliminary control model to obtain the control data of the target operating condition; and to perform reinforcement learning on the preliminary control model based on the pre-constructed evaluation function and the control data of the target operating condition to obtain the final control model.

[0121] The control module is used to perform end-to-end control of the unmanned logistics vehicle based on the final control model.

[0122] This application presents a dedicated end-to-end planning and control method and system for unmanned logistics vehicles. Employing an end-to-end design, this application integrates perception, decision-making, planning, and control functions into a unified model, simplifying the system architecture, reducing development complexity, and improving system real-time performance and consistency. This application innovatively combines imitation learning and reinforcement learning methods. Imitation learning is used for model initialization, enabling the model to quickly acquire a safe and gentle driving style; reinforcement learning is used for policy optimization, allowing the model to surpass the level of human drivers and discover better driving strategies. This collaborative optimization approach balances training efficiency and performance. Specifically designed for unmanned logistics vehicles, this application fully considers their unique characteristics in urban delivery scenarios. Targeted optimizations have been made in model design, training data selection, and reward function design to better adapt to the application requirements of unmanned logistics vehicles.

[0123] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0124] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0125] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0126] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0127] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0128] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0129] In the above embodiments, although the present application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. The embodiments of the present application are intended to cover all such substitutions, modifications, and variations falling within the broad scope thereof.

[0130] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by this application.

Claims

1. A dedicated end-to-end planning and control method for unmanned logistics vehicles, characterized in that, Including the following steps: Acquire driving sample data, wherein the driving sample data includes perception data samples and control data samples. The perception data samples include image sequences from multiple high-definition cameras, radar data, and the vehicle's own motion state information; the control data samples include steering wheel angle and acceleration / deceleration commands. Training data is constructed using the perceived data samples as labels, and a pre-constructed neural network model is trained based on the training data to obtain a preliminary control model that represents vehicle perception information and control commands. The neural network model includes an input layer for inputting the perceived data, a backbone network for extracting features from the perceived data, a fusion layer for fusing the extracted features to obtain fused features, a time-series processing layer for constructing time-series feature vectors based on the fused features, and an output layer for outputting control data based on the time-series feature vectors. Constructing a simulation environment for unmanned logistics vehicles and generating multiple target operating conditions for unmanned logistics vehicles within the simulation environment, as well as extracting target perception data from the target operating conditions; constructing the simulation environment for unmanned logistics vehicles and generating multiple target operating conditions within the simulation environment includes: building a simulation environment model based on a physics engine, wherein the simulation environment model includes a road network, traffic elements, and a weather system; inputting multiple frames of real driving data and calculating the occurrence probability of various operating conditions in the real driving data, wherein the real driving data includes sensor logs and control commands; filtering high-risk operating conditions with an occurrence probability lower than a preset probability threshold and extracting rules for the high-risk operating conditions; and generating random operating conditions based on the simulation environment model, and using the random operating conditions that conform to the rules of the high-risk operating conditions as target operating conditions; The process involves inputting the target perception data into a preliminary control model to obtain control data for the target operating condition; performing reinforcement learning on the preliminary control model based on a pre-constructed evaluation function and the control data for the target operating condition to obtain a final control model; and performing reinforcement learning on the preliminary control model based on a pre-constructed evaluation function and the control data for the target operating condition to obtain a final control model, including: simulating the operation of an unmanned logistics vehicle in a simulation environment model according to the control data for the target operating condition, and extracting various parameters and target events during the operation; calculating an evaluation score for the operation of the unmanned logistics vehicle based on a pre-constructed evaluation function, a penalty function, various parameters, and target events, and adjusting the parameters of the preliminary control model based on the evaluation score to obtain the final control model. The unmanned logistics vehicle is controlled end-to-end based on the final control model.

2. The dedicated end-to-end planning and control method for unmanned logistics vehicles according to claim 1, characterized in that, The backbone network includes a ResNet-50 network for extracting visual features from image sequences and a PointNet network for extracting 3D geometric features from LiDAR point cloud data, wherein both the ResNet-50 network and the PointNet network are obtained through pre-training.

3. The dedicated end-to-end planning and control method for unmanned logistics vehicles according to claim 1, characterized in that, The pre-built neural network model is trained based on training data to obtain a preliminary control model that represents vehicle perception information and control commands, including: S1, input the sensor data samples in the training data into the input layer, and obtain the corresponding prediction control data from the output layer; S2, calculate the loss between the predicted control data and the control data samples in the training data based on the pre-built loss function, and adjust the parameters of the neural network model based on the loss and the gradient descent method; S3. Repeat steps S1-S2 until training is complete, and a preliminary control model representing vehicle perception information and control commands is obtained.

4. The dedicated end-to-end planning and control method for unmanned logistics vehicles according to claim 3, characterized in that, The mathematical expression for the loss function is: In the formula, Indicates loss, This represents the number of sample frames in each training round. Indicates the frame number of the training sample. Indicates the weighting coefficient. This represents the steering wheel angle in the predictive control data. This indicates acceleration / deceleration commands in the predictive control data. This indicates the steering wheel angle in the control data sample. This indicates acceleration / deceleration commands in the control data sample.

5. A dedicated end-to-end planning and control method for unmanned logistics vehicles according to claim 1, characterized in that, The mathematical expression for the evaluation function is: In the formula, Indicates the evaluation score. Indicates the reward score. Indicates the penalty score. Represents time frame The corresponding state, Represents time frame The corresponding action, This indicates a reward for achieving the goal. This represents the indicator function for achieving the goal. Indicates an efficiency reward. Represents time frame The corresponding speed, Indicates a comfort reward. Indicates the comfort threshold. This represents the step size penalty coefficient. Indicates the intensity of the collision penalty. This represents the collision indication function. Indicates the severity of penalties for violations. This indicates a violation indicator function.

6. A dedicated end-to-end planning and control system for unmanned logistics vehicles, characterized in that, include: The acquisition module is used to acquire driving sample data, wherein the driving sample data includes perception data and control data. The perception data includes image sequences from multiple high-definition cameras, radar data, and the vehicle's own motion state information; the control data includes steering wheel angle and acceleration / deceleration commands. The imitation learning module is used to construct training data using the steering wheel angle and the acceleration / deceleration command as labels, and to train a pre-constructed neural network model based on the training data to obtain a preliminary control model that represents vehicle perception information and control commands. The neural network model includes an input layer for inputting perception data, a backbone network for extracting features from the perception data, a fusion layer for fusing the extracted features to obtain fused features, a time-series processing layer for constructing time-series feature vectors based on the fused features, and an output layer for outputting control data based on the time-series feature vectors. An environment simulation module is used to construct a simulated environment for unmanned logistics vehicles, generate multiple target operating conditions for unmanned logistics vehicles within the simulated environment, and extract target perception data from the target operating conditions. The construction of the simulated environment for unmanned logistics vehicles and the generation of multiple target operating conditions within the simulated environment include: constructing a simulated environment model based on a physics engine, wherein the simulated environment model includes a road network, traffic elements, and a weather system; inputting multiple frames of real driving data and calculating the probability of occurrence of various operating conditions in the real driving data, wherein the real driving data includes sensor logs and control commands; filtering high-risk operating conditions with a probability below a preset probability threshold and extracting rules for the high-risk operating conditions; and generating random operating conditions based on the simulated environment model, and using the random operating conditions that conform to the rules of the high-risk operating conditions as target operating conditions. The reinforcement learning module is used to input the target perception data into the preliminary control model to obtain control data for the target operating condition; perform reinforcement learning on the preliminary control model based on a pre-constructed evaluation function and the control data for the target operating condition to obtain a final control model; and perform reinforcement learning on the preliminary control model based on a pre-constructed evaluation function and the control data for the target operating condition to obtain a final control model, including: simulating the operation of the unmanned logistics vehicle in the simulation environment model according to the control data for the target operating condition, and extracting various parameters and target events during the operation; calculating the evaluation score of the unmanned logistics vehicle's operation process based on the pre-constructed evaluation function, penalty function, various parameters, and target events, and adjusting the parameters of the preliminary control model based on the evaluation score to obtain the final control model; The control module is used to perform end-to-end control of the unmanned logistics vehicle based on the final control model.

7. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images