Path planning method and system based on imitation learning and verification method

Abstract

The present disclosure provides a path planning system and method based on imitation learning, and a verification method, the path planning system comprising: a feature extractor configured to extract feature vectors from a first data set based on a driving command and form a combined vector; and a trajectory generator configured to pass the combined vector through a fully connected layer and a long short term memory network to generate a predicted trajectory and uncertainty.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving, and in particular, to a path planning method and system based on imitation learning and a verification method. Background Art

[0002] Globally, approximately 1.3 million people die on our roads each year, with nearly 94% of these deaths related to human error. Autonomous vehicles (AVs) can play a major role in reducing this number, while easing traffic congestion, reducing air pollution and lowering transportation energy consumption by up to 90%.

[0003] To achieve autonomous driving, the vehicle needs to perceive and understand the surrounding environment, and then use the extracted information to generate a collision-free trajectory to the target location. However, due to the high complexity of the environment and the inability to test the system in a wide variety of scenarios, achieving general autonomous driving remains a challenge, especially in realistic urban environments where trajectory generation is crucial. In this framework, the solution trajectory can be expressed as a time-parameterized function π(t):[0,T]→X, where T is the planning horizon and X is the configuration space of the vehicle. Planning and decision-making methods for autonomous vehicles can be divided into three major categories: traditional sequential planning, end-to-end control, and end-to-end planning, such as Figure 1As shown. Traditional methods are usually constructed as a pipeline consisting of independent components (such as perception, planning, and control) that link sensor inputs with actuator outputs. However, this approach has some major disadvantages: 1) Each component needs to be specified and tuned separately, and it is difficult to generalize to various scenarios due to the complex interdependencies between them; 2) The cost function in most works must be carefully designed with complex parameters; 3) It is prone to error propagation. Accumulated errors between unstable modules may lead to false positives or false negatives. For example, in the unfortunate Tesla accident in 2016, an error in the perception module misclassified the white trailer as the sky, and the error propagated along the pipeline until it caused a failure. Recently, a new paradigm based on deep learning integrates perception, planning, and control, and has achieved impressive results in vehicle navigation. This approach, called end-to-end control, formulates the driving problem as learning a mapping from perception inputs (such as RGB images) directly to vehicle control outputs (such as steering angle and throttle). Another similar paradigm is end-to-end planning, which maps observations into future trajectories. Deep learning enables these systems to perform well and generalize to new environments by self-optimizing (learning) their behavior from data, allowing the rule-based parameters to be freely adjusted in all foreseeable scenarios. However, these works have several shortcomings: 1) the ability to recover from eccentricity and deviation errors is not considered or verified, which is critical for real-world deployment; 2) most works only evaluate their models in simple environments with limited complexity in terms of dynamic obstacles and weather / lighting conditions; 3) In addition, generalization performance to different types of vehicles with changing physical characteristics is rarely evaluated; 4) the uncertainty of the generated driving actions or trajectories is ignored, making it impossible for users to check the correctness of the network output, which is crucial in safety-critical driving tasks; 5) Finally, many works heavily rely on post-processed detailed environment maps for decision making, which are expensive to create and transfer to new scenarios. Summary of the invention

[0004] On the one hand, the present disclosure provides a path planning system based on imitation learning, comprising: a feature extractor, which is configured to extract feature vectors from a first data set based on driving commands and form a combined vector; and a trajectory generator, which is configured to pass the combined vector through a fully connected layer and a long short-term memory network to generate a predicted trajectory and uncertainty.

[0005] In an embodiment, the feature extractor includes: an image module configured to extract and output a first feature vector from a plurality of camera images included in the first data set; a balancing module configured to expand the dimension of a motion vector to form and output a second feature vector, wherein the motion vector comes from motion information included in the first data set; and a first cascade module configured to cascade the first feature vector and the second feature vector at each historical time step to form a combined vector for each of a plurality of motion states.

[0006] In an embodiment, the cascade module performs the cascade by the following formula:

[0007]

[0008] Where <·> represents cascade operation, F i (I k ) represents the kth image output by the image module, F b (m k ) represents the kth motion state of the output of the balancing module, and represents the kth combination vector.

[0009] In an embodiment, the image module is implemented by any one of a convolutional neural network, a multilayer perceptron, and a graph neural network.

[0010] In an embodiment, the image module is implemented by MobileNet-V2.

[0011] In an embodiment, the balancing module is implemented by a first fully connected layer.

[0012] In an embodiment, the first data set includes the camera image captured by a camera installed on the top of the autonomous driving vehicle and the motion information of the autonomous driving vehicle obtained through GPS, and the motion information of the autonomous driving vehicle includes the position and speed of the autonomous driving vehicle.

[0013] In an embodiment, the camera images and the motion information are configured to be used for training and testing the path planning system.

[0014] In an embodiment, the trajectory generator comprises:

[0015] a second concatenation module configured to concatenate the combined vectors for the plurality of historical time steps with each other to form a first concatenation vector;

[0016] A second fully connected layer is configured to generate attention information (a1, a2, ..., a12) by processing the first cascade vector, wherein the attention information satisfies

[0017] A three-layer long short-term memory network configured to receive a product of a combination vector and attention information for each of the plurality of historical time steps to generate a spatiotemporal feature of a surrounding environment;

[0018] A third fully connected layer, configured to generate the predicted trajectory at a future time according to the spatiotemporal features; and

[0019] A fourth fully connected layer is configured to generate the uncertainty of the future time according to the spatiotemporal features.

[0020] In an embodiment, the driving command includes a left turn command, a right turn command, and a keep straight command.

[0021] On the other hand, the present disclosure provides a path planning method based on imitation learning, including: extracting feature vectors from a first data set based on driving commands and forming a combination vector; and passing the combination vector through a fully connected layer and a long short-term memory network to generate a predicted trajectory and uncertainty.

[0022] In an embodiment, the step of extracting feature vectors from a first data set and forming a combined vector based on a driving command includes: extracting a first feature vector from a plurality of camera images included in the first data set and outputting it; expanding the dimension of a motion vector to form a second feature vector and outputting it, wherein the motion vector comes from motion information included in the first data set; and concatenating the first feature vector and the second feature vector at each historical time step to form a combined vector for each of a plurality of historical time steps.

[0023] In an embodiment, the cascading is performed by the following formula:

[0024]

[0025] Where <·> represents cascade operation, F i (I k ) represents the kth image output, F b (m k ) represents the kth motion state of the output, and f c k represents the kth combination vector.

[0026] In an embodiment, the path planning method also includes collecting camera images through a camera installed on the top of the autonomous driving vehicle; obtaining motion information of the autonomous driving vehicle through GPS, the motion information of the autonomous driving vehicle including the position and speed of the autonomous driving vehicle, and forming the first data set based on the camera image and the motion information.

[0027] In an embodiment, the path planning method further includes cascading the combined vectors for the multiple historical time steps to form a first cascade vector; generating attention information (a1, a2, ..., a12) by processing the first cascade vector, wherein the attention information satisfies Receive the product of the combined vector of each of the multiple historical time steps and the attention information to generate the spatiotemporal characteristics of the surrounding environment; generate the predicted trajectory of the future time according to the spatiotemporal characteristics; and generate the uncertainty of the future time according to the spatiotemporal characteristics.

[0028] On the other hand, the present disclosure provides a verification method for a path planning system, wherein the path planning system is a path planning system according to an embodiment of the present disclosure, and the verification method includes: adding noise or disturbance to at least one of the left turn command and the right turn command, the noise or disturbance representing a random amount of corner added at a first time interval, the random amount of corner lasting for a second duration, and the second duration is less than the first time interval; inputting the predicted trajectory and the uncertainty into an autonomous driving vehicle equipped with the path planning system, so that the autonomous driving vehicle travels according to the predicted trajectory and the uncertainty; recording the situation where the autonomous driving vehicle reaches the end point without collision as a success, and counting the success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Illustrative, non-limiting example embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0030] Figure 1 Diagram of traditional planning and decision-making approaches for autonomous vehicles.

[0031] Figure 2 is an architectural diagram of a path planning system based on imitation learning according to an embodiment of the present disclosure.

[0032] Figure 3 The attention distribution in two cases according to an embodiment of the present disclosure is shown.

[0033] Figure 4 The sample qualitative results of the method of the present disclosure in 5 typical environments are shown.

[0034] Figure 5 Some driving snapshots according to the method of the present disclosure are shown in FIG.

[0035] Figure 6 A comparison of models M1 to M3 and VTGNet (M0) of the present application is shown. DETAILED DESCRIPTION

[0036] Traditional autonomous driving methods are implemented through many building blocks of perception, planning, and control, which make it difficult to generalize to various scenarios due to complex assumptions and interdependencies. Recently, end-to-end driving methods have emerged that perform well and generalize to new environments by learning directly from the data provided at the output. However, many existing methods ignore checking the confidence of driving behavior and the ability to recover from driving errors. In this context, the present disclosure develops an uncertainty-aware end-to-end trajectory generation method based on imitation learning. It can extract spatiotemporal features from sensor inputs and then generate collision-free trajectories for the next few seconds. Maneuvers and reference lanes are introduced as conditions of the planning model, which effectively reduces the workload of parameter tuning and data acquisition.

[0037] This paper combines the advantages of imitation learning and reinforcement learning, and uses neural networks to generate collision-free planning results for the next few seconds from the original or after a small amount of feature extraction of sensor data. This method is implemented in low-visibility scenarios in the factory area, is quickly deployed through imitation learning, and improves data utilization efficiency and autonomous driving safety by outputting planning result uncertainty, inputting operation instructions and setting reference lines.

[0038] This disclosure proposes a novel imitation learning-based path planning method that takes into account uncertainty, maneuverability, and reference lane priors.

[0039] This disclosure proposes an end-to-end driving network for AVs based on imitation learning, which directly takes raw camera images as input instead of using highly engineered environment maps to generate trajectories.

[0040] This disclosure proposes a new benchmark, AddNoise, to examine the driving ability to recover from off-center and off-direction errors.

[0041] This paper compares VTGNet (Vision-based Trajectory Generation Network for Autonomous Vehicles) with different baselines and demonstrates its advantages in various weather / lighting conditions (e.g., snow, rain, and fog) on ​​a large-scale driving dataset and a high-fidelity driving simulator with dynamic obstacles.

[0042] This paper demonstrates VTGNet's cross-scenario / platform and error-resilient driving capabilities on the AddNoise benchmark.

[0043] This paper verifies the validity of the estimated uncertainty through ablation studies and closed-loop driving tests

[0044] The planning control system disclosed herein includes a deep neural network module and a data set generation and analysis module. In an embodiment, the deep neural network module includes a feature extractor, a trajectory generator, and a loss function. In an embodiment, the feature extractor may include an encoder, and the trajectory generator may include a decoder.

[0045] The architecture of the deep neural network module is as follows Figure 2 As shown in Figure 2, a feature extractor (e.g., encoder) conditioned on waypoints or primitive actions first encodes the sensor input into a latent feature vector. Then a decoder (e.g., trajectory generator) generates future expected states and their covariances.

[0046] At each time step t, the input of the deep neural network module includes: camera images, such as the camera images in the past 12 frames It = {I 1 ,...,I 12}; and the motion information of the autonomous driving vehicle, such as the motion information of the autonomous driving vehicle in the past 12 frames mt = {m 1 ,...,m 12}, which may include the trajectory of the autonomous vehicle. The output of the deep neural network module is the predicted trajectory (also referred to as the generated trajectory) Tt and the trajectory uncertainty σt, which depends on the high-level driving command ct and the observations (e.g., the camera images and motion information mentioned above). Encoders include but are not limited to convolutional neural networks, multilayer perceptrons, and graph neural networks.

[0047] In an embodiment, the preview range of the generated trajectory can be set to 3s (ie, 22 frames), which is twice the average braking reaction time (RT) of human perception of 1.5 seconds. Therefore, the generated trajectory can be expressed as Tt = {T 1 ,T 2 …T 22 Here m k ∈R 3 And T k ∈R 3 . m k and T k It can contain the velocity and position in the current subject frame, where the position can be expressed as coordinates in a plane, for example, m k and T k can be represented by (x, y, v). Similarly, trajectory uncertainty can be expressed as σt = {σ 1 ,σ 2 …σ 22}, where σ k ∈R 3 , contains the uncertainty in velocity and x, y position, i.e., the covariance of x, y, and v.

[0048] In the implementation, three sub-networks may be included, which can be selected by a given driving command ct to perform different tasks, namely, turn left, turn right, and keep going straight. In practice, the network is trained to predict the logarithmic variance logσ 2 , because it is numerically more stable than the regression variance.

[0049] Deep Neural Network Module

[0050] Feature Extractor: Once the driving command ct selects a specific branch, the images It are first processed individually and then by an image module Fi implemented with a CNN (Convolutional Neural Network), which extracts a feature vector f from each image i ∈R 512 . In an embodiment, the image module can also be implemented by a multi-layer perceptron and a graph neural network. In an embodiment, MobileNet-V2 is used as Fi to extract the visual feature vector. MobileNet-V2 consists of 17 bottleneck convolution (Bottleneck Conv) blocks, and each block uses a depth-separable convolution instead of a full convolution operator. Then, each historical motion vector m is expanded by a balancing module Fb implemented with a fully connected (FC) layer k Dimension (For example, from 3 to 128) to balance the influence of the visual feature vector and the motion feature vector. The outputs of the image module and the balancing module are then concatenated together at each historical time step to become a combined vector of length 640, expressed as follows:

[0051]

[0052] Where <·> represents cascade operation, F i (I k ) is the first eigenvector represented by the kth image output by the image module, F b (m k ) is the second eigenvector represented by the kth motion state output by the balancing module, and f c k is the kth combination vector.

[0053] In an embodiment, a cascade operation may be performed by a cascade module. Specifically, the first feature vector and the second feature vector at the k-th time step may be connected end to end to form a k-th combination vector.

[0054] Trajectory generator: In order to better utilize the sequential information, we first transform the set f c ={f c 1 ,...,f c 12} are cascaded and processed through the FC layer to generate attention a1,...,a12 to measure the relative importance of past information. Figure 3 The attention distribution in two cases according to an embodiment of the present disclosure is shown. Figure 3 (a) represents the turning situation, (b) represents the driving along the lane, the horizontal axis represents the time step (or frame), and the vertical axis represents the attention level. For example, for the historical information in the past 12 frames (for example, camera images and motion information of the autonomous driving vehicle), the camera images and / or motion information obtained when the autonomous driving vehicle turns can be given greater attention, because the autonomous driving vehicle contains more information about the surrounding environment at the turn. For example, when the historical information includes image frames when the autonomous driving vehicle is driving along the lane on the road, the most recent camera images and / or motion information can be given greater attention, and the distant camera images and / or motion information can be given less attention.

[0055] According to an embodiment of the present disclosure, a softmax function is used to activate the last layer so that Then, the modulated feature {a1f c 1 ,...,a12f c 12} Input three layers of LSTM (Long Short-Term Memory Network) to generate the spatiotemporal features f of the surrounding environment st ∈R 256 , represents the feature vector (including position and velocity) at each future time step. It is further compressed through 2 FC layers to obtain two vectors of size 22×3, which represent the trajectory Tt and uncertainty σt generated at time step t for the next 22 frames.

[0056] Loss function: As the name suggests, the basic idea of ​​imitation learning is to train a network that imitates human behavior, which can be solved by supervised learning. Let T' t represents the trajectory at sample time t in the training dataset, is the reference data, θ is the learnable parameter of the neural network F, and represents the weight in the neural network. Then, the optimal parameter θ can be trained by minimizing the average prediction loss * :

[0057]

[0058] where L is the per-sample loss at sample time t, I t 、m t 、c t They represent the camera image acquired at time t, the motion information of the autonomous driving vehicle, and the driving command, respectively. t 、m t 、c t) is the output data of the neural network. According to the training method for data uncertainty, L is defined as

[0059]

[0060] Dataset generation and analysis module

[0061] In an embodiment, a camera can be mounted on top of an autonomous vehicle to obtain images in front. In an embodiment, motion information of the autonomous vehicle can be obtained by fusion of GPS / inertial solutions. In an embodiment, VTGNet can be trained and tested with camera images and motion information.

[0062] Camera images (e.g., It) and motion information (e.g., mt) are extracted from the RobotCar dataset to train and test VTGNet. In an embodiment, the RobotCar dataset comes from images captured by a camera mounted on the top of an autonomous vehicle and the ground truth position and speed obtained by fusing a GPS / inertial solution. Therefore, data without GPS information is first filtered out, and a total of some driving routes are collected. Another issue that needs to be considered is data balance. Specifically, it is necessary to balance the distribution of driving samples under different driving situations (e.g., following a vehicle, decelerating to avoid collisions, etc.) and under different environmental conditions.

[0063] Then, in this work, these data are reconstructed for training and testing, requiring each image to be equipped with the trajectory of the autonomous vehicle within the past 1.5s and the future 3s.

[0064] First, the ground truth Universal Transverse Mercator (UTM) position and velocity sequences are interpolated into image timestamps recorded at 15Hz, and then their coordinates are transformed into the vehicle coordinate system relative to the driver. Thereafter, each image is manually labeled with the corresponding intention command indicating the driving direction of the autonomous vehicle based on the ground truth trajectory. For example, when the autonomous vehicle approaches an intersection and plans to turn right, the command is changed from keep straight to turn right. When the turn is completed, the command to keep straight is restored. Finally, the original image is cropped to an image with a specific number of pixels (e.g., 1247*384) by removing the sky and awning at the top / bottom of the image, because these areas have less information for scene understanding. Thereby improving the efficiency of the learning process.

[0065] Figure 4The sample qualitative results of the method disclosed in this paper in 5 typical environments are shown. In the sample qualitative results of the model VTGNet disclosed in this paper in various environments, column (a) shows the results of left turn, column (d) shows the results of right turn, column (b) shows the results of following the car, and column (c) shows the results of lane keeping. The solid line represents the planning result, the dotted line represents the actual situation, and the shaded area marked with A represents the uncertainty of the estimate.

[0066] The last row shows a challenging scenario: rainy night. The halo effect in this scene is sometimes noticeable due to raindrops on the camera lens. It is worth noting that this scene was not included in the training set and was only used to qualitatively test the generalization performance. In general, it can be seen that the VTGNet of the present disclosure is able to generate collision-free trajectories under various lighting and weather conditions. The performance is also reliable in some difficult scenarios. For example, in Night - (d), the bus in the left area exhibits severe motion artifacts, and VTGNet generates a safe right turn trajectory. In Rainy Night - (c), the motion artifacts are more severe, but the generated results are still reasonable and similar to the ground truth (with higher uncertainty). In an embodiment, a highly balanced training dataset with high diversity in lighting and weather conditions helps to achieve this generalizable performance.

[0067] Closed-loop experiments were conducted in the CARLA driving simulator to further demonstrate its performance. CARLA is an open source simulator that provides a high-fidelity dynamic world and different autonomous vehicles with realistic physics.

[0068] Figure 5 Some driving snapshots according to the method of the present disclosure are shown in , showing the closed-loop evaluation results of VTGNet in the AddNoise benchmark. The AddNoise benchmark refers to adding noise or disturbance to test the ability of the path planning system according to the present disclosure to return to normal after being disturbed. Specifically, the noise or disturbance can be a random amount of turning angle added to the steering control instruction of the autonomous driving vehicle at a first time interval (e.g., every 5s) for a second duration (e.g., 0.2s to 1.0s), and the second duration can be less than the first time interval.

[0069] exist Figure 5 In Figure 2, the first row shows some normal driving behaviors: (a, b) turning at an intersection, (c) vehicle following, and (d) stopping for pedestrians. The second row shows the recovery behaviors for off-center / direction errors under different weather conditions. The black line represents the planning result, and the shaded area indicated by A represents the estimated uncertainty.

[0070] This shows that the disclosed method is not only able to adapt to different weather conditions, but also has the ability to perform reactive driving, such as following a vehicle ( Figure 5 -(c)) and parking for pedestrians ( Figure 5 -(d)). In addition, Figure 5 -(b) shows the recovery behavior, where the ego vehicle is autonomously recovering from the center deviation / direction deviation error disturbed by periodic steering noise.

[0071] Comparing the VTGNet disclosed in the present invention with the CNN-FC model, CNN-LSTM model, and CNNState-FC model in the related art, VTGNet exhibits good trajectory smoothness and produces more accurate trajectories than the CNN-FC model, CNN-LSTM model, and CNNState-FC model, that is, it performs best in terms of average velocity and acceleration error.

[0072] In addition, VTGNet achieves stable results under different conditions and performs best in snowy environments.

[0073] In addition, by comparing the VTGNet design (M0) of the present invention with models M1 to M3, M1 removes the uncertainty estimation in M0, M2 removes the calculation of attention from M1, and M3 is a variation of M2 that uses two LSTM modules to process image sequences and motion sequences respectively. Figure 6 The results of models M1 to M3 and VTGNet (M0) of the present application are shown. The dotted line represents the planned result, the solid line represents the actual situation, and area B represents the estimated uncertainty. The results show that the performance gradually decreases from M0 to M3, which proves that the use of uncertainty estimation in the present application improves the prediction performance. In addition, processing multiple types of information together through a single LSTM module helps to generate more reasonable outputs. Therefore, the VTGNet design of the present disclosure is accurate and the trajectory is smooth.

[0074] In an embodiment, a verification method for a path planning system is also provided. The verification method includes: obtaining a predicted trajectory and uncertainty when adding noise or disturbance as described above; inputting the predicted trajectory and uncertainty to an autonomous driving vehicle equipped with a path planning system so that the autonomous driving vehicle drives according to the predicted trajectory and uncertainty; recording the case where the autonomous driving vehicle reaches the end point without collision as a success, and counting the success rate.

[0075] This disclosure introduces uncertain maneuvers and reference lanes into path planning based on imitation learning for the first time. This method greatly reduces the number of manual adjustment processes and training data sets and has higher reliability.

[0076] The foregoing is an illustration of example embodiments and should not be construed as limiting thereof. Although some example embodiments have been described, those skilled in the art will readily appreciate that many modifications may be made in the example embodiments without substantially departing from the novel teachings and advantages of the example embodiments. Therefore, all such modifications are intended to be included within the scope of the example embodiments defined in the claims. Therefore, it should be understood that the foregoing is an illustration of various example embodiments and should not be construed as being limited to the specific example embodiments disclosed, and modifications to the disclosed example embodiments and some example embodiments are intended to be included within the scope of the appended claims.

Claims

1. A path planning system based on imitation learning, comprising: a feature extractor configured to extract a feature vector from the first data set based on the driving command and form a combined vector; as well as A trajectory generator is configured to pass the combined vector through a fully connected layer and a long short-term memory network to generate a predicted trajectory and uncertainty.

2. The path planning system according to claim 1, wherein: The feature extractor comprises: An image module, configured to extract and output a first feature vector from a plurality of camera images included in the first data set; a balancing module configured to expand the dimension of a motion vector to form and output a second feature vector, wherein the motion vector is derived from motion information included in the first data set; and A first concatenation module is configured to concatenate the first feature vector and the second feature vector at each historical time step to form the combined vector for each of a plurality of historical time steps.

3. The path planning system according to claim 2, wherein: The cascade module performs the cascade by the following formula: Where <·> represents cascade operation, F i (I k ) represents the kth image output by the image module, F b (m k ) represents the kth motion state of the output of the balancing module, and represents the kth combination vector.

4. The path planning system according to claim 2, wherein: The image module is implemented by any one of a convolutional neural network, a multilayer perceptron and a graph neural network.

5. The path planning system according to claim 2, wherein: The image module is implemented by MobileNet-V2.

6. The path planning system according to claim 2, wherein: The balancing module is implemented by a first fully connected layer.

7. The path planning system according to claim 2, wherein: The first data set includes the camera image captured by the camera installed on the top of the autonomous driving vehicle and the motion information of the autonomous driving vehicle obtained through GPS, and the motion information of the autonomous driving vehicle includes the position and speed of the autonomous driving vehicle.

8. The path planning system according to claim 2, wherein: The camera images and the motion information are configured to be used for training and testing the path planning system.

9. The path planning system according to claim 3, wherein: The trajectory generator comprises: a second concatenation module configured to concatenate the combined vectors for the plurality of historical time steps with each other to form a first concatenation vector; A second fully connected layer is configured to generate attention information (a1, a2, ..., a12) by processing the first cascade vector, wherein the attention information satisfies A three-layer long short-term memory network configured to receive a product of a combination vector and attention information for each of the plurality of historical time steps to generate a spatiotemporal feature of a surrounding environment; A third fully connected layer, configured to generate the predicted trajectory at a future time according to the spatiotemporal features; and A fourth fully connected layer is configured to generate the uncertainty of the future time according to the spatiotemporal features.

10. The path planning system according to claim 1, wherein: The driving commands include a left turn command, a right turn command, and a keep straight command.

11. A path planning method based on imitation learning, comprising: Extracting feature vectors from the first data set based on the driving command and forming a combined vector; as well as The combined vector is passed through a fully connected layer and a long short-term memory network to generate a predicted trajectory and uncertainty.

12. The path planning method according to claim 11, wherein: The step of extracting a feature vector from the first data set based on the driving command and forming a combined vector comprises: Extracting a first feature vector from a plurality of camera images included in the first data set and outputting the first feature vector; Expanding the dimension of a motion vector to form a second feature vector and outputting the second feature vector, wherein the motion vector is derived from motion information included in the first data set; and The first feature vector and the second feature vector are concatenated at each historical time step to form a combined vector for each of a plurality of historical time steps.

13. The path planning method according to claim 12, wherein: The cascade is performed by the following formula: Where <·> represents cascade operation, F i (I k ) represents the kth image output, F b (m k ) represents the kth motion state of the output, and represents the kth combination vector.

14. The path planning method according to claim 11, further comprising: The camera image is collected by a camera mounted on the top of the autonomous vehicle; obtaining motion information of the autonomous driving vehicle through GPS, wherein the motion information of the autonomous driving vehicle includes the position and speed of the autonomous driving vehicle, and The first data set is formed based on the camera image and the motion information.

15. The path planning method according to claim 12, further comprising: concatenating the combined vectors for the plurality of historical time steps with each other to form a first concatenated vector; The attention information (a1, a2, ..., a12) is generated by processing the first cascade vector, and the attention information satisfies receiving a product of the combination vector and attention information for each of the plurality of historical time steps to generate a spatiotemporal feature of the surrounding environment; Generate the predicted trajectory for the future time according to the spatiotemporal features; as well as The uncertainty at the future time is generated according to the spatiotemporal characteristics.

16. A verification method for a path planning system, wherein the path planning system is the path planning system according to claim 10, the verification method comprising: adding noise or disturbance to at least one of the left turn command and the right turn command, the noise or disturbance representing a random amount of turning angle added at a first time interval, the random amount of turning angle lasting for a second duration, and the second duration being less than the first time interval; Inputting the predicted trajectory and the uncertainty to an autonomous driving vehicle equipped with the path planning system, so that the autonomous driving vehicle drives according to the predicted trajectory and the uncertainty; The situation where the autonomous driving vehicle reaches the end point without collision is recorded as a success, and the success rate is counted.

Images