Lane changing scene automatic driving planning method based on curve path space

Abstract

The invention discloses a curve path space-based lane changing scene automatic driving planning method, which comprises the following steps of: firstly, acquiring scene information, forming a scene information vector, acquiring a shortest feasible route from a current position to a destination according to the scene information, and acquiring a space coordinate point sequence of the shortest feasible route; then projecting the scene information vector to a curve path of the space coordinate point sequence to obtain scene feature information, performing high-dimensional mapping and local feature processing on the scene feature information in sequence, and executing cross attention based on a planning Transform to obtain track features; and finally, regressing the trajectory characteristics to an executable trajectory of vehicle automatic driving, executing in a simulator, collecting results, generating Reward and feeding back to a planner, training and updating a parameterized planner network by using a continuous action space Actor-critic algorithm of DDPG until convergence, and then receiving scene information in real time in an actual automatic driving vehicle, so as to realize real-time automatic driving of the vehicle. And an optimal lane changing track is generated, and automatic driving planning of the lane changing scene is realized.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving planning technology, and in particular to an autonomous driving planning method for lane-changing scenarios based on curved path space. Background Technology

[0002] In the development of autonomous driving technology, the rational planning of lane-changing scenarios is crucial for achieving efficient and safe driving. During operation, vehicles need to dynamically decide whether and how to change lanes based on complex and ever-changing traffic environments. This involves the accurate processing and rapid decision-making of a large amount of information. Factors such as the speed, position, and driving intentions of surrounding vehicles, road curvature, lane markings, traffic signs and signal light status, and the vehicle's own dynamic limitations all intertwine, making lane-changing planning an extremely challenging task.

[0003] Rule-based planning methods rely on pre-defined, fixed rules to guide lane-changing decisions. However, the diversity and uncertainty of real-world traffic scenarios make it difficult for rules to cover all possible situations. When faced with special road conditions, unexpected situations, or complex traffic interactions, rule-based methods often fail to make flexible and optimal decisions, potentially leading to low vehicle efficiency or increased safety risks. Sampling-based planning methods search for feasible lane-changing trajectories by conducting extensive sampling within a certain range. While theoretically, they can find relatively optimal solutions, but the computational complexity is extremely high. In autonomous driving scenarios with strict real-time requirements, the sampling process may consume too much time, resulting in the inability to provide timely and effective lane-changing solutions for the vehicle. Especially during heavy traffic, the optimal lane-changing opportunity may be missed, affecting traffic flow and safety. Some applications of deep reinforcement learning in autonomous driving lane-changing planning fail to fully utilize the characteristics of curved path spaces. During path planning, the processing of key information such as road curvature is insufficient, resulting in generated lane-changing trajectories that are not ideal under actual curved road conditions, potentially affecting vehicle stability, comfort, and safety.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide an autonomous driving planning method based on the curve path space. This method aims to solve the problem that in autonomous driving scenarios with strict real-time requirements, the sampling process may take too long, resulting in the inability to provide the vehicle with an effective lane-changing plan in a timely manner. Especially in busy traffic, the optimal lane-changing opportunity may be missed, affecting traffic flow and safety.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: An autonomous driving planning method for lane-changing scenarios based on curved path space, comprising: Step 1: Input the autonomous driving scenario, obtain the scenario information, preprocess the scenario information to form a scenario information vector, obtain the shortest feasible route from the current location to the destination based on the scenario information, and then perform discretization processing to obtain the spatial coordinate point sequence of the shortest feasible route. Step 2: Project the scene information vector onto the curve path of the spatial coordinate point sequence to obtain scene feature information. Perform high-dimensional mapping and local feature processing on the scene feature information in sequence, and perform cross attention based on the planning Transformer to obtain trajectory features. Step 3: Regress the trajectory features to the executable trajectory of the vehicle's autonomous driving, execute it in the simulator, collect the results and generate a reward to feed back to the planner. Use the DDPG's continuous action space Actor-critic algorithm to train and update the parameterized planner network until convergence. Then, receive scene information in real time from the actual autonomous vehicle to generate the optimal lane-changing trajectory and realize autonomous driving planning for lane-changing scenarios.

[0007] Furthermore, in step 1, the scene information includes positioning information obtained from vehicle sensors, vehicle historical status information, scene dynamic participant information, and map information read from a high-precision map database. Specifically, the positioning information undergoes coordinate transformation and error correction to ensure the accuracy of the vehicle's position; the vehicle historical status information undergoes data cleaning and preprocessing to remove outliers and noise; for the scene dynamic participant information, sensor fusion technology is used to fuse data obtained from different sensors; the map information is organized according to a specific format, and key road features are extracted; then, the map information, positioning information, vehicle historical status information, and scene dynamic participant information are arranged sequentially according to a preset order and filled into a fixed-size vector to form a scene information vector.

[0008] Furthermore, in step 1, a graph structure is constructed based on the map information of the scene information, with road nodes represented as vertices in the graph and road connections represented as edges. The weight of the edges is determined according to the road length, traffic flow, and speed limit. Then, the Dijkstra algorithm or A* algorithm is used to search for the shortest path from the location information to the destination on the constructed graph. During the search process, the feasibility of the path is judged according to traffic rules and vehicle dynamics constraints, and infeasible paths are eliminated. The shortest path found is discretized and divided into a sequence of equally spaced spatial coordinate points according to a preset distance interval to form information.

[0009] Furthermore, in step 2, for each element of the scene information, the projection point of the element in the spatial coordinate point sequence is obtained according to the position and information of the element in the map. The projection method adopts an algorithm based on distance or geometric relationship to obtain the features of the projection point. The features of the projection point include: arc length, lateral offset, rate of curvature change, and relative positional relationship and velocity difference between the projection point and other scene element projection points. The features of the projection point are repackaged into new scene feature information in order and format.

[0010] Furthermore, in step 2, the high-dimensional mapping of scene feature information includes: A tokenized network is constructed using a convolutional neural network (CNN). The convolutional layers of the CNN are set with different kernel sizes and strides to extract features at different scales. Pooling layers are used to reduce the data dimensionality, and fully connected layers map the extracted features to a high-dimensional space. Scene feature information is input into the tokenized network and processed through multiple layers of convolution, pooling, and fully connected layers to obtain a representation in the high-dimensional feature space.

[0011] Furthermore, in step 2, local feature processing of the scene feature information includes: The design employs a local feature processing network. Convolutional layers extract local features from the high-dimensional feature space, focusing on features in different local regions by setting appropriate kernel sizes and numbers. Pooling layers further reduce data dimensionality and computational cost while preserving key feature information. Finally, fully connected layers integrate and abstract the local features to generate a scene token. Multiple local feature processing operations are performed on the scene feature information to progressively extract and aggregate higher-level local features, ultimately generating the scene token.

[0012] Furthermore, in step 2, cross-attention is performed on the scene feature information based on the planning Transformer to obtain trajectory features, including: The vehicle's historical state and spatial coordinate sequence are combined as the Query, and the remaining parts are used as the Key and Value. In the Transformer planning, a multi-head attention mechanism is used to calculate the attention weights between the Query and Key. Each head's attention calculation uses dot product attention, considering the correlation between the Query and Key vectors, assigning greater weight to parts with higher correlation. The number of heads in the multi-head attention mechanism is set to , for the ... Size Attention weight The calculation formula is: ; in It is the representation of the i-th head of the Query vector after a linear transformation. It is the representation of the i-th head of the Key vector after a linear transformation. It is the dimension of the key vector; Then, the value vector is weighted and summed to obtain the output of the i-th head. : ; Finally, the outputs of each head are concatenated to obtain the final output, namely the trajectory feature. : ; After multiple cross-attention layers, trajectory features that incorporate key scene information are obtained.

[0013] Furthermore, in step 3, the generated trajectory is executed in the simulator, the results are collected, and a reward is generated and fed back to the planner, including: Execute the generated trajectory in the simulator It monitors the relative position changes of the vehicle with surrounding vehicles and obstacles, whether collisions occur, and whether traffic rules are violated. It collects real-time environmental data after the execution trajectory and generates a reward based on preset rules, which is then fed back to the planner. The reward signal calculation formula is: ; in , and These are the weighting coefficients for safety, efficiency, and comfort, respectively. Safety items Considering the risk of collisions between the vehicle and surrounding objects and its distance from the road boundary, efficiency The comfort item measures the vehicle's speed and trajectory smoothness as it approaches its destination. Pay attention to changes in vehicle acceleration.

[0014] Furthermore, in step 3, the parameterized planner network is trained and updated using the DDPG continuous action space Actor-critic algorithm until convergence, including: The parameterized planner network is updated iteratively using the continuous action space Actor-critic algorithm of DDPG. The Actor network outputs lane change decisions and speed adjustment actions based on the current state, while the Critic network evaluates the value of the current state and actions. The policy function of the Actor network outputs the action. ; represents the value function of the Critic network. During training, the parameters are updated by minimizing the difference between the value predicted by the Critic network and the actual reward obtained, and maximizing the expected reward of the Actor network's output action. The loss function of the Critic network is... for: ; in , Discount factor; The Actor network is updated through policy gradient ascent, and the gradient calculation formula is: ; The parameters of the Actor and Critic networks are updated based on the gradient using the backpropagation algorithm.

[0015] The technical solution adopted in this invention has the following beneficial effects: This invention projects scene information onto a curved path space and performs feature calculations, enabling more accurate capture of key information in lane-changing scenarios, improving the accuracy of lane-changing decisions, and effectively reducing safety risks caused by decision-making errors. Utilizing the Transformer's cross-attention mechanism, it fully integrates different information from the scene, enhancing the model's understanding and response to complex traffic conditions, generating more reasonable and safer lane-changing trajectories, and improving vehicle stability and comfort. Through the Actor-critic architecture of the DDPG algorithm, it achieves efficient model learning and optimization, enabling precise policy search in a continuous action space, allowing the vehicle to better balance safety, efficiency, and comfort during lane changes, thus improving the overall performance of the autonomous driving system. The method of this invention is trained and validated in a simulation environment, allowing for thorough testing and optimization of the model before actual deployment, reducing risks in practical applications, and providing reliable technical support for lane-changing operations of autonomous vehicles in complex traffic scenarios. Attached Figure Description

[0016] Figure 1 A flowchart of an autonomous driving planning method for lane-changing scenarios based on curved path space provided by the present invention; Figure 2 A schematic diagram of the framework structure of an autonomous driving planning method for lane changing scenarios based on curved path space provided by the present invention; Figure 3 This is a visualization result of RDRL(ours) in an embodiment of the present invention; Figure 4 This is a visualization result of TRPO in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] This invention provides an autonomous driving planning method for lane-changing scenarios based on curved path space, specifically including: Step 1: Input the autonomous driving scenario, obtain the scenario information, preprocess the scenario information to form a scenario information vector, obtain the shortest feasible route from the current location to the destination based on the scenario information, and then perform discretization processing to obtain the spatial coordinate point sequence of the shortest feasible route. Step 2: Project the scene information vector onto the curve path of the spatial coordinate point sequence to obtain scene feature information. Perform high-dimensional mapping and local feature processing on the scene feature information in sequence, and perform cross attention based on the planning Transformer to obtain trajectory features. Step 3: Regress the trajectory features to the executable trajectory of the vehicle's autonomous driving, execute it in the simulator, collect the results and generate a reward to feed back to the planner. Use the DDPG's continuous action space Actor-critic algorithm to train and update the parameterized planner network until convergence. Then, receive scene information in real time from the actual autonomous vehicle to generate the optimal lane-changing trajectory and realize autonomous driving planning for lane-changing scenarios.

[0019] The following is in conjunction with the appendix Figure 1 The present invention will be described in further detail below.

[0020] Example: An autonomous driving planning method for lane-changing scenarios based on deep reinforcement learning in curved path space, such as... Figure 1 As shown, it includes the following steps: 1) Scene Information Preprocessing and Encapsulation: Location information, vehicle historical status information, and scene dynamic participant information are acquired from onboard sensors (such as GPS, inertial measurement unit, camera, LiDAR, millimeter-wave radar, etc.), while map information is read from a high-precision map database. Coordinate transformation and error correction are performed on the location information to ensure vehicle position accuracy. The vehicle historical status information undergoes data cleaning and preprocessing to remove outliers and noise, and smooth data curves such as speed and acceleration. For scene dynamic participant information, sensor fusion techniques (such as Kalman filtering or particle filtering) are used to fuse data from different sensors, improving the accuracy of target detection and tracking. The processed map information is organized according to a specific format (such as grid map or topological map), and key road features (such as curvature and lane line positions) are extracted. Then, map information, location information, vehicle historical status information, and scene dynamic participant information are arranged sequentially according to a preset order and filled into a fixed-size vector. For missing or incomplete information, appropriate default values ​​or interpolation methods are used to supplement it.

[0021] 2) Shortest Feasible Route Calculation: A graph structure based on map information is constructed, representing road nodes as vertices and road connections as edges. The weight of each edge can be determined based on factors such as road length, traffic flow, and speed limits. The Dijkstra algorithm or A* algorithm is used to search for the shortest path from the current vehicle location (location information) to the destination on the constructed graph. During the search, the feasibility of the path is assessed based on traffic rules and vehicle dynamics constraints, eliminating infeasible paths (such as turns that violate traffic rules or paths exceeding the vehicle's minimum turning radius). The found shortest path is discretized, dividing it into a sequence of equally spaced spatial coordinate points according to a preset distance interval. The discretization distance interval should be reasonably selected based on actual application requirements and computational resources, ensuring both the accuracy of the path representation and avoiding excessive computation.

[0022] 3) Curved Path Spatial Feature Calculation: For each element in the vehicle's historical information, map information, and scene dynamic participant information, calculate its projection point in the curved path space based on its position and information on the map. Projection methods can employ distance-based or geometric relationship-based algorithms to ensure that the element's projection on the curved path accurately reflects its relative position to the path. Calculate the features of the projection point, such as arc length (calculated by following the curved path from the starting point to the projection point), lateral offset (distance perpendicular to the curved path direction), and rate of curvature change (reflecting the change in the curvature of the curved path near the projection point). Simultaneously, calculate the relative positional relationships (such as distance and angle) and speed differences between the projection point and the projection points of other scene elements. These features provide important information for subsequent decision-making. Repackage the calculated features into new scene feature information according to a certain order and format, and store them using data structures such as vectors, matrices, or tensors for subsequent network processing.

[0023] 4) High-Dimensional Feature Space Mapping: Construct a tokenized network. The network structure can employ a Convolutional Neural Network (CNN) or other network architectures suitable for feature extraction. CNN convolutional layers can be configured with different kernel sizes and strides to extract features at different scales. Pooling layers reduce data dimensionality, and fully connected layers map the extracted features to a high-dimensional space. Scene feature information is input into the tokenized network and processed through multiple convolutional, pooling, and fully connected layers to obtain a representation in the high-dimensional feature space. During network training, network parameters (such as convolutional kernel weights and biases) are adjusted to enable the network to learn the most representative feature mappings, improving its ability to represent scene information.

[0024] 5) Local Feature Processing and Scene Token Generation: A local feature processing network is designed, including components such as convolutional layers, pooling layers, and fully connected layers. Convolutional layers are used to extract local features from the high-dimensional feature space, focusing on features in different local regions by setting appropriate kernel sizes and numbers. Pooling layers further reduce data dimensionality and computational cost while retaining key feature information. Fully connected layers integrate and abstract local features to generate scene tokens. Multiple local feature processing operations are performed to progressively extract and aggregate higher-level local features, ultimately generating scene tokens. The dimensionality of the scene tokens should be reasonably designed based on model requirements and computational resources, effectively representing the key features of the scene while avoiding excessive computational complexity due to high dimensionality.

[0025] 6) Cross-Attention Operation and Trajectory Feature Generation: The vehicle's historical state and spatial coordinate point sequence are merged into a Query, and the remaining parts are used as Key and Value. In the Transformer planning, a multi-head attention mechanism is used to calculate the attention weights between the Query and Key. The attention calculation for each head can use dot-product attention or other variations. During the calculation, the correlation between the Query and Key vectors is considered, and higher-correlation parts are assigned greater weights. Let the number of heads in the multi-head attention mechanism be... For the first head Attention weight The calculation formula is: in It is the representation of the i-th head of the Query vector after a linear transformation. It is the representation of the i-th head of the Key vector after a linear transformation. This is the dimension of the Key vector. Then, a weighted sum is performed on the Value vector to obtain the output of the i-th head. : Finally, the outputs of each head are concatenated to obtain the final output, namely the trajectory feature. : After multiple cross-attention layers, trajectory features that incorporate key scene information are obtained, providing a foundation for accurate trajectory prediction.

[0026] 7) Trajectory Regression and Generation: A multilayer perceptron network is constructed, consisting of an input layer, multiple hidden layers, and an output layer. The input layer receives trajectory features, and the hidden layers perform feature transformation using appropriate activation functions (such as ReLU or other nonlinear activation functions) to increase the network's expressive power. The output layer is designed according to the prediction task, outputting trajectory parameters such as the vehicle's future position, velocity, and acceleration.

[0027] 8) Environmental interaction and reward feedback: Execute the generated trajectory in the simulator. The environmental observation module collects real-time environmental data after the vehicle completes its execution trajectory. It monitors changes in the relative positions of the vehicle and surrounding vehicles and obstacles, detects collisions, and tracks traffic violations (such as speeding, running red lights, and crossing lane lines). Based on preset rules, it generates a reward signal that is fed back to the planner. Taking into account factors such as safety, efficiency, and comfort, the calculation formula is as follows: in , and These are the weighting coefficients for safety, efficiency, and comfort, respectively. Safety items Consider the risk of collision between the vehicle and surrounding objects, as well as its distance from road boundaries, for example, when the distance between the vehicle and surrounding vehicles or obstacles is less than a safety threshold. hour, (This represents the actual distance and is a proportional coefficient); when the vehicle approaches the road boundary and deviates laterally... Exceeding a certain threshold hour, ( (This is another proportionality coefficient). Efficiency This measure assesses the vehicle's speed and the smoothness of its trajectory as it approaches its destination; for example, if the vehicle's speed... greater than expected speed , If the rate of change of curvature of the trajectory Too large Comfort items Pay attention to changes in vehicle acceleration ,For example, ( For the acceleration of the previous moment, (as a coefficient) to avoid sudden acceleration or braking.

[0028] 9) Model Update Iteration: The parameterized planner network is updated iteratively using the DDPG continuous action space Actor-critic algorithm. The Actor network outputs actions such as lane change decisions and speed adjustments based on the current state, while the Critic network evaluates the value of the current state and actions. Let... The policy function of the Actor network ( (its parameters), output action ; The value function of the Critic network ( (Its parameters). During training, the parameters are updated by minimizing the difference between the value predicted by the Critic network and the actual reward obtained, and by maximizing the expected reward of the Actor network's output action. The loss function of the Critic network. for: in ( (This is the discount factor). The Actor network is updated through policy gradient ascent, and its gradient calculation formula is: The parameters of the Actor and Critic networks are updated using the backpropagation algorithm based on the gradients described above, enabling the model to continuously learn and optimize lane-changing strategies.

[0029] 10) Model Deployment: Once the model reaches convergence during training, the planner network is deployed. Convergence is determined by monitoring the stability of the loss function and reward values ​​during training, as well as performance evaluation metrics on the validation set (such as lane change success rate and average reward). The deployed planner network can receive scene information in real-time in actual autonomous vehicles, generate optimal lane change trajectories, and achieve safe and efficient autonomous lane change operations.

[0030] The following is the verification experiment of this embodiment: The method was compared on the open-source simulator CARLA. We randomly initialized 40,000 lane-change scenarios to form training and testing datasets, and the statistics of the datasets are shown in Table 1: Table 1 shows the data. The implementation details are as follows: Our experimental setup is as follows: The operating system used is Ubuntu 22.04, and the deep learning framework is PyTorch 1.13.0, accelerated by CUDA 11.3 on the GPU. At the hardware level, NVIDIA RTX 4090 graphics cards, a 12th generation Intel i9 processor, and 64GB of DDR4 memory are used as computing resources. Specific parameters and conditions are set for the relevant tasks. Historical information is taken from 20 frames of data at 0.1-second intervals over the past 2 seconds, and the model outputs 80 planned trajectory points at 0.1-second intervals over the next 8 seconds. For model learning, the Adam optimizer is used to adjust network parameters with a learning rate of 0.003. To evaluate the model's performance on the Carla lane-change scenario dataset, we use Speed, Comfort, and Safety metrics for testing and compare it with a series of advanced related methods. Specific data are shown in Table 2.

[0031] Table 2. Comparison results of Speed, Comfort, and Safety on the Carla lane change scenario dataset test set. Note: Bold text represents the optimal value for each line.

[0032] The numerical comparison results are as follows: On the Carla lane-change scenario dataset, a comprehensive performance evaluation was conducted on various models, focusing on key dimensions such as speed, comfort, and safety. The Lattices model showed specific performance values ​​across these three dimensions: 75 for speed, 81 for comfort, and 87 for safety. While its safety performance was acceptable, there was room for improvement in other areas. The DQN model achieved scores of 79, 72, and 83, with a relatively strong speed score, but its comfort score was unsatisfactory, impacting its overall performance. The TRPO model achieved scores of 75, 81, and 82, showing a relatively balanced performance across all aspects, but lacking any significant highlights. The A3C model achieved scores of 75, 82, and 84, demonstrating good comfort performance, but its speed and safety need improvement. The RDRL(ours) model achieved scores of 80, 83, and 84, approaching leading levels in speed, exhibiting excellent comfort, and maintaining a high level of safety. A comprehensive comparison reveals that the RDRL(ours) model outperforms other models across the board in terms of speed, comfort, and safety. Its unique design and algorithm enable a better balance of various requirements in autonomous driving lane-changing scenarios, providing a strong guarantee for efficient, safe, and comfortable autonomous driving lane-changing operations, demonstrating its application potential and advantages in this field.

[0033] Furthermore, the cornering lane-changing performance of TRPO, which showed better overall performance, and the method of this invention were compared. Visualization results were analyzed, such as... Figure 3 , 4 As shown.

[0034] The scene visualization results are as follows: In a comparative experiment involving lane changing and overtaking on a curve, Algorithms A and B exhibited significantly different performance. Specifically, Algorithm B, failing to adequately consider the unique characteristics of curves, revealed numerous problems in actual operation. Due to the curvature of the curve, the vehicle using Algorithm B would straddle the roadside upon entering the curve, severely impacting driving stability and safety. Furthermore, during lane changes, its distance control from the target vehicle was inadequate, getting too close, ultimately leading to lane change failure and preventing a successful overtaking maneuver. In contrast, Algorithm A, by incorporating relative coordinate information, could accurately calculate its relative position to the surrounding environment during lane changing and overtaking. Based on this precise information, Algorithm A could plan its lane change path more rationally, driving smoothly in curves while maintaining an appropriate and safe distance from the target vehicle, thus successfully and safely completing the lane changing and overtaking task, demonstrating superior performance and reliability compared to Algorithm B.

Claims

1. A curve path space-based automatic driving planning method for lane changing, characterized in that, The application relates to an automatic driving planning method and device. Step 1, inputting an automatic driving scene, obtaining scene information, and preprocessing the scene information to form a scene information vector, and obtaining a shortest feasible route from a current position to a destination according to the scene information, and then performing discretization processing to obtain a spatial coordinate point sequence of the shortest feasible route; Step 2, projecting the scene information vector onto the curve path of the spatial coordinate point sequence to obtain scene feature information, sequentially performing high-dimensional mapping and local feature processing on the scene feature information, and performing cross attention based on a planning Transformer to obtain trajectory features; Step 3, the trajectory features are regressed to an executable trajectory of vehicle automatic driving, and are executed in a simulator to collect results and generate Reward feedback to the planner, and a continuous action space Actor-critic algorithm of DDPG is used to train and update the parameterized planner network until convergence, and then real-time scene information is received in an actual automatic driving vehicle to generate an optimal lane-changing trajectory, so that automatic driving planning in a lane-changing scene is realized.

2. The curve path space based lane change scenario automatic driving planning method according to claim 1, characterized in that, In step 1, the scene information includes positioning information obtained from a vehicle-mounted sensor, vehicle historical state information, scene dynamic participant information and map information read from a high-precision map database, wherein the positioning information is subjected to coordinate conversion and error correction to ensure the accuracy of the vehicle position; the vehicle historical state information is subjected to data cleaning and preprocessing to remove abnormal values and noises; the scene dynamic participant information is fused by using a sensor fusion technology to fuse data obtained by different sensors; the map information is organized according to the format, and the key features of the road are extracted; and then the map information, the positioning information, the vehicle historical state information and the scene dynamic participant information are sequentially arranged according to a preset order and filled into a fixed-size vector to form a scene information vector. 3.The curve path space based automatic driving planning method for lane change scene according to claim 1, wherein, In step 1, a graph structure is constructed based on the map information of the scene information, road nodes are represented as vertices in the graph, road connection relationships are represented as edges, and the weights of the edges are determined according to road lengths, traffic flows and speed limits; then, the Dijkstra algorithm or the A* algorithm is used to search for a shortest path from the positioning information to the destination on the constructed graph; in the searching process, the path is subjected to feasibility judgment according to traffic rules and vehicle dynamics constraints, and unfeasible paths are excluded, the found shortest path is subjected to discretization processing, the path is segmented into equidistant spatial coordinate point sequences according to a preset distance interval, and information is formed.

4. The curve path space based lane change scenario automatic driving planning method according to claim 1, characterized in that, In step 2, for each element of the scene information, a projection point of the element under the spatial coordinate point sequence is obtained according to the position and information of the element in the map, a projection method based on distance or geometric relationship is adopted, the features of the projection point are obtained, the features of the projection point include arc length, lateral offset, curvature change rate and relative position relationship and speed difference between the projection point and other scene element projection points, and the features of the projection point are re-encapsulated into new scene feature information according to the order and format.

5. The curve path space based lane change scenario automatic driving planning method according to claim 1, characterized in that, In step 2, the high-dimensional mapping of the scene feature information includes: The tokenization network is constructed, and the network structure adopts a convolutional neural network. The convolutional layers of the convolutional neural network are set to different kernel sizes and steps to extract features of different scales. The pooling layer is used to reduce the data dimension. The full connection layer maps the extracted features to a high-dimensional space. The scene feature information is input into the tokenization network. After the processing of multiple convolution, pooling and full connection layers, the representation in the high-dimensional feature space is obtained.

6. The curve path space based lane change scenario automatic driving planning method according to claim 1, characterized in that, In step 2, the local feature processing of the scene feature information includes: A local feature processing network is designed. The information in the high-dimensional feature space is processed by a convolutional layer to extract local features. By setting appropriate convolution kernel size and number, the features of different local regions are focused on. Then, a pooling layer is used to further reduce the data dimension, reduce the calculation amount, and retain the key feature information. Finally, the local features are integrated and abstracted by a full connection layer to generate scene tokens. The scene feature information is processed multiple times to extract and aggregate higher-level local features, and finally generate scene tokens.

7. The curve path space based lane change scenario automatic driving planning method according to claim 1, characterized in that, In step 2, the cross-attention of the scene feature information based on the planning Transformer includes: The vehicle historical state and the spatial coordinate point sequence are combined as a Query, and the remaining part is as a Key and a Value. In the planning Transformer, the attention weight between the Query and the Key is calculated through a multi-head attention mechanism. The attention calculation of each head adopts dot product attention, and the correlation between the Query and the Key vectors is considered, and a higher weight is given to the part with higher correlation. The number of heads of the multi-head attention mechanism is set to For the first head , the calculation formula of the attention weight is as follows. ； wherein is a representation of the i-th head of the Query vector after linear transformation, is a representation of the i-th head of the Key vector after linear transformation, is the dimension of the Key vector; The output of the i-th head is then obtained by a weighted sum over the Value vectors : ； Finally, the outputs of each head are spliced to obtain the final output, i.e., the track feature : ； After multiple cross-attention layers, the trajectory features fused with the key information of the scene are obtained. 8.The curve path space based automatic driving planning method for lane change scene according to claim 1, wherein, In step 3, the generated trajectory is executed in the simulator, the results are collected, and the Reward feedback is generated to the planner, including: Executing the generated trajectory in the simulator And monitor the relative position changes of the vehicle and surrounding vehicles, obstacles, whether a collision occurs, whether a traffic rule is violated, collect the results of the environment after the trajectory is executed in real time, and generate Reward feedback to the planner according to the preset rules. The Reward signal calculation formula is: ； wherein , and are weight coefficients for the safety, efficiency and comfort items, respectively, and ; the safety item takes into account the risk of collision of the vehicle with surrounding objects and the distance from the road boundaries, the efficiency item measures the speed of the vehicle approaching the destination and the smoothness of the trajectory, the comfort item focuses on the variation of the vehicle acceleration.

9. The curve path space based lane change scenario automatic driving planning method according to claim 1, characterized in that, In step 3, the continuous action space Actor-critic algorithm of DDPG is used to train and update the parameterized planner network until convergence, including: The parameterized planner network is updated iteratively using the continuous action space Actor-critic algorithm with DDPG, the Actor network outputs lane changing decision and speed adjustment action according to the current state, and the Critic network evaluates the value of the current state and action, and the Actor network is the policy function of the Actor network, and the output action ; ; The value function of the Critic network, in the training process, the parameters are updated by minimizing the difference between the value predicted by the Critic network and the actual obtained Reward, and maximizing the expected Reward of the action output by the Actor network, and the loss function of the Critic network is: ； wherein , is a discount factor; The update of the Actor network is realized by the policy gradient ascent. The gradient calculation formula is: ； The parameters of the Actor and Critic networks are updated according to the gradient using the back propagation algorithm.

Images