An automatic driving lane-changing trajectory planning method based on deep learning

Abstract

The application relates to the technical field of automatic driving and discloses an automatic driving lane-changing trajectory planning method based on deep learning, which comprises the following steps: collecting and preprocessing multi-modal sensor data; adopting a CNN-LSTM hybrid architecture to perform spatial feature extraction and time sequence modeling on the pretreated multi-modal data, performing feature fusion through an attention mechanism, and outputting a first feature extraction vector; constructing a traffic graph by taking the detected vehicles as graph nodes, learning the interaction relationship between the vehicles through a graph attention network and a message passing mechanism, and calculating a scene urgency score and a safety score; generating a lane-changing decision based on a deep Q network and a policy gradient; and generating a trajectory based on a generative adversarial network. The technical problems that the existing lane-changing trajectory planning method cannot adapt to a dynamic traffic environment, lacks complex multi-vehicle interaction understanding ability, and is difficult to balance the safety and urgency conflict demands are solved, and intelligent, safe and efficient automatic driving lane-changing trajectory planning is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, more particularly, it relates to an automatic driving lane-changing trajectory planning method based on deep learning. BACKGROUND

[0002] In the field of automatic driving, deep learning technology has been widely applied to core functional modules such as environment perception, target detection, and path planning, significantly improving the intelligent level of the system.

[0003] Due to the complexity and variability of the driving environment, there are many uncertainties. When the vehicle has an important task, if it does not change lanes in time, it may miss a key intersection, which may result in a significant increase in time and thus cause huge economic losses and time costs.

[0004] Current automatic driving lane-changing trajectory planning methods mainly include rule-based methods, optimization-based methods, and traditional machine learning-based methods.

[0005] Rule-based methods determine the lane-changing opportunity by preset rules, such as detecting safety gaps, speed differences, and other threshold conditions. However, rules are rigid and cannot adapt to dynamic changes in the traffic environment, and cannot dynamically adjust lane-changing strategies according to the urgency of the scene, lacking the ability to understand complex multi-vehicle interactions.

[0006] Optimization-based methods model the lane-changing problem as a mathematical optimization problem and generate trajectories by minimizing the objective function. However, this method has a single optimization goal, making it difficult to balance the conflicting demands of safety and urgency; it has high computational complexity and cannot meet the real-time requirements of emergency scenarios; and it lacks a dynamic adjustment mechanism for different scene priorities.

[0007] Traditional machine learning-based methods such as support vector machines and decision trees have some learning ability, but they perform poorly in complex lane-changing scenarios: feature engineering relies on human design and is difficult to capture deep features of complex traffic environments; and they lack effective modeling of temporal information and spatial relationships.

[0008] Therefore, there is an urgent need for an intelligent lane-changing trajectory planning method that can accurately identify and balance time and safety, and achieve more safe and efficient lane-changing decisions and trajectory generation. SUMMARY

[0009] The present application provides an automatic driving lane-changing trajectory planning method based on deep learning, which solves the technical problems in the related art.

[0010] The present application provides an automatic driving lane-changing trajectory planning method based on deep learning, which includes the following steps:

[0011] Multi-modal sensor data acquisition and preprocessing: Real-time acquisition of surrounding environment information through laser radar, camera, millimeter wave radar and GPS, and preprocessing of the collected data to form a unified multi-modal data tensor;

[0012] Feature extraction based on convolutional neural network and long short-term memory network: Adopting CNN-LSTM hybrid architecture for spatial feature extraction and time series modeling of preprocessed multi-modal data, and outputting the first feature extraction vector through attention mechanism for feature fusion;

[0013] Environment understanding based on attention mechanism and graph neural network: Constructing a traffic graph by taking the detected vehicles as graph nodes, learning the interaction relationship between vehicles through graph attention network and message passing mechanism, and calculating the scene urgency score and safety score;

[0014] Lane changing decision based on deep Q network and policy gradient: Constructing state space and action space, designing a reward function that considers safety, comfort, efficiency and urgency, and adopting DQN and Actor-Critic architecture for lane changing decision;

[0015] Trajectory generation based on generative adversarial network: Using conditional generative adversarial network to generate smooth lane changing trajectory, using generator network to synthesize trajectory, using discriminator network to evaluate authenticity, and performing physical constraint verification and correction, and finally outputting the optimal trajectory.

[0016] Further, the multi-modal sensor data preprocessing includes: laser radar data processing, converting three-dimensional point cloud data into two-dimensional bird's eye view representation; visual data processing, correcting distortion, normalizing brightness and standardizing size of RGB images collected by multiple cameras; millimeter wave radar data processing, extracting distance, azimuth angle, radial velocity and radar cross section information of target objects; positioning data fusion, combining GPS, IMU and odometry data to obtain accurate vehicle state information.

[0017] Further, the CNN feature extraction adopts ResNet backbone network, uses 3D convolutional layer to extract three-dimensional spatial features for point cloud data, uses 2D convolutional layer to extract visual features for image data, encodes radar features and vehicle state through fully connected layer, and fuses multi-modal features.

[0018] Further, the LSTM time series modeling inputs the CNN feature vector of continuous multiple time steps into the bidirectional LSTM network, dynamically allocates the importance weight of different time steps through the time series attention mechanism, and learns the time series dependency relationship and motion pattern.

[0019] Further, the traffic graph is constructed according to the spatial distance and relative speed to establish the edge connection between vehicles, and the node features include the position, speed, acceleration, heading angle, lane ID, vehicle type and vehicle geometry of the vehicle.

[0020] Further, the graph attention network adopts a multi-head attention mechanism to learn the importance weight between different vehicles, and the information is transmitted and updated between adjacent nodes through multi-layer graph convolution operation.

[0021] Further, the scene urgency evaluation is calculated based on the congestion degree of the target lane, the distance to the target exit, the difference between the current speed and the expected speed, and the safety evaluation is calculated based on the distribution of surrounding vehicles, the relative speed and the predicted trajectory.

[0022] Further, the lane changing decision adopts a double DQN architecture combined with an Actor-Critic method, the action space includes keeping the current lane, changing to the left lane, changing to the right lane, accelerating lane changing and decelerating lane changing, and the Q value output by DQN and the probability output by the policy gradient are weighted and fused as the final lane changing decision.

[0023] Further, the conditional generative adversarial network includes a generator network and a discriminator network, the generator network takes the lane changing decision, vehicle state, target lane information and random noise as conditional input, generates a trajectory sequence through a full connection layer and a one-dimensional deconvolution network, and the discriminator network evaluates the authenticity of the generated trajectory.

[0024] An automatic driving lane changing trajectory planning system based on deep learning is used to perform any one of the above-mentioned automatic driving lane changing trajectory planning methods based on deep learning.

[0025] The beneficial effects of the present application are:

[0026] Through the multi-level safety evaluation mechanism, including dynamic safety calculation based on collision time, trajectory verification based on physical constraints and risk prediction based on multi-vehicle interaction, compared with the traditional rule method based on fixed safety distance, the collision risk in the lane changing process can be reduced, and the driving safety can be effectively guaranteed.

[0027] The graph neural network is used to model the complex interaction relationship between vehicles, and the attention mechanism is used to dynamically allocate importance weights, compared with the simplified model which only considers the front and rear vehicles, the cooperative motion mode of up to 10 surrounding vehicles can be accurately understood, and the decision accuracy in complex traffic scenes is improved.

[0028] By combining deep Q-networks and policy gradients, a multi-dimensional reward function that integrates safety, comfort, efficiency, and urgency is designed. Compared with traditional single-objective optimization methods, this method can improve lane-changing success rate and shorten average lane-changing time while ensuring safety.

[0029] Conditional generative adversarial networks are used to generate smooth trajectories that conform to physical constraints and driving habits. Through an adversarial training mechanism between the generator and the discriminator, the smoothness of the generated trajectory is improved compared to traditional polynomial fitting or spline interpolation methods, resulting in a higher passenger comfort score. Attached Figure Description

[0030] Figure 1 This is a flowchart of a deep learning-based autonomous driving lane-changing trajectory planning method according to the present invention. Detailed Implementation

[0031] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0032] At least one embodiment of the present invention discloses a deep learning-based lane-changing trajectory planning method for autonomous driving, such as... Figure 1 As shown, it includes the following steps:

[0033] Step 1: Multimodal sensor data acquisition and preprocessing;

[0034] The multimodal sensor data acquisition module collects and preprocesses ambient information in real time using sensors such as LiDAR, cameras, millimeter-wave radar, and GPS. Specifically, this includes:

[0035] LiDAR data processing: The 3D point cloud data is converted into a 2D bird's-eye view representation after height filtering and meshing, with a resolution of 0.1m × 0.1m and a detection range of 100m in front, behind, to the left and right of the vehicle;

[0036] Visual data processing: The RGB images captured by multiple cameras are subjected to distortion correction, brightness normalization and size standardization, and uniformly adjusted to 640×480 pixels;

[0037] Millimeter-wave radar data processing: After FFT processing, the data is converted into feature vectors in Cartesian coordinates to extract the target object's range, azimuth, radial velocity, and radar cross-section information, forming a structured target list;

[0038] Location data fusion: Combining GPS, IMU (Inertial Measurement Unit) and odometer data, accurate vehicle status information is obtained through an extended Kalman filter;

[0039] Vehicle status information includes: vehicle position in the global coordinate system, vehicle heading angle, vehicle speed, vehicle acceleration, and vehicle angular velocity.

[0040] It should be noted that the global coordinate system is the WGS84 coordinate system.

[0041] After preprocessing, a unified multimodal data tensor is formed as follows:

[0042] D input ={BEV lidar ,I processed ,F radar ,X t}

[0043] Among them: BEV lidar Aerial view of LiDAR; I processed : Camera image; F radar Radar target characteristics; X t Vehicle status.

[0044] Step 2: Feature extraction based on convolutional neural networks and long short-term memory networks;

[0045] The feature extraction layer adopts a CNN-LSTM hybrid architecture, specifically including the following steps:

[0046] Step 201, CNN feature extraction;

[0047] ResNet-50 was used as the backbone network to extract spatial features from the preprocessed multimodal data. For point cloud data, 3D convolutional layers were used to extract three-dimensional spatial features; for image data, 2D convolutional layers were used to extract visual features. The feature map size decreased layer by layer, resulting in a 512-dimensional high-level semantic feature vector.

[0048] 3D convolutional feature extraction from LiDAR data:

[0049] The input data for the LiDAR bird's-eye view is 2000×2000×3 in dimension, and a modified ResNet-50 network structure is used for feature extraction.

[0050] The first stage, the convolutional layer, receives the bird's-eye view data. Initial feature extraction is performed using a 7×7×3 convolutional kernel with a stride of 2, resulting in 64 output channels. The activation function is ReLU, followed by 3×3 max pooling with a stride of 2. This stage reduces the input dimension from 2000×2000×3 to 500×500×64.

[0051] The second stage residual block consists of three residual blocks, each composed of two 3×3 convolutional layers, with a total output channel count of 64. The residual connections employ identity mapping to ensure effective gradient propagation. The output dimension at this stage is 500×500×64.

[0052] The third stage residual block consists of four residual blocks. The first convolutional layer in each residual block has a stride of 2 and 128 output channels. Dimension-matched residual connections are performed using 1×1 convolutions. This stage reduces the feature map size to 250×250×128.

[0053] The fourth stage residual block consists of 6 residual blocks. The first convolutional layer of the first residual block has a stride of 2 and 256 output channels. This stage reduces the feature map size to 125×125×256.

[0054] The fifth stage residual block contains three residual blocks. The first convolutional layer of the first residual block has a stride of 2 and 512 output channels. This stage reduces the feature map size to 63×63×512.

[0055] Global average pooling layer: Performs global average pooling on the 63×63×512 feature map to output a 512-dimensional LiDAR feature vector.

[0056] 2D convolutional feature extraction from multi-camera visual data:

[0057] Feature extraction was performed on images from the six cameras, with each image having an input dimension of 480×640×3.

[0058] Feature extraction network structure: Each camera uses the same ResNet-50 network structure, but the weight parameters are trained independently. The network contains 5 stages of convolutional layers and residual blocks.

[0059] First stage: 7×7 convolution kernel, stride 2, output channels 64, followed by 3×3 max pooling, output dimensions 240×320×64.

[0060] Stages two through five contain 3, 4, 6, and 3 residual blocks respectively, with output channels of 64, 128, 256, and 512 respectively, and the final output dimension is 15×20×512.

[0061] Multi-view feature fusion: The feature maps from 6 cameras are stitched together along the channel dimension to obtain a fused feature map of 15×20×3072. The number of channels is reduced to 512 using a 1×1 convolution kernel to obtain a visual feature map of 15×20×512.

[0062] Global average pooling: Perform global average pooling on the fused visual feature map to output a 512-dimensional visual feature vector.

[0063] Fully connected feature extraction from millimeter-wave radar data:

[0064] The radar target feature matrix has a dimension of 4×K (K is the number of detected targets):

[0065] Feature flattening: The 4×K radar feature matrix is ​​flattened into a 4K-dimensional one-dimensional vector. When K varies, zero-padding or truncation is used to uniformly process it into a fixed length of 400 dimensions (maximum number of targets is 100).

[0066] Fully connected layer processing: Feature extraction is performed through a three-layer fully connected network.

[0067] First layer: 400-dimensional input, 1024-dimensional output, ReLU activation function, dropout rate 0.3;

[0068] Second layer: Input 1024 dimensions, output 512 dimensions, activation function ReLU, dropout rate 0.2;

[0069] Third layer: 512-dimensional input, 512-dimensional output, ReLU activation function;

[0070] Feature encoding of vehicle status data:

[0071] The vehicle state vector is 6-dimensional and includes information on position, heading angle, velocity, acceleration, and angular velocity.

[0072] Normalization: Standardize the data for each dimension by dividing position coordinates by 100 (meters), angles by π, velocity by 30 (m / s), and acceleration by 5 (m / s²). 2 ), angular velocity divided by π.

[0073] Feature encoding network: Feature encoding is performed through a two-layer fully connected network.

[0074] First layer: 6-dimensional input, 256-dimensional output, ReLU activation function;

[0075] Second layer: Input 256 dimensions, output 512 dimensions, activation function ReLU;

[0076] Multimodal feature fusion:

[0077] The 512-dimensional feature vectors of the four modalities are fused: the lidar features, visual features, radar features and vehicle status features are concatenated in dimensions to obtain a 2048-dimensional fused feature vector.

[0078] Dimensionality reduction: Dimensionality reduction is achieved through a two-layer fully connected network.

[0079] First layer: Input 2048 dimensions, output 1024 dimensions, activation function ReLU, dropout rate 0.3;

[0080] Second layer: Input 1024 dimensions, output 512 dimensions, activation function ReLU, dropout rate 0.2;

[0081] Output feature vector: The final output is a 512-dimensional high-level semantic feature vector, which serves as the result of CNN feature extraction.

[0082] Network training parameter settings:

[0083] Optimizer: The Adam optimizer was used, with a learning rate of 0.001, β1 = 0.9, and β2 = 0.999.

[0084] Batch size: 32;

[0085] Weight initialization: Convolutional layer weights are initialized using He, and fully connected layer weights are initialized using Xavier;

[0086] Regularization: The L2 regularization coefficient is set to 0.0001;

[0087] Training rounds: 200 rounds, with the learning rate decreasing by 0.1 times every 50 rounds.

[0088] Step 202, LSTM timing modeling;

[0089] The CNN feature vectors from five consecutive time steps are input into a bidirectional LSTM network with a hidden layer dimension of 256. This network learns temporal dependencies and motion patterns, specifically including:

[0090] The CNN feature vectors from the current moment and the previous four time steps are arranged in chronological order to form a temporal sequence of length 5, with each feature vector having a dimension of 512, resulting in a 5×512 temporal feature matrix. This temporal window length of 5 time steps is based on empirical values ​​for motion pattern recognition in autonomous driving scenarios, effectively capturing short-term motion trends without introducing excessive noise.

[0091] Bidirectional LSTM network structure:

[0092] A bidirectional LSTM network is used to model temporal features, with the hidden layer dimension set to 256. The bidirectional LSTM includes processing in both forward and backward directions:

[0093] Forward LSTM processing: Feature vectors are processed sequentially from time step 1 to time step 5. LSTM units employ a standard gating mechanism, including a forget gate, an input gate, and an output gate. The forget gate determines which information is discarded from the cell state, the input gate determines which new information is stored, and the output gate controls the hidden state of the output.

[0094] Reverse LSTM processing: The feature vectors are processed in reverse chronological order from the 5th time step to the 1st time step, using the same gating mechanism but with independent weight parameters.

[0095] Bidirectional feature fusion: The hidden states of the forward and backward LSTMs at all time steps are concatenated to obtain the bidirectional LSTM features at all time steps;

[0096] A temporal attention mechanism is used to dynamically assign importance weights to different time steps. The attention mechanism calculates an attention score for the bidirectional features of each time step using a learnable weight vector, and then normalizes it using a softmax function to obtain the attention weights. During vehicle acceleration, more recent time steps have higher weights; during constant-speed driving, more distant time steps may also receive higher weights.

[0097] Calculate the attention weights at each time step:

[0098]

[0099] in: Transpose of the attention weight vector; W h : Hidden state transformation matrix; b h : Change the bias; α t Attention weights at time step t, satisfying H t : Bidirectional LSTM features at time step t; tanh: hyperbolic tangent function; e t Attention score at time step t; e i : Attention score at time step i; Time: Total length of time window.

[0100] It should be noted that in this invention, exp refers to exponential functions.

[0101] By weighting the bidirectional features at five time steps using attention weights, a 512-dimensional weighted temporal feature vector F is obtained. temporal .

[0102] Feature fusion: Features from different modalities are weighted and fused using an attention weighting mechanism to obtain a comprehensive feature representation of 1024 dimensions. The specific steps are as follows:

[0103] Define a multimodal feature set:

[0104] F = {F temporal ,F current}

[0105] Wherein: F temporal : Weighted temporal feature vector; F current H5: The bidirectional LSTM feature at the current time; F: The set of multimodal features.

[0106] Calculate the attention weights between modalities:

[0107]

[0108] in: Transpose of the attention query vector; W proj : Feature projection matrix; b proj : Projection bias; F temporal : Weighted temporal feature vector; F current : Bidirectional LSTM features at the current time; e temporal Attention weights of the bidirectional LSTM features at the current time step; e current : Attention weights of the weighted temporal feature vector; tanh: Hyperbolic tangent function.

[0109] Normalized attention weights:

[0110]

[0111]

[0112] Weighted feature fusion:

[0113] F fused =β temporal ·F temporal +β current ·F current

[0114] Wherein: F fused : The weighted feature vector after fusion; F temporal : Weighted temporal feature vector; F current : Bidirectional LSTM features at the current time; β temporal Attention weights of the bidirectional LSTM features at the current time step after normalization; β current Attention weights of the normalized weighted temporal feature vector.

[0115] The extended feature vector is obtained by expanding the dimension of the fused weighted feature vector through a fully connected layer; the extended feature vector is then subjected to L2 normalization to improve the model's generalization ability and training stability; the first feature extraction vector obtained by L2 normalization is the final output of the feature extraction layer.

[0116] Step 3: Environmental understanding based on attention mechanisms and graph neural networks;

[0117] Step 301, Traffic Graph Construction: The detected vehicles are used as graph nodes, and edge connections are established based on spatial distance and relative speed;

[0118] Node set definition:

[0119] The detected set of vehicles is used as graph nodes; the vehicle itself is used as a special central node, and all other detected vehicles around it are used as ordinary nodes.

[0120] Edge connection conditions:

[0121] Conditions for establishing an edge connection between two vehicles:

[0122]

[0123] Where: e ij : The edge connection marker between nodes i and j; d ij : The Euclidean distance between nodes i and j; v rel,ij : The relative velocity between nodes i and j; d th Distance threshold; v th Relative velocity threshold.

[0124] In one embodiment of the present invention, a connection edge is established when the distance between the two vehicles is less than 50 meters and the relative speed difference is less than 15 m / s.

[0125] Adjacency matrix construction:

[0126] e ij =1 indicates that there is a connection between nodes i and j, e ij =0 indicates that no connection is established between nodes i and j, and the adjacency matrix of the graph is constructed;

[0127] Step 302, Node feature encoding: Each node contains the vehicle's position, speed, acceleration, heading angle, lane ID, vehicle type (0: car, 1: truck, 2: bus, etc.) and the vehicle's width, length and height attributes, forming a 12-dimensional node feature vector;

[0128] The first node feature matrix is ​​formed by concatenating the node feature vectors of each connected node in the traffic map.

[0129] Step 303, Graph Attention Network: A multi-head attention mechanism is adopted, with 8 attention heads, each with a 64-dimensional feature dimension, to learn the importance weights between different vehicles. The specific steps are as follows:

[0130] Multi-head attention calculation: For each of the 8 attention heads, calculate the query matrix, key matrix, and value matrix respectively.

[0131] The query, key, and value matrices are obtained by multiplying the first node's feature matrix by the corresponding weight matrix, with each head's feature dimension set to 64 dimensions.

[0132] Attention weight calculation: Attention weights are calculated using a scaled dot product attention mechanism. First, the dot product of the query vector and the key vector is calculated. Then, the result is scaled by dividing by the square root of the feature dimension. Finally, the result is normalized using the softmax function on the neighboring nodes of each node.

[0133] By calculating attention weights only within the range of neighboring nodes connected in the graph, computational complexity can be reduced while preserving the locality of vehicle interactions in traffic scenarios.

[0134] Attention output: The output of each attention head is obtained by weighting and summing the value vectors of its neighboring nodes using attention weights.

[0135] Multi-head concatenation: The outputs of the 8 attention heads are concatenated along the feature dimension to obtain a 512-dimensional node feature representation;

[0136] The node feature representations of each connected node in the traffic map are concatenated to form the second node feature matrix.

[0137] Step 304, Message Passing: Information is passed between adjacent nodes through graph convolution operations, the second node feature matrix of the node is updated, and the process is repeated for 3 graph convolutional layers. The specific steps are as follows:

[0138] Calculation of convolutional layers (l = 1, 2, 3):

[0139]

[0140] Wherein: H (l) : Feature matrix of the second node in layer l; H (0) =Z: Feature matrix of the second node; Normalized adjacency matrix; D: degree matrix; W (l) : Weight matrix of layer l; σ: Activation function, such as ReLU.

[0141] To ensure training stability and gradient propagation effectiveness, a residual connection is added after each graph convolution layer, which adds the output of the current layer to the output of the previous layer after dimension matching.

[0142] The final graph features H of all vehicles after 3 layers of graph convolution. final H (3) This represents the feature matrix of the second node in the third layer;

[0143] Extract graph features h from vehicle (node ​​0) ego :

[0144] Scenario urgency assessment: Based on factors such as the congestion level of the target lane, the distance to the target exit, and the difference between the current speed and the expected speed, a scenario urgency score is calculated, ranging from 0 to 1. The calculation formula is as follows:

[0145]

[0146] Urgency=w1·Congestion+w2·DF+w3·SF

[0147] Where: ρ max Maximum vehicle density; ρ target Vehicle density of the target lane; Congestion: Normalized value of congestion level; d exit : Distance from the target exit; d critical Critical distance; DF: Distance urgency factor; v current : Current speed; v desired : Expected speed; v max : Maximum permissible speed; |v current -v desired |: The absolute value of the difference between the current speed and the expected speed; SF: Speed ​​difference factor; w1, w2, and w3 represent the weight coefficients of the first, second, and third urgency scores, respectively, and w1+w2+w3=1; Urgency: Scenarios urgency score.

[0148] Safety assessment: Based on the distribution of surrounding vehicles, relative speeds, and predicted trajectories, a lane-changing safety score is calculated, ranging from 0 to 1, considering the influence of vehicles in front, behind, and in the target lane. The vehicle is denoted as vehicle i, and the calculation formula is as follows:

[0149]

[0150] Safety = min(Safety) front Safety rear Safety target )

[0151] Among them: TTC j : The collision time between vehicle j and vehicle d; ij : The distance between the vehicle and vehicle j; v rel,ij d: The relative speed between the vehicle and vehicle j;safe Minimum safe distance; v ego Vehicle speed; t reaction : Reaction time, default value is 1.5; a max Maximum braking deceleration, default value is 8; TTC min Minimum safe collision time, default value is 3. Safety front Safety score of the vehicle ahead; rear Rear vehicle safety score; Safety target Target lane safety score; TTC front Time between collision between the vehicle and the vehicle in front; TTC rear : The time of collision between the vehicle and the vehicle behind; d target : Actual distance between the vehicle and the target lane vehicle; min: minimum value; Safety: safety score.

[0152] Step 4: Lane switching decision based on deep Q-network and policy gradient;

[0153] The lane-changing decision-making method combines Deep Q-Network (DQN) and policy gradient, and includes the following steps:

[0154] Step 401, State space definition: State vector s t It includes the first feature extraction vector, the vehicle's graph features, the scene urgency score, and the safety score;

[0155] Step 402, Action Space Design: Define 5 discrete actions: Maintain current lane, change lane to left, change lane to right, accelerate lane change, and decelerate lane change;

[0156] Step 403, Reward Function Design: Taking into account four dimensions—safety, comfort, efficiency, and urgency—the formula is as follows:

[0157] Total reward function:

[0158] R t =λ1R safety +λ2R comfort +λ3R efficiency +λ4R urgency

[0159]

[0160] R comfort =-α1|a lon |-α2|a lat |-α3|j lon |

[0161]

[0162] Where: R t : Total reward value at time t; λ1, λ2, λ3, and λ4 represent the weight coefficients of the first, second, third, and fourth total reward functions, respectively; R safety Safety reward; R comfort Comfort reward; R efficiency Efficiency-based rewards; R urgency Urgent reward; d min : Distance to the nearest vehicle; d safe : Safety distance threshold; d critical : Critical distance threshold; a lon and a lat These represent longitudinal and lateral accelerations, respectively; j lon : longitudinal acceleration; α1, α2, α3 represent the first, second, and third comfort reward / penalty coefficients, respectively; v current : The vehicle's actual speed at the current moment; v desired : Expected driving speed under current road conditions; The indicator function for whether the vehicle is oriented toward the target lane; β1 and β2 represent the first and second efficiency reward coefficients, respectively; γ: urgency reward coefficient; Indicator function for whether to perform a lane change (1 indicates to perform a lane change, 0 indicates to stay in the current lane); Urgency: Urgency score.

[0163] Step 404, DQN network structure: adopt a dual DQN architecture, with both the main network and the target network containing 3 fully connected layers, and the hidden layer dimensions are 512, 256, and 128 respectively; obtain the Q value of each action according to the DQN network.

[0164] The dual DQN architecture consists of a main network and a target network. The main network is responsible for real-time value function estimation and consists of a three-layer fully connected structure: the first layer maps 1536-dimensional states to a 512-dimensional hidden representation, the second layer compresses it to 256 dimensions, and the third layer further compresses it to 128 dimensions. Finally, it outputs the Q-values ​​of 5 actions. Each layer uses the ReLU activation function to ensure non-linear expressive power.

[0165] Target network update strategy:

[0166] Soft update: For each update step, the target network parameter = 0.001 × main network parameter + 0.999 × target network parameter;

[0167] Hard update: Every 1000 steps, the parameters of the master network are completely copied to the target network.

[0168] Use the main network to select the optimal action: Optimal action = argmax(main network Q value);

[0169] The value of the action is evaluated using the target network: Target value = Instant reward + Discount factor (default value is 0.99) × Target network Q value;

[0170] Loss function = (Target value - Main network Q-value) 2 Expectations;

[0171] Step 405, Policy Gradient Optimization: Using the Actor-Critic architecture, the Actor network outputs the action probability of each action, the Critic network evaluates the state value, and the decision policy is optimized through the policy gradient method.

[0172] The Actor network outputs the probability distribution of actions, employing a two-layer structure (256-dimensional and 128-dimensional hidden layers), and finally outputs the probabilities of five actions through the Softmax function. The Critic network evaluates state value, having the same two-layer network structure as the Actor network, but outputs a single state value estimate.

[0173] The advantage function is calculated as: immediate reward + discount factor × next state value - current state value;

[0174] Actor network learning rate: 0.001;

[0175] Critic network learning rate: 0.002;

[0176] Actor update: Parameters = Parameters + 0.001 × Policy gradient;

[0177] Critic update: Parameters = Parameters - 0.002 × Gradient of Value Function;

[0178] Step 406: The Q-value and probability of each action are weighted and summed in a 7:3 ratio, and the action with the largest weighted sum is selected as the final lane-changing decision.

[0179] Experience replay and target network update: Maintain an experience replay buffer with a capacity of 100,000. Update the target network parameters every 1,000 steps. The target network adopts a soft update strategy with an update coefficient of 0.001. At the same time, a hard update strategy is also implemented, completely copying the master network parameters to the target network every 1,000 steps.

[0180] Each sample in the experience replay buffer is formatted as: (state, action, reward, next state);

[0181] Sampling method: 32 samples are randomly selected for training each time;

[0182] Step 5: Trajectory generation based on generative adversarial networks.

[0183] The trajectory generation layer uses a conditional generative adversarial network (cGAN) to generate smooth lane-change trajectories. Specifically, it includes the following steps:

[0184] Step 501, Construction of the conditional input vector for the generator network;

[0185] Lane change decision encoding: Convert the lane change decision output in step 4 into a 5-dimensional one-hot encoded vector;

[0186] Current vehicle status extraction: Obtain the vehicle's 6-dimensional status information, including current position coordinates, heading angle, speed, acceleration, and angular velocity;

[0187] Target lane information acquisition: Extract 4-dimensional target lane features, including lane ID, lane width, road curvature, and slope information;

[0188] Randomness introduction: A 64-dimensional Gaussian random noise vector is introduced to enable the network to generate diverse trajectories with reasonable randomness, simulating the uncertainty of real driving;

[0189] Conditional vector concatenation: The above four parts of information are connected in sequence to form a 79-dimensional comprehensive conditional vector, which contains all the constraints and guidance information required for trajectory generation.

[0190] Step 502, generator network trajectory synthesis;

[0191] Trajectory synthesis is performed using a trained generator network.

[0192] Fully connected feature mapping: The 79-dimensional conditional vector is mapped to 512-dimensional hidden features through a fully connected layer, and batch normalization and ReLU activation are performed to extract high-level semantic representations of the conditional information;

[0193] Temporal dimension reconstruction: The 512-dimensional features are reshaped into an 8×64 two-dimensional tensor, and a temporal structure is introduced to prepare for the subsequent generation of temporal trajectories. This step transforms the abstract features into a representation with temporal meaning.

[0194] One-dimensional deconvolution upsampling: Temporal upsampling is performed using a three-layer one-dimensional deconvolution network.

[0195] The first layer expands the 8 time steps to 16 time steps, and the feature dimension is reduced from 64 to 32;

[0196] The second layer expands the 16 time steps to 32 time steps, and the feature dimension is reduced from 32 to 16;

[0197] The third layer expands the 32 time steps to 64 time steps, and the feature dimension is reduced from 16 to 8;

[0198] Batch normalization and ReLU activation are used in each layer to ensure training stability;

[0199] Trajectory output layer generation: The 64×8 feature tensor is transformed into a 50×7 trajectory matrix through the final one-dimensional convolutional layer, where 50 represents the number of trajectory points in the next 5 seconds at 0.1-second intervals, and 7 represents the attribute dimension of each trajectory point (timestamp, x-coordinate, y-coordinate, heading angle, velocity, acceleration, curvature).

[0200] Step 503, Discriminator network authenticity assessment;

[0201] Trajectory and condition fusion: The generated 50×7 trajectory sequence is concatenated with the repeatedly expanded 15-dimensional condition information (extracting key parts from the 79-dimensional condition vector) in the feature dimension to form a 50×22 discriminator input, ensuring that the discriminator can perceive the consistency between trajectory and condition;

[0202] One-dimensional convolutional feature extraction: Temporal feature patterns of trajectories are extracted using a three-layer one-dimensional convolutional network.

[0203] The first layer compresses 50 time steps to 25, and expands the feature dimension to 32;

[0204] The second layer compresses 25 time steps to 12, and expands the feature dimension to 64.

[0205] The third layer compresses 12 time steps into 6, and expands the feature dimension to 128;

[0206] Each layer uses LeakyReLU activation and batch normalization;

[0207] Authenticity probability output: The final 768-dimensional feature vector is mapped to a single authenticity probability value, ranging from 0 to 1, through a fully connected layer and a Sigmoid activation function. This probability value guides the generator to improve trajectory quality.

[0208] Step 504, Physical constraint verification and correction;

[0209] Physical constraints are verified and corrected in real time during the generation process to ensure the executability of the trajectory:

[0210] Kinematic constraint check: Verify that the velocity, acceleration, and curvature of each point in the generated trajectory are within the specified range. Values ​​exceeding the range are corrected to boundary values ​​using a truncation function.

[0211] Road boundary constraint verification: Check whether the y-coordinate of the trajectory point is within the left and right boundaries of the target lane. Trajectory points that exceed the boundary are adjusted to the lane boundary using the nearest point projection method to ensure that the vehicle is always within the legal driving area.

[0212] Continuity constraint correction: Verify whether the positional change between adjacent trajectory points exceeds the maximum allowable displacement (calculated based on velocity limit) and whether the velocity change exceeds the maximum allowable acceleration change. Trajectory segments that violate continuity are regenerated using cubic spline interpolation to ensure the smoothness of the trajectory.

[0213] Constraint loss feedback: The degree of violation of three types of constraints—kinematics, path, and continuity—is calculated to form a weighted constraint loss. This loss information is fed back to the generator to improve the quality of subsequent trajectory generation.

[0214] Step 505: Generation and optimal selection of diverse trajectories.

[0215] Diverse sampling strategy: By changing the random noise vector, 10 different candidate trajectories are generated. Each trajectory meets the same lane-changing action requirements but has different detailed features, simulating the diverse choices in real driving.

[0216] Comprehensive score calculation: A three-dimensional score is calculated for each candidate trajectory:

[0217] Safety score: calculated based on the minimum distance between each point on the trajectory and surrounding vehicles; the greater the distance, the higher the score.

[0218] Comfort score: calculated based on the smoothness of changes in acceleration and curvature of the trajectory; the smoother the change, the higher the score.

[0219] Efficiency score: calculated based on the degree of closeness between the average speed during trajectory execution and the expected speed;

[0220] Weighted comprehensive evaluation: The three scores are weighted and summed according to the set weights to obtain the comprehensive quality score of each trajectory;

[0221] Optimal trajectory selection: Select the trajectory with the highest comprehensive score as the optimal trajectory output.

[0222] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A deep learning-based lane-changing trajectory planning method for autonomous driving, characterized in that, Includes the following steps: Multimodal sensor data acquisition and preprocessing: Real-time acquisition of surrounding environmental information through lidar, camera, millimeter-wave radar and GPS, and preprocessing of the acquired data to form a unified multimodal data tensor; Feature extraction based on convolutional neural networks and long short-term memory networks: A CNN-LSTM hybrid architecture is used to perform spatial feature extraction and temporal modeling on the preprocessed multimodal data, and feature fusion is performed through an attention mechanism to output the first feature extraction vector; Environment understanding based on attention mechanism and graph neural network: The detected vehicles are used as graph nodes to construct a traffic map. The interaction relationship between vehicles is learned through graph attention network and message passing mechanism, and the scene urgency score and safety score are calculated. The urgency score for the scenario ranges from 0 to 1, and the calculation formula is as follows: in: Maximum vehicle density; Vehicle density in the target lane; Normalized value of congestion level; Distance from the target exit; Critical distance; : Distance urgency factor; Current speed; Expected speed; Maximum permissible speed; The absolute value of the difference between the current speed and the desired speed; Speed ​​difference factor; , , These represent the weighting coefficients for the first, second, and third urgency scores, respectively. ; : Score for the urgency of the scenario; Lane-changing decision based on DQN and policy gradient: Construct state space and action space, design a reward function that comprehensively considers safety, comfort, efficiency and urgency, and use DQN and Actor-Critic architecture for lane-changing decision; Trajectory generation based on generative adversarial networks: A conditional generative adversarial network is used to generate a smooth lane-changing trajectory. The trajectory is synthesized through a generator network, and the discriminator network evaluates the realism of the trajectory and verifies and corrects the physical constraints to output the optimal trajectory.

2. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The multimodal sensor data preprocessing includes: lidar data processing, converting 3D point cloud data into a 2D bird's-eye view representation; visual data processing, performing distortion correction, brightness normalization, and size standardization on RGB images acquired by multiple cameras; millimeter-wave radar data processing, extracting distance, azimuth, radial velocity, and radar cross-section information of the target object; and positioning data fusion, combining GPS, IMU, and odometer data to obtain accurate vehicle status information.

3. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The CNN uses a ResNet backbone network for feature extraction. It uses 3D convolutional layers to extract three-dimensional spatial features for point cloud data and 2D convolutional layers to extract visual features for image data. It encodes radar features and vehicle status through fully connected layers and fuses multimodal features.

4. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The LSTM temporal modeling inputs the CNN feature vectors from multiple consecutive time steps into a bidirectional LSTM network, dynamically assigns importance weights for different time steps through a temporal attention mechanism, and learns temporal dependencies and motion patterns.

5. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The traffic map is constructed by establishing edge connections between vehicles based on spatial distance and relative speed. Node features include vehicle position, speed, acceleration, heading angle, lane ID, vehicle type, and vehicle geometric attributes.

6. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The graph attention network employs a multi-head attention mechanism to learn the importance weights between different vehicles, and updates node features by transmitting information between adjacent nodes through multi-layer graph convolution operations.

7. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The urgency assessment is calculated based on the congestion level of the target lane, the distance to the target exit, and the difference between the current speed and the expected speed. The safety assessment is calculated based on the distribution of surrounding vehicles, their relative speeds, and the predicted trajectory.

8. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The lane-changing decision adopts a dual DQN architecture combined with the Actor-Critic method. The action space includes maintaining the current lane, changing lanes to the left, changing lanes to the right, accelerating lane-changing, and decelerating lane-changing. The Q-value output by DQN and the probability output by the policy gradient are weighted and fused to obtain the final lane-changing decision.

9. The deep learning-based lane-changing trajectory planning method for autonomous driving according to claim 1, characterized in that, The conditional generative adversarial network includes a generator network and a discriminator network. The generator network takes lane-changing decisions, vehicle status, target lane information and random noise as conditional inputs, and generates trajectory sequences through fully connected layers and a one-dimensional deconvolutional network. The discriminator network evaluates the authenticity of the generated trajectory.

10. A deep learning-based lane-changing trajectory planning system for autonomous driving, characterized in that, It is used to execute the deep learning-based lane-changing trajectory planning method for autonomous driving as described in any one of claims 1-9.

Images