Deep Reinforcement Learning Lane Changing Model for Unmanned Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional self-driving vehicle decision-making systems face challenges in complex and unstructured environments due to high computational complexity and instability from uncertain factors like bumpy roads, making it difficult to output optimal control variables from sensing data using classical optimization-based algorithms.
Innovation Solution
A method for generating a lane changing decision-making model using a deep reinforcement learning network, which includes a learning-based prediction network and a pre-trained rule-based target network, trained with a training sample set containing state and control variables of vehicles, allowing for correlation of state variables with control variables and improved planning efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If classical optimization-based algorithms are used for decision-making control, then the system structure is simple, but the computational complexity is high and cannot solve complex decision-making tasks
Solution Approach 1:
The patent replaces classical optimization-based algorithms with a deep reinforcement learning network. The neural network learns optimal lane-changing decisions through training with state variables and control variables, substituting the mechanical optimization process with a learned predictive model that outputs decisions directly without complex computational optimization during execution.
2Ease of operation
If model-based algorithms with SLAM are used to obtain trajectories, then the processing framework is established, but instability increases due to uncertainty of height and environmental factors
Solution Approach 1:
The patent extracts and removes the SLAM-based mapping and trajectory generation components from the decision-making system. Instead of using environmental maps and trajectory planning, the system directly processes sensor data through the deep reinforcement learning network to make lane-changing decisions, eliminating the source of instability associated with height uncertainty and map-based processing.
3Adaptability or versatility
If pure deep neural networks are used for decision-making, then adaptability to specific scenarios improves, but planning efficiency decreases
Solution Approach 1:
The patent implements a hybrid dynamic system that switches between two networks based on the decision context. The DQN network handles scenarios requiring high adaptability and learning, while the rule-based network handles scenarios requiring fast, efficient planning. This dynamic allocation optimizes both adaptability and planning efficiency depending on the specific situation.
4Measurement precision
If complex sensors like cameras and laser rangefinders are used, then sensing capability improves, but the complexity of processing sensor data and handling environmental uncertainty increases
Solution Approach 1:
The patent designs a unified deep reinforcement learning network that processes multiple types of sensor inputs (camera images, laser ranging data, vehicle state information) through a single integrated architecture. The network accepts diverse sensor data as input features and uniformly processes them to generate control decisions, reducing the need for separate processing pipelines for different sensor types.
Data Source
AI summary
Provided are a method for generating a lane changing decision-making model and a method and an apparatus for lane changing decision-making of an unmanned vehicle. The method for generating a lane changing decision-making model includes: obtaining a training sample set of vehicular lane changing, wherein the training sample set includes a plurality of training sample groups, each of the training sample groups includes a training sample under each time step length in a process that the vehicle completes lane changing based on a planned lane changing trajectory, the training sample includes a group of state variables and corresponding control variables; obtaining the lane changing decision-making model by training a decision-making model based on deep reinforcement learning network by use of the training sample set, wherein the lane changing decision-making model enables the state variable of the target vehicle and the corresponding control variable to be correlated.