Autonomous Driving Policy Learning With Two-Stage Neural Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autonomous vehicle systems face limitations in robustness, sensing ability, efficiency, and extendibility due to separate object detection, tracking, and prediction models, leading to significant information loss and decreased reliability, particularly in crowded conditions, and require significant computing resources for deep neural networks.
Innovation Solution
A deep neural network is trained in two stages: first using object detection loss for feature learning, then reinforced with reinforcement learning to determine optimal driving policies, incorporating sensor data from multiple vehicles for enhanced sensing range and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If separate object detection, tracking, and prediction models are used, then each model can be optimized independently, but significant information loss occurs and reliability decreases
Solution Approach 1:
The patent merges separate object detection, tracking, and prediction models into a unified deep neural network that processes sensor data end-to-end. This integration eliminates information loss between separate models while maintaining the ability to learn optimal driving policies directly from raw sensor inputs, thereby improving reliability without sacrificing optimization capabilities.
2Extent of automation
If deep neural networks are used for autonomous vehicle systems, then decision-making capability is improved, but significant computing power and resources are required
Solution Approach 1:
The patent segments the training process into two distinct stages: first stage training that pre-trains the deep neural network using object detection loss, and second stage training that fine-tunes the network using reinforcement learning. This segmentation reduces computational complexity by breaking down the resource-intensive training into manageable phases, allowing the system to achieve high decision-making capability while optimizing resource utilization.
3Device complexity
If sensor data from only one vehicle is used, then the system is simpler to implement, but sensing range and field of view are limited
Solution Approach 1:
The patent implements a multi-vehicle data fusion system where sensor data from multiple vehicles is integrated into a unified deep neural network model. This approach allows the system to maintain a single standardized processing architecture (preserving simplicity) while simultaneously achieving extended sensing range and improved field of view by incorporating data from multiple vehicle sources.
4Loss of time
If a single-stage training approach is used, then training is faster, but the system cannot achieve optimal performance for both feature extraction and decision-making
Solution Approach 1:
The patent applies preliminary action by implementing a two-stage training process where the first stage pre-trains the deep neural network using object detection loss to learn robust feature representations. This preliminary training establishes a strong foundation that enables the second stage to efficiently fine-tune the model using reinforcement learning for optimal decision-making, thereby achieving both time efficiency and optimal performance.
Data Source
AI summary
A system for learning optimal driving behavior for autonomous vehicles comprises a deep neural network, a first stage training module, and a second stage training module. The deep neural network comprises a feature learning network configured to receive sensor data from a vehicle as input and output spatial temporal feature embeddings and a decision action network configured to receive the spatial temporal feature embeddings as input and output an optimal driving policy for the vehicle. The first training stage module is configured to, during a first training stage, train the feature learning network using object detection loss. The second stage training module is configured to, during a second training stage, train the decision action network using reinforcement learning.