Autonomous Driving Policy Learning from Historical Task Paths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current self-driving technologies face inefficiencies in learning driving policies for new tasks, requiring extensive training data and causing wear and tear on vehicles, as previously learned policies are not applicable to similar but distinct tasks, leading to high costs and time overheads.
Innovation Solution
A method involving a self-driving apparatus that determines a task feature vector from historical paths using a task abstraction network and a shared policy network to quickly learn and formulate driving policies for various tasks, allowing for efficient adaptation to similar tasks without extensive retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If reinforcement learning is used to learn a driving policy for a specific task, then the self-driving vehicle can perform that specific task, but efficiency of reinforcement learning is greatly reduced when encountering similar but distinct tasks requiring new policy training
Solution Approach 1:
The patent applies universality by training a single reinforcement learning model on a diverse set of driving tasks simultaneously. The model learns task-agnostic policies that can be transferred to multiple driving scenarios (e.g., parking, lane changing, intersection navigation) without requiring separate training for each task. This multi-functional approach allows the self-driving vehicle to adapt to different driving tasks while avoiding repeated training time overhead.
2Reliability
If reinforcement learning is used to learn a driving policy for a specific task, then the self-driving vehicle can perform that specific task, but costs and time overheads are high due to extensive training data requirements
Solution Approach 1:
The patent applies preliminary action by pre-training the reinforcement learning model on a comprehensive dataset covering multiple driving tasks before deployment. The model is prepared in advance with generalized knowledge from diverse scenarios, enabling it to quickly adapt to new tasks with minimal additional training. This preliminary preparation reduces both the time overhead and computational costs associated with training while maintaining high driving policy accuracy.
3Adaptability or versatility
If extensive training data is collected for each new driving task, then the self-driving vehicle can learn the task, but great wear and tear is caused to the self-driving vehicle
Solution Approach 1:
The patent applies copying by using simulated driving environments to generate virtual training data instead of collecting extensive real-world data through physical vehicle operation. The reinforcement learning model is trained on copies of driving scenarios created in simulation, which eliminates the wear and tear associated with extensive real-world testing while maintaining the model's ability to learn and adapt to various driving tasks.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A self-driving method, a training method, and a related apparatus are provided. The method includes: A self-driving apparatus (100) determines a task feature vector (401) of a self-driving task based on M groups of historical paths of the self-driving task. The task feature vector is a vector representing features of the self-driving task, and M is an integer greater than 0. The self-driving apparatus (100) determines, based on the task feature vector and a status vector, a target driving operation (402) that needs to be performed. The status vector is configured to represent a driving status of the self-driving apparatus (100). The self-driving apparatus (100) performs the target driving operation (403). The self-driving apparatus (100) determines the task feature vector of the self-driving task by using the historical paths of the self-driving task, so as to formulate a targeted driving policy based on the task feature vector. In this way, an existing historical path can be effectively used to formulate a better driving policy.