Personalized Reward Network for Autonomous Vehicle Path Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles often fail to provide personalized path planning, leading to dissatisfaction among passengers as the vehicle may prioritize speed over comfort, resulting in unstable driving experiences.
Innovation Solution
A learning method using human driving data as training data to support Reinforcement Learning (RL) algorithms, generating personalized reward functions that adjust common reward functions to optimize driving experiences, reducing computing resource usage by employing an adjustment reward network and estimation network to learn parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a common reward function is used for path planning in autonomous vehicles, then the vehicle can achieve efficient and stable driving, but it cannot provide personalized driving experiences that match individual passenger preferences
Solution Approach 1:
The reward function is segmented into two independent components: a common reward function that handles basic driving stability and efficiency, and a personalized reward function that adapts to individual passenger preferences. This segmentation allows the system to maintain computational efficiency while providing personalized driving experiences.
Solution Approach 2:
The patent combines the common reward function and personalized reward function into a unified reward calculation mechanism. The total reward is computed as the sum of both functions, allowing the system to leverage both general driving expertise and individual passenger preferences simultaneously.
2Adaptability or versatility
If personalized path planning is implemented for each passenger, then passenger satisfaction can be improved, but computing resource usage increases significantly
Solution Approach 1:
The system performs preliminary learning of personalized reward functions during offline training phases using collected passenger feedback data. This preliminary action allows the model to pre-adapt to individual preferences without requiring intensive real-time computing resources during actual vehicle operation.
Solution Approach 2:
The personalized reward function automatically adapts to passenger preferences through continuous learning from feedback signals (such as acceleration preferences, braking behavior, and route choices). The system serves itself by using operational data to refine its personalized models without requiring manual reconfiguration or intensive external computation.
3Measurement precision
If extensive training data is collected to improve personalized path planning accuracy, then model performance can be enhanced, but data privacy and security risks increase
Solution Approach 1:
The patent extracts only the essential preference signals from raw driving data, such as acceleration patterns, braking behaviors, and route preferences, rather than storing or processing complete raw sensor data. This extraction approach maintains modeling accuracy while minimizing privacy risks by removing personally identifiable information.
Solution Approach 2:
The system transforms raw driving trajectory data into aggregated statistical parameters that capture passenger preferences without revealing individual behavior patterns. By changing the data representation from detailed trajectories to summarized preference parameters, the system maintains personalization accuracy while reducing privacy exposure.
Data Source
AI summary
A learning method for acquiring at least one personalized reward function, used for performing a Reinforcement Learning (RL) algorithm, corresponding to a personalized optimal policy for a subject driver is provided. And the method includes steps of: (a) a learning device performing a process of instructing an adjustment reward network to generate first adjustment rewards, by referring to the information on actual actions and actual circumstance vectors in driving trajectories, a process of instructing a common reward module to generate first common rewards by referring to the actual actions and the actual circumstance vectors, and a process of instructing an estimation network to generate actual prospective values by referring to the actual circumstance vectors; and (b) the learning device instructing a first loss layer to generate an adjustment reward and to perform backpropagation to learn parameters of the adjustment reward network.