Neural Network Training for Autonomous Parking via Pre-calculated Q-values
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep reinforcement learning-based AI systems for autonomous vehicle operations, such as autonomous parking and driving, require extensive data and training time, with algorithms like DQN being inefficient and prone to poor policy convergence due to rapidly changing targets during training iterations.
Innovation Solution
A method for training neural networks that generates policy values based on sample data sets, using an approximate action-value function to calculate training targets and minimize errors, allowing for simultaneous parking spot localization and execution in various scenarios, including forward, backward, and parallel parking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep reinforcement learning algorithms like DQN are used for autonomous vehicle operations, then the system can learn complex policies, but the training time and data requirements become excessively large
Solution Approach 1:
The patent pre-calculates Q-values for all possible actions at each state before training the neural network. This preliminary computation creates a complete reference policy table that guides the network training process, allowing the system to learn effective policies with significantly fewer training iterations and less data than traditional DQN approaches.
2Reliability
If deep reinforcement learning algorithms like DQN are used for autonomous vehicle operations, then the system can learn complex policies, but the amount of training data required becomes excessively large
Solution Approach 1:
The patent pre-calculates Q-values for all possible actions at each state before training the neural network. This preliminary computation creates a complete reference policy table that guides the network training process, allowing the system to learn effective policies with significantly fewer training iterations and less data than traditional DQN approaches.
3Ease of manufacture
If traditional rule-based parking control systems are used, then the control logic is simple and easy to implement, but the parking performance is poor and requires human drivers to position the vehicle close to the parking spot
Solution Approach 1:
The patent replaces traditional rule-based mechanical control logic with a neural network-based intelligent control system. The neural network learns optimal parking control strategies through training on pre-calculated Q-values, enabling accurate automatic parking without requiring human intervention for initial positioning, while maintaining implementation feasibility through standardized neural network architectures.
Data Source
AI summary
A method of generating training data for training a neural network, method of training a neural network and using a neural network for autonomous operations, related devices and systems. In one aspect, a neural network for autonomous operation of an object in an environment is trained. Policy values are generated based on a sample data set. An approximate action-value function is generated from the policy values. A set of approximated policy values is generated using the approximate action-value function for all states in the sample data set for all possible actions. A training target for the neural network is calculated based on the approximated policy values. A training error is calculated as the difference between the training target and the policy value for the corresponding state-action pair in the sample data set. At least some of the parameters of the neural network are updated to minimize the training error.