Neural Network Training via Output Variable Exclusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks trained for vehicle automation tend to overfit on specific details present in training data, leading to poor recognition of vehicles with different features or under varying conditions, and require extensive labeling efforts due to considering all output variables in backpropagation.
Innovation Solution
A method where a subset of output variables is excluded from backpropagation, focusing the training on more general features and reducing labeling time by only considering a portion of the data for parameter updates, with the excluded variables introducing uncertainty and improving robustness against overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all output variables are considered in backpropagation during training, then the neural network achieves high accuracy on training data, but it overfits to specific details and performs poorly on unseen data
Solution Approach 1:
The patent extracts and excludes a subset of output variables from backpropagation consideration. By removing specific output variables (e.g., those corresponding to less important classes or features) from the backpropagation process, the network is prevented from overfitting to details in those dimensions while still learning from the remaining variables, thus improving generalization performance.
Solution Approach 2:
Instead of applying backpropagation to all output variables (excessive action), the patent applies it only to a selected subset (partial action). This partial application of the training mechanism allows the network to learn from important features while ignoring less critical ones, balancing training accuracy with generalization capability.
2Adaptability or versatility
If all output variables are included in backpropagation, then comprehensive learning occurs, but labeling effort and training time increase significantly
Solution Approach 1:
The patent extracts only the essential output variables that need to be considered in backpropagation, excluding others from the training process. This extraction approach maintains learning comprehensiveness for critical features while dramatically reducing the labeling effort required for non-critical variables.
Solution Approach 2:
The patent applies backpropagation partially to only those output variables that contribute most to learning effectiveness. This partial application reduces the total labeling time and computational resources required while preserving the essential learning comprehensiveness needed for good performance.
Data Source
AI summary
Training an artificial neural network, ANN, which translates one or more input variables into one or more output variables, using learning data sets including learning input variable values having measurement data, and associated learning output variable values, by: mapping learning input variable values from a learning data set onto output variable values using the ANN; processing deviations of the output variable values from the respective learning output variable values using a cost function to form a measure of the error of the ANN when processing the learning input variable values; determining from the error, by backpropagation, changes in parameters, the execution of which, when learning input variable values are further processed by the ANN, improve the evaluation of the obtained output variable values by the cost function, and applying said changes to the ANN; wherein a subset of the output variable values is excluded from consideration in the backpropagation.