Neural Network Training with Feature Map Dependency Regularization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training neuronal networks are susceptible to overfitting due to the large number of parameters, which can lead to poor performance in safety-critical applications such as autonomous vehicles.
Innovation Solution
The method involves regularizing the neuronal network during training by determining a first layer loss value based on the dependence between feature maps, allowing for unsupervised training and adaptation of parameters without requiring specific layer structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of parameters in the neuronal network is increased to improve classification performance, then the network can learn more complex patterns, but the network becomes susceptible to overfitting
Solution Approach 1:
The patent applies parameter changes by introducing regularization terms that modify the loss function parameters. Specifically, it adds L1 and L2 regularization penalties to the original loss function, changing the parameter optimization landscape to prevent overfitting while maintaining classification performance. The regularization strength parameter controls the balance between fitting the training data and maintaining generalization capability.
2Reliability
If regularization is applied to mitigate overfitting, then the classifier performance is improved, but additional computational overhead is introduced during training
Solution Approach 1:
The patent applies local quality by selectively applying different regularization techniques to different parts of the network. It uses L1 regularization for sparse feature selection in certain layers and L2 regularization for weight normalization in other layers, optimizing the computational cost-benefit ratio locally rather than uniformly across the entire network.
3Adaptability or versatility
If flow-based models with predefined layer structures are used, then the training process is constrained, but the classifier can be adapted to specific data characteristics
Solution Approach 1:
The patent applies dynamics by making the network architecture flexible and adaptive during training. It employs dynamic layer configurations where the number and type of layers can be adjusted based on the specific characteristics of the training data, allowing the classifier to adapt its structure rather than being constrained by a fixed predefined architecture.
Data Source
AI summary
A method for training a neural network. The neural network comprises a first layer which includes a plurality of filters to provide a first layer output comprising a plurality of feature maps. Training of the classifier includes: receiving, by a preceding layer, a first layer input in the first layer, wherein the first layer input is based on the input signal; determining the first layer output based on the first layer input and a plurality of parameters of the first layer; determining a first layer loss value based on the first layer output, wherein the first layer loss value characterizes a degree of dependency between the feature maps, the first layer loss value being obtained in an unsupervised fashion; and training the neural network. The training includes an adaption of the parameters of the first layer, the adaption being based on the first layer loss value.