Neural Network Bias Control via Sensitivity Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in controlling network bias after training, leading to inconsistent confidence levels in classification tasks, requiring careful parameter tuning that is not always effective.
Innovation Solution
A method is introduced to control network bias by incorporating a sensitivity input that alters the loss function during training, allowing real-time sensitivity adjustments without retraining, using a weighted loss function that prioritizes certain classes, and applying class-dependent weights to balance network sensitivity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional training methods are used to control network bias, then the model achieves basic classification performance, but the model requires careful parameter tuning and cannot adjust sensitivity in real-time
Solution Approach 1:
The patent introduces a dynamic sensitivity parameter that can be adjusted in real-time during inference without requiring retraining or complex parameter tuning. This sensitivity parameter dynamically modifies the loss function applied to different classes, allowing the model to adapt its bias towards underrepresented classes on-the-fly. The dynamic nature of this parameter resolves the contradiction by enabling real-time adaptability while keeping the training process simple and unified.
Solution Approach 2:
The patent changes the parameter space by introducing a sensitivity parameter that controls the weighting of different classes in the loss function. Instead of fixing class weights during training, the sensitivity parameter allows continuous adjustment of class importance during inference. This parameter change enables real-time control of network bias without requiring complex retraining procedures, thus resolving the technical contradiction.
2Reliability
If the model is trained to be confident in its decisions, then classification accuracy improves for majority classes, but the model becomes overly confident and fails to recognize minority classes
Solution Approach 1:
The patent applies local quality by introducing class-specific sensitivity weights that allow different parts of the model (different class predictions) to have different confidence levels. Instead of applying a uniform confidence threshold to all classes, the model can locally adjust sensitivity for each class based on the sensitivity parameter, enabling high confidence for well-represented classes while maintaining sensitivity for underrepresented classes.
Solution Approach 2:
The patent uses counterweight by introducing sensitivity parameters that act as counterbalancing weights to the natural bias towards majority classes. When the sensitivity parameter is adjusted, it counterweights the overconfidence of the model for majority classes and compensates for the underrepresentation of minority classes, achieving balanced reliability across all classes.
3Measurement precision
If the loss function is modified to prioritize minority classes during training, then sensitivity to minority classes improves, but the training process becomes more complex and time-consuming
Solution Approach 1:
The patent applies preliminary action by pre-defining sensitivity parameters and loss function weights during the training phase that encode the desired sensitivity to minority classes. Instead of requiring complex iterative adjustments or multiple training runs with different hyperparameters, the sensitivity parameters are prepared in advance and can be activated during inference. This preliminary configuration achieves minority class sensitivity without extending training time.
Solution Approach 2:
The patent uses parameter changes to control minority class detection by modifying the loss function weights through sensitivity parameters. Instead of requiring extensive retraining with different weight configurations, the sensitivity parameters can be changed to adjust minority class detection accuracy. This parameter-based control achieves high measurement precision for minority classes without the time penalty of retraining.
Data Source
AI summary
A method for controlling a bias of a neural network, the method may include training the neural network by using a loss function that is responsive to classes of a classification process and sensitivity input values thereby setting the bias.