Dynamic Confidence Threshold for Low-Complexity AI Model Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep neural network (DNN) learning requires a large amount of computation, leading to long learning times and high energy consumption, as it involves significant computation for both significant and insignificant changes in weights during mini-batch gradient descent.
Innovation Solution
An electronic device and method for learning a low-complexity artificial intelligence model that selects a dynamic prediction confidence threshold, allowing for the omission of backward propagation operations for sub-input data with high prediction confidence, thereby reducing unnecessary computation and energy usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If backward propagation operation is performed on all sub-input data in mini-batch gradient descent, then learning accuracy is maintained, but computation amount and energy consumption increase significantly
Solution Approach 1:
The patent segments the mini-batch data into two groups: high-confidence samples (prediction confidence ≥ threshold) and low-confidence samples (prediction confidence < threshold). Different propagation operations are applied to each segment, with high-confidence samples using simplified operations and low-confidence samples using full backward propagation, thereby reducing overall computation while maintaining accuracy.
Solution Approach 2:
The patent applies partial action by performing complete backward propagation only on necessary low-confidence samples while using simplified or skipped propagation for high-confidence samples. This selective approach performs less than full propagation on all data, yet maintains sufficient learning accuracy by focusing computational resources where they are most needed.
2Measurement precision
If backward propagation operation is performed on all sub-input data, then weight gradient calculation is accurate, but learning time increases
Solution Approach 1:
The patent divides the mini-batch into segments based on prediction confidence levels. High-confidence samples are processed with simplified gradient calculation or skipped entirely, while low-confidence samples receive full backward propagation. This segmentation reduces the number of samples requiring computationally intensive operations, thereby reducing learning time while maintaining gradient accuracy for critical samples.
Solution Approach 2:
The patent applies different quality levels of computation to different data samples based on their prediction confidence. High-confidence samples receive lower-quality (simplified or omitted) computation, while low-confidence samples receive high-quality (full) computation. This local differentiation optimizes the trade-off between learning time and gradient accuracy.
3Measurement precision
If prediction confidence threshold is set low, then more samples undergo backward propagation improving accuracy, but computation amount increases
Solution Approach 1:
The patent employs a dynamic threshold adjustment mechanism where the prediction confidence threshold is not fixed but adapts during the learning process. The threshold can be adjusted based on learning progress, data characteristics, and computational constraints, allowing the system to dynamically balance between accuracy and computation amount rather than being constrained by a static threshold.
Solution Approach 2:
The patent changes the threshold parameter dynamically during learning based on various factors including learning epoch, data distribution, and computational resources available. By adjusting this key parameter, the system can shift the balance between the number of samples undergoing full backward propagation and those using simplified operations, thereby controlling the trade-off between accuracy and computation.
Data Source
AI summary
The present invention is related to the electronic device and learning method for learning of low complexity artificial intelligence model based on selecting dynamic prediction confidence threshold.