DNN Training Accelerator Skip Logic for Energy Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network (DNN) training based on mini-batch gradient descent requires extensive computational resources and time, necessitating a method to differentiate between crucial and non-critical operations to apply effective approximations.
Innovation Solution
A deep neural network training accelerator that performs a first operation using a forward propagation algorithm and determines input data as either skip or training data based on a confidence matrix. The accelerator skips the second operation (backward propagation and weight update) for skip data, reducing unnecessary computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If mini-batch gradient descent is used for DNN training, then training can be performed with batch processing, but the training time and energy consumption remain excessively long and high
Solution Approach 1:
The patent applies partial action by selectively performing the second operation (backward propagation and weight update) only on a subset of input data identified as training data, rather than processing all input data. The determination unit identifies skip data that can be excluded from the second operation, thereby reducing the total computational workload and energy consumption while maintaining training effectiveness.
Solution Approach 2:
The determination unit extracts and identifies specific input data as skip data based on the confidence matrix from the first operation. By separating skip data from training data, the system removes unnecessary computations from the processing pipeline, reducing both training time and energy consumption without compromising the quality of gradient descent updates.
2Reliability
If all input data undergo both first and second operations, then complete training is achieved, but the computational complexity and time consumption increase significantly
Solution Approach 1:
The system performs the computationally intensive second operation only on a partial subset of input data (training data) rather than all input data. The determination unit enables this selective processing by identifying skip data that can be excluded, thereby reducing computational complexity while maintaining sufficient training reliability through gradient accumulation from representative samples.
Solution Approach 2:
The first operation (forward propagation) is performed on all input data beforehand to generate confidence matrices. This preliminary action enables the determination unit to identify skip data before the second operation, allowing the system to pre-filter the dataset and avoid unnecessary complex computations in subsequent processing stages.
3Manufacturing precision
If the second operation is performed on all input data, then comprehensive weight updates are achieved, but the operation time increases substantially
Solution Approach 1:
The second operation (backward propagation and weight update) is applied only to training data identified by the determination unit, rather than all input data. This selective partial processing reduces operation time substantially while maintaining weight update precision through accumulated gradients from the selected subset, leveraging the statistical representativeness of the training data.
Solution Approach 2:
The system skips the second operation for identified skip data, rushing through these samples without performing computationally intensive backward propagation and weight updates. This skipping strategy reduces overall operation time while maintaining training effectiveness, as the confidence matrix from the first operation already provides sufficient information for these samples.
Data Source
AI summary
A deep neural network training accelerator includes an operational unit sequentially performing first and second operations on a plurality of input data of a sub-set according to a mini-batch gradient descent, a determination unit determining each of the input data as one of skip data and training data based on a confidence matrix obtained by the first operation, and a control unit controlling the operational unit to skip the second operation with respect to the skip data.