Time-Difference Feature Augmentation for Low-Precision Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Low-precision neural networks face a trade-off between reducing hardware resource consumption and maintaining prediction accuracy, as reducing computation precision decreases prediction accuracy.
Innovation Solution
A data feature augmentation system that uses a time difference calculation unit, comprising a sample-and-hold circuit and a subtractor, to generate a time difference signal from the input signal, which is then inputted to the low-precision neural network, enhancing prediction accuracy without increasing hardware resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If the precision of the neural network is reduced to decrease computation amount, then hardware resource consumption is reduced, but prediction accuracy is decreased
Solution Approach 1:
The patent introduces a new dimension of time by computing time difference signals (current signal minus previous time unit signal) and feeding them as additional input features to the low-precision neural network. This temporal dimension augmentation enriches the input data without increasing computational precision, allowing the network to capture dynamic changes while maintaining low-precision arithmetic operations.
Solution Approach 2:
The patent performs preliminary processing of input signals by computing time difference values before they enter the neural network. The sample-and-hold circuits and subtractors pre-compute the temporal differences, so that when data enters the low-precision neural network, the temporal dynamics are already encoded, eliminating the need for complex high-precision computations during inference.
2Measurement precision
If high-precision neural network is used to maintain prediction accuracy, then prediction accuracy is improved, but hardware resource consumption increases
Solution Approach 1:
The patent introduces time difference signals as an intermediary representation that bridges the gap between simple low-precision inputs and the need for accurate prediction. By computing and appending temporal difference features, the system provides the neural network with enriched information that compensates for the reduced precision of the arithmetic operations, achieving accurate predictions with low-precision hardware.
3Device complexity
If low-precision neural network is used to reduce hardware resource consumption, then hardware resource consumption is reduced, but computation accuracy is decreased
Solution Approach 1:
The patent changes the parameters of the input data by introducing time difference features (temporal dynamics) rather than changing the precision parameters of the neural network itself. This allows the system to maintain low-precision arithmetic operations while improving the quality of computations through enriched input features that capture temporal patterns.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The proposed solution effectively increases the prediction accuracy of low-precision neural networks while minimizing hardware resource consumption by leveraging time difference signals processed through low-resolution analog-to-digital converters.
Implementation Method 1
The first signal is related to a first leakage rate of the first sample-and-hold circuit and is the first signal the signal generated by delaying the input signal by one time unit
Implementation Method 2
The subtractor is used for performing subtraction on the input signal and the first signal to obtain a time difference signal
Data Source
AI summary
A data feature augmentation system and method for a low-precision neural network are provided. The data feature augmentation system includes a first time difference unit. The first time difference unit includes a first sample-and-hold circuit and a subtractor. The first sample-and-hold circuit is used for receiving an input signal and obtaining a first signal according to the input signal. The first signal is related to a first leakage rate of the first sample-and-hold circuit and the first signal is the signal generated by delaying the input signal by one time unit. The subtractor is used for performing subtraction on the input signal and the first signal to obtain a time difference signal. The input signal and the time difference signal are inputted to the low-precision neural network.