Neural Network Arithmetic Device with Dynamic Activation Function Circuit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing arithmetic devices for neural networks face challenges in efficiently processing and optimizing activation functions, particularly in managing errors and improving accuracy across different input ranges.
Innovation Solution
The proposed arithmetic device incorporates a multiplying-accumulating (MAC) operator and an activation function (AF) circuit with a look-up table. The AF circuit adjusts the number of logic level combinations of the input distribution signal based on the input range of the activation function, using a table input signal generator and an output distribution signal selector to optimize the output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a fixed look-up table is used for activation function processing, then device complexity is reduced, but measurement precision and manufacturing precision deteriorate due to error accumulation across different input ranges
Solution Approach 1:
The activation function input range is divided into multiple segments or regions. Each segment has its own optimized look-up table or processing parameters. This segmentation allows the system to maintain high precision across the entire input range by treating each segment with specialized handling, rather than using a single fixed table that must compromise for all ranges.
Solution Approach 2:
The look-up table structure is made dynamic by adjusting the number of logic level combinations based on the input range. The system dynamically selects or configures different table configurations depending on the current input distribution, allowing optimization for each specific range while maintaining overall system accuracy.
2Device complexity
If the number of logic level combinations is uniformly distributed, then device complexity is minimized, but manufacturing precision worsens due to error variations across different input ranges
Solution Approach 1:
Different input ranges are assigned different numbers of logic level combinations based on their error characteristics. Ranges with higher error rates receive more logic level combinations for finer granularity and better precision, while ranges with lower error rates use fewer combinations. This local optimization approach improves overall accuracy without uniformly increasing complexity.
Solution Approach 2:
The system changes the parameter of logic level combination count based on the input range being processed. By dynamically adjusting this parameter according to the specific input distribution and error characteristics of each range, the system optimizes precision for each segment while managing overall complexity.
3Productivity
If a simplified activation function processing method is used, then productivity is improved, but measurement precision deteriorates due to increased error rates
Solution Approach 1:
The system applies partial precision optimization by using variable numbers of logic level combinations only where needed - specifically in input ranges that exhibit higher error rates. For ranges with acceptable error margins, a simpler processing approach is used. This selective application of complexity maintains high productivity while improving precision only where necessary.
Data Source
AI summary
An arithmetic device includes a multiplying-accumulating (MAC) operator and an activation function (AF) circuit. The MAC operator performs a MAC arithmetic operation for weight data and vector data to generate an arithmetic result signal. The AF circuit stores a look-up table for an activation function, adjusts a number of logic level combinations of an input distribution signal that correspond to each logic level combination of the output distribution signal, among a plurality of logic level combinations of the output distribution signal, based on an input range of the activation function. The AF circuit selects and outputs the output distribution signal that corresponds to the input distribution signal based on the look-up table. The input range of the activation function is based on a relative number of errors that occur.