Neural Network Activation Function Circuit Using Bit Group Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network arithmetic devices face challenges in efficiently processing and distributing activation functions, leading to suboptimal performance in extracting useful information from input data.
Innovation Solution
The implementation of a multiplying-accumulating (MAC) operator and an activation function (AF) circuit that extracts bit groups from arithmetic results, generates input distribution signals, and selects output distribution signals based on activation functions, enhancing data processing and distribution within the neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a MAC operator and AF circuit are implemented to extract bit groups and generate distribution signals, then the speed and accuracy of activation operations is improved, but the device complexity increases
Solution Approach 1:
The arithmetic result signal is divided into multiple bit groups (first bit group, second bit group, third bit group) that are processed separately. The AF circuit extracts these bit groups independently and generates corresponding distribution signals for each group, enabling parallel processing that improves accuracy while managing complexity through modular organization.
Solution Approach 2:
Input distribution signals and output distribution signals are introduced as intermediary signals between the MAC operator and the final activation output. These distribution signals serve as mediators that carry processed bit group information through the AF circuit, enabling complex activation operations to be broken down into manageable signal transformation steps.
2Productivity
If bit groups are extracted and distribution signals are generated to enhance activation function applications, then the productivity of neural network processing is improved, but the device complexity increases
Solution Approach 1:
The activation function application process is segmented into multiple parallel paths, each handling a specific bit group. The first bit group generates an input distribution signal, the second bit group generates another input distribution signal, and the third bit group generates an output distribution signal. This segmentation enables simultaneous processing of multiple bit groups, significantly improving productivity while organizing complexity into parallel modular units.
Solution Approach 2:
The AF circuit performs preliminary extraction of bit groups from the arithmetic result signal before the main activation function computation. By pre-processing the arithmetic result to separate and organize bit groups into distribution signals, the system prepares data in advance for more efficient activation function application, improving overall processing throughput.
Data Source
AI summary
An arithmetic device includes a multiplying-accumulating (MAC) operator and an activation function (AF) circuit. The MAC operator performs a MAC arithmetic operation for weight data and vector data to generate an arithmetic result signal. The AF circuit extracts a first bit group and a second bit group from the arithmetic result signal. In addition, the AF circuit generates an input distribution signal based on the first bit group and the second bit group. Moreover, the AF circuit selects and outputs an output distribution signal that corresponds to the input distribution signal based on an activation function.