Neural Network Arithmetic Device Activation Function Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network arithmetic devices face challenges in efficiently processing and distributing activation data, particularly in selecting and applying various activation functions without design changes, which affects the speed and accuracy of neural network operations.
Innovation Solution
The proposed arithmetic device includes an activation function control circuit, data storage circuit, and output distribution signal generation circuit, which generate column addresses, data selection signals, and internal control signals to select and output activation data from a memory cell array, allowing for the application of different activation functions without requiring design changes by storing these functions in look-up tables.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If activation functions are hardwired into the arithmetic device, then the processing speed is improved, but the adaptability to different activation functions deteriorates
Solution Approach 1:
The patent implements dynamic configurability of activation functions through control circuits that can select and apply different activation functions based on input conditions. The arithmetic device transitions from static hardwired functions to dynamic selectable functions, allowing the system to adapt its processing characteristics in real-time while maintaining high processing speed through optimized control logic.
Solution Approach 2:
The patent changes the parameter of activation function selection by introducing control signals and selection circuits that can dynamically alter which activation function is applied. This allows the arithmetic device to switch between different activation functions (e.g., ReLU, Sigmoid, Tanh) without physical redesign, resolving the contradiction between speed optimization for specific functions and adaptability to multiple functions.
2Measurement precision
If the arithmetic device is designed for specific activation functions, then the processing accuracy is improved, but the ease of manufacture for different configurations deteriorates
Solution Approach 1:
The patent creates a universal arithmetic device architecture that can handle multiple activation functions through a common structure with selectable configuration. The control circuits and data paths are designed to accommodate different activation functions without requiring separate hardware designs, thereby improving ease of manufacture while maintaining processing accuracy through optimized control logic for each function type.
Solution Approach 2:
The patent implements dynamic configurability that allows the same hardware to be optimized for different activation functions through control signals. This enables a single manufacturing process to produce devices that can be configured for high-accuracy processing of various activation functions, eliminating the need for multiple specialized manufacturing lines.
3Adaptability or versatility
If activation data is stored in look-up tables, then the adaptability to different activation functions is improved, but the device complexity increases
Solution Approach 1:
The patent extracts the activation function data from the core arithmetic logic and stores it in separate look-up tables. This separation allows the activation functions to be easily changed by simply updating the table contents rather than modifying the arithmetic logic, thereby improving adaptability while managing complexity through modular design. The control circuits manage the complexity by providing simple address generation and data retrieval mechanisms.
Data Source
AI summary
An arithmetic device includes an activation function (AF) control circuit, a data storage circuit, and an output distribution signal generation circuit. The AF control circuit generates a column address, a data selection signal, and an internal control signal based on an arithmetic result signal during an activation operation. The data storage circuit outputs activation data from a memory cell array that is selected by the column address and a row address. The output distribution signal generation circuit generates an output distribution signal from the activation data based on the data selection signal and the internal control signal.