Hardware Approximation of Activation Functions for ML Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processor architectures face challenges in efficiently implementing non-linear activation functions for machine learning inference and other processing tasks, leading to suboptimal computational efficiency, processing time, and energy consumption.
Innovation Solution
A processing unit is designed to perform hardware approximation of activation functions without memory lookups, utilizing a lookup table for data aggregation and supporting vector operations for functions like sigmoid and tanh, with each element of the input vector processed in a single clock cycle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If memory lookup tables are used for activation function computation, then accuracy is improved, but processing time and energy consumption increase
Solution Approach 1:
The patent extracts the essential computational pattern of activation functions (piecewise linear approximation) and implements it directly in hardware circuitry, removing the dependency on memory lookup tables. This extraction of the core computational logic into dedicated hardware paths enables direct computation without memory access delays.
Solution Approach 2:
The patent replaces the mechanical memory lookup process with an electronic hardware approximation circuit that directly computes activation function values. The hardware circuit uses piecewise linear approximation with conditional logic to compute sigmoid, tanh, and other activation functions without requiring memory reads, thus substituting a slower mechanical process with faster electronic computation.
2Measurement precision
If memory lookup tables are used for activation function computation, then accuracy is improved, but energy consumption increases
Solution Approach 1:
The patent extracts the essential computational pattern of activation functions and implements it directly in hardware circuitry, removing the dependency on memory lookup tables. This extraction of the core computational logic into dedicated hardware paths eliminates the energy-intensive memory access operations.
Solution Approach 2:
The patent replaces the energy-consuming memory lookup process with an electronic hardware approximation circuit that directly computes activation function values. The hardware circuit uses piecewise linear approximation with conditional logic to compute activation functions without requiring memory reads, thus reducing energy consumption.
3Adaptability or versatility
If complex non-linear activation functions are implemented, then computational capability is improved, but device complexity increases
Solution Approach 1:
The patent changes the computational approach from exact non-linear computation to piecewise linear approximation. By representing complex non-linear activation functions as sequences of linear segments with different slopes and intercepts, the patent simplifies the hardware implementation while maintaining sufficient computational accuracy for machine learning applications.
Solution Approach 2:
The patent segments complex non-linear activation functions into multiple linear segments. Each segment is defined by specific boundary conditions and linear parameters, allowing the hardware to process different ranges of input values through different simplified linear paths, thus reducing overall device complexity.
4Measurement precision
If vector operations are processed in multiple clock cycles, then computational accuracy is improved, but processing speed decreases
Solution Approach 1:
The patent performs preliminary organization of input data into vector format with proper alignment and preprocessing before the activation function computation. This preliminary action ensures that all necessary data is ready and properly formatted, enabling the hardware to process complete vectors in a single clock cycle without intermediate steps.
Solution Approach 2:
The patent merges multiple computational operations (data loading, preprocessing, activation function application, and output generation) into a single integrated hardware pipeline that processes complete vectors in one clock cycle. This merging of operations eliminates sequential processing delays and enables parallel computation across all vector elements.
Data Source
Figure 1A~1B
Figure 2
Figure 3
AI summary
Apparatuses, systems, and techniques to improve processing efficiency are provided. In at least one embodiment, a processing unit is described as including circuitry that receives an input vector and applies an activation function to the input vector by performing a hardware approximation of the activation function in a vector manner. The circuitry also generates an output vector based on the activation function.