Dynamic Input Skipping for Neural Network Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies, including CPUs, GPUs, and FPGAs, face challenges in efficiently processing artificial neural network (ANN) computations due to high computational loads, power consumption, and programming complexities, particularly in handling massive data movement and integration with AI frameworks.
Innovation Solution
A system comprising a controller, selector, and arithmetic unit dynamically selects input values from a stream based on criteria, such as comparison to a reference value, to perform mathematical operations only on necessary inputs, and includes a synchronization module to reorder neuron outputs for aligned processing, thereby reducing redundant operations and optimizing ANN computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPUs are used to process large ANNs due to higher throughput capacity, then computational speed is improved, but power consumption increases excessively
Solution Approach 1:
The patent extracts and processes only the non-zero input values from the neural network computations, separating them from the zero values that would otherwise require processing. This selective extraction approach allows the system to achieve high computational speed by focusing only on meaningful data while avoiding the power consumption associated with processing all inputs including zeros.
Solution Approach 2:
Instead of processing all input values (excessive action), the patent applies partial action by processing only the necessary non-zero inputs. The system dynamically determines which inputs require processing and applies computational resources only to those cases, optimizing the balance between speed and power consumption.
2Productivity
If FPGAs are configured to perform parallel computations for ANNs, then computational efficiency is improved, but programming complexity increases significantly
Solution Approach 1:
The patent creates a universal processing approach that can handle different neural network configurations and input patterns through a single dynamic selection mechanism. Rather than requiring custom FPGA configurations for different ANN architectures, the system uses a unified method of dynamically selecting and processing non-zero inputs, significantly reducing programming complexity while maintaining computational efficiency.
3Productivity
If all input values are processed in parallel within the same layer, then computational throughput is improved, but unnecessary operations on zero values increase power consumption
Solution Approach 1:
The patent implements skipping by dynamically identifying and bypassing zero input values during parallel processing. The system rushes through the selection and processing of only non-zero values, allowing parallel throughput to be maintained for meaningful computations while skipping the energy-wasting operations on zero inputs.
Data Source
AI summary
Systems and methods for accelerating artificial neural network computation are disclosed. An example may comprise selecting, by a controller communicatively coupled to a selector and an arithmetic unit and based on a criterion, an input value from the stream of input values of a neuron, configuring, by the controller, the selector to provide, dynamically, the selected input value to the arithmetic unit, providing, by the controller to the arithmetic unit, an information of the selected input value, acquiring, by the arithmetic unit and based on the information, a weight from a set of weights, and performing, by the arithmetic unit a mathematical operation on the selected input value and the weight to obtain a result, wherein the result is to be used to compute an output of the neuron. The criterion may include a comparison between the input value and a reference value. The reference value may include zero.