Neural Network Compute Units Using Indexed Non-Zero Activations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network compute units perform numerous unnecessary computations due to multiplying with zero input activation values, leading to inefficiency and energy wastage.
Innovation Solution
A method and system that identifies and stores only non-zero input activations, generating an index for memory locations with these values, and provides them to computational units while skipping computations with zeros, thereby optimizing memory usage and computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If all input activations are processed through matrix multiplication, then complete computational accuracy is maintained, but numerous unnecessary computations are performed multiplying by zero values
Solution Approach 1:
The patent extracts and processes only the non-zero input activations through the neural network compute unit, separating them from zero-valued activations. By identifying and isolating non-zero elements using an index structure, the system performs computations only on necessary data, eliminating wasteful multiplications by zero while maintaining computational accuracy.
Solution Approach 2:
The patent applies partial action by processing only a subset of input activations—specifically, only those with non-zero values. Instead of performing complete matrix multiplication on all input elements, the system selectively applies computations to relevant portions of the data, reducing overall computational workload while preserving essential functionality.
2Quantity of substance
If all input activations are stored in memory, then complete data availability is ensured, but memory storage space is wasted on zero values
Solution Approach 1:
The patent extracts non-zero input activations from the complete input dataset and stores only these extracted elements in memory. By using an index structure to track positions of non-zero values, the system maintains data availability for computations while significantly reducing the quantity of data that must be stored, thus optimizing memory utilization.
3Device complexity
If a brute force method is used for matrix multiplication, then computational simplicity is maintained, but the processor architecture limits parallelization capability
Solution Approach 1:
The patent segments the input activations into non-zero and zero-valued groups, with the index structure organizing non-zero elements for efficient access. This segmentation enables the computational system to process only relevant data portions in parallel, overcoming architecture limitations by restructuring data organization rather than changing the fundamental brute force computation approach.
Data Source
AI summary
A computer-implemented method includes receiving, by a computing device, input activations and determining, by a controller of the computing device, whether each of the input activations has either a zero value or a non-zero value. The method further includes storing, in a memory bank of the computing device, at least one of the input activations. Storing the at least one input activation includes generating an index comprising one or more memory address locations that have input activation values that are non-zero values. The method still further includes providing, by the controller and from the memory bank, at least one input activation onto a data bus that is accessible by one or more units of a computational array. The activations are provided, at least in part, from a memory address location associated with the index.