Multinomial Distribution Selection on Neural Network Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network processors face inefficiencies when making decisions based on multinomial distributions, as they often require sequential steps that disrupt parallel computation, leading to stalled operations and increased computational load.
Innovation Solution
An integrated circuit device, such as a neural network accelerator, is programmed to perform parallel computations by using multiple execution units to calculate cumulative sums of probabilities, allowing random selection based on multinomial distributions without stalling the main computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sequential steps are used to make decisions based on multinomial distributions, then selection accuracy is improved, but computational efficiency deteriorates due to stalled parallel operations
Solution Approach 1:
The patent divides the computation into separate parallel execution units: one unit handles the cumulative sum calculations while another unit performs the random selection based on the distribution. This segmentation allows both operations to proceed simultaneously without stalling, resolving the contradiction between maintaining selection accuracy and preserving computational efficiency
Solution Approach 2:
The patent pre-calculates the cumulative sums of the probability distribution before the random selection occurs. By preparing the cumulative sum data in advance, the selection process can proceed efficiently using the pre-computed values, avoiding the need for sequential computation during the critical selection phase
2Reliability
If sequential computation steps are implemented for multinomial distribution selection, then correctness of probability selection is improved, but computational load increases and parallel processing is disrupted
Solution Approach 1:
The patent merges the cumulative sum computation and the random selection operation into a unified parallel execution model. Both operations execute simultaneously in different execution units, reducing the overall computational load compared to sequential execution while maintaining the correctness of probability selection through the mathematical properties of the multinomial distribution
3Productivity
If random selection based on multinomial distributions is implemented, then computational parallelism is maintained, but additional computational load is introduced
Solution Approach 1:
The patent implements a self-service mechanism where the system generates its own random values and performs cumulative sum calculations using its internal execution units. This eliminates the need for external sequential processing while maintaining parallelism, and the computational load is distributed efficiently across multiple units working simultaneously rather than adding sequential overhead
Data Source
AI summary
An integrated circuit device such as a neural network accelerator can be programmed to select a numerical value based on a multinomial distribution. In various examples, the integrated circuit device can include an execution engine that includes multiple separate execution units. The multiple execution units can operate in parallel on different streams of data. For example, to make a selection based on a multinomial distribution, the execution units can be configured to perform cumulative sums on sets of numerical values, where the numerical values represent probabilities. In this example, to then obtain cumulative sums across the sets of numerical values, the largest values from the sets can be accumulated, and then added, in parallel to the sets. The resulting cumulative sum across all the numerical values can then be used to randomly select a specific index, which can provide a particular numerical value as the selected value.