SIMD Operation Selection in Neural Processing Units Without Predication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing techniques, particularly using single instruction multiple data (SIMD) processors, suffer from inefficiencies due to predication, leading to wasted computing cycles and increased energy usage, especially in tasks with predictable processing paths like convolutional neural networks.
Innovation Solution
Implementing techniques that select operations based on specific positions or stanzas within the neural network processing, using a program counter to limit execution to groups of operations defined by these stanzas, thereby avoiding the need for predication and reducing unnecessary computing cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If predication is used to selectively execute operations in SIMD processors, then operations can be conditionally performed based on data values, but computing cycles are wasted and energy consumption increases due to evaluating predicates for all operations
Solution Approach 1:
The patent applies preliminary action by pre-grouping operations into stanzas with predetermined beginning and ending pointers before execution. The compiler organizes operations into structured groups during compilation, so that during runtime the SIMD processor can directly jump to and execute specific stanzas without evaluating predicates for skipped operations. This eliminates the energy-wasting predicate evaluation while maintaining conditional execution capability through selective stanza invocation based on processing position.
2Productivity
If predication is used to skip operations based on conditions, then unnecessary operations are avoided, but fetch performance decreases due to skipped instructions and complex control flow
Solution Approach 1:
The patent applies segmentation by dividing the operation stream into discrete stanzas with clear beginning and ending pointers. Each stanza represents a self-contained group of operations that can be independently selected and executed. This segmentation eliminates complex control flow within stanzas, allowing the SIMD processor to efficiently fetch and execute contiguous instruction blocks without the performance penalty of scattered conditional jumps and predicate evaluations associated with traditional predication methods.
3Reliability
If all operations are executed in SIMD processors, then complete processing is ensured, but computing cycles are wasted on operations that should be skipped
Solution Approach 1:
The patent applies dynamics by making the execution scope of SIMD operations adaptive and position-dependent. Instead of statically executing all operations or using complex dynamic predication, the system dynamically adjusts the execution boundary to only include relevant stanzas based on the current processing position in the neural network. This dynamic stanza selection ensures complete processing of necessary operations while eliminating waste on irrelevant operations, achieving both reliability and productivity.
Data Source
AI summary
Systems and methods for efficient selection of single instruction multiple data operations for neural processing units. An example processor system comprises a matrix processor configured to perform convolutions associated with a neural network and single instruction multiple data (SIMD) processors in communication with the matrix processors, with the SIMD processors being configured to execute a group of operations based on a current position associated with processing the neural network, and with the group of operations being selected from multiple SIMD programs, and with the group of operations being selected from the SIMD programs according to the current position.