Neural Network Processor Conditional Branching via Tensor Arithmetic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processing systems lack efficient mechanisms for implementing control flow operations, such as conditional branching, which limits the flexibility and optimization of neural network processing tasks.
Innovation Solution
Incorporating a new type of conditional branching instruction that uses tensor arithmetic operations to determine whether a branching condition is satisfied, allowing the processor to trigger a branch operation without requiring additional dedicated hardware, by re-purposing the existing execution unit to perform these calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional neural network processing systems are used without conditional branching instructions, then the hardware implementation remains simple, but the flexibility and optimization capability of neural network processing tasks is limited
Solution Approach 1:
The execution unit is designed to perform multiple functions: it can execute both traditional tensor arithmetic operations and conditional branching instructions. The same execution unit that performs matrix multiplications and convolutions is also capable of evaluating branch conditions and controlling program flow, eliminating the need for separate dedicated hardware for control flow operations.
Solution Approach 2:
The patent merges the control flow functionality with the existing execution unit by incorporating conditional branching instructions into the tensor processing pipeline. The branch condition evaluation, target address calculation, and program counter update are integrated into the same hardware resources that handle tensor operations, combining what were traditionally separate functions into a unified execution architecture.
2Productivity
If dedicated hardware is added for control flow operations, then the processing efficiency and flexibility improve, but the hardware area and complexity increase significantly
Solution Approach 1:
The execution unit serves dual purposes by handling both tensor arithmetic and control flow operations. This multi-functionality allows the system to achieve improved processing efficiency for neural network tasks that require conditional branching without adding dedicated control flow hardware, thereby avoiding increases in hardware area.
Solution Approach 2:
The execution unit performs its own control flow management by evaluating branch conditions using its existing tensor processing capabilities and updating its own program counter. This self-service approach eliminates the need for external or dedicated control units, maintaining hardware area efficiency while enabling sophisticated control flow operations.
3Area of stationary object
If conditional branching instructions are implemented using existing execution units, then the hardware area remains small, but the processing overhead for determining branch conditions may increase
Solution Approach 1:
The patent combines control flow operations with the tensor processing pipeline by integrating branch condition evaluation into the existing execution unit. This merging allows the system to evaluate branch conditions using the same efficient tensor arithmetic hardware, avoiding the creation of separate slow control paths while maintaining compact hardware area.
Solution Approach 2:
The execution unit maintains continuous operation by evaluating branch conditions as part of the normal tensor processing flow rather than as separate interrupt-driven operations. The program counter updates and branch target calculations are performed continuously within the execution pipeline, minimizing idle cycles and reducing processing overhead.
Data Source
AI summary
Disclosed are methods of operating processor to perform neural network processing. The processor comprises a control circuit that is operable to cause an execution unit of the processor to perform tensor arithmetic operations. A program to perform neural network processing includes conditional branching instructions that cause the program execution to branch to a different part of the program when an associated branching condition is satisfied. The control circuit causes the execution unit to perform a set of one or more tensor arithmetic operations to determine a value indicative of whether a branching condition is satisfied, with the value then being communicated to the control circuit such that the control circuit, when processing the conditional branching instruction, triggers the branch (or not). Also disclosed are processors configured in this way and methods of compiling programs including such conditional branching instructions.