SOMAC Instruction Pruning via Macro-Instruction Iterator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computationally intensive Multiply-Accumulate (MAC) operations in neural networks, particularly during graph streaming processing, are resource-heavy and require optimization to reduce operational overhead.
Innovation Solution
A method and system that schedules and executes threads on a Graph Streaming processor, utilizing a thread arbiter and macro-instruction iterator to identify and optimize Sum-Of-Multiply-Accumulate (SOMAC) instructions by determining instruction size, skipping iterations where source operands are zero, and using an instruction mask to disable unnecessary iterations, thereby reducing the number of operations performed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If SOMAC instructions are executed with full iterations as indicated by instruction size, then computational completeness is maintained, but operational overhead increases
Solution Approach 1:
The patent applies partial action by executing only the necessary number of iterations for SOMAC instructions rather than always executing the full instruction size. The system determines the actual number of iterations needed and executes only that many, avoiding unnecessary operations while maintaining computational completeness.
Solution Approach 2:
The patent changes the iteration parameter dynamically based on the actual computational needs. Instead of using the fixed instruction size as the iteration count, the system adjusts the iteration parameter to match the actual number of operations required, thereby reducing operational overhead while maintaining reliability.
2Reliability
If all iterations of SOMAC instructions are executed, then processing thoroughness is ensured, but processing time increases
Solution Approach 1:
The system performs only the partial number of iterations that are actually necessary for correct processing. By analyzing the instruction and determining the minimal required iterations, the system avoids executing excessive iterations that would waste time while ensuring processing thoroughness is maintained for the necessary operations.
Solution Approach 2:
The patent performs preliminary analysis of the SOMAC instruction to determine the actual number of iterations needed before execution. This preliminary action allows the system to prepare the correct iteration count in advance, avoiding unnecessary iterations and reducing processing time while ensuring thoroughness.
3Reliability
If MAC operations are performed without optimization, then computational accuracy is maintained, but resource utilization decreases
Solution Approach 1:
The patent extracts and removes unnecessary iterations from SOMAC instructions. By identifying and eliminating redundant operations where the instruction size exceeds the actual needed iterations, the system maintains computational accuracy while improving resource utilization by reducing the number of MAC operations performed.
Solution Approach 2:
The system changes the iteration parameter from the original instruction size to an optimized value that reflects the actual computational needs. This parameter change ensures that enough iterations are performed to maintain accuracy while avoiding excessive iterations that would degrade resource utilization.
Data Source
AI summary
Methods, systems and apparatuses for reducing operations of Sum-Of-Multiply-Accumulate (SOMAC) instructions are disclosed. One method includes scheduling, by a scheduler, a thread for execution, executing, by a processor of a plurality of processors, the thread, fetching, by the processor, a plurality of instructions for the thread from a memory, selecting, by a thread arbiter of the processor, an instruction of the plurality of instructions for execution in an arithmetic logic unit (ALU) pipeline of the processor, and reading the instruction, and determining, by a macro-instruction iterator of the processor, whether the instruction is a Sum-Of-Multiply-Accumulate (SOMAC) instruction with an instruction size, wherein the instruction size indicates a number of iterations that the SOMAC instruction is to be executed.