Inline Load Inspection for Sparse Deep Learning Arithmetic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning applications face inefficiencies due to sparse data sparsity in activations or weights, leading to wasteful arithmetic operations on zero values, which can harm processing time and performance.
Innovation Solution
An inline data inspection technique that automatically detects zero or threshold values during data loading, eliminating unnecessary arithmetic operations without additional instructions, by using an inspection circuit to compute predicate values for subsequent instructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If arithmetic operations are performed on all data elements including zero values, then processing is simplified and uniform, but processing time increases and performance decreases
Solution Approach 1:
The inspection circuit performs preliminary inspection of data elements immediately after loading from memory, determining which elements are zero or below threshold before arithmetic operations are executed. This preliminary action enables subsequent instructions to skip unnecessary operations, resolving the contradiction by maintaining simple uniform processing for non-zero elements while avoiding wasteful operations on zero elements.
Solution Approach 2:
An inspection circuit is introduced as an intermediary component between memory and arithmetic processing units. This intermediary inspects data elements and generates predicate signals that control whether arithmetic operations are performed, allowing the system to maintain simple uniform processing logic while selectively avoiding operations on zero or threshold values to improve performance.
2Device complexity
If arithmetic operations are performed on zero values, then processing logic remains simple and uniform, but power consumption increases
Solution Approach 1:
The inspection circuit performs preliminary inspection of data elements to identify zero or threshold values before arithmetic operations are executed. This preliminary action enables the system to avoid unnecessary arithmetic operations on zero elements, reducing power consumption while maintaining relatively simple processing logic through the use of predicate signals that control operation execution.
Solution Approach 2:
An inspection circuit serves as an intermediary that inspects data elements and generates predicate signals to control arithmetic operations. This intermediary enables selective execution of operations based on data values, reducing power consumption by avoiding operations on zero elements while keeping the overall processing logic relatively simple through predicate-driven control.
3Productivity
If additional inspection instructions are added to detect zero values, then arithmetic operations on zero values can be eliminated, but instruction count increases and processing overhead increases
Solution Approach 1:
The inspection functionality is merged with the existing data loading operation. The inspection circuit inspects data elements as they are loaded from memory, combining the inspection function with the load operation itself. This eliminates the need for separate inspection instructions, maintaining arithmetic operation efficiency by avoiding operations on zero elements while keeping the instruction count low.
Solution Approach 2:
The data loading operation performs self-inspection through the inspection circuit that is integrated into the load/store unit. The inspection occurs automatically as part of the loading process without requiring additional instructions from the program, enabling elimination of arithmetic operations on zero elements while maintaining simple instruction sequences.
4Productivity
If inline data inspection is implemented, then processing performance improves by avoiding operations on zero values, but hardware complexity increases
Solution Approach 1:
The inspection circuit is designed with multi-functionality, serving both to inspect data elements for zero or threshold values and to generate predicate signals that control subsequent arithmetic operations. This universal component improves processing performance by enabling selective operation execution while minimizing hardware complexity through its dual role in inspection and control signal generation.
Solution Approach 2:
An inspection circuit is introduced as an intermediary component between memory and arithmetic processing units. This intermediary inspects data elements and generates predicate signals that control whether arithmetic operations are performed, allowing the system to improve processing performance by avoiding unnecessary operations while adding minimal hardware complexity through a dedicated inspection component.
Data Source
AI summary
A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.