Compiler for Neural Network Gating on Integrated Circuits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network implementations on integrated circuits face challenges with high latency and power consumption due to unnecessary clock cycles and power retention in components, particularly when dealing with numerous configuration options for each layer of the network.
Innovation Solution
A compiler is developed to optimize the implementation of machine-trained neural networks on integrated circuits by generating configuration data that leverages time- and power-saving features, including zero channel removal operations, sparsity-based optimizations, and efficient memory management to reduce the number of cores and channels required, thereby minimizing power consumption and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional neural network configuration methods are used, then complete neural network operations can be performed, but power consumption and latency increase due to unnecessary clock cycles and power retention in components
Solution Approach 1:
The compiler dynamically changes configuration parameters (number of active cores, channels, precision settings) based on the specific neural network layer and computational requirements. This allows the system to adapt power consumption and resource allocation to actual needs rather than maintaining fixed high-power configurations, directly resolving the contradiction between energy efficiency and operational completeness
Solution Approach 2:
The system transitions from static configuration to dynamic configuration where the compiler generates optimized settings for each layer. The number of active cores, channels, and precision levels are dynamically adjusted based on computational requirements, enabling the system to reduce power consumption when full resources are not needed while maintaining performance when required
2Productivity
If all cores and channels are activated for every layer, then computational completeness is maintained, but latency increases due to unnecessary clock cycles
Solution Approach 1:
The compiler extracts and activates only the necessary subset of cores and channels required for each specific neural network layer. By removing unnecessary computational units from active duty, the system avoids the latency associated with coordinating and clocking all available resources, while still maintaining computational completeness for the current layer's requirements
Solution Approach 2:
The system applies partial action by activating only the minimum necessary resources (cores, channels) required for each layer's computation. This prevents excessive resource allocation that would increase latency through unnecessary clock cycles, while ensuring sufficient resources are activated to maintain productivity and meet performance requirements
3Measurement precision
If high precision computation is used throughout the network, then computational accuracy is maintained, but power consumption increases
Solution Approach 1:
The compiler applies different precision levels to different layers and computational stages based on their specific requirements. Critical layers maintaining high precision while less critical layers use lower precision, allowing the system to maintain overall computational accuracy while reducing power consumption in areas where full precision is not necessary
Solution Approach 2:
The system dynamically changes precision parameters across different layers and computational stages. The compiler analyzes each layer's requirements and adjusts precision settings accordingly, enabling high precision where needed for accuracy while using lower precision where sufficient, thereby resolving the contradiction between measurement precision and energy consumption
Data Source
AI summary
Some embodiments provide a compiler for optimizing the implementation of a machine-trained network (e.g., a neural network) on an integrated circuit (IC). The compiler of some embodiments receives a specification of a machine-trained network including multiple layers of computation nodes and generates a graph representing options for implementing the machine-trained network in the IC. In some embodiments, the compiler also generates instructions for gating operations. Gating operations, in some embodiments, include gating at multiple levels (e.g., gating of clusters, cores, or memory units). Gating operations conserve power in some embodiments by gating signals so that they do not reach the gated element or so that they are not propagated within the gated element. In some embodiments, a clock signal is gated such that a register that transmits data on a rising (or falling) edge of a clock signal is not triggered.