Hierarchical Neural Network Processor for Dense Low-Power Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing engines face inefficiencies in computational unit density and power consumption, and existing weight quantization methods are limited to post-training stages, not fully exploring potential modifications in weights for improved performance.
Innovation Solution
A neural network processing engine with a hierarchical architecture that includes self-contained computational units, lean control logic, and dynamic resource assignment, allowing for efficient computation and reduced power consumption, along with the ability to modify weights during both training and inference stages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing neural network processing engines use traditional computational architectures, then implementation flexibility is maintained, but computational density and power efficiency are insufficient
Solution Approach 1:
The processing engine is divided into multiple computational units (CUs) organized in a hierarchical structure with different levels (e.g., CU level, core level, chip level). Each level performs specific computational aggregations, allowing fine-grained control over computational density and power consumption at different scales of the neural network.
Solution Approach 2:
The architecture enables dynamic resource allocation where computational units can be activated or deactivated based on the specific neural network model being executed. The hierarchical structure allows flexible configuration of which computational levels are active, optimizing the balance between computational density and power consumption for different workloads.
2Use of energy by stationary object
If computational units are aggregated in hierarchical levels, then power efficiency is improved, but control complexity increases
Solution Approach 1:
Control functions are segmented and distributed across the hierarchical levels. Each level has its own control logic that manages only the computational units at that level, reducing the control burden on higher levels. This distributed control approach simplifies the overall control architecture despite the hierarchical aggregation.
Solution Approach 2:
Computational units at different hierarchical levels serve multiple functions: they perform local computations, aggregate results from lower levels, and contribute to higher-level computations. This multi-functionality reduces the need for separate dedicated control mechanisms, simplifying the control architecture while maintaining power efficiency.
3Productivity
If dedicated memory elements are associated with computational units, then processing efficiency is improved, but memory usage increases
Solution Approach 1:
Different hierarchical levels have different memory requirements and characteristics. Lower levels with simpler computational units have smaller associated memory elements, while higher levels with more complex units have larger memory elements. This local differentiation optimizes the balance between processing efficiency and memory usage for each level.
Solution Approach 2:
The memory hierarchy is nested within the computational hierarchy, with each level containing memory elements that serve that level's computational units. Higher levels can access memory at lower levels, creating a nested memory structure that reduces overall memory requirements while maintaining efficient access patterns for the neural network computations.
Data Source
AI summary
A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs). The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.