Neural Processing System With Bridge Units For Power Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning architectures, such as Convolutional Neural Networks (CNNs), require significant hardware resources and high power consumption, making them costly and inefficient, especially in mobile systems where low cost and low power consumption are essential for implementing artificial intelligence.
Innovation Solution
A neural processing system is designed with multiple neural processing units operating in different clock domains and utilizing a bridge unit for data transmission between them, allowing for selective control of power consumption and processing speeds, and a workload manager to allocate data and control clock frequencies, thereby reducing overall power usage and costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a CNN system is implemented with many parameters and operations to achieve high-level abstraction and deep learning functionality, then the intelligence capability is improved, but the hardware resource cost and power consumption increase significantly
Solution Approach 1:
The system divides the CNN processing into multiple independent neural processing units (NPUs), each handling specific feature extraction operations. This segmentation allows parallel processing of different feature maps and weights, reducing the time and energy required for each individual operation while maintaining overall intelligence capability.
Solution Approach 2:
The patent introduces a multi-dimensional architecture with multiple NPUs operating in parallel, adding a spatial dimension to the processing. Each NPU processes different portions of the data simultaneously, transforming the sequential processing bottleneck into a parallel multi-dimensional operation that reduces total execution time and power consumption.
2Adaptability or versatility
If a CNN system is implemented with many parameters and operations to achieve high-level abstraction and deep learning functionality, then the intelligence capability is improved, but the hardware resource cost increases
Solution Approach 1:
The system segments the CNN architecture into multiple specialized NPUs, each dedicated to specific feature extraction tasks. This segmentation distributes the hardware resources across multiple units, preventing any single unit from requiring excessive resources while collectively achieving the needed intelligence capability.
Solution Approach 2:
Each NPU is designed as a universal processing unit capable of handling different feature maps and weight combinations. This multi-functionality allows the same hardware structure to be reused across multiple operations, reducing overall hardware resource requirements compared to having dedicated specialized units for each operation.
3Use of energy by moving object
If multiple neural processing units operate in different clock domains to allow selective control of power consumption and processing speeds, then power efficiency is improved, but the system complexity increases due to bridge units and clock domain coordination
Solution Approach 1:
The system employs dynamic clock domain management where each NPU can operate at different clock frequencies based on its current workload and power requirements. This dynamic adjustment allows the system to optimize power efficiency by reducing clock speeds for less critical operations while maintaining high performance where needed, with bridge units managing the coordination between different clock domains.
Data Source
AI summary
A neural processing system includes a first frontend module, a second frontend module, a first backend module, and a second backend module. The first frontend module executes a feature extraction operation using a first feature map and a first weight, and outputs a first operation result and a second operation result. The second frontend module executes the feature extraction operation using a second feature map and a second weight, and outputs a third operation result and a fourth operation result. The first backend module receives an input of the first operation result provided from the first frontend module and the fourth operation result provided from the second frontend module via a second bridge to sum up the first operation result and the fourth operation result. The second backend module receives an input of the third operation result provided from the second frontend module and the second operation result provided from the first frontend module via a first bridge to sum up the third operation result and the second operation result.