Scalable Multi-Die DNN Package with Ground Reference Signaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face challenges in scalability and efficiency due to high design complexity and manufacturing costs, particularly in custom inference accelerators for diverse applications like image recognition and autonomous driving, which require balancing performance, energy efficiency, and accuracy within specific constraints.
Innovation Solution
A scalable multi-die package design with a tiled architecture, utilizing ground reference signaling transceivers for inter-die communication and flexible weight and activation tiling across processing elements, dice, and package levels, optimizing data locality and reducing communication costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If custom inference accelerators are designed for specific performance constraints, then performance and accuracy are improved, but design complexity and manufacturing costs increase
Solution Approach 1:
The system is divided into multiple identical processing cores that can be replicated across multiple dies. Each die contains a standardized set of processing elements, allowing the system to be segmented into modular units that can be manufactured independently and then combined through inter-die interconnects to achieve the desired performance level.
Solution Approach 2:
The patent employs a universal processing core design that can be used across all dies in the system. This standardized core architecture serves multiple functions and can be configured for different performance levels by simply changing the number of cores and their interconnection pattern, rather than designing custom architectures for each performance requirement.
2Reliability
If custom inference accelerators are designed for specific performance constraints, then performance and accuracy are improved, but manufacturing costs increase
Solution Approach 1:
By segmenting the system into identical reusable die units, the manufacturing process benefits from standardization. Each die can be manufactured using the same process flow and design rules, enabling volume production and reducing per-unit costs through economies of scale, while still achieving high performance through parallel execution across multiple dies.
Solution Approach 2:
The patent implements a copy-based architecture where identical processing cores are replicated across multiple dies. This copying approach allows the same proven design to be manufactured repeatedly with consistent quality and performance characteristics, reducing development costs and enabling efficient manufacturing through standardization.
3Productivity
If multiple dice are used to scale the system, then processing capacity is improved, but inter-die communication overhead increases
Solution Approach 1:
The patent merges multiple identical dies into a unified processing system through standardized interconnects. By combining the processing capacity of multiple dies while maintaining a regular interconnection pattern, the system achieves high processing capacity without proportionally increasing communication overhead, as the inter-die communication follows the same efficient patterns as intra-die communication.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables efficient and scalable DNN processing, balancing performance, energy efficiency, and accuracy across various applications by distributing weights and activations effectively, thereby reducing engineering costs and design cycles for custom hardware.
Implementation Method 1
low-energy, high-bandwidth ground reference signaling (GRS) transceivers for inter-die communication
Data Source
AI summary
A distributed deep neural net (DNN) utilizing a distributed, tile-based architecture implemented on a semiconductor package. The package includes multiple chips, each with a central processing element, a global memory buffer, and processing elements. Each processing element includes a weight buffer, an activation buffer, and multiply-accumulate units to combine, in parallel, the weight values and the activation values.