Pre-Configurable HBM Array Layout for Adaptive GPU Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional semiconductor devices face challenges in meeting increasing demands for faster, more efficient, and smaller form factors while maintaining circuit robustness and failure detectability, particularly in GPU-centric systems with limited flexibility to adapt to varying bandwidth requirements for AI model workloads.
Innovation Solution
A pre-configurable system architecture that allows flexible connection and communication between GPUs, high bandwidth memory (HBM) cubes, and high bandwidth storage (HBS) cubes via an interposer, utilizing physical layer circuits and secondary communication circuits to dynamically adjust device configurations based on real-time computing needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If device footprint is reduced to meet market demands, then device size decreases, but circuit robustness and failure detectability deteriorate
Solution Approach 1:
The patent segments the semiconductor device into multiple independent die (compute die, memory die, storage die) that are stacked and interconnected via silicon interposer. This segmentation allows each die to be optimized independently while maintaining overall system reliability through modular architecture and distributed interconnect structures.
Solution Approach 2:
The patent transitions from planar 2D layout to 3D stacked architecture, moving circuits vertically across multiple layers and interposers. This dimensional change enables significant reduction in footprint while maintaining circuit robustness through spatial distribution and redundant interconnect paths in the vertical dimension.
2Productivity
If circuit density is increased to improve device performance, then processing capacity increases, but manufacturing complexity and failure risk increase
Solution Approach 1:
The patent divides the high-density processing functionality across multiple specialized die (compute die with ALUs, memory die with SRAM/DRAM, storage die with Flash/NAND). This segmentation concentrates density where needed while simplifying manufacturing of individual die types using optimized process nodes for each function.
Solution Approach 2:
The silicon interposer provides universal interconnect functionality that can be configured through routing patterns to support various compute-memory-storage configurations. This multi-functional interposer simplifies manufacturing by using a single standardized component rather than custom interconnect structures for each device variant.
3Speed
If operating speed is increased to meet performance demands, then processing speed increases, but power consumption increases
Solution Approach 1:
The patent implements local quality by placing high-speed SRAM cache closer to compute ALUs on compute die, medium-speed DRAM in separate memory die at intermediate distances, and lower-speed Flash storage in storage die farther away. This spatial differentiation of storage speeds allows fast operations near the processor while using slower, lower-power storage for less time-critical data.
Solution Approach 2:
The patent creates a nested hierarchy of storage layers (SRAM cache within compute die, DRAM memory in adjacent memory die, Flash storage in external storage die) where each layer provides different speed-capacity tradeoffs. This nested structure enables the system to use high-speed storage for frequent accesses and low-speed storage for bulk data, optimizing the speed-power balance.
4Adaptability or versatility
If device footprint is reduced to integrate more functionality, then integration capacity increases, but failure detectability and robustness decrease
Solution Approach 1:
The patent segments integrated functionality into separate fail-isolated die (compute die, memory die, storage die) connected through an interposer. This segmentation contains potential failures to individual die, making them easier to detect and isolate. The interposer provides dedicated test access points for each die, improving failure detectability despite high integration capacity.
Solution Approach 2:
The silicon interposer acts as an intermediary layer that provides test structures, routing, and isolation between functional die. This intermediary enables independent testing of each die type while maintaining their integrated functionality, improving failure detectability without sacrificing integration capacity.
Data Source
AI summary
An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The high bandwidth memory circuit can include two or more physical layer circuits to communicate with neighboring devices. The high bandwidth memory circuit can broadcast a status to the neighboring devices. The neighboring devices can be configured according to the operating demands of the high bandwidth memory circuit.