Shared Extended Memory Mapping Across GPU and PIM Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices face challenges in balancing memory capacity, bandwidth, and cost efficiency, particularly in AI applications where large datasets and advanced algorithms require efficient data processing.
Innovation Solution
A computing system with a GPU and PIM architecture integrated with HBM and LPDDR memory, allowing for flexible usage modes and shared memory allocation across processing devices, enhancing memory capacity and bandwidth while minimizing costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If memory capacity is increased to handle large datasets for AI applications, then data processing capability is improved, but cost and power consumption increase
Solution Approach 1:
The system segments memory into two distinct types: HBM2e memory for high-bandwidth AI processing workloads and LPDDR4x memory for lower-bandwidth operations. This segmentation allows each memory type to be optimized for its specific use case, enabling the system to handle large datasets with improved power efficiency by using the appropriate memory type for each task rather than relying solely on high-capacity HBM2e for all operations.
Solution Approach 2:
The memory management device provides universal memory management capabilities that handle both HBM2e and LPDDR4x memory types through a unified interface. The system can dynamically allocate and manage memory resources across different memory types, allowing a single system to serve multiple functions and workload types while optimizing power consumption based on the specific requirements of each AI application.
2Productivity
If memory bandwidth is increased to accelerate AI computations, then processing speed is improved, but device complexity increases
Solution Approach 1:
The memory management device acts as an intermediary between the CPU/GPU and the dual memory system (HBM2e and LPDDR4x). It translates memory access requests into appropriate commands for the target memory type, manages data transfers between different memory spaces, and handles address mapping. This intermediary layer simplifies the overall system architecture by providing a unified memory interface while enabling complex dual-memory operations, thereby improving processing speed without proportionally increasing device complexity.
3Adaptability or versatility
If dual memory architecture is implemented to balance capacity and efficiency, then cost efficiency is improved, but memory management complexity increases
Solution Approach 1:
The memory management device implements self-service capabilities through automated memory allocation, address mapping, and data transfer management between HBM2e and LPDDR4x memory spaces. The system automatically determines the optimal memory type for each workload based on bandwidth requirements and data size, eliminating the need for manual configuration or complex software management. This self-managing approach provides versatile usage flexibility while keeping memory management complexity contained within the dedicated memory management hardware.
Data Source
AI summary
Systems and methods for extended memory are disclosed. An apparatus for extended memory may include a first processing device; a second processing device; a first memory device; and a second memory device. A first logical memory space and a second logical memory space are configured to be allocated for respectively the first processing device and the second processing device. The first logical memory space and the second logical memory space are further configured to be mapped to a first physical memory space of one of the first memory device or the second memory device.