Local Data Share Unit for Flexible Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems lack a flexible shared memory architecture that allows for easy adjustment of memory banks, conflict resolution, and full accessibility, limiting performance and power efficiency.
Innovation Solution
A local data share (LDS) unit enables multiple threads to share data cooperatively, using a conflict state machine to manage memory access and a shared memory architecture that allows repeated access with low latency, enabling higher memory bandwidth and power reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing shared memory architecture is used, then data sharing between threads is possible, but the architecture cannot be easily changed and lacks flexibility
Solution Approach 1:
The shared memory is divided into multiple banks, allowing independent access to different segments. This segmentation enables flexible configuration where each bank can be independently managed and accessed by different thread groups, providing architectural flexibility without requiring complete redesign.
Solution Approach 2:
The system dynamically configures thread groups and their access patterns to memory banks based on runtime requirements. Thread groups can be dynamically formed and dissolved, and memory bank assignments can be adjusted on-the-fly, providing adaptability while maintaining a stable underlying architecture.
2Productivity
If more memory banks are added to increase bandwidth, then memory bandwidth increases, but device complexity and cost increase
Solution Approach 1:
Each memory bank is designed to serve multiple purposes and be accessible by multiple thread groups. The same memory bank can be shared across different thread groups at different times, allowing the system to achieve high bandwidth through temporal multiplexing rather than requiring proportionally more physical memory banks.
Solution Approach 2:
The system maintains continuous memory access by overlapping operations across multiple thread groups. While one thread group accesses memory bank A, another thread group simultaneously accesses memory bank B, ensuring continuous useful action and high effective bandwidth without requiring excessive memory banks.
3Productivity
If repeated memory access is enabled for higher bandwidth, then memory bandwidth increases, but power consumption increases
Solution Approach 1:
Data is pre-loaded into the shared memory banks by thread groups before the main computation phase. This preliminary action ensures that frequently accessed data is already in the fast shared memory, eliminating the need for repeated high-power accesses to external memory during computation, thus reducing overall power consumption while maintaining high bandwidth for essential operations.
4Reliability
If full accessibility and conflict resolution are implemented, then data sharing reliability improves, but device complexity increases
Solution Approach 1:
Thread groups autonomously manage their own access to memory banks without requiring complex centralized arbitration. Each thread group independently selects and accesses assigned memory banks, and the system provides basic conflict detection and resolution through simple protocols, achieving reliable data sharing with minimal complexity overhead.
Data Source
AI summary
Embodiments for a local data share (LDS) unit are described herein. Embodiments include a co-operative set of threads to load data into shared memory so that the threads can have repeated memory access allowing higher memory bandwidth. In this way, data can be shared between related threads in a cooperative manner by providing a re-use of a locality of data from shared registers. Furthermore, embodiments of the invention allow a cooperative set of threads to fetch data in a partitioned manner so that it is only fetched once into a shared memory that can be repeatedly accessed via a separate low latency path.