Scratchpad Memory Access Across Data Lines for Latency Hiding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory access systems face challenges in efficiently managing data access in limited scratchpad memories and off-chip data latency, particularly in the context of compute-intensive tasks like convolutional neural networks, leading to inefficiencies in data transfer and latency.
Innovation Solution
A memory access device and method that allows multiple processing elements to access data lines based on command-specific thresholds, using a controller to manage access and storage of data across data lines, enabling concurrent operations by different processing elements on the same data lines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a scratchpad memory is used to reduce off-chip data access latency, then access speed is improved, but memory capacity is limited
Solution Approach 1:
The scratchpad memory is divided into multiple banks, and each bank is further divided into multiple data lines. This segmentation allows multiple processing elements to access different data lines simultaneously, effectively increasing the usable memory capacity while maintaining fast access speeds through parallel operations.
Solution Approach 2:
The patent introduces a new dimension of parallelism by allowing multiple processing elements to access the same bank's different data lines concurrently. This multi-dimensional access approach transforms the traditional single-processing-element-per-bank model into a many-to-many mapping, effectively expanding capacity without sacrificing speed.
2Productivity
If multiple processing elements access the same data line concurrently, then productivity is improved, but access control complexity increases
Solution Approach 1:
The memory is segmented into multiple banks, with each bank containing multiple data lines. This segmentation isolates access conflicts to the data line level rather than the bank level, simplifying control logic. The controller only needs to manage access at the data line granularity, reducing overall system complexity while enabling concurrent access.
Solution Approach 2:
The system dynamically assigns data lines to processing elements based on access patterns and thresholds. The controller monitors access counts and automatically switches between processing elements when thresholds are reached, providing adaptive resource allocation without requiring complex static scheduling mechanisms.
3Productivity
If data is transferred faster across data lines, then productivity is improved, but data transfer management complexity increases
Solution Approach 1:
The system implements self-service through automatic threshold monitoring and processing element switching. The controller automatically tracks access counts for each data line and switches processing elements when thresholds are reached, eliminating the need for external management intervention and simplifying overall data transfer management while maintaining high speed.
Solution Approach 2:
The system incorporates feedback mechanisms where the controller continuously monitors access patterns and adjusts processing element assignments based on real-time data. This feedback loop enables automatic optimization of data transfer operations, reducing management complexity by allowing the system to self-regulate based on observed access patterns.
Data Source
AI summary
Provided is an operating method of a memory access device, the operating method including allowing an access of a first processing element to a plurality of data lines of a memory based on a first command, identifying a number of accesses of the first processing element to the plurality of data lines, and when a number of accesses of the first processing element to a first data line among the plurality of data lines reaches a predetermined number of accesses, allowing an access of a second processing element to the first data line based on a second command.