Memory Unit Step Cache Scratchpad ESM Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing emulated shared memory architectures face challenges in optimizing memory access and multi-prefixed operations due to complex hardware requirements and inefficiencies in cache coherence, leading to performance limitations and increased complexity.
Innovation Solution
A memory unit with a step cache and scratchpad configuration that reduces the number of requests per memory location by filtering out subsequent references, allowing sequential processing and simplifying multi-operation execution, while utilizing a hash and compose unit to manage memory references and a reply receive buffer to handle incoming data, thereby reducing latency and hardware complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If distributed shared memory architecture with multiple processor-cache pairs is used, then memory capacity and parallelism are improved, but cache coherency maintenance cost and latency increase
Solution Approach 1:
The patent introduces a centralized memory controller as an intermediary component that manages memory access and coherency for multiple processor-cache pairs. This controller acts as a mediator between processors and memory, handling coherency protocols and reducing the complexity burden on individual processor-cache units.
Solution Approach 2:
The memory controller is designed as a universal component that handles multiple functions including memory arbitration, coherency management, and address translation for all processor-cache pairs, consolidating what would otherwise be duplicate functionality in each processor unit.
2Reliability
If step cache associativity is increased to handle concurrent accesses, then memory access correctness is improved, but hardware area and complexity increase
Solution Approach 1:
The patent employs a low associativity step cache that only tracks a limited number of concurrent accesses (e.g., 2-4 ways) rather than tracking all possible processor accesses. This partial tracking approach is sufficient for most practical workloads and dramatically reduces hardware area while maintaining correctness for the majority of cases.
Solution Approach 2:
The system dynamically adjusts step cache behavior based on workload characteristics, changing parameters such as associativity level and validation strictness to balance between correctness and hardware resource usage depending on the specific access patterns detected.
3Productivity
If multiport step caches and scratchpads are used to support concurrent reads and writes, then memory operation throughput is improved, but device complexity and port requirements increase
Solution Approach 1:
The patent implements periodic validation cycles where the step cache checks for conflicts between concurrent reads and writes at regular intervals rather than continuously monitoring every access. This periodic validation approach maintains throughput while reducing the complexity of continuous multiport arbitration logic.
Solution Approach 2:
The memory system is segmented into separate read and write paths with independent validation logic, allowing concurrent operations to proceed independently when no conflicts exist, while only requiring coordination when read-write or write-write conflicts are detected.
4Reliability
If clock cycle is lengthened to accommodate step cache validation, then memory access accuracy is improved, but processing speed decreases
Solution Approach 1:
The step cache performs preliminary validation of concurrent accesses during the address generation phase, before the actual memory access occurs. This preliminary checking allows the system to identify and handle potential conflicts early, avoiding the need for lengthy validation cycles during the critical memory access window.
Solution Approach 2:
When the step cache detects no conflicts during validation, the system skips detailed validation steps and proceeds directly with the memory access, rushing through the operation at full speed. Detailed validation is only performed when conflicts are detected, maintaining accuracy without sacrificing normal processing speed.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A memory unit (500) for handling data memory references of a multi-threaded processor provided with interleaved inter-thread pipeline in emulated shared memory (ESM) architectures, comprising a step cache (504) defining associative cache memory array in which data stays valid till the end of ongoing step of multithreaded execution, said memory array incorporating a plurality of cache lines with data fields, each line, preferably exclusively, containing a data field for address tag and a data field thread id of the first thread referring to a data memory location specified by the address tag, a scratchpad (506) defining a memory buffer for storing internal data of multi-operations, such as intermediate results, said buffer including, preferably exclusively, a single data field for each thread of the processor, wherein the memory unit is configured to access the step cache for a cache search and scratchpad for retrieving and/or storing said internal data at different clock cycles and different stages of the processor pipeline during multioperation execution involving data memory (508) reference by the processor. A corresponding method for handling memory references is also presented.