Tag and Thread Logic for Concurrent Memory Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computer systems, out-of-order processors face challenges in efficiently managing memory access scheduling, particularly when the processor runs faster than memory, leading to inefficiencies in handling memory operations and potential reordering of instructions, which can result in suboptimal performance and increased latency.
Innovation Solution
The implementation of tag and thread logic within a memory scheduler that allows for per-thread and per-tag memory access scheduling, enabling concurrent management of tags and threads to reorder memory operations based on overall memory efficiency and Quality-of-Service latency requirements, optimizing the servicing order of memory requests out of their initial issue order.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If out-of-order processing is used to avoid processor stalling, then processor productivity is improved, but memory access scheduling complexity increases
Solution Approach 1:
The memory scheduler is segmented into multiple independent queues, each dedicated to a specific thread. This segmentation allows the scheduler to manage multiple threads concurrently without complex inter-thread coordination, reducing overall scheduling complexity while maintaining high processor throughput through parallel queue processing.
Solution Approach 2:
The patent introduces a new dimension of scheduling by organizing memory requests into a two-dimensional structure: thread dimension (multiple thread queues) and request dimension (individual request queues within each thread). This dimensional organization simplifies the scheduling problem by separating thread management from request management, reducing complexity while improving productivity.
2Device complexity
If memory access scheduling is performed in issue order, then scheduling simplicity is maintained, but processor latency increases due to processor-memory speed mismatch
Solution Approach 1:
The patent implements preliminary action by maintaining ready queues for each thread that pre-organize memory requests before they need to be serviced. This allows the processor to continuously issue requests from the ready queue without stalling, reducing latency while keeping scheduling logic simple through the use of straightforward queue data structures.
Solution Approach 2:
The patent ensures continuity of useful action by maintaining multiple ready queues that can be continuously serviced by the memory controller. As long as at least one queue has ready requests, the memory interface remains active, eliminating idle cycles and reducing average access latency without requiring complex dynamic scheduling decisions.
3Reliability
If dedicated buffer space is allocated for each thread, then thread isolation and reliability are improved, but buffer space utilization efficiency decreases
Solution Approach 1:
The buffer space is segmented into separate queues for each thread, providing logical isolation and reliability. Each thread has its own dedicated queue structure that prevents interference from other threads, ensuring reliable thread isolation while maintaining efficient space utilization through shared physical buffer resources managed by the segmented queue logic.
Data Source
AI summary
A method, apparatus, and system in which an integrated circuit comprises an initiator Intellectual Property (IP) core, a target IP core, an interconnect, and a tag and thread logic. The target IP core may include a memory coupled to the initiator IP core. Additionally, the interconnect can allow the integrated circuit to communicate transactions between one or more initiator Intellectual Property (IP) cores and one or more target IP cores coupled to the interconnect. A tag and thread logic can be configured to concurrently perform per-thread and per-tag memory access scheduling within a thread and across multiple threads such that the tag and thread logic manages tags and threads to allow for per-tag and per-thread scheduling of memory accesses requests from the initiator IP core out of order from an initial issue order of the memory accesses requests from the initiator IP core.