Network Controller Memory Request Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrated circuit devices face performance issues due to significant memory latency in DRAM, which negatively impacts memory throughput, and existing solutions do not effectively address this problem.
Innovation Solution
A network controller is configured to determine response times for memory requests, associate them with pick times, and use a request scheduler to prioritize memory requests based on running averages of response times, issuing requests to memory banks with the lowest average response times and addressing starvation through counters and thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory requests are serviced in first-come-first-served order, then fairness is maintained, but memory latency increases and throughput decreases
Solution Approach 1:
The patent implements dynamic scheduling that adapts to changing memory bank performance characteristics. The system continuously monitors response times for each memory bank and adjusts request prioritization in real-time, transitioning from static FCFS ordering to dynamic performance-based scheduling. This allows the system to optimize throughput by directing requests to faster-response banks while maintaining fairness through starvation count mechanisms.
Solution Approach 2:
The patent employs feedback loops where the scheduler continuously monitors response times from memory banks and uses this information to adjust future request prioritization. The running average response time calculation and starvation count tracking create closed-loop control that adapts to actual memory performance, enabling the system to learn from past behavior and optimize scheduling decisions accordingly.
2Productivity
If requests are prioritized based on running average response times, then memory throughput increases, but system complexity increases due to additional counters and scheduling logic
Solution Approach 1:
The patent divides the scheduling function into separate modular components: pick time counter circuits for each memory bank, running average calculators, starvation count counters, and arbitration logic. This segmentation allows each component to be independently optimized and implemented, reducing overall system complexity while enabling sophisticated scheduling behavior through coordinated operation of simple modular units.
Solution Approach 2:
The scheduling system performs self-adjustment through automated monitoring and adaptation. The running average response times are automatically calculated from actual performance data, and the scheduler autonomously prioritizes requests based on these measurements without external intervention. This self-service capability reduces the need for complex external control logic while maintaining high throughput optimization.
3Loss of time
If memory requests are issued out-of-sequence based on response time predictions, then latency is reduced, but reliability may be impacted by incorrect predictions
Solution Approach 1:
The patent performs preliminary actions by pre-calculating running average response times for each memory bank before making scheduling decisions. The system maintains historical performance data and uses this pre-computed information to guide out-of-sequence request issuance. This preliminary preparation enables faster real-time scheduling decisions while maintaining reliability through the use of accumulated performance knowledge.
Solution Approach 2:
The patent implements a conservative scheduling approach where requests are reordered based on running averages rather than instantaneous measurements. By using averaged historical data rather than raw recent performance, the system applies partial reordering that balances latency reduction with prediction reliability. The starvation count mechanism provides an additional safety layer that prevents excessive reordering that could compromise reliability.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Apparatuses and method for an integrated circuit device are described. In an apparatus thereof, there is a plurality of memory controllers (103, 200) coupled to a plurality of memory banks (104). A network of switches (102) is coupled to the plurality of memory controllers (103, 200). A plurality of data processing devices (105, 300) is coupled to the network of switches (102) and is configured to generate memory requests (305). A network controller (101, 400) is coupled to the network of switches (102) and is configured to queue the memory requests (305) and selectively issue requests to memory (422) from the memory requests (305) queued responsive to corresponding response times (417) associated with the plurality of memory banks (104).