Wavefront Instruction Scheduling for SIMD Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The throughput of SIMD micro-architectures is hindered by pipeline stalls due to the latency of memory access instructions, leading to inefficient instruction scheduling in parallel data processing circuits.
Innovation Solution
A scheduler assigns priority levels to wavefronts based on both the count of issued vector arithmetic instructions and the age of vector memory access instructions, using a first-in-first-out data storage arrangement to prioritize older memory access instructions, thereby balancing workload and hiding latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory access instructions are executed in order, then instruction correctness is maintained, but pipeline stalls occur due to latency reducing throughput
Solution Approach 1:
The scheduler performs preliminary classification of instructions into arithmetic and memory access types, and assigns priority levels to wavefronts based on their memory access instruction ages before execution. This allows the system to proactively prepare and issue instructions in an optimized sequence, preventing pipeline stalls rather than reacting to them after they occur.
Solution Approach 2:
The scheduler dynamically adjusts priority levels of wavefronts based on real-time instruction status and age of memory access instructions. Rather than using a static scheduling approach, the system continuously monitors instruction progress and reassigns priorities to maintain optimal pipeline utilization, allowing arithmetic instructions to be issued ahead when memory access instructions are pending.
2Productivity
If priority is assigned based only on wavefront age, then simple scheduling is maintained, but arithmetic instruction execution efficiency is reduced
Solution Approach 1:
The scheduler applies different priority assignment rules to different instruction types within wavefronts. Arithmetic instructions receive priority based on wavefront age, while memory access instructions receive priority based on their individual age and status. This localized differentiation optimizes execution efficiency for each instruction type without requiring complete redesign of the scheduling mechanism.
Solution Approach 2:
The system changes the priority parameter from a single wavefront age metric to a composite metric that includes wavefront age, instruction type, and individual instruction age. This parameter transformation allows the scheduler to differentiate between arithmetic and memory access instructions, optimizing arithmetic instruction execution while maintaining manageable scheduling complexity through structured parameter management.
Data Source
AI summary
An apparatus and method for efficiently scheduling instructions for a parallel data processing circuit. In various implementations, a computing system includes a parallel data processing circuit with multiple compute circuits, each uses multiple single instruction multiple data (SIMD) circuits. Each compute circuit includes a scheduler for selecting instructions to issue to the SIMD circuits. The scheduler assigns priority levels to wavefronts based on two factors. The first factor includes balancing execution of instructions of a first instruction type across the multiple wavefronts. For example, the scheduler maintains a count of issued instructions of the first type for each wavefront. The second factor includes satisfying urgency of execution of instructions of a second instruction type across the plurality of wavefronts. The scheduler combines the two factors to create priority levels for each of the wavefronts.