Vector Coprocessor Scheduling to Hide Load Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vector processors face challenges in efficiently handling vector data loads and stores with long latency, particularly in AI and ML applications, requiring high-bandwidth operations and parallel execution while avoiding costly register renaming techniques.
Innovation Solution
A vector register scoreboard with a vector time counter is used to statically dispatch vector load and store instructions based on a programmable time count, allowing for out-of-order execution with known throughput and latency times, and a load-store buffer to accommodate variable latencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If register renaming technique is used for out-of-order execution, then instruction execution flexibility is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent extracts the renaming functionality specifically for scalar registers while maintaining original vector register names. This allows out-of-order execution for scalar operations without the overhead of renaming large vector registers, resolving the contradiction by selectively applying renaming only where necessary for execution flexibility.
Solution Approach 2:
The register file is segmented into scalar registers and vector registers with different handling mechanisms. Scalar registers use renaming for out-of-order execution while vector registers maintain their original names, dividing the problem into manageable segments that each use the most appropriate technique.
2Productivity
If vector load latency is reduced, then instruction throughput is improved, but device complexity increases due to additional buffering mechanisms
Solution Approach 1:
The load-store buffer pre-fetches vector data from memory before the vector instructions are ready to execute. By performing the load operation in advance and holding data in the buffer, the system hides memory latency without requiring complex renaming or scheduling mechanisms, thus improving throughput with moderate complexity.
3Device complexity
If static scheduling based on time counter is used, then device complexity is reduced, but ability to handle variable load latencies worsens
Solution Approach 1:
The system uses a dynamic time counter that can be selectively frozen when variable load latencies occur. The static scheduling framework remains simple, but the time counter's ability to pause and resume provides the necessary adaptability to handle cases where load times vary, resolving the contradiction between simplicity and flexibility.
4Productivity
If vector processor executes instructions in parallel, then productivity is improved, but difficulty of managing data dependencies increases
Solution Approach 1:
The load-store buffer acts as an intermediary between memory and the vector execution units. It manages data dependencies by holding loaded data until dependent instructions are ready to execute, enabling parallel execution without requiring complex dependency tracking logic in the main execution path.
Data Source
AI summary
A processor includes a time counter and a vector coprocessor for executing vector instructions for statically dispatching vector instructions with preset execution times based on a write time of a register in a coprocessor register scoreboard and a time counter provided to a vector execution pipeline. The processor also provides a method for hiding the latency of the vector load instructions.