Scratchpad Memory Instruction Scheduling for AI Processor Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory models and processors are not fully adaptable to cache-less scratchpad memory structures used in artificial intelligence processors, leading to inefficient handling of hazardous conflicts between memory instructions, which affects performance and execution efficiency.
Innovation Solution
An instruction executing method that cooperatively handles hazardous conflicts between memory instructions using both hardware and software, allowing non-memory instructions to be executed during the cycle of a memory instruction, ensuring programming correctness and improving overall processor performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional memory models and processors are used with cache-less scratchpad memory structures, then hardware simplicity and programmer control are improved, but handling of hazardous conflicts between memory instructions becomes inefficient
Solution Approach 1:
The processor divides instruction handling into separate segments: memory instructions are processed through a memory instruction queue with hazard detection, while non-memory instructions are processed separately. This segmentation allows independent optimization of each instruction type and enables parallel processing paths, resolving the contradiction between structural simplicity and execution efficiency.
Solution Approach 2:
A hazard detection unit is introduced as an intermediary component between the instruction queue and execution units. This mediator detects hazardous conflicts between memory instructions and prevents incorrect executions, thereby maintaining correctness without requiring complex memory management hardware throughout the entire system.
2Reliability
If memory instructions are executed sequentially to maintain order consistency, then programming correctness is improved, but processor performance deteriorates
Solution Approach 1:
The processor dynamically adjusts instruction execution based on hazard detection results. When no hazardous conflicts are detected, memory instructions can be executed in parallel with non-memory instructions or out of order. When hazards are detected, the execution order is dynamically adjusted to maintain correctness. This dynamic approach resolves the contradiction between sequential execution reliability and parallel execution performance.
Solution Approach 2:
The hazard detection unit performs preliminary analysis of incoming memory instructions against previously issued instructions before execution. By detecting potential hazardous conflicts in advance, the system can proactively reorder or serialize instructions to prevent correctness issues, rather than relying on purely sequential execution or complex post-execution verification.
3Reliability
If non-memory instructions are stalled during memory instruction cycles to maintain order consistency, then memory instruction correctness is improved, but overall execution efficiency deteriorates
Solution Approach 1:
The instruction stream is segmented into memory instructions and non-memory instructions with separate processing paths. Non-memory instructions can be executed independently of memory instruction hazards, eliminating unnecessary stalls. This segmentation allows the system to maintain memory instruction correctness while maximizing overall execution throughput by keeping execution units busy with non-memory instructions during memory operation cycles.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Example embodiments of the present application provide an instruction executing method and apparatus, an electronic device, and a computer-readable storage medium that may be applied in the field of artificial intelligence. The instruction executing method may include: executing an instruction sequence that includes memory instructions and non-memory instructions, the instructions in the sequence executed starting to be executed in order; determining that execution of a first memory instruction needs to be completed before a second memory instruction starts to be executed, the second memory instruction being a next memory instruction following the first memory instruction in the instruction sequence; and executing non-memory instructions between the first memory instruction and the second memory instruction without executing the second memory instruction, during a cycle of executing the first memory instruction.