Compiler Scheduling for Memory Locality to Reduce Spilling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Compiler optimizations face challenges in resource allocation and instruction scheduling, leading to anti-dependencies that result in spilling and increased data latency, particularly in hardware architectures with limited physical resources, such as neural network accelerators.
Innovation Solution
The technique involves sorting memory objects based on their dependencies and scheduling instructions accordingly, allowing for efficient allocation of physical resources and minimizing spilling by keeping instructions that operate on the same memory object together.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If instructions are scheduled before resource allocation, then scheduling freedom is improved, but resource allocation difficulty increases leading to more spilling
Solution Approach 1:
The patent segments the scheduling process into two distinct phases: a pre-scheduler that operates before resource allocation to establish instruction order based on memory object dependencies, and a resource allocator that operates afterward to assign physical resources. This segmentation allows each phase to focus on its specific optimization goal without being constrained by the other's requirements.
Solution Approach 2:
The pre-scheduler performs preliminary scheduling of instructions based on memory object dependencies before resource allocation occurs. By establishing the instruction execution order in advance, the system gains scheduling freedom while the subsequent resource allocator can then efficiently assign resources without being constrained by scheduling decisions.
2Productivity
If the same physical resource is allocated to both read and write instructions, then resource utilization is improved, but anti-dependencies are created forcing read before write scheduling
Solution Approach 1:
The patent inverts the traditional approach by performing scheduling before resource allocation rather than after. This reversal allows the system to establish instruction ordering based on memory dependencies without being constrained by physical resource assignments, thereby maintaining both high resource utilization and scheduling flexibility.
Solution Approach 2:
The patent introduces memory object dependencies as an intermediary concept that mediates between read and write instructions. By scheduling instructions based on their relationships to memory objects rather than direct resource conflicts, the system can allocate the same physical resources to both read and write instructions while maintaining correct execution order through the memory dependency framework.
3Adaptability or versatility
If spilling is increased to accommodate resource allocation, then resource allocation flexibility is improved, but data latency increases and execution slows
Solution Approach 1:
The pre-scheduler performs preliminary scheduling that groups instructions operating on the same memory objects together, which minimizes the need for spilling by ensuring that related operations are executed contiguously. This preliminary organization of instructions reduces data latency while maintaining resource allocation flexibility.
Solution Approach 2:
By scheduling instructions that operate on the same memory objects to execute contiguously without interruption, the patent ensures continuous useful action on memory data. This continuity reduces the frequency of spilling operations and associated data latency, while the resource allocator can still flexibly assign physical resources to these instruction groups.
Data Source
AI summary
A technique for scheduling instructions includes obtaining a set of instructions that operate on memory objects, and determining the dependencies of the memory objects. The memory objects are then sorted into a sequence of memory objects based on the dependencies of the memory objects, and the set of instructions are scheduled into a sequence of instructions according to the sequence of memory objects. Sorting memory objects allows instructions that operate on the same memory object to be kept together. This helps minimize spilling conditions because intervening instructions that do not operate on the same memory object can be avoided.