Reinforcement Learning Scheduler for CGRA Workload Mapping

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional compilation tools face challenges in programming operations for new architectures like the streaming engine (SE) implemented using a coarse grained reconfigurable array (CGRA), requiring a new assembly language that can describe data flows and synchronous schedules to avoid corrupting the synchronous flow of operations.

Innovation Solution

An assembly language is developed that describes programs as graphs representing data flows, allowing for the configuration of dispatch interface, memory interface, and tile memory blocks, enabling a scheduler to map instructions for execution on CGRA tiles with pipelined time-multiplexed processing units.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If traditional compilation tools are used for new architectures like CGRA, then existing programming paradigms can be maintained, but the synchronous flow of operations may be corrupted and manual configuration effort increases

Engineering Contradiction:

Improveprogramming easeVSAvoidsynchronous flow integrity

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

A new assembly language is introduced as an intermediary between traditional high-level programming and CGRA hardware execution. This assembly language includes explicit synchronization constructs and memory fence instructions that mediate between programmer intent and hardware behavior, preventing corruption of synchronous flows while maintaining programming accessibility.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The programming model is segmented into distinct components: data flow description, synchronization points, and execution schedule specification. This segmentation allows each aspect to be handled separately with appropriate abstractions, making the system manageable while ensuring correct synchronous execution through dedicated synchronization primitives.

Inventive Principle:

Principle #1Segmentation

2Manufacturing precision

If manual configuration is used for CGRA execution, then execution precision can be controlled, but configuration time and complexity increase

Engineering Contradiction:

Improveexecution configuration precisionVSAvoidconfiguration time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

Execution configurations are prepared and validated in advance through formal verification of the assembly language program. The compiler performs preliminary analysis to generate optimized execution schedules and memory mappings before runtime, eliminating the need for manual configuration adjustments and reducing runtime configuration overhead.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The CGRA system performs self-configuration through automated compiler techniques that generate optimal execution schedules, register allocations, and memory mappings based on the assembly language program characteristics. The system automatically adapts to different workloads without requiring manual reconfiguration, maintaining precision through algorithmic optimization.

Inventive Principle:

Principle #25Self-service

3Productivity

If parallel execution is implemented across multiple tiles, then productivity increases, but scheduling complexity and device complexity increase

Engineering Contradiction:

Improveexecution throughputVSAvoidscheduling complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

Manual scheduling mechanisms are replaced with automated reinforcement learning-based scheduling algorithms. The system uses machine learning models to predict optimal instruction placement and tile assignment, substituting complex manual scheduling processes with intelligent automated decision-making that scales to parallel architectures.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The scheduling problem is transformed from a two-dimensional tile assignment problem to a multi-dimensional optimization problem that includes time, data dependencies, and resource constraints. This dimensional expansion enables more sophisticated parallel execution strategies while managing complexity through structured optimization approaches.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS20230306272A1Mapping Workloads to Circuit Units in a Computing Device via Reinforcement Learning

Publication Date: 2023.09.28 MICRON TECHNOLOGY INC

AI summary

An artificial neural network is trained via reinforcement learning to receive first data representative of execution dependency conditions of instructions of a program, second data representative of a schedule of a first portion of the instructions of the program for execution in a device having a plurality of circuits units operable in parallel, and third data identifying a next instruction selected from a second portion of the instructions of the program remaining to be scheduled for execution in the device. The artificial neural network selects a placement of the next instruction in one of the circuit units from a plurality of possible placements of the next instruction in the device. Performance of placements of instructions being tested in search for a valid schedule for running the program in the device can be measured to generate samples to train the artificial neural network via reinforcement learning.