Statically Scheduled Data Transfer in Machine Learning Accelerators

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Machine learning networks require significant data transfer and computations, often necessitating large resources typically found in cloud-based facilities, which can be inefficient for edge devices that need real-time processing and reduced hardware requirements.

Innovation Solution

A machine learning accelerator (MLA) with a compiler that generates statically scheduled data transfer instructions, allocating computations and data transfers across processing elements to eliminate the need for hardware routing tables and congestion arbitration, allowing for efficient data transfer and computation execution on edge devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If hardware routing tables and congestion arbitration are implemented to facilitate data transfer, then data transfer reliability is improved, but device complexity and hardware requirements increase

Engineering Contradiction:

Improvedata transfer reliabilityVSAvoidhardware complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces hardware-based routing tables and congestion arbitration mechanisms with a software-based compiler that generates statically scheduled instructions. The compiler performs data transfer scheduling at compile-time, eliminating the need for complex hardware routing logic and arbitration circuits, thus reducing hardware complexity while maintaining data transfer reliability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent applies preliminary scheduling of data transfers at compile-time before the program is executed. The compiler determines the exact timing and routing of all data transfers in advance, creating a static schedule that eliminates the need for runtime hardware arbitration and routing decisions, thereby simplifying hardware requirements

Inventive Principle:

Principle #10Preliminary action

2Productivity

If cloud-based facilities with significant resources are used, then machine learning network performance is improved, but loss of time and efficiency for real-time processing worsen

Engineering Contradiction:

Improvemachine learning processing capabilityVSAvoidreal-time processing delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent extracts the machine learning accelerator from remote cloud-based facilities and embeds it directly on edge devices. This extraction eliminates the need for continuous data transmission to remote clouds, enabling real-time processing locally while maintaining high computational performance through dedicated hardware acceleration

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces an intermediary machine learning accelerator that bridges the gap between edge devices and cloud facilities. The accelerator is embedded on the edge device, providing local processing capability that eliminates communication delays with remote clouds while maintaining the ability to perform complex machine learning computations

Inventive Principle:

Principle #24Intermediary (Mediator)

3Speed

If edge devices perform real-time processing, then response time is improved, but device complexity and power consumption increase

Engineering Contradiction:

Improvereal-time processing speedVSAvoidpower consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The patent segments the processing functions by separating the machine learning accelerator from the general-purpose processor. The accelerator is a dedicated hardware component that handles specific machine learning computations efficiently, while the main processor handles other tasks. This segmentation enables real-time processing without requiring the entire system to consume high power continuously

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent employs a simplified architecture that uses basic routing logic without complex hardware tables or arbitration mechanisms. By using simpler, less resource-intensive components for data transfer management, the system achieves real-time processing capability with reduced power consumption and lower hardware costs

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Data Source

PatentUS11886981B2Inter-processor data transfer in a machine learning accelerator, using statically scheduled instructions

Publication Date: 2024.01.30 SIMA TECHNOLOGIES INC

AI summary

A compiler generates a computer program implementing a machine learning network on a machine learning accelerator (MLA) including interconnected processing elements. The computer program includes data transfer instructions for non-colliding data transfers between the processing elements. To generate the data transfer instructions, the compiler determines non-conflicting data transfer paths for data transfers based on a topology of the interconnections between processing elements, on dependencies of the instructions and on a duration for execution of the instructions. Each data transfer path specifies a routing and a time slot for the data transfer. The compiler generates data transfer instructions that specify routing of the data transfers and generates a static schedule that schedules execution of the data transfer instructions during the time slots for the data transfers. The static schedule also schedules execution of compute instructions for computations using transferred data that implement the machine learning network.