ANN Compiler Optimization for Deep Learning Accelerator Memory Bottlenecks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies face challenges in efficiently implementing Artificial Neural Networks (ANNs) on Deep Learning Accelerators (DLAs) due to insufficient logic operation capabilities, inadequate memory capacity, and high communication bandwidth requirements, leading to increased energy consumption and computation time.

Innovation Solution

A compiler is used to optimize instructions for DLAs, leveraging an ANN to predict optimal compilation strategies, utilizing a combination of local memory, random access memory, and high-bandwidth connections to reduce data access bottlenecks and enhance computation efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If existing technologies are used to implement ANNs on DLAs, then the system can execute neural network computations, but energy consumption increases and computation time extends due to insufficient logic operation capabilities and inadequate memory capacity

Engineering Contradiction:

Improvecomputation speedVSAvoidenergy consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent segments the neural network computation into distinct phases (data loading, computation, result storage) and optimizes each phase separately using different memory hierarchies and processing units, thereby improving overall computation speed while managing energy consumption efficiently

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary memory hierarchy (including scratchpad memory and buffer memory) between the DLA and external memory, reducing communication bandwidth requirements and energy consumption while maintaining high computation speed through localized data access

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If existing technologies are used to implement ANNs on DLAs, then the system can process neural network data, but communication bandwidth requirements increase leading to data access bottlenecks

Engineering Contradiction:

Improvedata processing efficiencyVSAvoidcommunication bandwidth

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent implements preliminary action by pre-loading data into local memory buffers before computation phases begin, and by pre-configuring the memory hierarchy to anticipate data access patterns, thereby reducing real-time communication bandwidth requirements and eliminating data access bottlenecks

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent employs a nested memory hierarchy where smaller, faster memory structures (registers, scratchpad memory) are nested within larger, slower memory structures (buffer memory, external memory), allowing data to be accessed at the most appropriate level and reducing the need for high-bandwidth communication between memory levels

Inventive Principle:

Principle #7Nested doll (Nesting)

3Use of energy by moving object

If minimal CPU intervention is used to optimize resource utilization, then energy consumption is reduced, but instruction optimization capabilities are limited

Engineering Contradiction:

Improveenergy consumptionVSAvoidinstruction optimization complexity

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

The patent implements self-service by incorporating an integrated ANN compiler directly within the DLA system, allowing the compiler to automatically optimize instruction sequences for neural network operations without requiring external CPU intervention, thereby reducing energy consumption while maintaining optimization capabilities

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent creates a universal compiler architecture that handles multiple types of neural network operations (convolutions, fully connected layers, activation functions) through a single integrated system, reducing the need for specialized optimization routines and minimizing CPU intervention requirements across diverse workloads

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20260044734A1Compiler with an artificial neural network to optimize instructions generated for execution on a deep learning accelerator of artificial neural networks

Publication Date: 2026.02.12 MICRON TECHNOLOGY INC

AI summary

Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory (RAM). A compiler has an artificial neural network configured to identify an optimized compilation option for an artificial neural network to be compiled by the compiler and/or for a hardware platform of Deep Learning Accelerators. The artificial neural network of the compiler can be trained via machine learning to identify the optimized compilation option based on the features of the artificial neural network to be compiled and/or features of the hardware platform on which the compiler output will be executed.