Compiler Hardware Abstraction Layer Neural Network Accelerator

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep neural network (DNN) systems are computationally complex, making general-purpose CPUs inefficient for executing neural networks due to high latency and memory usage, especially in convolution layers.

Innovation Solution

A method for mapping and porting neural networks to integrated circuits (ICs) involves receiving a network description, generating a framework-independent network graph, and configuring the IC based on an execution sequence vector, utilizing a compiler and hardware abstraction layer to optimize and schedule operations on programmable ICs like FPGAs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If a general-purpose CPU is used to execute neural network operations, then the system is easier to operate and more versatile, but latency increases and memory usage increases

Engineering Contradiction:

Improveease of operationVSAvoidlatency

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The patent introduces a compiler and hardware abstraction layer as intermediary components between the neural network framework and the IC. The compiler translates framework-independent network graphs into execution sequence vectors, while the HAL manages the interface between software and hardware, enabling efficient execution on specialized ICs without requiring users to directly handle hardware complexity

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical/general-purpose CPU execution system with a specialized IC execution system. By substituting the general-purpose CPU with a dedicated IC that has hardware optimized for neural network operations, the system achieves lower latency and memory usage while maintaining ease of operation through software abstraction layers

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of operation

If a general-purpose CPU is used to execute neural network operations, then the system is easier to operate, but memory usage increases

Engineering Contradiction:

Improveease of operationVSAvoidmemory usage

Core Design Contradiction:

Ease of operationVSQuantity of substance

Solution Approach 1:

The hardware abstraction layer acts as an intermediary that manages memory resources efficiently. It translates high-level neural network operations into optimized hardware commands that utilize the IC's memory architecture, reducing overall memory usage while maintaining ease of operation through abstraction

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent substitutes the CPU's memory management mechanism with an IC-optimized memory system. The specialized IC architecture provides more efficient memory access patterns and reduced memory requirements for neural network operations, achieved through hardware-level optimizations managed by the HAL

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Loss of time

If neural networks are executed on specialized ICs, then latency is reduced and memory usage is reduced, but device complexity increases

Engineering Contradiction:

ImprovelatencyVSAvoiddevice complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The patent extracts the complexity of IC configuration and optimization into a separate compiler component. The compiler handles the complex tasks of translating network graphs into execution sequences and optimizing for the specific IC architecture, while the runtime system remains simple and ease-to-use

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The hardware abstraction layer serves as an intermediary that shields users from IC complexity. It provides a unified interface that works across different IC architectures, translating high-level operations into IC-specific commands without requiring users to understand the underlying hardware complexity

Inventive Principle:

Principle #24Intermediary (Mediator)

4Quantity of substance

If neural networks are executed on specialized ICs, then memory usage is reduced, but device complexity increases

Engineering Contradiction:

Improvememory usageVSAvoiddevice complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent extracts memory management complexity into the compiler and HAL. These components handle the complex tasks of optimizing memory usage for the specific IC architecture, while the runtime system maintains simple and efficient memory access patterns

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent replaces general-purpose memory management with IC-optimized memory management. The specialized IC architecture, when paired with the compiler and HAL, provides more efficient memory utilization by leveraging hardware-specific features and optimization techniques

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS10789402B1Compiler and hardware abstraction layer architecture for a neural network accelerator

Publication Date: 2020.09.29 XILINX INC

AI summary

Examples herein describe a method for a compiler and hardware-abstraction-layer architecture for a programmable integrated circuit (IC). In one embodiment, a method for mapping and porting a neural network to an integrated circuit (IC) is disclosed. The method includes receiving a network description of the neural network; generating a framework independent network graph based on the network description; performing a plurality of back-end operations on the network graph to generate an execution sequence vector; and configuring the IC based on the execution sequence vector.