Functional Neural Network Accelerator for Predictable Debugging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current accelerator circuits for neural network computations, while offering high-performance at lower costs compared to GPUs, are difficult to integrate with CPU programming interfaces and debug due to their non-functional implementations.
Innovation Solution
Implementing accelerator circuits within a purely functional framework that allows direct programming and debugging, utilizing memory-to-memory instructions, custom call instructions, and a MAC matrix circuit for parallel computation, with CPU control and monitoring to ensure predictable performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If accelerator circuits are implemented with non-functional implementations to achieve high performance, then computational performance is improved, but debugging difficulty and integration complexity with CPU programming interfaces increases
Solution Approach 1:
The patent introduces a functional framework layer as an intermediary between the CPU programming interfaces and the accelerator circuit implementation. This framework provides memory-to-memory instructions and custom call instructions that enable high-performance computation while maintaining standard programming interfaces, thus resolving the contradiction between performance and ease of operation.
Solution Approach 2:
The accelerator circuit is segmented into multiple independent functional units that can execute pure functional instructions in parallel. Each unit operates independently with well-defined inputs and outputs, which improves performance through parallelism while also making debugging easier by isolating functional errors to specific units.
2Speed
If accelerator circuits use non-functional implementations for high performance, then computational speed is improved, but integration with CPU programming interfaces becomes difficult
Solution Approach 1:
The functional framework implements a universal interface layer that supports multiple CPU programming interfaces and instruction formats. The memory-to-memory instructions and custom call instructions provide versatile adaptation mechanisms that allow the accelerator to integrate with different CPU architectures while maintaining high computational speed through optimized functional execution.
3Productivity
If parallel execution is enabled to improve performance, then productivity is improved, but control and monitoring complexity increases
Solution Approach 1:
The patent implements feedback mechanisms through custom call instructions that allow the CPU to monitor and control the parallel execution of functional units. The feedback loop enables the CPU to track execution status, manage data flow, and coordinate multiple parallel operations, thereby managing control complexity while maintaining high productivity through parallelism.
Data Source
AI summary
An accelerator circuit includes a control interface to receive a stream of instructions, a first memory to store an input data, and an engine circuit. The engine circuit includes a dispatch circuit to decode an instruction of the stream of instructions into a plurality of commands and a plurality of queue circuits. Each of the plurality of queue circuits supports a queue data structure to store a respective one of the plurality of commands decoded from the instruction, and a plurality of command execution circuits. Each of the plurality of command execution circuits is to receive and execute a command extracted from a corresponding one of the plurality of queues.