Neural Engine Circuit for 3D Convolution Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems relying solely on CPUs for neural network operations face significant bandwidth consumption and increased power usage due to the resource-intensive nature of machine learning computations, particularly in performing multi-dimensional convolution operations.
Innovation Solution
A neural engine circuit within a neural processor that includes multiple multiply-add circuits and an accumulator circuit, designed to perform three-dimensional convolution operations efficiently by dividing accumulators into batches to process and store output data for each output depth plane and channel, thereby reducing the computational load on CPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If CPU and main memory are used to instantiate and execute machine learning systems, then ease of configuration is improved, but bandwidth consumption and power consumption increase significantly
Solution Approach 1:
The system is divided into two parts: a configurable control layer (CPU) that handles high-level decision-making and model selection, and a dedicated neural processing unit (NPX) that handles intensive computational operations. This segmentation allows the CPU to remain lightweight while the NPX handles power-intensive computations efficiently.
Solution Approach 2:
A neural processing engine acts as an intermediary between the CPU and main memory, performing computations locally within the NPX unit. This intermediary architecture reduces the need for frequent data transfers to and from main memory, thereby reducing bandwidth consumption and power usage while maintaining computational flexibility.
2Ease of operation
If CPU is used for various operations of machine learning systems, then ease of operation is improved, but bandwidth consumption increases
Solution Approach 1:
The system separates control functions (handled by CPU) from computation functions (handled by NPX). The CPU manages data flow and coordinates operations, while the NPX performs intensive multiply-accumulate operations, reducing the bandwidth burden on the CPU-memory interface.
Solution Approach 2:
The neural processing engine serves as an intermediary computation unit that processes data locally before results need to be returned to the CPU. This local processing reduces the volume of data that must be transmitted over the memory bus, thereby reducing bandwidth consumption while maintaining ease of operation through centralized control.
3Productivity
If multi-dimensional convolution operations are performed, then processing capability is improved, but computational load on CPU increases
Solution Approach 1:
The neural processing engine is divided into multiple independent neural processing elements (NPEs), each capable of performing convolution operations independently. This segmentation distributes the computational load across multiple specialized units rather than burdening the CPU, thereby improving processing capability while reducing CPU complexity.
Solution Approach 2:
The patent replaces general-purpose CPU-based computation with specialized hardware circuits (neural processing elements) that are optimized for specific neural network operations such as convolution, activation functions, and pooling. This substitution of specialized hardware for general-purpose software execution improves processing capability while reducing the effective computational load on the CPU.
Data Source
AI summary
Embodiments of the present disclosure relate to a neural engine of a neural processor circuit having multiple multiply-add circuits and an accumulator circuit coupled to the multiply-add circuits. The multiply-add circuits perform multiply-add operations of a three dimensional convolution on a work unit of input data using a kernel to generate at least a portion of output data in a processing cycle. The accumulator circuit includes multiple batches of accumulators. Each batch of accumulators receives and stores, after the processing cycle, the portion of the output data for each output depth plane of multiple output depth planes. A corresponding batch of accumulators stores, after the processing cycle, the portion of the output data for a subset of the output channels and for each output depth plane.