Planar Engine Circuit Broadcasting for Neural Processor Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing systems rely heavily on central processing units (CPUs), leading to significant bandwidth consumption and increased power usage due to the resource-intensive nature of neural network operations.
Innovation Solution
A neural processor architecture that incorporates multiple neural engine circuits and a planar engine circuit, where the planar engine performs broadcasting and elementwise operations to efficiently process input data of different ranks and sizes, reducing the computational load on neural engines and optimizing data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU and main memory are used to execute machine learning systems, then ease of configuration and instantiation is improved, but power consumption and CPU bandwidth consumption increase significantly
Solution Approach 1:
The system segments neural network processing into two distinct parts: configuration management handled by the CPU and actual computational operations handled by dedicated neural engine circuits. This segmentation allows the CPU to remain lightweight while offloading power-intensive computations to specialized hardware, resolving the contradiction between ease of configuration and power consumption.
Solution Approach 2:
The patent introduces intermediate memory buffers between the CPU and neural engine circuits to facilitate efficient data transfer and processing. These buffers act as intermediaries that cache data and results, reducing the need for continuous CPU intervention and minimizing bandwidth consumption while maintaining configuration flexibility.
2Ease of operation
If CPU and main memory are used to execute machine learning systems, then ease of configuration and instantiation is improved, but CPU bandwidth consumption increases significantly
Solution Approach 1:
The system segments computational responsibilities by separating configuration instantiation (CPU) from data processing (neural engine circuits). This division reduces the bandwidth burden on the CPU by eliminating the need for continuous data transfer between CPU and memory during execution, while maintaining configuration flexibility through software updates.
Solution Approach 2:
The patent implements preliminary loading of neural network models and data into dedicated memory buffers associated with neural engine circuits before execution begins. This preliminary action reduces real-time bandwidth requirements by pre-positioning data close to the computational units, allowing the CPU to focus solely on configuration management.
3Productivity
If planar engine circuit performs broadcasting operation to expand data, then processing efficiency is improved, but device complexity increases
Solution Approach 1:
The planar engine circuit is designed as a multi-functional unit that can perform broadcasting operations, elementwise operations, and data expansion within a single integrated architecture. This universality reduces overall device complexity by consolidating multiple specialized components into one versatile circuit that handles various data processing tasks efficiently.
Solution Approach 2:
The patent merges the broadcasting operation and elementwise operation into a unified processing pipeline within the planar engine circuit. By combining these operations rather than implementing them as separate stages or components, the system achieves processing efficiency while minimizing the increase in device complexity through integrated hardware design.
Data Source
AI summary
Embodiments relate to a neural processor that includes one or more neural engine circuits and planar engine circuits. The neural engine circuits can perform convolution operations of input data with one or more kernels to generate outputs. The planar engine circuit is coupled to the plurality of neural engine circuits. A planar engine circuit can be configured to multiple modes. In an elementwise mode, the planar engine circuit may combine two tensors by performing operations element by element. The planar engine circuit may support elementwise operation for two tensors that are in different sizes and ranks. The planar engine circuit may perform a broadcasting operation to duplicate one or more values across one or more channels to make a smaller tensor matching the size of the larger tensor.