Neural Processor Binary Comparison and Patch-Wise Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems, such as neural networks, rely heavily on central processing units (CPUs) for operations, leading to significant bandwidth consumption and increased power usage, especially when performing complex computations like convolution and accumulation.
Innovation Solution
A neural processor circuit with a neural engine and a planar engine is introduced, capable of performing binary comparison and reduction operations without software control, including a binary comparator circuit and a filter circuit to generate conditional tensors and reduced values, thereby offloading CPU-intensive tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU is used to perform neural network operations, then ease of operation is improved, but power consumption increases
Solution Approach 1:
The patent replaces the mechanical/CPU-based processing system with a specialized neural processing unit that uses dedicated hardware circuits (neural engine and planar engine) to perform neural network operations. This substitution of general-purpose CPU with purpose-built hardware reduces power consumption while maintaining operational capability.
2Ease of operation
If CPU is used to perform neural network operations, then ease of operation is improved, but bandwidth consumption increases
Solution Approach 1:
The patent replaces CPU-based processing with a dedicated neural processing unit that has integrated memory and processing capabilities. This eliminates the need for continuous data transfer between CPU and main memory, thereby reducing bandwidth consumption.
Solution Approach 2:
The patent introduces an intermediary neural processing unit with its own local memory (buffer memory) that sits between the CPU and the neural network operations. This intermediary structure allows data to be processed locally without constant CPU intervention and main memory access, reducing bandwidth requirements.
3Use of energy by stationary object
If specialized neural processing unit is introduced, then power consumption decreases, but device complexity increases
Solution Approach 1:
The patent divides the neural processing unit into distinct functional segments: a neural engine for convolution operations, a planar engine for activation and pooling operations, and buffer memory. This segmentation allows each component to be optimized for its specific function while working together as an integrated low-power processing unit.
Solution Approach 2:
The patent designs the neural processing unit with multi-functional engines that can handle various neural network operations. The neural engine performs convolutions, the planar engine handles activation functions and pooling, making the overall unit versatile enough to process different types of neural network layers without requiring separate dedicated hardware for each operation type.
Data Source
AI summary
Embodiments of the present disclosure relate to binary comparison operations (e.g., Boolean operations) and reduction operations in a neural processor circuit to enable implementation of conditional operations without software control. The neural processor circuit includes a neural engine circuit and a planar engine circuit coupled to the neural engine circuit. The neural engine circuit performs a convolution operation to generate output data. The planar engine circuit includes a binary comparator circuit and a filter circuit coupled to the binary comparator circuit. The binary comparator circuit performs a binary comparison operation on a tensor from the output data to generate a conditional tensor. The filter circuit performs a reduction operation for each patch of the conditional tensor to generate a respective reduced value of multiple reduced values associated with a corresponding channel of multiple channels of the conditional tensor.