Neural Processor Transposer for Width-Last and Channel-Last Layouts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems rely heavily on central processing units (CPUs) for neural network operations, leading to significant bandwidth consumption and increased power usage, which is inefficient and resource-intensive.
Innovation Solution
A neural processor circuit that switches between width-last and channel-last modes for data layout formats, using a transposer to optimize data processing by transposing input data when necessary, thereby reducing CPU load and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU is used for neural network operations, then ease of operation is improved, but power consumption increases
Solution Approach 1:
The patent replaces the general-purpose CPU (mechanical/electronic computing system) with a specialized neural processor circuit that is optimized for neural network operations. This substitution maintains ease of operation through automated hardware acceleration while significantly reducing power consumption by eliminating the overhead of general-purpose processing for specialized tasks.
Solution Approach 2:
The patent extracts neural network processing functionality from the CPU and implements it as a separate, dedicated neural processor circuit. This extraction allows the CPU to focus on control and coordination tasks while the neural processor handles computationally intensive operations, thereby reducing overall system power consumption while maintaining operational simplicity.
2Productivity
If data transposing is performed, then processing efficiency is improved, but device complexity increases
Solution Approach 1:
The patent introduces a transposer circuit as an intermediary component between the data buffer and neural processing units. This dedicated transposing mechanism simplifies the overall system architecture by providing specialized hardware for data format conversion, thereby improving processing efficiency without significantly increasing overall device complexity.
Solution Approach 2:
The patent performs data transposing in advance before neural network processing begins. By pre-processing the input data to match the required format, the system avoids complex real-time transformations during processing, thereby improving efficiency while keeping the processing pipeline simple and manageable.
Data Source
AI summary
Embodiments of the present disclosure relate to a neural processor circuit configured to switch between a width-last mode and a channel-last mode of input data for more efficient processing of tasks. A compiler may determine whether the neural processor circuit is likely to perform a task more efficiently by using the input data in a width-last format or the channel-last format and compiles instructions to enable or disable a transposer circuit in the neural processor circuit. When the neural processor circuit is in a mode that uses the channel-last format, the input data in the width-last format is transposed into transposed input data in the channel-last format before being fed into one or more neural engines of the neural processor circuit, and output data generated by the one or more neural engines are also transposed back into the width-last format.