Neural Processor Data Reshaping for Non-Power-of-Two Work Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural processors often struggle to optimally process non-power-of-two shaped work units, leading to inefficient utilization and increased power consumption when performing machine learning operations.
Innovation Solution
A neural processor circuit that includes a data processor circuit to reshape non-power-of-two shaped source data into power-of-two shaped work units, enhancing the performance of neural engines by optimizing data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural processors process non-power-of-two shaped work units directly, then the processors can handle diverse machine learning operations, but the utilization rate decreases and power consumption increases
Solution Approach 1:
The data processor circuit segments non-power-of-two shaped source data into multiple power-of-two shaped work units. Each work unit is processed independently by neural engine circuits, enabling efficient utilization of the neural processor while maintaining the ability to handle diverse machine learning operations with varying data shapes
Solution Approach 2:
The data processor circuit acts as an intermediary between the source data and neural engine circuits. It performs reshaping operations to convert non-power-of-two shaped data into power-of-two shaped work units, mediating the compatibility issue between diverse data formats and the fixed architecture of neural processors
2Device complexity
If neural processors process non-power-of-two shaped work units directly, then the processors can maintain simple architecture, but power consumption increases due to inefficient processing
Solution Approach 1:
The data processor circuit performs preliminary reshaping actions on source data before it reaches the neural engine circuits. By pre-converting non-power-of-two shaped data into power-of-two shaped work units, the system avoids the need for complex runtime adaptations in the neural processor architecture while optimizing power consumption through efficient processing
3Productivity
If neural processors are configured to work optimally with specific algorithms, then processing efficiency improves, but adaptability to different algorithms decreases
Solution Approach 1:
The data processor circuit provides a universal interface that can handle various machine learning algorithms and data formats. By converting diverse non-power-of-two shaped data from different algorithms into standardized power-of-two shaped work units, the system maintains both high processing efficiency for the neural processor and broad adaptability to different machine learning algorithms
Data Source
AI summary
A neural processor includes one or more neural engine circuits for performing convolution operations on input data corresponding to one or more tasks to generate output data. The neural engine circuits process the input data having a power-of-two (P2) shape. The neural processor circuit also includes a data processor circuit. The data processor circuit fetches source data having a non-power-of-two (NP2) shape. The source data may correspond to data of a machine learning model. The data processor circuit also reshapes the source data to generate reshaped source data with the P2 shape. The data processor circuit further sends the reshaped source data to the one or more neural engine circuits as the input data for performing convolution operations. In some cases, the data processor circuit may also perform padding on the source data before the source data is reshaped to the P2 shape.