Universal Arithmetic Operator for CNN Bit Width Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for processing convolutional neural networks (CNNs) in embedded systems face inefficiencies due to the need for different bit widths in each layer, leading to reduced hardware use efficiency and increased processing time for intermediate bit widths.
Innovation Solution
The proposed technique involves an information processing apparatus that uses a data processing unit capable of handling feature data of varying bit widths by employing a multiplier-accumulator and a shift operator that can process data in parallel across different bit widths, optimizing the bit width for each layer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If different kinds of arithmetic operators are used to process CNN with different bit widths for each layer, then processing capability for various bit widths is improved, but hardware complexity and device complexity increase
Solution Approach 1:
The patent employs a universal arithmetic operator that can process multiple bit widths (2-bit, 4-bit, 8-bit, 16-bit, 32-bit) through a single unified hardware structure. The operator uses configurable parameters including data bit width selection, MAC operation enable/disable control, and shift amount configuration to adapt to different layer requirements without requiring separate dedicated operators for each bit width, thereby reducing hardware complexity while maintaining versatility
Solution Approach 2:
The arithmetic operator's functionality is adjusted by changing operational parameters rather than hardware structure. Key parameters include: data bit width (selectable from 2, 4, 8, 16, 32 bits), MAC operation enable/disable flag, and shift amount (0-31 bits). These parameter changes allow the same hardware to efficiently process different bit widths appropriate for each layer's calculation amount
2Productivity
If convolution operation units dedicated to specific bit widths are used, then processing efficiency for that bit width is improved, but hardware use efficiency decreases when calculation amounts differ across layers
Solution Approach 1:
The arithmetic operator is designed with dynamic configurability to adapt its behavior to different processing requirements. The operator can dynamically adjust its data bit width handling, enable or disable MAC operations based on layer requirements, and configure shift amounts dynamically. This dynamic adaptability allows efficient processing across layers with varying calculation amounts using a single flexible hardware unit rather than multiple static dedicated units
3Adaptability or versatility
If SIMD configuration is used to process feature data sets with plurality of bit widths, then adaptability is improved, but processing time increases due to temporary memory holding and re-execution of SIMD commands
Solution Approach 1:
The arithmetic operator maintains continuous processing capability by handling different bit widths within the same operational flow without requiring data to be temporarily held in memory and re-processed. The operator continuously processes feature data sets with various bit widths (2, 4, 8, 16, 32 bits) in a single pass by configuring its internal parameters appropriately for each data type, eliminating the need for interruptive memory storage and command re-execution that plagues SIMD approaches
4Adaptability or versatility
If 8-bit data MAC operation unit is used for intermediate bit widths such as 4 bits, then processing capability is maintained, but efficiency degrades due to mismatched bit width processing
Solution Approach 1:
The arithmetic operator implements local quality adaptation by allowing different parts of the processing pipeline to operate at their optimal bit widths. The operator can be configured to process 2-bit, 4-bit, 8-bit, 16-bit, or 32-bit data according to the specific requirements of each layer, rather than forcing all processing through a fixed 8-bit width. This localized optimization maintains processing capability for intermediate bit widths while maximizing efficiency by matching the operator's operational width to the data being processed
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An information processing apparatus comprises a control unit configured to set a shift amount based on a bit width of data, for each layer of a network including a plurality of layers, a plurality of MAC (multiply-accumulate) units configured to execute MAC operations on a plurality of data and a plurality of filter coefficients of the layer, a plurality of shift operation units configured to shift a plurality of MAC operation results obtained by the plurality of MAC units based on the shift amount, and an adding unit configured to calculate a total sum of the plurality of MAC operation results shifted by the plurality of shift operation units.