Memory Layout for CNN Weight Coefficients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for storing weight coefficients in convolutional neural networks (CNNs) face inefficiencies due to memory alignment issues, leading to increased memory capacity requirements and the need for complex circuitry for address calculation, especially when bit widths do not match memory word alignment or when different bit precision formats are used.
Innovation Solution
An apparatus with a memory that stores weight coefficients according to the output channels' order and memory word alignment, using a transfer unit to efficiently transfer coefficients to parallel product-sum operators, thereby optimizing memory usage and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If weight coefficients are stored in different formats in two memories, then memory usage efficiency is improved, but device complexity increases due to the need for separate address calculation circuits
Solution Approach 1:
The patent merges the storage of weight coefficients with different bit precisions into a single memory by organizing them in a transposed format where coefficients are arranged by output channel groups. This eliminates the need for separate memories and their associated address calculation circuits, while maintaining efficient memory utilization through the structured arrangement that allows simple sequential access patterns.
2Productivity
If weight coefficients are arranged by output channel groups in transposed format, then memory access efficiency is improved, but storage layout complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-organizing weight coefficients into groups corresponding to output channels during the storage phase. This transposed arrangement, where coefficients are grouped by output channel rather than by input channel, is prepared in advance to enable efficient parallel access patterns during convolution operations, allowing multiple product-sum operators to simultaneously retrieve their required coefficients without complex access logic.
3Adaptability or versatility
If bit width of weight coefficients does not match memory word alignment, then processing flexibility is improved, but memory capacity increases due to unused storage areas
Solution Approach 1:
The patent applies the nested doll principle by organizing weight coefficients of varying bit precisions (1-bit, 8-bit, 16-bit) into a hierarchical structure where smaller precision coefficients are grouped together and larger precision coefficients are grouped separately. This nested arrangement allows the memory to efficiently pack coefficients of different sizes without wasting space, as each group can be accessed as a unit matching its bit width requirements, thereby maintaining processing flexibility while optimizing memory utilization.
Data Source
AI summary
An apparatus comprises: an operation unit including M product-sum operators that can operate in parallel; a memory configured to hold a plurality of coefficients used by the operation unit; and a transfer unit configured to transfer the coefficients held in the memory to the operation unit. The memory holds the plurality of coefficients in accordance with an order of output channels of the operation unit and word alignment of the memory. In a case where the operation unit parallelly executes N arithmetic processes, the transfer unit transfers at least N coefficients to the operation unit in a predetermined transfer format.