CIM Array Architecture for Phased Depthwise Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computation-in-memory (CIM) processes struggle to efficiently process complex machine learning model architectures like depthwise separable convolutional neural networks without additional hardware elements, which increase space, power, and complexity, and data movement introduces latency and power consumption.
Innovation Solution
Implementing CIM cells configured as different kernels for depthwise and pointwise neural network computations on a CIM array, performing DW convolution operations in multiple phases and using the same columns, with outputs fed back for PW convolution, reducing the need for additional hardware and minimizing data movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If additional hardware elements (DMACs and peripherals) are added to enable complex model architectures in CIM processes, then processing capability is improved, but space, power, and complexity increase
Solution Approach 1:
The patent makes the CIM array universally applicable to both simple and complex model architectures by enabling it to perform diverse operations (MAC, DW convolution, PW convolution, ReLU, max pooling, average pooling) without requiring additional specialized hardware. The same CIM array is reconfigured through different operation modes to handle various neural network operations, eliminating the need for separate DMAC units and peripherals for each operation type.
2Adaptability or versatility
If data is moved to and from ancillary aspects with DMACs for processing, then processing capability is improved, but time and power increase
Solution Approach 1:
The patent merges the functions of separate processing units (DMACs) into the CIM array itself. By combining multiple processing capabilities (MAC, depthwise convolution, pointwise convolution, activation functions, pooling) into a single reconfigurable CIM array, the system eliminates the need to move data between separate hardware components. All processing operations are performed in-place within the memory array, significantly reducing data movement time and associated power consumption.
3Productivity
If data is moved across common data busses, then processing capability is improved, but power usage increases
Solution Approach 1:
The patent extracts the processing functions from separate DMAC units and relocates them directly into the CIM array structure. This extraction eliminates the need for data to traverse external data busses between memory and processing units. By embedding all necessary processing capabilities within the memory array itself, the system removes the energy-consuming data movement pathway while maintaining full processing capability.
Data Source
AI summary
Certain aspects provide an apparatus for signal processing in a neural network. The apparatus generally includes first computation in memory (CIM) cells configured as a first kernel for a neural network computation, the first set of CIM cells comprising one or more first columns and a first plurality of rows of a CIM array. The apparatus also include a second set of CIM cells configured as a second kernel for the neural network computation, the second set of CIM cells comprising the one or more first columns and a second plurality of rows of the CIM array. The first plurality of rows may be different than the second plurality of rows.