CIM Array Kernel Layout for Depthwise CNN Dataflow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computation in memory (CIM) processes struggle to efficiently process complex machine learning model architectures like depthwise separable convolutional neural networks without additional hardware elements, such as digital multiply-and-accumulate circuits, which increases space, power, and complexity, and requires significant data movement, leading to power usage and latency issues.
Innovation Solution
The implementation of a CIM array with distinct sets of CIM cells configured as different kernels on varying columns and rows, allowing for parallel operation and analog-to-digital conversion, enabling efficient depthwise and pointwise computations within the memory, reducing the need for additional hardware and minimizing data movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional CIM processes are used to process depthwise separable convolutional neural networks, then processing capability is maintained, but additional hardware elements (DMACs) are required which increase space, power, and complexity
Solution Approach 1:
The memory device is designed to perform multiple functions: it serves as both storage memory and compute unit. The memory cells directly perform multiply-accumulate operations for depthwise separable convolution, eliminating the need for separate DMAC hardware. This multi-functionality allows the same hardware structure to handle both conventional CNNs and depthwise separable CNNs without additional specialized components.
2Adaptability or versatility
If additional hardware elements like DMACs are added to conventional CIM processes, then complex model architectures can be processed, but space, power, and complexity increase
Solution Approach 1:
The patent merges the compute function with the memory structure by implementing multiply-accumulate operations directly within the memory device. The memory cells perform the computational operations that would otherwise require separate DMAC hardware, consolidating storage and computation into a single integrated structure. This merging eliminates the need for data movement between separate compute and memory units, significantly reducing power consumption.
3Adaptability or versatility
If data is moved to and from ancillary aspects with DMACs, then processing that cannot be directly performed in memory is achieved, but time and power are consumed
Solution Approach 1:
The patent extracts the computational functionality from separate DMAC hardware and integrates it directly into the memory device structure. By taking out the compute function and embedding it within the memory cells themselves, the system eliminates the need to move data between memory and separate compute units. The memory device performs all necessary computations in-place, removing data movement bottlenecks and time delays.
4Adaptability or versatility
If data is moved across common data busses, then data sharing between processes is enabled, but significant power usage and latency are introduced
Solution Approach 1:
The memory device serves itself by performing computational operations internally without requiring external compute hardware. The multiply-accumulate operations are executed within the memory structure using its own stored data, eliminating the need to transfer data across external data busses for processing. This self-service capability reduces power consumption on shared data busses by keeping data local to the memory device throughout the computation process.
Data Source
AI summary
Certain aspects provide an apparatus for signal processing in a neural network. The apparatus generally includes a first set of computation in memory (CIM) cells configured as a first kernel for a neural network computation, the first set of CIM cells comprising on one or more first columns and a first plurality of rows of a CIM array, and a second set of CIM cells configured as a second kernel for the neural network computation, the second set of CIM cells comprising on one or more second columns and a second plurality of rows of the CIM array. In some aspects, the one or more first columns are different than the one or more second columns, and the first plurality of rows are different than the second plurality of rows.