Computing in Memory Kernel Segmentation for Stride Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In deep learning training, traditional computing in memory (CIM) methods suffer from high energy consumption due to excessive data movement caused by duplicate feeding of input feature maps during stride operations, especially when the stride step is small, leading to increased energy consumption and reduced operation speed.
Innovation Solution
The method involves unfolding a kernel into multiple sub-kernels and complement sub-kernels based on a stride parameter, writing weights into target memory cells, and performing stride operations within the memory array while temporarily storing partial sums to efficiently reuse input data, thereby reducing data movement and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional computing in memory methods are used with small stride steps, then stride operations can be performed with fine-grained control, but data movement increases and energy consumption rises due to duplicate feeding of input feature maps
Solution Approach 1:
The kernel is segmented into multiple sub-kernels based on the stride parameter. Each sub-kernel processes a specific portion of the input feature map, allowing the system to avoid reprocessing the same data multiple times. This segmentation reduces duplicate data feeding while maintaining fine-grained stride operation control.
Solution Approach 2:
The kernel is pre-processed into sub-kernels and complement sub-kernels before the actual stride operation. This preliminary segmentation allows the system to efficiently reuse input data across multiple stride operations, reducing the need for repeated data movement and lowering energy consumption.
2Adaptability or versatility
If traditional computing in memory methods are used with small stride steps, then stride operations can be performed with fine-grained control, but operation speed decreases due to repeated data movement
Solution Approach 1:
By dividing the kernel into sub-kernels corresponding to different stride positions, the system can process multiple output positions in parallel without repeatedly fetching the same input data. This segmentation enables faster stride operations while maintaining precise control over the convolution process.
Solution Approach 2:
Complement sub-kernels are combined with regular sub-kernels to form a complete processing unit. This merging allows the system to handle both regular and boundary cases in a unified manner, improving operation speed by reducing the overhead of separate processing paths.
3Reliability
If input data is repeatedly fed for each stride operation cycle, then complete stride operations can be performed, but data movement increases significantly
Solution Approach 1:
The kernel is pre-segmented into sub-kernels that correspond to different stride positions. This preliminary action allows input data to be reused across multiple stride operations without being repeatedly fed, reducing data movement while ensuring complete and accurate stride operation execution.
Solution Approach 2:
Instead of discarding input data after each stride operation cycle, the system recovers and reuses the same input data for subsequent stride operations by having pre-segmented sub-kernels ready to process. This recovering approach significantly reduces redundant data movement while maintaining operational completeness.
Data Source
AI summary
A computing in memory method for a memory device is provided. The computing in memory method includes: based on a stride parameter, unfolding a kernel into a plurality of sub-kernels and a plurality of complement sub-kernels; based on the sub-kernels and the complement sub-kernels, writing a plurality of weights into a plurality of target memory cells of a memory array of the memory device; inputting an input data into a selected word line of the memory array; performing a stride operation in the memory array; temporarily storing a plurality of partial sums; and summing the stored partial sums into a stride operation result when all operation cycles are completed.