In-Memory Convolution Data Transform for Faster Low-Power Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for performing convolution operations require significant computational resources and time due to the separation of memory and operation processors, leading to inefficiencies in processing speed and power consumption, especially when using accelerators like GPUs or NPUs.
Innovation Solution
A memory device with a function processor that transforms input and filter data based on convolution operation parameters during read/write commands, using algorithms like Winograd or Strassen to reduce computational complexity, and performs operations within the memory device itself, thereby reducing the need for extensive processing by high-speed operation processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is moved from memory to high-speed operation processors for convolution operations, then processing speed is improved, but power consumption and computational resources increase significantly
Solution Approach 1:
The patent combines the memory device and function processor into an integrated system where the function processor is directly coupled to the memory. This merging eliminates the need to move data between separate memory and processing units, performing convolution operations, preprocessing, and postprocessing within the integrated structure, thereby reducing power consumption while maintaining high processing speed.
Solution Approach 2:
The function processor is segmented into specialized circuits including input data transform circuit, filter data transform circuit, MAC circuit, and output data transform circuit. Each circuit performs specific functions (preprocessing, filtering, multiplication-accumulation, postprocessing) to reduce overall computational complexity and power consumption while maintaining processing speed.
2Adaptability or versatility
If traditional convolution operations are performed using separate memory and processors, then computational flexibility is maintained, but processing time and computational quantity increase
Solution Approach 1:
The input data transform circuit performs preprocessing on input data before it reaches the MAC circuit, and the output data transform circuit performs postprocessing after multiplication-accumulation. These preliminary and follow-up actions reduce the computational quantity of the main convolution operation, thereby reducing processing time while maintaining computational flexibility through configurable transform circuits.
Solution Approach 2:
The transform circuits are configured based on parameters of the convolution operation (such as filter size, input data size, stride, padding) to optimize processing. By changing the configuration parameters of the transform circuits according to different convolution requirements, the system maintains adaptability while reducing processing time through optimized computational paths.
3Use of energy by moving object
If more computational operations are performed within the memory device, then power efficiency improves, but device complexity increases
Solution Approach 1:
The function processor serves multiple functions including data transformation, filtering, multiplication-accumulation operations, and result transformation. This multi-functionality allows the integrated memory-processor system to perform various computational tasks (convolution, matrix operations, etc.) within the memory device, improving power efficiency without proportionally increasing complexity through shared hardware resources.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A memory device includes a memory configured to store input data and filter data for a convolution operation, and a function processor configured to, in response to a read command of at least a portion of data from among the input data and the filter data, transform the at least a portion of the data based on a parameter of the convolution operation during a clock cycle corresponding to the read command and output a corresponding transformation result as transformed data.