Function-in-Memory Convolution Processing to Cut Data Movement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for performing convolution operations require significant computational resources and processing time, as they typically move data from memory to a high-speed operation processor for processing, leading to inefficiencies in both time and power consumption, especially when using accelerators like GPUs or NPUs.
Innovation Solution
A memory device with a function processor that transforms input and filter data based on convolution operation parameters using algorithms like Winograd or Strassen, reducing the computational quantity of convolution operations by preprocessing and postprocessing data within the memory device during clock cycles, thereby reducing the need for extensive processing by high-speed operation processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is moved from memory to high-speed operation processor for convolution operation, then processing speed is improved, but power consumption and computational resources increase significantly
Solution Approach 1:
The patent combines the memory device and function processor into an integrated architecture where the function processor is directly coupled to the memory. This merging eliminates the need for data movement between separate components, reducing power consumption while maintaining processing speed. The convolution operation is performed within the integrated unit, keeping computation and storage closely coupled.
Solution Approach 2:
The function processor performs preprocessing of input data and filter data before the main convolution operation. By transforming the data in advance using algorithms like Winograd or Strassen, the system reduces the computational quantity required for the actual convolution, thereby reducing power consumption while maintaining or improving processing speed.
2Measurement precision
If traditional convolution operation is performed by moving data to operation processor, then computation accuracy is maintained, but processing time increases
Solution Approach 1:
The patent changes the parameters of the convolution operation by applying transformation algorithms (Winograd, Strassen) that modify how the computation is performed. These parameter changes allow the function processor to reduce the number of computational steps while maintaining the same mathematical result, thereby reducing processing time without sacrificing computation accuracy.
Solution Approach 2:
The patent replaces the traditional mechanical data movement and processing system with a function-in-memory architecture. Instead of physically moving data between memory and processor, the computation is performed within the memory device itself, substituting the traditional von Neumann architecture with a more efficient computational model that reduces processing time.
3Productivity
If more computational resources are allocated to high-speed operation processor, then processing capability is improved, but device complexity and cost increase
Solution Approach 1:
The function processor is designed with multi-functionality, capable of performing various data transformation operations and convolution computations directly within the memory device. This universal function processor eliminates the need for separate dedicated hardware components, reducing device complexity while maintaining or improving processing capability.
Solution Approach 2:
The memory device with integrated function processor serves itself by performing computational tasks internally without requiring external high-speed operation processors. The function processor uses algorithms like Winograd or Strassen to reduce computational quantity, allowing the system to maintain high processing capability while reducing overall device complexity and eliminating the need for separate GPU or NPU components.
Data Source
AI summary
A memory device includes a memory configured to store input data and filter data for a convolution operation, and a function processor configured to, in response to a read command of at least a portion of data from among the input data and the filter data, transform the at least a portion of the data based on a parameter of the convolution operation during a clock cycle corresponding to the read command and output a corresponding transformation result as transformed data.