Convolution Processing System Zero-Value Skip Mechanism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolution neural networks face inefficiencies due to high sparsity in image and filter data, leading to increased latency and current consumption in convolution operations, as many operations involve zero values that do not influence results.
Innovation Solution
A processing system with a temporary memory and a flow controller that skips overwriting points with zero values, allowing for efficient storage and processing of features and weights, thereby reducing unnecessary operations and current consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional matrix multiplication units are used to perform convolution operations, then the operations can be completed, but latency increases and current consumption increases due to processing zero values
Solution Approach 1:
The patent extracts and removes zero values from the feature map and filter data before storing them in temporary memory. The flow controller detects zero values and prevents their storage, effectively extracting the harmful zero elements from the data stream. This reduces the number of unnecessary multiplication operations the processing element must perform, directly reducing latency and improving convolution operation speed.
Solution Approach 2:
The flow controller implements a skipping mechanism that detects zero values in the input data and skips overwriting them in temporary memory. When a zero value is detected, the flow controller advances to the next data element without performing the write operation. This skipping of zero value processing reduces the operational cycle time and improves overall productivity of the convolution operation.
2Reliability
If conventional matrix multiplication units process all data including zero values, then complete operations are performed, but current consumption increases due to unnecessary operations
Solution Approach 1:
The patent extracts zero values from the data stream before they reach the processing element. By preventing zero values from being stored in temporary memory, the system eliminates unnecessary multiplication and addition operations that would consume current. The processing element only performs operations on non-zero values, maintaining operation completeness for meaningful data while reducing energy consumption.
Solution Approach 2:
The system discards zero values by preventing their storage in temporary memory through the flow controller's detection and skipping mechanism. These discarded zero values would have required current-consuming operations but contribute nothing to the final result. By discarding them upfront, the system recovers current that would have been wasted on null operations.
3Stability of the object's composition
If zero values are stored in temporary memory, then data structure is maintained, but unnecessary operations increase latency
Solution Approach 1:
The patent implements a dynamic data structure in temporary memory that adapts its size and organization based on the actual content being processed. Instead of allocating fixed memory positions for all possible data elements including zeros, the system dynamically adjusts the memory structure to only accommodate non-zero values. This dynamic approach maintains data structure integrity for meaningful operations while eliminating the latency caused by processing fixed-structure zero placeholders.
Data Source
AI summary
An image filtering processing system includes a temporary memory; a flow controller configured to control a feature and a weight to be stored at a point of the temporary memory, and, when at least one of the feature and the weight has a value of 0, to control next feature and next weight to be overwritten at the point; and a processing element configured to perform a multiplication operation on a value of the feature and a value of the weight in the temporary memory and an addition operation on the multiplication result.