CNN Inference Engine Parallel MAC Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Fully connected neural networks face challenges in efficiently processing large datasets due to high computational workload and noise in data, particularly in image recognition tasks where the number of parameters (N) is large, leading to delays and inaccuracies.
Innovation Solution
A CNN inference engine is developed that includes a slice buffer to cache input data, a system of MAC elements for parallel processing, and a method for operating a data processing system to compute scalar products efficiently, utilizing integer weights and a trinary coding scheme to reduce computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fully connected neural network is used to process large datasets, then the network can perform comprehensive computations, but the computational workload becomes excessively high and processing time increases significantly
Solution Approach 1:
The patent divides the fully connected neural network into multiple layers of convolutional operations, where each layer processes a subset of the computation. This segmentation allows the overall computational task to be distributed across multiple smaller, parallelizable operations, reducing the time complexity from O(N³) to a more manageable level while maintaining comprehensive computation through the stacked convolutional layers.
Solution Approach 2:
The patent introduces spatial dimensionality through convolutional kernels that operate on local regions of the input data. Instead of treating all N inputs as a single flat vector requiring full pairwise connections, the convolutional structure adds spatial organization, allowing computations to be performed on local patches and then combined, dramatically reducing the computational burden while preserving information from all inputs.
2Measurement precision
If the number of parameters N is increased to improve classification accuracy, then more information can be processed, but noise in the data overwhelms the computation and prevents accurate weight determination
Solution Approach 1:
The patent applies local quality by using convolutional kernels that operate on local regions of the input data rather than treating all data points uniformly. Each kernel learns to detect specific local patterns (edges, textures, shapes) that are robust to noise. This local processing approach allows the network to accumulate reliable signals from multiple local features while averaging out random noise, improving classification accuracy even with large N.
Solution Approach 2:
The patent merges multiple convolutional kernels and layers to combine local features into increasingly abstract representations. By stacking convolutional layers and combining their outputs through pooling and non-linear activations, the network integrates information from many local regions in a way that reinforces true patterns while canceling out random noise, enabling accurate classification despite noisy individual measurements.
3Productivity
If parallel computation engines are used to reduce computation time, then processing speed improves, but memory transfer delays between different levels of data storage become significant
Solution Approach 1:
The patent implements preliminary action by pre-loading weight parameters into on-chip memory before computation begins, and by organizing the computational pipeline to overlap memory access with computation operations. The convolutional structure allows weights to be reused across multiple input locations, so weights are loaded once and then applied repeatedly, minimizing memory access frequency. Additionally, the regular structure of convolutional operations enables efficient memory access patterns that reduce transfer delays.
Solution Approach 2:
The patent achieves universality through the convolutional operation itself, which performs the same computational pattern (weighted sum of local inputs) across the entire input data set. This uniform computational structure allows the same hardware circuit to be reused repeatedly for different locations and different layers, maximizing resource utilization and reducing the need for multiple specialized computation units, thereby improving overall productivity while minimizing memory access overhead.
Data Source
AI summary
A CNN inference engine that convolves an input data set with a weight data set is disclosed together with components that facilitate such computation. The engine includes a plurality of multiply and accumulate processors (MACs), each MAC causing a value in the accumulator to be augmented by a product of a data value received on an input data port, a weight value received on a weight port. The engine also includes a slice buffer having a plurality of output ports, each output port being connected to one of the MAC input data value ports. The engine causes the slice buffer to connect one of the slices to the plurality of slice buffer output ports, and causes a weight received on an inference engine weight port to be input to each MAC weight port. The MACs process the input data values on the output ports in the slice in parallel.