CNN Pipelining for Throughput and Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Convolutional Neural Networks (CNNs) face challenges in increasing throughput while minimizing power consumption and cost, particularly in accelerating execution across various computation layers without increasing CPU frequency or adding additional CPUs.
Innovation Solution
The implementation involves pipelining data flow processing operations across convolution, RELU, and pooling layers, allowing data flow processors to process convolution/RELU layer outputs immediately, and executing fully connected layer operations in parallel, thereby reducing power consumption and enhancing processing speed without increasing costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If more SIMD MAC units and higher frequency DSPs are added to accelerate CNN throughput, then computation speed is improved, but switching power consumption increases
Solution Approach 1:
The patent segments the CNN computation into distinct layers (convolution, pooling, fully connected) and processes them using different strategies. The convolution layer uses traditional SIMD MAC units while the pooling and fully connected layers use alternative processing approaches that consume less power, thereby segmenting the energy consumption profile across different computational stages.
Solution Approach 2:
The patent changes the computational parameters by replacing standard multiplication-accumulation operations in pooling and fully connected layers with alternative operations that require fewer switching events. This parameter change reduces switching power consumption while maintaining computational throughput for these specific layers.
2Speed
If CPU frequency is increased to accelerate CNN execution, then processing speed is improved, but power consumption increases
Solution Approach 1:
The patent segments the CNN processing pipeline into parallel stages that can operate independently at different frequencies. By separating convolution, pooling, and fully connected layer processing, the system can optimize frequency settings for each stage, avoiding the need to increase the entire CPU frequency while still achieving overall speedup.
Solution Approach 2:
The patent implements dynamic processing where different layers of the CNN are processed at different frequencies based on their computational requirements. The pooling and fully connected layers are processed at lower frequencies using alternative operations, while only the computationally intensive convolution layer operates at higher frequencies, creating a dynamic frequency distribution that reduces overall power consumption.
3Productivity
If additional CPUs are added to increase CNN throughput, then computation capacity is improved, but cost increases
Solution Approach 1:
The patent merges the processing of pooling and fully connected layers with the convolution layer processing by implementing a unified pipeline architecture. This consolidation allows multiple CNN layers to be processed within a single CPU using optimized instruction sequences, thereby achieving increased computation capacity without proportionally increasing the number of CPUs required.
Solution Approach 2:
The patent makes the CPU universally capable of handling different CNN layer types through a unified processing approach. The same CPU infrastructure is used to process convolution, pooling, and fully connected layers, with the system dynamically adapting its processing strategy based on the layer type, thereby maximizing the utility of each CPU and reducing the total quantity needed.
4Duration of action of moving object
If sequential processing of CNN layers is used, then implementation simplicity is maintained, but processing time increases
Solution Approach 1:
The patent implements preliminary action by pre-organizing the CNN layers into a structured pipeline with predetermined data flow paths. The pooling and fully connected layers are prepared to receive data from previous layers in advance, allowing immediate processing upon data arrival. This preliminary structuring enables parallel execution without requiring complex dynamic coordination during runtime.
Solution Approach 2:
The patent establishes continuity of useful action by creating an uninterrupted data flow pipeline where the output of one layer immediately becomes the input of the next layer without intermediate storage or sequential waiting. The processing architecture maintains continuous operation across convolution, pooling, and fully connected layers, eliminating idle time and enabling sustained high-speed processing through parallel execution.
Data Source
AI summary
Convolutional neural network (CNN) components can operate to provide various speed-ups to improve upon or operate as part of an artificial neural network (ANN). A convolution component performs convolution operations that extract data from one or more images, and provides the data to one or more rectified linear units (RELUs). The RELUs are configured to generate non-linear convolution output data. A pooling component generates pooling outputs in parallel with the convolution operations via a pipelining process based on a pooling window for a subset of the non-linear convolution output data. A fully connected (FC) component configured to form an artificial neural network (ANN) that provides ANN outputs based on the pooling outputs and enables a recognition of a pattern in the one or more images based on the ANN outputs. Layers of the FC component are also able to operate in parallel in another pipelining process.