Shallow CNN Ensemble for Reduced Computational Complexity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Convolutional Neural Networks (CNNs) are computationally intensive due to a large number of nodes and parameters, which becomes problematic with limited computational power and memory in computing devices.
Innovation Solution
An improved CNN architecture is proposed that reduces the number of nodes and parameters by using a down-sampled input image and shallow CNN networks imitating a deep neural network, thereby reducing computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional CNN architecture with many nodes and parameters is used, then feature detection capability is improved, but computational complexity and memory requirements increase
Solution Approach 1:
The patent segments the deep CNN into multiple shallow CNNs that process different regions of the input image independently. Each shallow CNN handles a specific spatial region, reducing the computational burden on any single network while maintaining comprehensive feature detection across the entire image through parallel processing of segmented regions
Solution Approach 2:
The patent applies partial action by using multiple shallow CNNs that each process only a portion (region) of the input image rather than requiring a single deep CNN to process the entire image. This partial processing approach reduces computational complexity while the collective output of multiple shallow networks achieves the feature detection capability of a full deep network
2Measurement precision
If deep CNN with many layers is used, then processing accuracy is improved, but processing time and energy consumption increase
Solution Approach 1:
The patent segments the deep processing task into multiple shallow processing stages, where each shallow CNN processes a specific region in parallel. This segmentation reduces the sequential processing time while maintaining accuracy through the collective contribution of multiple specialized shallow networks
Solution Approach 2:
The patent implements periodic action through parallel processing of multiple shallow CNNs that operate simultaneously on different image regions. This parallel periodic processing reduces overall processing time compared to sequential deep network processing, while maintaining accuracy through aggregated results
3Productivity
If high capacity neural network is used, then machine learning performance is improved, but memory requirements exceed available resources
Solution Approach 1:
The patent segments the large-scale neural network into multiple smaller shallow CNNs, each requiring significantly less memory than the original deep network. This segmentation enables deployment on devices with limited memory resources while maintaining overall machine learning performance through the combined capability of multiple specialized networks
Solution Approach 2:
The patent uses multiple copies of shallow CNN architectures instead of a single deep CNN. Each shallow network is a simplified copy that processes specific regions, reducing individual memory requirements while the ensemble of copied shallow networks achieves the performance of a full deep network
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Methods and systems are disclosed using improved Convolutional Neural Networks (CNN) for image processing. In one example, an input image is down-sampled into smaller images with a smaller resolution than the input image. The down-sampled smaller images are processed by a CNN having a last layer with a reduced number of nodes than a last layer of a full CNN used to process the input image at a full resolution. A result is outputted based on the processed down-sampled smaller images by the CNN having a last layer with a reduced number of nodes. In another example, shallow CNN networks are built randomly. The randomly built shallow CNN networks are combined to imitate a trained deep neural network (DNN).