CNN Hardware Structure Optimization for FHD to UHD Upscaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face challenges in hardware implementation due to high computational complexity, particularly in achieving the required throughput for image up-scaling from FHD to UHD, which necessitates a large number of operators and is impractical for a single chip set, leading to inefficiencies and bottlenecks in operation.
Innovation Solution
A method to optimize the hardware structure of CNNs by minimizing the number of operators through a pipeline structure, where the operation size of previous layers is adjusted based on the minimum operation size of the final layer, and the number of operators is allocated and reallocated to ensure uniform operation time across layers, thereby eliminating bottlenecks and optimizing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large number of operators are used to achieve the required throughput for real-time image up-scaling, then the throughput is satisfied, but the device complexity and hardware implementation difficulty increase significantly
Solution Approach 1:
The CNN is divided into multiple convolutional layers with different numbers of operators, creating a segmented pipeline structure. Each layer processes a portion of the computational workload, allowing the system to achieve high throughput without requiring all operators to be implemented simultaneously in a single chip set.
Solution Approach 2:
The patent implements dynamic operation time allocation across different layers, where the operation time is adjusted based on the number of operators in each layer. This dynamic timing approach allows efficient pipeline utilization while accommodating varying computational demands across layers.
2Productivity
If parallel computation processing is used to increase throughput, then the required processing speed is achieved, but the bottleneck in operation time between layers increases
Solution Approach 1:
The patent changes the operational parameters of different layers by assigning different operation times based on their respective numbers of operators. This parameter adjustment balances the pipeline stages, minimizing idle time and eliminating bottlenecks while maintaining high throughput performance.
3Power
If all operators are integrated into a single chip set, then the computational power is sufficient, but the hardware implementation becomes practically impossible
Solution Approach 1:
Instead of integrating all operators into a single chip set, the patent segments the computational workload across multiple layers that can be implemented sequentially or distributed across multiple hardware components. This segmentation makes the hardware implementation practically feasible while maintaining sufficient computational power through pipeline processing.
Data Source
AI summary
A method for optimizing a hardware structure of a convolutional neural network including: searching an initial feature value group by which a final convolution layer located at a final stage among a plurality of convolution layers, setting an initial fusing network by analyzing a feature value group having a size corresponding to the initial feature value group; computing an operation time for each layer by allocating the number of operators corresponding to the size of the feature value group to each of the plurality of convolution layers, and dividing the size of the feature value group by determining a layer having a minimum operation time; resetting the fusing network by changing the size of the feature value group, determining the size of the feature value group having the smallest number of operators, determining the number of operators to be included in each of the plurality of convolution layers.