Integrated Upscaling and Convolution Processing for CNN Memory Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image upscaling techniques for convolutional neural networks (CNNs) require a line buffer for temporary image storage, leading to increased power consumption for memory access and higher memory costs as image size grows.
Innovation Solution
An information processing apparatus and method that integrates upscaling and convolution processing within the same layer, eliminating the need for temporary storage of upscaled feature data and reducing memory access power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image upscaling processing is performed using a line buffer for temporary storage, then upscaling accuracy is improved, but power consumption for memory access increases and memory cost increases
Solution Approach 1:
The patent combines upscaling processing and convolution processing into a single integrated operation within the same computational unit. The upscaling unit performs upscaling on feature data, and the convolution processing unit immediately processes the upscaled data without requiring temporary storage in a line buffer. This merging eliminates the need for intermediate memory storage, reducing power consumption while maintaining processing accuracy.
Solution Approach 2:
The patent extracts the upscaling function from a separate processing stage that required line buffer storage, and integrates it directly into the convolution processing pipeline. By taking out the upscaling operation and embedding it within the convolution unit, the system eliminates the harmful line buffer component while preserving the beneficial upscaling function.
2Measurement precision
If image upscaling processing is performed using a line buffer for temporary storage, then upscaling accuracy is improved, but memory cost increases
Solution Approach 1:
The patent merges upscaling and convolution processing into a single integrated operation, eliminating the need for line buffer memory. The computational unit processes feature data through upscaling and convolution in one continuous operation, removing the requirement for temporary storage memory and thereby reducing memory cost.
Solution Approach 2:
The patent extracts the upscaling function from a separate processing stage that required line buffer storage, and integrates it directly into the convolution processing pipeline. This extraction eliminates the need for line buffer memory while preserving upscaling accuracy.
3Adaptability or versatility
If separate upscaling and convolution processing stages are used, then processing flexibility is improved, but device complexity increases
Solution Approach 1:
The patent combines upscaling processing and convolution processing into a single integrated computational unit. The upscaling unit and convolution processing unit operate together in one processing stage, reducing device complexity while maintaining the flexibility to process different feature maps through the same integrated pipeline.
Data Source
AI summary
An information processing apparatus that processes feature data in a plurality of feature maps in accordance with a network structure including a plurality of layers, the information processing apparatus comprising: a data holding unit configured to hold feature data; a first reading unit configured to read the feature data based on a read pattern; an upscaling unit configured to upscale the read feature data; and a convolution processing unit configured to perform convolution processing on the feature data upscaled by the upscaling unit, wherein the upscaling unit and the convolution processing unit execute upscaling and convolution processing on the feature data in one feature map, and then execute upscaling and convolution processing on the feature data in the next feature map.