Image Processing Apparatus Memory Bandwidth Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In image processing, the cost of apparatuses increases due to high memory bandwidth requirements for encoding and decoding large images, particularly when using predictive encoding methods like AVC and HEVC, which necessitate large memory access for decoding and reusing reference images.
Innovation Solution
An image processing apparatus and method that sets restriction information to limit block sizes and prediction methods, generating prediction images based on these restrictions, and encoding and transmitting the images using these settings, thereby reducing memory bandwidth demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If predictive encoding methods like AVC and HEVC are used for large images, then encoding efficiency is improved, but memory bandwidth requirements increase leading to higher apparatus costs
Solution Approach 1:
The patent segments the image processing into multiple threads (first thread for encoding, second thread for decoding and reuse). This allows parallel processing where decoded images are immediately available for prediction without blocking the encoding process, reducing memory bandwidth requirements while maintaining encoding efficiency.
Solution Approach 2:
The patent implements preliminary decoding of image blocks before they are needed for prediction. The second thread decodes and stores images in advance, so when the first thread needs reference images for predictive encoding, they are already available in memory, reducing peak memory bandwidth demands.
2Quantity of substance
If large block sizes are used for encoding, then compression ratio is improved, but memory access requirements increase
Solution Approach 1:
The patent dynamically adjusts the block size used for prediction based on the available memory bandwidth and processing requirements. Different block sizes can be used for different regions or different prediction directions, optimizing the balance between compression ratio and memory access requirements.
Solution Approach 2:
The patent applies different prediction methods and block sizes to different regions of the image. Regions with high spatial correlation use larger blocks for better compression, while regions with motion or detail use smaller blocks or alternative prediction methods, reducing overall memory bandwidth requirements.
3Productivity
If reference images are stored and reused for prediction, then encoding efficiency is improved, but memory bandwidth consumption increases
Solution Approach 1:
The patent merges the encoding thread and decoding thread into a unified processing flow. The decoding of reference images is combined with the encoding process, allowing shared memory access patterns and reducing redundant memory operations. This integration reduces overall memory bandwidth consumption while maintaining encoding efficiency.
Solution Approach 2:
The system uses its own decoded output as input for prediction without requiring external memory access. By immediately reusing decoded images within the same processing pipeline, the system reduces memory bandwidth consumption and creates a self-sufficient encoding process.
Data Source
AI summary
The present technology relates to an image processing apparatus and method capable of preventing an increase in a cost of the apparatus. A setting unit sets restriction information for restricting a size of a block of an image and a prediction method to be applied to the block having the size. An inter-prediction unit generates a prediction image according to the restriction information. An encoder 1000 encodes the block using the prediction image and generates an encoded stream. Then, the encoder 1000 transmits the encoded stream and the restriction information. The present technology can be applied to a case of encoding/decoding an image, and the like.