Image Processing Pixel Calculation Order for Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies face inefficiencies in re-sampling and rotating input images to meet fixed size requirements for subsequent processing, particularly in neural networks, due to varying subject distances and orientations, which lead to excessive interpolation and memory access issues.
Innovation Solution
An image processing apparatus that determines an optimal order for calculating output image pixels to maximize data reuse in local memory, allowing for simultaneous data reading and computation, and employs a sampling module to read input image data in bursts across a required width, rather than an entire area, thereby minimizing storage and memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If output image pixels are calculated in order of pixel positions in the output image, then the output image can be generated systematically, but the amount of storage required for local memory and the number of input image read requests increase
Solution Approach 1:
The patent inverts the traditional calculation order by determining pixel calculation sequences based on input data availability rather than output image spatial order. This allows the system to process pixels in an order that maximizes data reuse from input stripes, reducing memory storage requirements while maintaining processing efficiency
Solution Approach 2:
The patent performs preliminary organization of input image data into stripes before processing. By pre-organizing the input data structure and determining which stripes are needed in advance, the system can efficiently reuse the same stripe data for multiple output pixels without requiring large amounts of local memory storage
2Productivity
If the same set of data is reused for multiple output pixels, then memory access efficiency improves, but the calculation order becomes more complex
Solution Approach 1:
The patent implements a dynamic calculation order that adapts based on the spatial relationship between output pixels and input stripes. Rather than using a fixed calculation sequence, the system dynamically determines which output pixels can be calculated from currently available input stripes, allowing flexible data reuse while managing complexity through adaptive processing
3Loss of time
If input image data is read in bursts covering required width, then memory access time is reduced, but the data reading complexity increases
Solution Approach 1:
The patent segments the input image into horizontal stripes that can be read in efficient bursts. By dividing the input data into manageable stripe segments corresponding to processor width requirements, the system can read data in optimized batches rather than individual pixels, reducing memory access time while the segmentation structure itself manages the reading complexity
Data Source
AI summary
The present invention relates to an image processing apparatus which determines an order for calculating output image pixels that maximally reuses data in a local memory for computing all relevant output image pixels. Thus, the same set of data is re-used until it is no longer necessary. Output image pixel locations are browsed to determine pixel values in an order imposed by available input data, rather than in an order imposed by pixel positions in the output image. Consequently, the amount of storage required for local memory as well as the number of input image read requests and data read from memory containing the input image is minimized.