Configurable Image Processing Pipeline for SoC Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image processing devices in mobile communication devices face significant memory latency and power consumption issues due to the need to read image data from memory for scaling operations, which affects the efficiency of image scaling methods.
Innovation Solution
A System on Chip (SoC) with a configurable image processing pipeline that includes multiple bypass and scaler paths, allowing for dynamic selection and connection of scalers based on control signals to optimize image data processing and reduce memory access, thereby minimizing power consumption and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image data is read from memory for scaling operations, then image scaling can be performed, but memory latency increases and power consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-fetching image data from memory into an internal buffer before scaling operations are needed. The memory controller proactively loads data based on predicted access patterns, so that when the image processing device requests data, it is already available in the buffer, eliminating wait time and reducing memory latency.
Solution Approach 2:
The patent introduces an intermediary buffer between the memory and the image processing device. This buffer acts as a mediator that decouples the memory access timing from the processing timing, allowing the processor to operate continuously without waiting for memory reads, thus reducing the impact of memory latency on overall productivity.
2Productivity
If image data is read from memory for scaling operations, then image scaling can be performed, but power consumption increases
Solution Approach 1:
The system performs preliminary data loading into an internal buffer during periods when processing demand is low, utilizing idle time to pre-fetch data. This reduces the frequency and intensity of high-power memory access operations, thereby lowering overall power consumption while maintaining the capability to perform scaling operations on demand.
Solution Approach 2:
The memory controller operates in a periodic manner, batching data transfers into structured intervals rather than continuous operations. By organizing memory accesses into periodic cycles with active transfer phases and idle phases, the system reduces peak power consumption and allows the memory interface to enter low-power states during idle periods.
3Adaptability or versatility
If multiple scalers are configured in the image processing pipeline, then processing flexibility increases, but device complexity increases
Solution Approach 1:
The patent implements dynamic configurability where the image processing pipeline can be reconfigured at runtime based on the specific processing requirements. The system includes control logic that dynamically enables or disables scaler stages and adjusts pipeline topology according to the desired scaling operations, providing adaptability without permanently incorporating all possible configurations, thus managing device complexity.
Solution Approach 2:
The patent employs universal scaler blocks that can perform multiple scaling functions (upscaling, downscaling, different aspect ratios) through programmable control parameters. Rather than implementing separate dedicated hardware for each scaling function, a single multi-functional scaler unit handles diverse scaling requirements, reducing overall device complexity while maintaining high processing flexibility.
Data Source
AI summary
A system on chip (SoC) including a configurable image processing pipeline is provided. The SoC includes a bus; a first image processing module configured to be connected to the bus and to process image data; a first image processing stage configured to transmit either first image data or second image data received from the bus to at least one of the bus and the first image processing module through a first bypass path in response to first control signals; and a second image processing stage configured to transmit either third image data received from the first image processing module or fourth image data received from the bus to the bus through one of a second bypass path and a second scaler path in response to second control signals.