Dual-ISP Display Pipeline for Low-Latency Camera Image Output
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current display technologies experience significant delays in image data transmission, resulting in a lag of two or more frames between the real-time image captured by a camera sensor and the image viewed by the human eye, which is detrimental in applications requiring precise timing, such as sighting devices and mixed reality systems.
Innovation Solution
A display control apparatus with a Mini ISP module for real-time lens correction, defective pixel correction, noise filtering, and format conversion, and a compensation ISP module for additional processing steps like black level compensation and color enhancement, allowing simultaneous display of preliminary and advanced image data to reduce processing delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If comprehensive image processing is performed to improve image quality, then image processing time increases causing display delay
Solution Approach 1:
The patent divides the image processing system into two independent modules: a mini ISP module that performs basic processing (lens correction, defective pixel correction, noise filtering, format conversion) and a compensation ISP module that performs advanced processing (black level compensation, color interpolation, white balance correction, color correction, gamma correction, color space conversion, color noise filtering, edge enhancement, color and contrast enhancement, automatic exposure control). This segmentation allows the basic processing to complete quickly for real-time display while advanced processing can be performed separately without blocking the display pipeline.
Solution Approach 2:
The mini ISP module performs preliminary image processing operations before the image data is sent to the display control module. By completing essential corrections and conversions in advance, the system ensures that basic image quality requirements are met while the image data is still in transit or being processed by the compensation ISP module, thereby reducing overall processing delay.
2Manufacturing precision
If multiple processing steps are performed sequentially, then image processing completeness improves but processing delay increases
Solution Approach 1:
The patent segments processing steps into two groups: essential processing (lens correction, defective pixel correction, noise filtering, format conversion) handled by the mini ISP module, and advanced processing (black level compensation, color interpolation, white balance correction, color correction, gamma correction, color space conversion, color noise filtering, edge enhancement, color and contrast enhancement, automatic exposure control) handled by the compensation ISP module. This allows essential processing to complete quickly for real-time display while advanced processing occurs in parallel or subsequently without blocking the display pipeline.
Solution Approach 2:
The patent merges the outputs of both the mini ISP module and compensation ISP module into a unified display pipeline. The display control module receives image data from the mini ISP module and can incorporate additional processing results from the compensation ISP module, combining both processing streams to achieve complete image processing while maintaining real-time display capability.
3Device complexity
If a single ISP module is used, then device complexity is reduced but processing speed decreases
Solution Approach 1:
The patent implements segmentation by creating two specialized ISP modules with distinct processing responsibilities. The mini ISP module handles time-critical basic processing operations, while the compensation ISP module handles computationally intensive advanced processing. This segmentation enables parallel processing of different image data streams or different processing stages, significantly improving overall processing speed compared to a single sequential ISP module.
Solution Approach 2:
The system dynamically selects which processing module outputs to use based on real-time requirements. The display control module can prioritize mini ISP output for immediate display needs while incorporating compensation ISP output when available, creating a dynamic processing architecture that adapts to varying performance requirements and optimizes processing speed.
Data Source
AI summary
Display control apparatus, display control method and display device are provided, belonging to the field of displaying. The apparatus includes: Mini ISP module configured to obtain original image data of camera sensing component in real time, to obtain first image data by performing lens correction, defective pixel correction, noise filtering and format conversion on the original image data; compensation ISP module, configured to obtain second image data by performing lens correction, defective pixel correction, noise filtering, format conversion and first processing on the original image data; display control module, configured to process the first image data to obtain image data for displaying before receiving the second image data, to process the second image data to obtain image data for displaying after receiving the second image data and to control display screen to display according to the image data for displaying. The delay in image data transmission can be reduced.