Low-Light Image Enhancement for Real-Time XR Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Extended reality (XR) systems, particularly video see-through (VST) XR devices, face challenges in low-light environments due to dark and noisy image frames, which affect user discernment and comfort.
Innovation Solution
A low-light visibility enhancement model is trained using a dataset from imaging sensors to improve brightness and contrast of captured image frames, applying specific models based on sensor properties and environmental conditions to enhance image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If image processing is performed in real-time for low-light enhancement, then image visibility and quality are improved, but processing time and computational complexity increase
Solution Approach 1:
The system performs preliminary actions by capturing multiple image frames at different exposure levels before the actual enhancement is needed. These pre-captured frames are stored and then selectively processed based on the low-light detection, avoiding the need for real-time processing of every frame and reducing overall processing time.
Solution Approach 2:
The system dynamically adjusts the image processing pipeline based on detected low-light conditions. When low-light conditions are detected, the system activates the enhanced processing path using pre-captured multi-exposure data. When lighting is adequate, it uses the standard processing path, creating a dynamic adaptation that optimizes processing time based on actual conditions.
2Manufacturing precision
If multiple image frames are captured at different exposure levels, then low-light visibility enhancement is improved, but device complexity and data processing requirements increase
Solution Approach 1:
The system segments the image capture process into multiple independent exposure levels (e.g., short exposure, long exposure). Each exposure level captures specific information optimized for different lighting conditions. This segmentation allows the system to process only the relevant exposure data needed for low-light enhancement rather than processing all captured data uniformly, reducing overall system complexity.
Solution Approach 2:
The system applies different quality processing to different parts of the image data based on their exposure characteristics. High-exposure frames are processed differently from low-exposure frames, with the system selectively applying enhancement algorithms only to the portions of data that are most useful for low-light conditions, thereby managing complexity through localized processing strategies.
3Manufacturing precision
If image enhancement algorithms are applied to all image frames, then image visibility is improved, but computational energy consumption increases
Solution Approach 1:
The system performs preliminary detection of low-light conditions using a simple threshold check on the captured image frames. Only when low-light conditions are detected does the system activate the computationally intensive enhancement algorithms. This preliminary filtering action ensures that energy-intensive processing is performed only when necessary, significantly reducing overall computational energy consumption while maintaining image visibility where needed.
Solution Approach 2:
The system applies partial action by selectively processing only the subset of image frames that require enhancement based on low-light detection. Rather than applying the full enhancement algorithm to every frame, the system processes only the necessary frames (those with low-light conditions), reducing computational energy consumption while still providing adequate image visibility enhancement for the critical low-light scenarios.
Data Source
AI summary
A method includes obtaining, using at least one imaging sensor of an electronic device, a first image frame of a scene. The method also includes determining, using at least one processing device of the electronic device, a low-light image score indicative of a brightness of the first image frame. The method further includes, in response to the low-light image score indicating that the brightness of the first image frame is below a threshold, applying, using the at least one processing device, a low-light visibility enhancement model to the first image frame in order to generate a second image frame having a higher brightness than the first image frame. The low-light visibility enhancement model is trained using at least one dataset that includes image frames obtained using the at least one imaging sensor of the electronic device.