Head-Mounted Display Colorization for Low-Light Passthrough
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users of head-mounted displays (HMDs) experience fatigue due to long exposure to black and white passthrough feeds in low light conditions, and capturing accurate color imagery without active illumination is typically not feasible.
Innovation Solution
A head-mounted display system that includes a processor, camera, inertial measurement unit, and memory, configured to apply denoising and filtering techniques to low light images, identify objects, assign colors, and modify images based on pose data to generate color images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If active illumination is used to capture color imagery in low light conditions, then color accuracy is improved, but the desirability of the application decreases due to detection risk and operational constraints
Solution Approach 1:
The patent uses an intermediary approach by capturing images in the visible spectrum without active illumination, then using computational methods (deep learning models, denoising algorithms) as intermediaries to reconstruct color information from the captured low-light images, eliminating the need for active illumination while maintaining color accuracy
Solution Approach 2:
The patent replaces the mechanical/optical system of active illumination (flashlights, LEDs) with a computational system that processes captured images through denoising algorithms, deep learning colorization models, and image processing techniques to generate color imagery without physical light sources
2Object-affected harmful factors
If black and white passthrough feed is used in low light conditions, then the system operates without active illumination, but user fatigue increases due to lack of color information
Solution Approach 1:
The patent applies color changes by transforming black-and-white images into color images using deep learning colorization models that predict and assign color information to objects and scenes, making the display more visually appealing and reducing user fatigue while maintaining operational constraints
Solution Approach 2:
The patent changes parameters by applying denoising algorithms and image processing techniques to enhance image quality, adjust brightness and contrast, and improve overall visual appearance of low-light images, making them more comfortable for extended viewing
3Measurement precision
If image processing techniques are applied to enhance color and reduce noise, then image quality is improved, but processing time and computational complexity increase
Solution Approach 1:
The patent applies preliminary action by pre-processing captured images through denoising algorithms and preparing them for colorization before the main color reconstruction process, optimizing the workflow to reduce overall processing time while maintaining image quality
Solution Approach 2:
The patent uses partial action by applying image processing techniques selectively to regions or frames where they are most needed, rather than processing all images uniformly, thereby reducing computational complexity and processing time while maintaining quality where critical
Data Source
AI summary
In an embodiment, a method includes receiving a first image captured via a head-mounted display (HMD) under a low light condition. The method further includes applying at least one of a denoising technique or a filtering technique to the first image to generate a first modified image. The method further includes identifying a first object in the first modified image. The method further includes, in response to determining that the first object does not have a color assignment, assigning a first color to the first object at the first modified image. The method further includes, in response to determining that the first object is assigned the first color at the first modified image, assigning the first color to the first object at a second modified image.