Monocular Depth Prediction in Low Light via Synthetic Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systems using monocular cameras face difficulties in generating accurate depth maps in low-light conditions, particularly at nighttime, and combining information from monocular cameras with other sensors increases complexity.
Innovation Solution
A two-stage training approach for a prediction system that uses synthetic images to simulate daytime representations of low-light environments, employing pose and flow losses to improve semantic consistency, and then applies these models to real low-light images using an adversarial network to generate clearer depth maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a monocular camera is used to predict depth without relying on LIDAR or stereo cameras, then device complexity is reduced, but measurement precision of depth map deteriorates
Solution Approach 1:
The system performs preliminary training using synthetic images with ground truth depth maps to pre-train the neural network. This preliminary action allows the model to learn depth prediction patterns before being deployed with real monocular images, thereby improving measurement precision while maintaining device simplicity.
Solution Approach 2:
The system creates synthetic copies of real-world scenes with known depth information to train the model. These synthetic images serve as proxies for real depth data, allowing the system to learn from simulated environments and apply the learned patterns to real monocular images, thus improving depth prediction accuracy without additional sensors.
2Reliability
If the system is trained for low-light conditions using real data, then reliability in low-light environments is improved, but loss of information increases due to noise and poor visibility
Solution Approach 1:
The system uses synthetic images that copy the appearance and lighting conditions of low-light environments but without the noise and information loss of real low-light photographs. These synthetic copies provide clean training data that teaches the model how to handle low-light scenarios without being corrupted by actual low-light degradation.
Solution Approach 2:
The system changes the lighting parameters in synthetic images to simulate various low-light conditions (moonlight, streetlights, twilight) while maintaining perfect image quality. By adjusting illumination parameters in the synthetic data rather than using degraded real low-light images, the model learns robust low-light prediction without information loss.
3Measurement precision
If pose and flow losses are computed to account for object motion between frames, then measurement precision of depth map is improved, but loss of time increases due to additional computations
Solution Approach 1:
The system computes pose and flow losses during the training phase rather than during real-time operation. By performing these additional computations in advance during training, the model internalizes the motion compensation knowledge, allowing for faster real-time depth prediction without sacrificing accuracy.
Solution Approach 2:
The system uses pose and flow losses as feedback signals during training to guide the optimization of depth prediction. These loss functions provide directional guidance to the gradient descent algorithm, improving convergence efficiency and reducing overall training time while maintaining high measurement precision.
Data Source
AI summary
System, methods, and other embodiments described herein relate to training a prediction system for improving depth perception in low-light. In one embodiment, a method includes computing, in a first training stage, losses associated with predicting a depth map for a synthetic image of a low-light scene, wherein the losses include a pose loss, a flow loss, and a supervised loss. The method also includes adjusting, according to the losses, a style model and a depth model. The method also includes training, in a second training stage, the depth model using a synthetic representation of a low-light image. The method also includes providing the depth model.