Image Relighting via 3D Reconstruction and Pixel Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in generating images of an environment under different lighting conditions without requiring vehicles to re-travel the same path, which is essential for training machine-learning algorithms for object recognition tasks that depend on interpreting shadows.
Innovation Solution
A computer system processes first images taken in a specific lighting condition, classifies pixels, masks specific categories, generates a three-dimensional representation, and creates artificial images of the environment in various lighting conditions, allowing for the generation of images from the same perspective without the need for extensive viewpoint collection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If vehicles re-travel the same path to collect images under different lighting conditions, then the dataset diversity is improved, but the time consumption and operational complexity increase significantly
Solution Approach 1:
The system performs preliminary actions by collecting images from multiple viewpoints in advance and constructing a three-dimensional representation of the environment. This pre-processing enables subsequent generation of images under different lighting conditions without requiring actual re-traversal of the path, thus resolving the contradiction between dataset diversity and time consumption
Solution Approach 2:
The system creates artificial copies of the environment from the three-dimensional representation to generate images under various lighting conditions. These synthetic images replicate the diversity needed for training machine-learning algorithms without requiring physical re-collection, thereby eliminating time consumption while maintaining dataset versatility
2Adaptability or versatility
If vehicles collect multiple views around the environment, then the lighting condition variety is improved, but the device complexity and operational difficulty increase
Solution Approach 1:
The system replaces the mechanical approach of physically collecting multiple views with a computational approach using three-dimensional representation and image synthesis algorithms. This substitution eliminates the need for complex multi-view collection operations while achieving the same lighting condition variety through virtual rendering
3Manufacturing precision
If images are processed with pixel classification and masking, then the image quality for specific categories is improved, but the processing time increases
Solution Approach 1:
The system segments the image processing task into distinct stages: pixel classification, masking of specific categories, and three-dimensional representation construction. This segmentation allows for optimized processing of each stage independently, improving overall efficiency while maintaining high image quality for specific categories of interest
Data Source
AI summary
A computer includes a processor and a memory storing instructions executable by the processor to receive a plurality of first images of an environment in a first lighting condition, classify pixels of the first images into categories, mask the pixels belonging to at least one of the categories from the first images, generate a three-dimensional representation of the environment based on the masked first images, and generate a second image of the environment in a second lighting condition based on the three-dimensional representation and on a first one of the first images.