Shadow Removal for Local Feature Detection Using Sensor Sensitivity Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision techniques face inefficiencies in training descriptor and semantic segmentation networks due to the need for extensive data collection and processing, particularly when dealing with shadows in images, which can impede the identification of keypoints and pixel classification.
Innovation Solution
The use of a camera sensor sensitivity model to generate shadow-invariant images, which are then utilized to train descriptor and semantic segmentation networks, allowing for more efficient detection of keypoints and improved pixel classification by removing the impact of shadows from the training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional training data collection methods are used to train descriptor and semantic segmentation networks, then comprehensive training data can be obtained, but the process becomes time-consuming and computationally intensive due to the need to handle shadows in images
Solution Approach 1:
The patent applies preliminary action by pre-processing images to remove shadows before training data collection. A shadow removal module processes captured images to generate shadow-free versions, which are then used for training descriptor and semantic segmentation networks. This preliminary shadow elimination step prevents shadows from interfering with the training process, thereby reducing the time and computational resources needed for data collection and processing while maintaining training data quality
Solution Approach 2:
The patent applies the extraction principle by separating and removing the shadow component from images. The shadow removal module specifically extracts and eliminates shadow regions from captured images, creating shadow-free images that contain only the relevant visual information for training. This extraction of harmful shadow elements allows the training process to focus solely on meaningful features, reducing computational overhead and time requirements
2Adaptability or versatility
If shadows are included in training images, then realistic scene conditions are represented, but keypoint identification and pixel classification accuracy deteriorate
Solution Approach 1:
The patent applies the 'blessing in disguise' principle by converting the harmful effect of shadows into a benefit. Instead of allowing shadows to degrade training quality, the system uses a shadow removal module to eliminate shadows, thereby transforming images with potentially harmful shadow regions into high-quality shadow-free training images. This approach maintains adaptability to various scene conditions while improving measurement precision for keypoint detection and pixel classification by ensuring that shadows do not interfere with feature identification
3Adaptability or versatility
If extensive data collection is performed to account for various lighting conditions including shadows, then training comprehensiveness is improved, but processing complexity and computational resources increase
Solution Approach 1:
The patent applies preliminary action by performing shadow removal as a pre-processing step before training data collection and processing. By eliminating shadows in advance, the system reduces the complexity of handling various lighting conditions during training. The shadow removal module standardizes all input images by removing shadow variations, thereby simplifying the training process while maintaining comprehensiveness across different lighting scenarios
Solution Approach 2:
The patent applies universality by creating a universal shadow removal process that handles all types of shadow conditions with a single module. Instead of developing separate processing pipelines for different lighting conditions, the shadow removal module provides a unified solution that works across various scene conditions, thereby reducing processing complexity while maintaining adaptability to diverse lighting environments
Data Source
AI summary
Training a descriptor network includes obtaining a first image of a scene from an image capture device, applying a sensor sensitivity model to the first image to obtain shadow-invariant image data for the first image, and selecting a first patch from the image. Training a descriptor network also includes obtaining a subset of the shadow-invariant image data corresponding to the first patch, and training the descriptor network to provide localization data based on the first patch and the subset of the shadow-invariant image data.