Plenoptic Camera Annotation Matching Light Field Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in efficiently annotating scenes due to high bandwidth requirements, delays in computing annotations, and the tedious process of building and registering 3D models, which are sensitive to viewing angles and lighting conditions.
Innovation Solution
A method and apparatus utilizing a plenoptic camera to capture and annotate scenes by matching light field data with reference data, performing registration entirely in the plenoptic domain without converting to 2D or 3D images, leveraging the direction and intensity of light rays for precise annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional 2D image capture is used, then device complexity is low, but recognition performance and registration accuracy are insufficient under varying viewing angles and lighting conditions
Solution Approach 1:
The patent transitions from conventional 2D image capture to plenoptic camera capture, adding a new dimension of light field data that includes both spatial and angular information. This enables robust recognition and registration under varying viewing angles and lighting conditions without significantly increasing device complexity, as the plenoptic camera integrates multiple views into a single capture.
2Adaptability or versatility
If 3D reference models are used for annotation, then adaptability to different viewing angles improves, but device complexity and manufacturing difficulty increase due to the need for 3D or stereo cameras
Solution Approach 1:
Instead of using complex 3D reference models, the patent employs 2D reference images that are enhanced with light field data. This approach maintains adaptability to different viewing angles by capturing multiple angular views in the plenoptic domain while avoiding the complexity of building and managing 3D models.
Solution Approach 2:
The patent creates a virtual copy of the light field data from the captured plenoptic image, allowing the system to reference and compare with stored 2D reference images without needing physical 3D models. This copying approach simplifies the reference database while maintaining the ability to handle various viewing angles.
3Ease of manufacture
If 2D reference images are used for annotation, then ease of manufacture is high, but recognition performance deteriorates due to sensitivity to viewing angle and lighting conditions
Solution Approach 1:
The patent enhances 2D reference images by incorporating light field data that captures angular information. This adds a new dimension to the reference data, enabling robust matching with captured images from various viewing angles and lighting conditions while maintaining the simplicity of using 2D image formats.
4Power
If images are transmitted to remote servers for annotation, then processing capability is enhanced, but loss of time increases due to transmission and computation delays
Solution Approach 1:
The patent extracts and processes only the essential light field features and descriptors locally on the mobile device before transmission to the server. This preliminary action reduces the amount of data that needs to be transmitted and processed remotely, significantly reducing communication time and computational delay while still benefiting from server-side processing capabilities.
Data Source
AI summary
The present invention relates to an annotating method including the steps of capturing (100) data representing a light field with a plenoptic image capture device (4); matching (101) the captured data with a corresponding reference data; retrieving an annotation associated with an element of the reference data (102); rendering (103) a view generated from the captured data and including at least one annotation.