Registration Error Map Using Light-Field Refocusing for Visual Servoing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual servoing techniques, whether Image-Based Control (IBC) or Position-Based Control (PBC), face challenges with maintaining the object within the camera's field of view and determining the relative pose accurately, leading to instability and convergence issues.
Innovation Solution
A processor computes a registration error map by intersecting a re-focusing surface derived from a three-dimensional model with a focal stack based on four-dimensional light-field data, determining a re-focusing distance for each pixel, and displaying a sharpness level map to guide the user or robot into the correct viewing position.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional visual servoing techniques (IBC or PBC) are used, then control can be implemented, but the object may get out of the camera field of view making it difficult to determine relative pose
Solution Approach 1:
The patent implements a feedback mechanism by computing a registration error map that provides visual feedback to the operator. The system continuously compares the captured image with a reference image, generates an error map indicating misalignment, and uses this feedback to guide adjustments until the object is properly registered within the field of view, thereby maintaining reliable visual servoing control.
Solution Approach 2:
The system performs preliminary action by pre-computing a registration error map based on a reference image before actual visual servoing operations begin. This allows the system to establish a reference framework in advance, enabling faster and more accurate determination of relative pose during subsequent operations without needing to search for the object from scratch.
2Ease of operation
If Image-Based Control is used, then visual feedback is directly defined in the image, but stability and convergence problems occur
Solution Approach 1:
The patent transitions from traditional 2D image-based control to a 3D-enhanced approach by incorporating depth information through light-field data. The system computes a registration error map that includes depth-aware pixel displacement, adding a dimensional aspect that enables more stable and convergent visual feedback while maintaining ease of operation through intuitive visual representation.
3Measurement precision
If Position-Based Control with 3D model is used, then control error function is computed in Cartesian space, but the object may still move out of field of view
Solution Approach 1:
The patent creates a universal registration error map that serves multiple functions simultaneously: it provides control error information for Position-Based Control, maintains object tracking within the field of view, and offers visual guidance for operators. The error map integrates Cartesian space computations with visual field monitoring, enabling one system to fulfill multiple objectives that were previously separate.
Data Source
Figure 1A~1B
Figure 2~4
Figure 5
AI summary
The present invention generally relates to an apparatus and a method for obtaining a registration error map representing a level of sharpness of an image. Many methods are known which allow determining the position of a camera with respect to an object, based on the knowledge of a 3D model of the object and the intrinsic parameters of the camera. However, regardless of the visual servoing technique used, there is no control in the image space and the object may get out of the camera field of view during servoing. It is proposed to obtain a registration error map relating to an image of the object of interest generated by computing an intersection of a re-focusing surface obtained from a 3D model of said object of interest and a focal stack based on acquired four-dimensional light-field data relating to said object of interest.