Robot Object Localization Using Fractional Occlusion Frustums
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous mobile robots face challenges in accurately localizing and identifying objects in environments due to occlusions, which lead to incorrect object detection and navigation issues, especially when objects are partially hidden by obstacles within the camera's field of view.
Innovation Solution
The system employs a controller circuit that calculates fractional visibility values using multiple images from different perspectives, incorporating an attenuation model to determine the probability of object visibility at various locations, allowing for accurate object localization and recognition even when partially occluded.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the robot uses a single camera view to detect objects, then the detection process is simple and fast, but object localization accuracy deteriorates when objects are occluded
Solution Approach 1:
The patent transitions from 2D image analysis to 3D spatial reasoning by calculating fractional visibility values across three-dimensional frustum volumes. The system projects 3D frusta from multiple camera positions and computes visibility probabilities at discrete 3D points within the environment, enabling accurate localization of occluded objects through volumetric analysis rather than planar image processing.
Solution Approach 2:
The system dynamically adapts the detection approach by adjusting the number of camera positions and frustum calculations based on occlusion detection. When occlusions are detected, the robot performs additional viewpoint changes and recalculates visibility values, creating a dynamic detection process that intensifies computational effort only when necessary to resolve occluded objects.
2Measurement precision
If the robot takes multiple images from different perspectives to handle occlusions, then object localization accuracy improves, but the time and computational resources required increase
Solution Approach 1:
The system performs preliminary occlusion detection by analyzing initial image data and identifying regions where objects may be hidden. Based on this preliminary assessment, it pre-determines the necessary number and positions of additional camera viewpoints required to resolve the occlusions, avoiding unnecessary image captures and computations for non-occluded regions.
Solution Approach 2:
The patent implements a graduated detection approach where the system first attempts object localization with minimal viewpoints, then progressively adds more camera positions and computational analysis only to the extent necessary to resolve detected occlusions. This partial action principle ensures that full computational resources are deployed only when occlusion complexity demands it.
3Measurement precision
If the robot assumes complete occlusion when an object is not visible in the current view, then the detection process is simple, but localization accuracy deteriorates due to incorrect assumptions
Solution Approach 1:
The patent introduces fractional visibility values as an intermediary metric between binary occlusion states. Instead of assuming complete occlusion or complete visibility, the system calculates continuous visibility probabilities (e.g., 0.3, 0.7, 0.9) that represent the degree to which an object is visible from each camera position. This intermediary measurement enables nuanced reasoning about occlusion states and improves localization accuracy.
Solution Approach 2:
The system transforms the detection problem from binary (occluded/not occluded) to continuous by changing the parameter from discrete occlusion state to continuous fractional visibility value. This parameter transformation allows the robot to quantify partial occlusions and use these graded measurements to infer object positions behind obstacles, rather than making binary assumptions that reduce accuracy.
Data Source
AI summary
Described herein are systems, devices, and methods for localizing and recognizing an object in an environment. In an example, a mobile cleaning robot comprises a drive system to move the mobile cleaning robot about an environment, an imaging sensor to take images of an object in the environment from different perspectives. The multiple observations include images of the object that is at least partially occluded by an obstacle. A controller circuit of the mobile robot can, for multiple different locations in a map of the environment, calculate respective fractional visibility values using the plurality of images. The fractional visibility values each represent a probability of the object being visible through the corresponding location. The controller circuit can localize and recognize the object based on the fractional visibility values at the multiple locations on the map.