Narrow-FOV Robot Vision Using Global Map Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic devices face challenges in differentiating between objects and surfaces, particularly when they blend into the background, leading to navigation errors and potential collisions, such as with transparent glass panels.
Innovation Solution
The integration of a sensor system that captures images and applies pre-trained machine learning models to both captured and rendered images, providing broader context for property determination, such as drivability, by using a map of the environment and sensor data from RGB, depth, or LIDAR sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a narrow field of view sensor is used on the robotic device, then the sensor can capture detailed images of the immediate environment, but the robotic device cannot obtain broader context information about the environment
Solution Approach 1:
The patent introduces a global map as an intermediary data structure that contains pre-captured images from multiple locations. This global map serves as a mediator between the narrow field of view sensor and the broader environmental context, allowing the robotic device to access wide-area information without requiring a wide field of view sensor. The global map is generated by capturing images at multiple locations and stitching them together to create a comprehensive environmental representation.
Solution Approach 2:
The patent applies preliminary action by pre-capturing images and constructing a global map of the environment before the robotic device performs its primary task. This pre-processing step creates a ready-to-use contextual framework that can be quickly referenced during operation, eliminating the need for the device to capture and process wide-area images in real-time while maintaining access to comprehensive environmental information.
2Productivity
If the robotic device uses only captured images from a narrow field of view sensor, then the processing load is low, but the ability to differentiate between objects and surfaces is reduced
Solution Approach 1:
The patent merges captured images from the narrow field of view sensor with corresponding images from the global map. By combining these two image sources, the system enriches the visual information available for analysis. The captured image provides current detailed view while the global map image provides contextual information about the broader environment, together enabling better object-surface differentiation without significantly increasing processing complexity.
3Loss of information
If the robotic device captures and processes images from multiple locations to create a global map, then broader environmental context is obtained, but the device complexity increases
Solution Approach 1:
The patent performs the complex task of multi-location image capture and map construction as a preliminary action before the robotic device begins its primary operation. This pre-processing approach consolidates the complexity into an initial setup phase, allowing the device to operate with simpler real-time processing by referencing the pre-built global map. The global map serves as a reusable resource that reduces ongoing computational requirements.
Data Source
AI summary
A method includes receiving, from a sensor on a robotic device, a captured image representative of an environment of the robotic device when the robotic device is at a location in the environment. The method also includes determining, based at least on the location of the robotic device, a rendered image representative of the environment of the robotic device. The method further includes determining, by applying at least one pre-trained machine learning model to at least the captured image and the rendered image, a property of one or more portions of the captured image.