Narrow-FOV Sensor Training With Rendered Map Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic devices face challenges in differentiating between objects and surfaces, particularly when they blend into the background, leading to navigation errors and potential collisions, such as with transparent glass panels.
Innovation Solution
The integration of a sensor system that captures images and applies pre-trained machine learning models to both captured and rendered images, providing broader context for property determination, such as drivability, by combining location data and semantic labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a robotic device uses a narrow field of view sensor to capture images, then the sensor can achieve high resolution and detailed object recognition, but the robotic device lacks broader context understanding of the environment leading to navigation errors and collisions
Solution Approach 1:
The patent combines multiple image sources (captured image from narrow field of view sensor and rendered image from global map) into a unified analysis framework. The machine learning model processes both images simultaneously to extract features from the detailed captured image while incorporating contextual information from the broader rendered image, thereby resolving the contradiction between high resolution and broad context.
Solution Approach 2:
The rendered image generated from the global map serves as an intermediary that provides broader environmental context. This intermediary image bridges the gap between the limited narrow field of view sensor data and the robot's navigation needs, allowing the system to make informed decisions without requiring the sensor itself to have a wide field of view.
2Device complexity
If the robotic device relies solely on captured images from the sensor, then the system remains simple and computationally efficient, but the robotic device cannot accurately differentiate objects that blend into the background
Solution Approach 1:
The system performs preliminary actions by generating a rendered image from the global map before analyzing the captured image. This pre-computed contextual information is then integrated with the sensor data, allowing the machine learning model to better distinguish objects that blend into the background by comparing them against the expected environmental context from the rendered image.
3Reliability
If the robotic device integrates both captured images and rendered images with machine learning models, then the robotic device achieves better environmental understanding and navigation accuracy, but the computational requirements and processing time increase
Solution Approach 1:
The rendered image from the global map is pre-computed and prepared before the robot needs to make navigation decisions. This preliminary preparation of contextual information reduces the real-time processing burden during actual navigation, as the machine learning model receives pre-processed data that requires less computational effort to analyze.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method includes receiving, from a sensor on a robotic device, a captured image representative of an environment of the robotic device when the robotic device is at a location in the environment. The method also includes determining, based at least on the location of the robotic device, a rendered image representative of the environment of the robotic device. The method further includes determining, by applying at least one pre-trained machine learning model to at least the captured image and the rendered image, a property of one or more portions of the captured image.