Robot Drivability Estimation Using Pre-Captured Vision Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robots face delays in decision-making, particularly in navigation, due to limitations in processing raw sensor data from unfamiliar terrains, which hinders real-time analysis of drivable surfaces.
Innovation Solution
Generating training instances that combine vision data from multiple sources, such as cameras and LiDAR devices, with corresponding drivability labels, to train machine learning models that can make real-time determinations of drivability in unfamiliar areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the robot uses heuristic approach with near-field vision data to identify drivable surfaces, then the robot can operate in familiar terrains efficiently, but the robot cannot reliably estimate drivability of distant surfaces in unfamiliar terrains
Solution Approach 1:
The system performs preliminary action by capturing and storing vision data of distant surfaces before the robot actually needs to traverse them. The robot captures vision data at multiple time points during approach, and this pre-captured data is used to train machine learning models that can predict drivability in advance, eliminating the need to wait until the robot is close to the surface.
Solution Approach 2:
The system creates a copy of the drivability assessment capability by training machine learning models on historical vision data and labels. Instead of directly analyzing distant surfaces in real-time, the system uses trained models that have learned from copied historical data, allowing accurate drivability estimation without direct real-time measurement of distant areas.
2Speed
If the robot processes raw sensor data in real-time, then the robot can make immediate navigation decisions, but the robot experiences delays in decision-making for unfamiliar terrains
Solution Approach 1:
The system performs preliminary training of machine learning models using historical vision data and drivability labels before actual navigation occurs. This preliminary action allows the robot to make accurate real-time decisions during navigation, as the complex analysis has already been performed and encoded in the trained models.
Solution Approach 2:
The system replaces direct real-time mechanical processing of raw sensor data with a substituted system of trained machine learning models. Instead of processing raw data in real-time, the robot queries pre-trained models that have already analyzed similar situations, substituting complex real-time computation with faster model inference.
3Measurement precision
If the robot approaches surfaces to estimate drivability, then the robot can accurately assess nearby surfaces, but the robot cannot accurately assess distant surfaces without further approaching
Solution Approach 1:
The system performs preliminary capture and storage of vision data at multiple time points during the robot's approach to distant surfaces. This preliminary action allows the robot to assess drivability of distant surfaces without completing the full approach, as the captured data during approach is sufficient for training accurate predictions.
Solution Approach 2:
The system uses feedback from the robot's actual traversal of surfaces to generate accurate drivability labels. The robot's successful or failed attempts to traverse surfaces provide feedback that is used to train machine learning models, allowing the robot to learn from its own experiences and improve future drivability assessments without repeated physical trial-and-error.
Data Source
AI summary
Implementations set forth herein relate to generating training data, such that each instance of training data includes a corresponding instance of vision data and drivability label(s) for the instance of vision data. A drivability label can be determined using first vision data from a first vision component that is connected to the robot. The drivability label(s) can be generated by processing the first vision data using geometric and/or heuristic methods. Second vision data can be generated using a second vision component of the robot, such as a camera that is connected to the robot. The drivability labels can be correlated to the second vision data and thereafter used to train one or more machine learning models. The trained models can be shared with a robot(s) in furtherance of enabling the robot(s) to determine drivability of areas captured in vision data, which is being collected in real-time using one or more vision components.