Monocular Image Height Estimation for Low-Complexity Object Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous systems such as vehicles and drones require accurate information about object sizes and locations for navigation and collision avoidance, but traditional 3D sensing technologies like LIDAR are expensive and computationally intensive, and limited by environmental constraints.
Innovation Solution
A convolutional neural network (CNN) architecture is used to estimate object sizes and locations from monocular images, reducing processing time and computational requirements by leveraging machine learning and a modified Visual Geometry Group (VGG) CNN with data augmentation techniques to improve accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LIDAR or radar sensors are used to provide location information of objects, then measurement precision is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent uses a camera to capture a two-dimensional image that copies the visual appearance of the object, then uses machine learning to extract three-dimensional information from this two-dimensional copy. Instead of directly measuring with complex 3D sensors, the system creates a 2D representation and computationally derives the needed spatial information.
Solution Approach 2:
The patent replaces complex mechanical 3D sensing systems (LIDAR, radar) with a simpler optical system (camera) combined with computational processing. The mechanical/optical complexity of multiple sensors is substituted with a single camera and machine learning algorithm that processes images to extract 3D characteristics.
2Measurement precision
If LIDAR or radar sensors are used to provide location information of objects, then measurement precision is improved, but cost increases significantly
Solution Approach 1:
The patent employs inexpensive camera systems instead of expensive LIDAR or radar sensors. Cameras are mass-produced, low-cost components compared to active sensing systems like LIDAR, making the overall system more affordable while achieving the same functional goal of object detection and measurement.
Solution Approach 2:
The system uses affordable 2D image captures as proxies for expensive 3D measurements. By creating and processing inexpensive visual copies of objects through cameras, the system avoids the need for costly dedicated 3D sensing hardware while still extracting accurate spatial information.
3Measurement precision
If traditional 3D sensing technologies are used, then object size and location information is obtained, but processing time and computational requirements increase
Solution Approach 1:
The patent pre-trains machine learning models with extensive datasets of objects and their three-dimensional characteristics. This preliminary training allows the system to make rapid predictions about object size and location from new images without requiring complex real-time calculations, significantly reducing processing time during actual operation.
Solution Approach 2:
The patent replaces computationally intensive real-time 3D processing with a streamlined pipeline that uses pre-trained neural networks to infer 3D information from 2D images. The heavy computational lifting is done during offline training rather than real-time operation, enabling faster deployment and lower processing demands during actual use.
Data Source
AI summary
Systems and methods for estimating a height of an object from a monocular image are described herein. Objects are detected in the image, each object being indicated by a region of interest. The image is then cropped for each region of interest and the cropped image scaled to a predetermined size. The cropped and scaled image is then input into a convolutional neural network (CNN), the output of which is an estimated height for the object. The height may be represented by a mean of a probability distribution of possible sizes, a standard deviation, as well as a level of confidence. A location of the object may be determined based on the estimated height and region of interest. A ground truth dataset may be generated for training the CNN by simultaneously capturing a LIDAR sequence with a monocular image sequence.