3D Scene Geometry Estimation Without Camera Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for estimating the real-world size of objects in images rely on calibration objects or pre-calibrated cameras, which may not always be available, leading to inaccurate scale information and difficulty in measuring object sizes in various scenes.
Innovation Solution
A method using uncalibrated cameras and machine learning models to generate 3D reconstructions of scenes, followed by determining relative object sizes and applying size probability distributions to convert these to absolute real-world measurements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If calibration objects are used to estimate real-world size, then measurement accuracy is improved, but device complexity and ease of operation deteriorate due to requiring additional equipment
Solution Approach 1:
The patent extracts the calibration function from physical objects and embeds it within the machine learning model itself. The model learns scale relationships directly from training data containing images with known object sizes, eliminating the need for separate calibration objects during operation.
Solution Approach 2:
The system performs self-calibration through the machine learning model that automatically learns and applies scale factors from training data. The model serves its own calibration needs by incorporating size information of known objects directly into the training process, without requiring external calibration equipment.
2Measurement precision
If pre-calibrated cameras are used, then measurement accuracy is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts the calibration data from the camera hardware itself and transfers it to the software domain. By training the machine learning model with scale information, the calibration requirements are moved from physical camera calibration to digital model training, simplifying hardware requirements.
Solution Approach 2:
The patent replaces the mechanical/optical calibration system with a computational approach. Instead of calibrating camera hardware through physical procedures, the system uses machine learning to learn scale relationships from training images, substituting computational processing for mechanical calibration.
3Measurement precision
If calibration objects are inserted into the scene, then measurement accuracy is improved, but the scene integrity deteriorates due to pollution
Solution Approach 1:
The patent extracts the calibration information requirement from the physical scene and places it in the training data domain. The model learns from training images that contain size information, then applies this knowledge without requiring calibration objects to be present in the actual measurement scene.
Solution Approach 2:
The patent performs calibration action in advance during the training phase. The machine learning model is trained beforehand with images containing known object sizes, so that when the model is deployed, calibration has already been completed and no additional calibration objects are needed in the measurement scene.
4Ease of operation
If object detection algorithms are used with known object sizes, then measurement process is simplified, but versatility deteriorates due to limited object recognition
Solution Approach 1:
The patent creates a universal measurement system that works across multiple object types and scenarios. The machine learning model is trained on diverse training data containing various objects with known sizes, enabling it to generalize to different object types in the measurement scene without requiring specific detection algorithms for each object category.
Solution Approach 2:
The patent changes the approach from object-specific detection to scale-factor-based measurement. Instead of relying on recognizing specific object types, the model learns general scale relationships and applies them to measure various objects, transitioning from categorical recognition to geometric parameter estimation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of obtaining real world scale information for a scene comprises obtaining at least one image of a plurality of objects in a scene; detecting at least some of the objects in the at least one image as corresponding to pre-determined objects; generating a 3D reconstruction of the scene based on the image content of the at least one image; determining a relative size of each object in the 3D reconstruction of the scene in at least one dimension, the relative size being defined in dimensions of the generated 3D reconstruction; wherein the relative size of each object is determined based on a distance between at least two points corresponding to that object as transformed into 3D space; obtaining a size probability distribution function for each object detected in the at least one image, each size probability distribution function defining a range of sizes in at least one dimension that a corresponding object is likely to possess in real world units; rescaling the size probability distribution function for each detected object based on a corresponding relative size of that object in the 3D reconstruction; and estimating a geometry of the scene in real world units by combining the re-scaled probability distribution function for at least one detected object with the re-scaled probability distribution function for at least one other detected object.