3D Object Measurement via Semantic Plane Intersection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in providing accurate and efficient automatic measurement of objects in physical environments using sensor data, particularly with incomplete or insufficient data, and fail to deliver real-time measurements using mobile devices.
Innovation Solution
The implementation generates three-dimensional (3D) representations of physical environments based on image and depth sensor data, employing semantic segmentation and labeling, and utilizes multiple neural networks to refine bounding boxes and determine class-specific measurements for various objects, such as furniture and appliances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automatic measurement techniques are implemented using sensor data, then measurement efficiency is improved, but measurement precision deteriorates due to incomplete or insufficient sensor data
Solution Approach 1:
The patent combines multiple sensor types (depth sensors, light intensity sensors, semantic segmentation data) and multiple measurement techniques (bounding box methods, plane intersection methods, machine learning models) into an integrated system. This fusion of diverse data sources and methods compensates for the limitations of individual sensors, maintaining high measurement precision while achieving automatic real-time measurement efficiency.
2Speed
If simple bounding box measurements are used, then measurement speed is improved, but measurement precision deteriorates due to lack of semantic information
Solution Approach 1:
The system performs semantic segmentation and object classification before final measurement calculation. By pre-processing the sensor data to identify object types and extract semantic features, the system prepares refined measurement parameters in advance, enabling both fast real-time measurement and high precision through class-specific measurement models.
3Measurement precision
If class-specific machine learning models are used for different object types, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent implements a unified measurement system that handles multiple object classes (furniture, appliances, electronics) through a common architecture. The system uses a single integrated pipeline that incorporates semantic segmentation, object detection, and class-specific measurement models, allowing one system to perform diverse measurement functions without requiring separate dedicated systems for each object type.
4Productivity
If real-time measurements are implemented using mobile devices, then productivity is improved, but measurement precision deteriorates due to mobile device constraints
Solution Approach 1:
The patent segments the measurement process into distinct modular components: sensor data acquisition, 3D point cloud generation, semantic segmentation, object detection, and measurement calculation. This segmentation allows the system to process data in manageable stages on mobile devices, maintaining real-time performance while improving precision through systematic refinement at each processing stage.
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that provide measurements of objects based on a location of a surface of the objects. An exemplary process may include obtaining a three-dimensional (3D) representation of a physical environment that was generated based on depth data and light intensity image data, generating a 3D bounding box corresponding to an object in the physical environment based on the 3D representation, determining a class of the object based on the 3D semantic data, determining a location of a surface of the object based on the class of the object, the location determined by identifying a plane within the 3D bounding box having semantics in the 3D semantic data satisfying surface criteria for the object, and providing a measurement of the object, the measurement of the object determined based on the location of the surface of the object.