Voxel Occupancy Mapping for Irregular 3D Object Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing imaging systems struggle with accurately classifying out-of-vocabulary objects and irregularly-shaped objects using 3D bounding box representations, leading to potential hazards in autonomous navigation, and require time-consuming manual annotation for semantic labeling.
Innovation Solution
Utilizing voxel-based object detection and mapping with machine learning models trained through multi-modal supervision to generate 3D occupancy prediction maps, which label voxels as occupied, unoccupied, or unobserved, and provide semantic labels, overcoming limitations of 3D bounding boxes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D bounding box representations are used for object detection, then object classification can be performed, but accuracy deteriorates for out-of-vocabulary objects and irregularly-shaped objects
Solution Approach 1:
The patent segments the 3D environment into a voxel grid, where each voxel can be independently classified. This segmentation allows the system to handle irregular shapes and out-of-vocabulary objects by assigning semantic labels to individual voxels rather than relying on predefined 3D bounding box categories. The segmentation enables flexible representation of objects regardless of their shape or whether they match predefined vocabularies.
Solution Approach 2:
The patent transitions from traditional 2D image-based object detection to a 3D volumetric representation using voxels. This dimensional change enables the system to capture spatial relationships and object shapes in three dimensions, improving accuracy for irregularly-shaped objects and providing a more versatile representation that can accommodate out-of-vocabulary objects through learned semantic patterns rather than predefined categories.
2Measurement precision
If manual annotation is used for semantic labeling, then labeling accuracy can be ensured, but time consumption increases
Solution Approach 1:
The patent implements a self-service approach where the system automatically generates semantic labels for voxels using a trained machine learning model. The model processes input images and directly predicts semantic labels for each voxel without requiring manual annotation. This self-service mechanism maintains high labeling accuracy through the model's training on labeled data while completely eliminating the time-consuming manual annotation process.
Solution Approach 2:
The patent creates a digital copy of the environment in the form of a voxel grid that mirrors the 3D space. Instead of manually annotating the actual environment, the system processes images of the environment and generates corresponding semantic labels for the voxel copy. This copying approach allows automated processing and labeling of the entire environment efficiently, maintaining accuracy while reducing time requirements.
3Adaptability or versatility
If voxel-based representation is used instead of 3D bounding boxes, then handling of irregular shapes improves, but computational complexity increases
Solution Approach 1:
The patent segments the environment into a manageable voxel grid, where each voxel is a simple cubic unit. This segmentation simplifies the representation of irregular shapes by composing them from basic voxel units, reducing the computational complexity compared to representing arbitrary 3D shapes with complex 3D bounding boxes. The standardized voxel structure enables efficient processing while maintaining versatility for irregular object representation.
4Measurement precision
If multi-modal supervision is used for training, then model accuracy improves, but training time and data requirements increase
Solution Approach 1:
The patent employs a multi-modal supervision approach where a single machine learning model processes multiple types of input data (images, depth information, semantic labels) to generate comprehensive voxel-based representations. This multi-functional model simultaneously performs object detection, segmentation, and semantic labeling tasks, improving overall accuracy while efficiently utilizing training data and time through unified processing rather than separate specialized models.
Data Source
AI summary
Imaging systems and techniques are described. In some examples, an imaging system extracts a plurality of features from the plurality of images of an environment. The plurality of images include different perspectives on the environment. The imaging system processes the plurality of features to generate a voxel-based representation of the environment. The voxel-based representation includes a plurality of voxels. The imaging system analyzes the plurality of images and the voxel-based representation to classify a first subset of the plurality of voxels into a first object category and to classify a second subset of the plurality of voxels into a second object category.