Object Classification Using RGB-D Bounding Polygons and ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for object localization and characterization in multi-dimensional image data, such as RGB-D data, are imprecise, leading to challenges in accurately detecting and identifying objects within these complex datasets.
Innovation Solution
The use of region and measurements-based convolutional neural networks, combined with binary and multi-class classifiers, to process RGB-D data by bifurcating color and depth information, generating bounding polygons, and applying machine learning models for enhanced object localization and characterization, including the selection of appropriate machine learning models based on the type of object within the bounding polygon.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional object detection methods are used on multi-dimensional image data, then the processing is computationally simpler, but the localization and identification precision remains imprecise
Solution Approach 1:
The patent segments the object detection process into multiple specialized components: a region proposal network that generates candidate bounding boxes, a binary classifier that filters valid objects, and multiple object classifiers that identify specific object types. This segmentation allows each component to specialize in one aspect of detection, improving overall precision while managing computational complexity through distributed processing
Solution Approach 2:
The patent processes multi-dimensional image data (RGB-D data with color and depth information) by utilizing the additional dimensional information beyond traditional 2D images. The region proposal network and classifiers leverage depth information to generate and evaluate three-dimensional bounding boxes, transforming the detection problem from two-dimensional to three-dimensional space to improve localization precision
2Measurement precision
If multiple machine learning models are applied for object classification, then the characterization accuracy is enhanced, but the processing time increases
Solution Approach 1:
The binary classifier performs preliminary filtering by determining whether a bounding box contains any object before applying the more computationally intensive object classifiers. This preliminary action eliminates false positives early in the pipeline, reducing the number of objects that require detailed classification and thereby reducing overall processing time while maintaining high characterization accuracy for valid objects
Solution Approach 2:
Different object classifiers are applied based on the specific type of object detected. The system selects appropriate classifiers tailored to different object categories (e.g., person, vehicle, animal), applying specialized classification methods only where needed rather than using a single universal classifier for all objects, thus optimizing processing time for each specific case
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Combined color and depth data for a field of view is received. Thereafter, using at least one bounding polygon algorithm, at least one proposed bounding polygon is defined for the field of view. It can then be determined, using a binary classifier having at least one machine learning model trained using a plurality of images of known objects, whether each proposed bounding polygon encapsulates an object. The image data within each bounding polygon that is determined to encapsulate an object can then be provided to a first object classifier having at least one machine learning model trained using a plurality of images of known objects, to classify the object encapsulated within the respective bounding polygon. Further, the image data within each bounding polygon that is determined to encapsulate an object is provided to a second object classifier having at least one machine learning model trained using a plurality of images of known objects, to classify the object encapsulated within the respective bounding polygon. A final classification for each bounding polygon is then determined based on the output of the first classifier machine learning model and the output of the second classifier machine learning model.