Depth Image Feature Vector Generator Using Seed Pixels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning algorithms are limited by their inability to process multi-dimensional image data directly, leading to inefficiencies in training and inference due to the need for vectorization, which results in large, computationally expensive data sets that are not compatible with all machine learning models, such as fully-connected neural networks.
Innovation Solution
A method and system that convert multi-dimensional depth image data and segmentation masks into unidimensional feature vectors by using seed pixels and sampling lines, reducing data size and computational load, making these data sets compatible with a broader range of machine learning algorithms like fully-connected neural networks, decision trees, and support vector machines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If multi-dimensional depth image data is directly used for machine learning, then the data contains complete spatial information, but the data size is large and computational cost is high
Solution Approach 1:
The patent extracts only the essential spatial information needed for machine learning by using seed pixels to identify foreground objects and sampling lines to capture depth attributes. This extraction process removes redundant background information while retaining critical spatial characteristics, thereby reducing data size without significant information loss.
Solution Approach 2:
The patent segments the depth image into foreground and background regions using a segmentation mask, then further segments the foreground into discrete seed pixels. This hierarchical segmentation allows the system to process only relevant regions with detailed sampling, rather than treating the entire image uniformly, thus reducing overall data quantity while preserving important spatial information.
2Adaptability or versatility
If conventional vectorization methods are used to convert image data, then the data becomes compatible with some machine learning algorithms, but the resulting vectors are large and not compatible with fully-connected neural networks
Solution Approach 1:
The patent changes the parameters of the feature representation by selecting specific depth attributes (such as mean depth, variance, or other statistical measures) from the sampling line pixels rather than using raw pixel values. This parameter transformation creates compact feature vectors with reduced dimensionality that are suitable for fully-connected neural networks while maintaining compatibility with various machine learning algorithms through standardized feature formats.
3Loss of information
If the entire depth image is processed for feature extraction, then all spatial information is captured, but the computational time and processing load increase significantly
Solution Approach 1:
The patent applies partial action by processing only a subset of pixels (sampling line pixels) rather than the entire depth image. The sampling lines are strategically positioned to pass through foreground objects identified by seed pixels, capturing essential spatial information while avoiding unnecessary processing of background regions. This selective processing significantly reduces computational time while maintaining adequate information capture for accurate machine learning inference.
Data Source
AI summary
Certain aspects provide a method, including: receiving a depth image from a depth sensor; receiving a segmentation mask corresponding to the depth image and segmenting the depth image into a set of foreground pixels and a set of background pixels; determining a set of seed pixels in the depth image; for each respective seed pixel of the set of seed pixels: determining a sampling line in the depth image that starts at the respective seed pixel and passes through a portion of the depth image; for each respective sampling line pixel in the sampling line having a value in the segmentation mask indicating a foreground object in the depth image: determining one or more data attribute values based on a depth value for the respective sampling line pixel in the depth image; and adding the one or more data attribute values to a feature vector.