3D Modeling from 2D Images Using Feature Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 2D-to-3D conversion techniques for agricultural applications are computationally expensive and produce large 3D models that are unsuitable for remote transmission and rendering on resource-constrained devices like VR/AR headsets, and users often require only specific features of interest from 3D models.
Innovation Solution
Implementing a method that filters out uninterested features from 2D images using machine learning models like CNNs to generate 3D models focusing on specific features of interest, reducing computational load and data size, allowing for efficient processing and transmission of filtered 3D data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If 2D-to-3D processing is performed on all features using SFM, then comprehensive 3D models are produced, but computational cost and processing time increase significantly
Solution Approach 1:
The patent segments the 3D modeling process by dividing features into multiple classes (e.g., fruit, leaves, stems, soil) and selectively applying 2D-to-3D processing only to specified feature classes of interest. This segmentation allows the system to process only relevant portions of the image data, reducing computational load while maintaining completeness for targeted features.
Solution Approach 2:
The patent extracts and isolates specific feature classes from the complete image data using filtering mechanisms. By taking out only the specified features (e.g., fruit only, or fruit and leaves) and excluding other features (e.g., soil, stems), the system reduces the volume of data requiring 2D-to-3D processing, thereby improving processing speed without losing information about the features of interest.
2Loss of information
If comprehensive 3D models capturing all features are generated, then complete object representation is achieved, but data size becomes too large for remote transmission and resource-constrained devices
Solution Approach 1:
The patent extracts only the specified feature classes from the complete object representation. By filtering out features not of interest (e.g., removing soil and stems when only fruit is needed), the system produces 3D model data that is compact enough for remote transmission and rendering on resource-constrained devices like VR/AR headsets, while preserving all information about the specified features.
Solution Approach 2:
The patent applies local quality by providing different levels of detail and representation for different feature classes. Specified features receive full 3D modeling treatment with high detail, while non-specified features are either excluded or represented minimally. This approach optimizes data size by allocating computational resources only where needed, enabling efficient transmission and rendering.
3Loss of information
If 2D-to-3D processing is performed on all image data, then complete feature coverage is achieved, but computational resources and processing time are excessive
Solution Approach 1:
The patent segments the image data processing by classifying pixels into multiple feature classes and directing 2D-to-3D processing only to segments corresponding to specified features. This segmentation strategy ensures that processing time is not wasted on irrelevant features while maintaining complete coverage of the features of interest.
Solution Approach 2:
The patent applies partial action by performing 2D-to-3D processing only on a subset of features rather than all features in the image. By applying processing selectively to specified feature classes (e.g., only fruit or only leaves), the system achieves sufficient feature coverage for the user's needs while dramatically reducing processing time compared to comprehensive processing.
4Manufacturing precision
If all feature classes are processed into 3D data, then full object detail is captured, but the resulting data is unsuitable for rendering on VR/AR headsets
Solution Approach 1:
The patent extracts only the specified feature classes that are relevant to the user's needs and suitable for VR/AR rendering. By filtering out excessive detail from non-specified features (e.g., removing soil and background elements), the system produces 3D data that maintains high detail accuracy for the features of interest while being compact and compatible with the rendering capabilities of VR/AR headsets.
Data Source
AI summary
Implementations are described herein for three-dimensional (“3D”) modeling of objects that target specific features of interest of the objects, and ignore other features of less interest. In various implementations, a plurality of two-dimensional (“2D”) images may be received from a 2D vision sensor. The plurality of 2D images may capture an object having multiple classes of features. Data corresponding to a first set of the multiple classes of features may be filtered from the plurality of 2D images to generate a plurality of filtered 2D images in which a second set of features of the multiple classes of features is captured. 2D-3D processing, such as structure from motion (“SFM”) processing, may be performed on the 2D filtered images to generate a 3D representation of the object that includes the second set of one or more features.