Neural Network for 3D Point Cloud Segmentation and Feature Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models struggle to process unstructured and unordered 3D point clouds, particularly those representing real-world scenes with multiple objects in various poses, as they are typically trained for canonical poses and fail to generate accurate numerical features for such complex scenarios.
Innovation Solution
A computer-implemented method is developed to train a machine learning model by executing a neural network on a 3D scene to generate key and value sets, computing spatial and quantized context features, and modifying the network based on the likelihood of reconstructing the point cloud, enabling the model to map 3D scenes to structured representations of 3D geometry instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a permutation-invariant neural network is trained to map scene point clouds to numerical features, then the model consistently maps different orderings of the same set of points to the same features, but the model is not capable of processing point clouds representing 3D scenes that include single objects in non-canonical poses or multiple objects
Solution Approach 1:
The patent segments the 3D scene point cloud into multiple object point clouds using a segmentation model. Each object point cloud is then processed separately by the permutation-invariant neural network, which was originally designed for single objects in canonical poses. This segmentation approach allows the model to handle complex scenes with multiple objects and various poses by treating them as separate instances, thereby resolving the contradiction between maintaining consistent mapping and expanding adaptability to diverse scenes.
2Productivity
If traditional ML models are trained to recognize patterns in structured and ordered data, then they can make decisions based on those patterns, but they are unable to process unstructured and unordered input data and generate inconsistent results when processing different orderings of the same input data
Solution Approach 1:
The patent introduces a permutation-invariant neural network as an intermediary component that sits between the unstructured point cloud input and the downstream task-specific ML models. This intermediary processes the unordered 3D point cloud data and transforms it into structured numerical features that are invariant to input ordering. This resolves the contradiction by enabling traditional pattern-recognition models to work effectively with unstructured 3D data while maintaining consistent results regardless of input ordering.
Data Source
AI summary
In various embodiments, an unsupervised training application executes a neural network on a first point cloud to generate keys and values. The unsupervised training application generates output vectors based on a first query set, the keys, and the values and then computes spatial features based on the output vectors. The unsupervised training application computes quantized context features based on the output vectors and a first set of codes representing a first set of 3D geometry blocks. The unsupervised training application modifies the first neural network based on a likelihood of reconstructing the first point cloud, the quantized context features, and the spatial features to generate an updated neural network. A trained machine learning model includes the updated neural network, a second query set, and a second set of codes representing a second set of 3D geometry blocks and maps a point cloud to a representation of 3D geometry instances.