Neural Network Keypoint Detection and Point Cloud Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for creating and using representations of physical environments, such as 3D models from images, lack accuracy and efficiency in detecting features and removing redundancy from multiple sources.
Innovation Solution
The use of neural networks with precomputed-weight layers for orientation and curvature assessments, allowing for efficient and accurate detection and descriptor generation of key points in images, and a fusing process to combine keypoints from multiple sources by identifying nearest neighbors and removing redundancy based on distance thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing CNN techniques are used to detect keypoints and create descriptors, then feature detection can be performed, but accuracy and efficiency are insufficient due to large network size and inability to accurately learn pixel orientations and curvature
Solution Approach 1:
The patent segments the neural network into distinct functional layers: a first CNN layer for detecting candidate keypoints, a second layer for computing orientations using precomputed weights, and a third layer for computing curvature also with precomputed weights. This segmentation allows each layer to specialize in specific tasks, improving overall accuracy while managing network complexity through modular design.
Solution Approach 2:
The patent applies preliminary action by precomputing the weights for the orientation and curvature assessment layers before runtime. These precomputed weights are stored and reused, eliminating the need for the network to learn these patterns from scratch during inference. This approach significantly improves efficiency and accuracy for orientation and curvature calculations while reducing the computational burden during actual keypoint detection.
2Reliability
If multiple image sources are used to create 3D point clouds, then more comprehensive environmental representation is achieved, but redundancy removal is inaccurate and inefficient
Solution Approach 1:
The patent replaces traditional mechanical or algorithmic redundancy removal methods with a learned neural network approach. The network automatically identifies and removes redundant keypoints by learning patterns of redundancy from multiple image sources, achieving both higher accuracy in maintaining unique features and improved efficiency in processing large numbers of points from multiple views.
Data Source
AI summary
Some implementations involve a process that identifies a subset of points in an image and creates descriptors for these points. The detection and descriptor process may use one or more neural networks. In some implementations, the process includes a neural network that uses one or more fixed (e.g., weight independent) neural network layers to perform certain functions that can be performed more accurately and/or efficiently than via non-fixed (e.g., weight-based) layers. In some implementations, for example, a neural network includes a layer that determines orientation formulaically within the neural network. Such orientations may be determined convolutionally (e.g., using sliding patches) but are not determined based on internal node weights within the layer that were determined during the training of the neural network.