Planar Surface Detection Through Self-Supervised Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting planar surfaces in images face challenges in acquiring ground truth labels, particularly in real environments, limiting their application in computer vision tasks such as scene understanding and object recognition, especially in augmented reality (AR) devices.
Innovation Solution
A method and apparatus utilizing a neural network-based unsupervised learning framework to detect planar surfaces by estimating pixelwise disparity and segment matching probabilities, enabling accurate detection without ground truth labels through a first network for disparity estimation and a second network for image segmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised learning is used for plane detection with ground truth labels, then detection accuracy can be improved, but it becomes difficult to acquire training data and apply to real environments
Solution Approach 1:
Instead of using ground truth labels to train the network (traditional supervised learning), the patent inverts the approach by using the network's own predictions to generate pseudo-ground truth labels for self-supervised training. The system creates synthetic training data from unlabelled real images by estimating depth and generating corresponding plane labels, eliminating the need for manually annotated ground truth data while maintaining detection accuracy in real environments
Solution Approach 2:
The system performs self-service by automatically generating its own training data without external supervision. The plane detection network generates depth estimates and plane labels from unlabelled images, which are then used to retrain and improve the network itself, creating a self-sustaining training loop that works directly with real-world unlabelled data
2Measurement precision
If depth information is used for plane detection, then detection accuracy is improved, but it requires additional information not available in monocular images
Solution Approach 1:
The patent replaces the mechanical/optical system requirement (stereo cameras or depth sensors) with a computational approach using neural networks. Instead of physically capturing depth information through multiple cameras or active sensors, the system uses a trained neural network to estimate depth from single monocular images, substituting physical depth acquisition with computational depth prediction
Solution Approach 2:
The neural network acts as an intermediary that bridges the gap between 2D monocular images and 3D depth information. The network learns to map from image features to depth values, serving as a computational mediator that translates visual information into spatial understanding without requiring direct depth measurement
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is a method and apparatus for detecting a planar surface, the method including acquiring, based on a pixelwise disparity of an input image estimated in a first network, a pixelwise plane parameter of the input image, determining a pixelwise segment matching probability of the input image based on a second network trained to perform a segmentation of an image, acquiring a segment-wise plane parameter based on the pixelwise plane parameter and the pixelwise segment matching probability, and detecting a planar surface in the input image based on the segment-wise plane parameter.