Single-Frame NOCS Pose Estimation for Real-Time AR

The patent segments the pose estimation task into two independent components: instance segmentation (identifying object boundaries) and NOCS map prediction (estimating 3D pose and size). By processing these separately and fusing results, the system achieves accurate pose estimation without the computational burden of a two-stage network, thus improving execution speed while maintaining precision.

The patent performs instance segmentation first to obtain object masks, then uses these masks to guide NOCS map prediction. This preliminary action of segmenting objects before pose estimation allows the system to focus computational resources on relevant regions, improving both speed and accuracy by avoiding unnecessary processing of background areas.

The patent divides the complex two-stage network into separate instance segmentation and NOCS prediction modules. Each module operates independently with its own optimized architecture, reducing the overall system complexity while maintaining the accuracy benefits of multi-stage processing through modular design.

The patent introduces instance segmentation masks as an intermediary between the input image and NOCS map prediction. This intermediary step simplifies the overall network structure by providing explicit object boundaries that guide subsequent pose estimation, avoiding the need for complex hierarchical architectures.

The patent uses instance segmentation masks as an intermediary that works effectively with both RGB and depth data. When depth data is available, the masks guide accurate pose estimation; when depth data is omitted or noisy, the masks provide robust boundaries that enable reliable pose prediction from RGB data alone, thus improving system reliability across different input conditions.

The patent adapts its processing parameters based on input data quality. When depth data is present, it utilizes depth information for accurate pose estimation; when depth data is noisy or missing, it switches to relying primarily on RGB data guided by instance segmentation masks, changing the weight and importance of different input parameters to maintain reliability.

The patent performs instance segmentation as a preliminary action on the single-frame RGB image to obtain accurate object boundaries and masks. These masks provide geometric information about object shape and size, enabling the system to estimate object dimensions without requiring multi-frame input or depth data, thus maintaining both speed and size estimation capability.

The patent transitions from 2D image analysis to 3D pose and size estimation by introducing NOCS map prediction. The instance segmentation masks provide 2D geometric constraints that are transformed into 3D size estimates through the NOCS framework, enabling accurate object dimension estimation from single-frame RGB images without additional temporal or depth information.