Fully convolutional interest point detection and description via homographic adaptation

Abstract

Systems, devices, and methods for training a neural network and performing image interest point detection and description using the neural network. The neural network may include an interest point detector subnetwork and a descriptor subnetwork. An optical device may include at least one camera for capturing a first image and a second image. A first set of interest points and a first descriptor may be calculated using the neural network based on the first image, and a second set of interest points and a second descriptor may be calculated using the neural network based on the second image. A homography between the first image and the second image may be determined based on the first and second sets of interest points and the first and second descriptors. The optical device may adjust virtual image light being projected onto an eyepiece based on the homography.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS[0001]This application claims priority to U.S. Provisional Patent Application No. 62 / 586,149 filed Nov. 14, 2017 titled “FULLY CONVOLUTIONAL INTEREST POINT DETECTION AND DESCRIPTION VIA HOMOGRAPHIC ADAPTATION,” and to U.S. Provisional Patent Application No. 62 / 608,248 filed Dec. 20, 2017 titled “FULLY CONVOLUTIONAL INTEREST POINT DETECTION AND DESCRIPTION VIA HOMOGRAPHIC ADAPTATION,” the entire disclosures of which are hereby incorporated by reference, for all purposes, as if fully set forth herein.BACKGROUND OF THE INVENTION[0002]Interest point detection is an important concept in computer vision. The first step in many geometric computer vision tasks such as pose estimation, simultaneous localization and mapping (SLAM), structure-from-motion, sparse three-dimensional (3D) mapping, camera calibration, and image matching is to extract interest points from images. Interest points are two-dimensional (2D) locations in an image which are stable a...

AI Technical Summary

Benefits of technology

The present invention provides a self-supervised framework for training interest point detectors and descriptors that operate on any set of single or multi-channel two-dimensional (2D) images. It can detect richer interest points than traditional corner detectors or pre-adapted deep models. It does not require explicit geometric correspondence information. These benefits make it suitable for a large number of multiple-view geometry problems and outperform other traditional descriptors such as ORB and SIFT on point matching accuracy and on the task of homography estimation. Overall, the present invention provides a more accurate and efficient solution for detecting and describing interest points in image data.

Problems solved by technology

However, the input to most real-world computer vision systems is not idealized point locations but is instead raw, unlabeled images.

Several approaches have been developed to detect the interest points in such images, with only limited success.

Images