Method and apparatus for device orientation tracking using a visual gyroscope

Abstract

A method for tracking device orientation on a portable device is disclosed. The method comprises initializing a device orientation to a sensor orientation, wherein the sensor orientation is based on information from an inertial measurement unit (IMU) sensor. It also comprises initiating visual tracking using a camera on the portable device and capturing a frame. Next, it comprises determining a plurality of visual features in the frame and matching the frame to a keyframe, wherein capture of the keyframe precedes capture of the frame. Subsequently, it comprises computing a rotation amount between the frame and the keyframe. Responsive to a determination that a rotational distance between the frame and the keyframe exceeds a predetermined threshold, promoting the frame to a keyframe status and adding it to a first orientation map and adjusting the frame with all prior captured keyframes.

Description

CROSS-REFERENCE TO RELATED APPLICATIONSRelated Applications[0001]The present application is related to U.S. patent application Ser. No. ______, filed ______, entitled “A METHOD AND APPARATUS FOR LONG TERM IMAGE EXPOSURE WITH IMAGE STABILIZATION ON A MOBILE DEVICE,” naming Syed Zahir Bokari, Josh Abbott, and Jim van Welzen as inventors, and having attorney docket number NVID-PDU-13-0254-US1. That application is incorporated herein by reference in its entirety and for all purposes.FIELD OF THE INVENTION[0002]Embodiments according to the present invention generally relate to augmented reality systems and more specifically to device orientation tracking for augmented reality systems.BACKGROUND OF THE INVENTION[0003]While augmented reality technologies have improved considerably in recent years, the improvements typically come at the cost of computationally intensive procedures implemented on expensive hardware. The high cost of implementing effective augmented reality systems is a barri...

AI Technical Summary

Benefits of technology

[0008]Accordingly, a need exists for a system and a method for tracking device orientation on a mobile device that has a smaller compute footprint. For example, embodiments of the present invention track device orientation without constructing an entire map of the environment or even trying to calculate the device's position. Further, embodiments of the present invention advantageously track device orientation without requiring a user to change the position of the camera on the mobile device. As a result, device orientation can be tracked more efficiently and quickly using smaller and more affordable electronic components.

Problems solved by technology

While augmented reality technologies have improved considerably in recent years, the improvements typically come at the cost of computationally intensive procedures implemented on expensive hardware.

The high cost of implementing effective augmented reality systems is a barrier to entry that prevents casual users from having access to such systems on everyday devices that have relatively low processing capability, e.g., tablet computers, phones and other hand-held devices.

Procedures implemented in conventional augmented reality applications for tracking device orientation with expected levels of robustness, speed and precision are computationally expensive.

Accordingly, they are not ideal for handheld mobile devices.

Building a 3D map of the environment can be significantly intensive from a computation standpoint, in part, because it involves tracking the device's position as well as orientation.

Because of the considerable computational requirements for implementing SLAM, it is not a suitable procedure for implementation on a mobile device, such as a smart phone.

Other conventional augmented reality systems are unsuitable for determining device orientation on mobile devices because they suffer from problems such as drift error.

A major disadvantage of using IMUs is that they typically suffer from accumulated error.

Because the guidance system is continually adding detected changes to its previously-calculated positions, any errors in measurement, however small, are accumulated from point to point.

This leads to drift error, or an ever-increasing difference between where the system thinks it is located and the actual location, where, as stated above, location includes both the device's position and also its orientation.

Stated differently, drift is a problem because integration of the orientation tracking's relative measurements accumulates the small errors in each measurement.

This can, consequently, create significant differences between the estimated and actual orientation.

Further, measurements from IMU sensors tend to be error prone and noisy.

Thus, IMU sensors are typically not suitable for an immersive augmented reality environment.

Conventional methods of tracking device orientation for augmented reality systems, therefore, are unsuitable for use on mobile devices because they are either too expensive and computationally intensive or simply unsuitable as a result of problems such as noise and drift error.

Images