Augmented reality surveillance and rescue system

Abstract

A system, method, and computer program product for automatically combining computer-generated imagery with real-world imagery in a portable electronic device by retrieving, manipulating, and sharing relevant stored videos, preferably in real time. A video is captured with a hand-held device and stored. Metadata including the camera's physical location and orientation is appended to a data stream, along with user input. The server analyzes the data stream and further annotates the metadata, producing a searchable library of videos and metadata. Later, when a camera user generates a new data stream, the linked server analyzes it, identifies relevant material from the library, retrieves the material and tagged information, adjusts it for proper orientation, then renders and superimposes it onto the current camera view so the user views an augmented reality.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims the benefit under 35 U.S.C. 119 of provisional application 61 / 411,053 filed on Nov. 8, 2010 entitled “An Augmented Reality Interface for Video Tagging and Sharing” which is hereby incorporated by reference in its entirety, and is related to seven other simultaneously-filed applications, including Attorney Docket No. S1162.1102US1 entitled “Augmented Reality Interface for Video”, Attorney Docket No. S1162.1102US2 entitled “Augmented Reality Interface for Video Tagging and Sharing”, Attorney Docket No. S1162.1102US3 entitled “Augmented Reality System for Communicating Tagged Video and Data on a Network”, Attorney Docket No. 51162.1102US4 entitled “Augmented Reality System for Position Identification”, Attorney Docket No. S1162.1102US5 entitled “Augmented Reality System for Supplementing and Blending Data”, Attorney Docket No. S1162.1102US6 entitled “Augmented Reality Virtual Guide System”, Attorney Docket No. S1162.1...

AI Technical Summary

Benefits of technology

[0036]In a fourth embodiment, the retrieved video may be compared with the currently observed real-world scene to enable navigation. The embodiment may visually verify a real-world path or a real-world destination for a portable electronic device user.

[0037]In a fifth embodiment, the man

Problems solved by technology

Traditionally augmented reality applications have been limited to expensive custom setups used in universities and academia, but with the advent of modem smartphones and powerful embedded processors, many of the algorithms that were once confined to the personal computer world are becoming a part of the mobile world.

One of the major drawbacks of using GPS based systems is that they require direct line-of-sight views to the satellites and are commonly blocked in urban areas, canyons, etc.

This limits their usability severely.

Their main drawback however is that they accumulate drift over a period of time.

The obvious disadvantage of these systems is that modifying the environment in this manner outdoors is usually not practical and restricts the user to the location of the active sources.

However most augmented reality applications refrain from using these algorithms since they are computationally intensive.

Even though these techniques have been deployed and used extensively in the mobile space, there are still several technical challenges that need to be addressed for a robust, usable augmented reality system.

Existing Mobile 3D solutions are cumbersome and impose limitations on seamless integration with live camera imagery.

User experience therefore degrades quickly when accuracy is lost.

However the use of OpenGL on smartphone platforms introduces other challenges.

One of the biggest disadvantages of using OpenGL is that once perspective-rendered content is displayed onscreen, it is hard

Images