System and process for viewing and navigating through an interactive video tour

Abstract

A system and process for providing an interactive video tour of a tour site to a user is presented. In general, the system and process provides an image-based rendering system that enables users to explore remote real world locations, such as a house or a garden. The present approach is based directly on filming an environment, and then using image-based rendering techniques to replay the tour in an interactive manner. As such, the resulting experience is referred to as Interactive Video Tours. The experience is interactive in that the user can move freely along a path, choose between different directions of motion at branch points in the path, and look around in any direction. The user experience is additionally enhanced with multimedia elements such as overview maps, video textures, and sound.

Description

BACKGROUND [0001] 1. Technical Field [0002] The invention is related to video tours, and more particularly to a system and process for providing an interactive video tour of a tour site using an image-based rendering technique that enables users to explore remote real world locations. [0003] 2. Background Art [0004] Interactive scene walkthroughs have long been an important application area of computer graphics, as evidenced by the efficient rendering algorithms that have been developed for visualizing large architectural databases [9,1]. In fact, many beautiful examples of 3D reconstruction from multiple images, video, and / or range data have been demonstrated by research groups around the world. The most ambitious of these is Seth Teller's City Scanning Project at MIT [2], in which dozens of buildings have been reconstructed from high-resolution, high dynamic range omnidirectional panoramas. While these systems allow the viewer to have unrestricted motion throughout the digitized s...

AI Technical Summary

Benefits of technology

The present invention is a system and process for creating interactive video tours of tour sites. The system uses high resolution continuous imagery and interactivity to provide viewers with an unprecedented sense of presence. The user interface allows users to pan the viewing position and choose the direction of motion to explore the tour site. The video data is input through scene data and user commands, and the resulting image of the tour site is displayed to the user. The user interface provides a seamless experience of continuous control over the viewing position and direction of motion. The video data may be compressed for storage and transmission, and decompressed before being processed. The technical effects of the invention include an enhanced sense of presence, interactivity, and unprecedented access to remote real-world locations.

Problems solved by technology

While these systems allow the viewer to have unrestricted motion throughout the digitized scene, they fail to capture the visual effects such as reflections in windows, specularities, and finely detailed geometry such as foliage, that make the scene look real.

However, to date none have been able to provide a viewer with continuous control over both position and viewpoint, as would be desirable to achieve the goal of creating a sense of being there.

In addition, these systems cannot achieve a fully interactive, photorealistic, video based tour on a personal computer at or above broadcast video resolutions and frame rates.

Images