Combined stereo camera and stereo display interaction

Abstract

One embodiment of the present invention provides a system that facilitates interaction between a stereo image-capturing device and a three-dimensional (3D) display. The system comprises a stereo image-capturing device, a plurality of trackers, an event generator, an event processor, and a 3D display. During operation, the stereo image-capturing device captures images of a user. The plurality of trackers track movements of the user based on the captured images. Next, the event generator generates an event stream associated with the user movements, before the event processor in a virtual-world client maps the event stream to state changes in the virtual world. The 3D display then displays an augmented reality with the virtual world.

Description

BACKGROUND[0001]1. Field[0002]The present disclosure relates to a system and technique for facilitating interaction with objects via a machine vision interface in a virtual world displayed on a large stereo display in conjunction with a virtual world server system, which can stream changes to the virtual world's internal model to a variety of devices, including augmented reality devices.[0003]2. Related Art[0004]During conventional assisted servicing of a complicated device, an expert technician is physically collocated with a novice to explain and demonstrate by physically manipulating the device. However, this approach to training or assisting the novice can be expensive and time-consuming because the expert technician often has to travel to a remote location where the novice and the device are located.[0005]In principle, remote interaction between the expert technician and the novice is a potential solution to this problem. However, the information that can be exchanged using exi...

AI Technical Summary

Benefits of technology

This patent describes a system that combines a device for capturing stereo images with a display that shows a virtual world. The system uses trackers to track the movement of the user and any objects around them, and an event processor to map the movement onto changes in the virtual world. This creates a more immersive experience for the user, as they can interact with the virtual world in real-time.

Problems solved by technology

However, this approach to training or assisting the novice can be expensive and time-consuming because the expert technician often has to travel to a remote location where the novice and the device are located.

However, the information that can be exchanged using existing communication techniques is often inadequate for such remotely assisted servicing.

For example, during a conference call audio, video, and text or graphical content are typically exchanged by the participants, but three-dimensional spatial relationship information, such as the spatial interrelationship between components in the device (e.g., how the components are assembled) is often unavailable.

This is a problem because the expert technician does not have the ability to point and physically manipulate the device during a remote servicing session.

Furthermore, the actions of the novice are not readily apparent to the expert technician unless the novice is able to effectively communicate his actions.

Typically, relying on the novice to verbally explain his actions to the expert technician and vice versa is not effective because there is a significant knowledge gap between the novice and the expert technician.

Consequently, it is often difficult for the expert technician and the novice to communicate regarding how to remotely perform servicing tasks.

Images