Quasi-three-dimensional method and apparatus to detect and localize interaction of user-object and virtual transfer device

Abstract

A system used with a virtual device inputs or transfers information to a companion device, and includes two optical systems OS1, OS2. In a structured-light embodiment, OS1 emits a fan beam plane of optical energy parallel to and above the virtual device. When a user-object penetrates the beam plane of interest, OS2 registers the event. Triangulation methods can locate the virtual contact, and transfer user-intended information to the companion system. In a non-structured active light embodiment, OS1 is preferably a digital camera whose field of view defines the plane of interest, which is illuminated by an active source of optical energy. Preferably the active source, OS1, and OS2 operate synchronously to reduce effects of ambient light. A non-structured passive light embodiment is similar except the source of optical energy is ambient light. A subtraction technique preferably enhances the signal / noise ratio. The companion device may in fact house the present invention.

Description

RELATION TO PREVIOUSLY FILED APPLICATION [0001] Priority is claimed from applicants' co-pending U.S. provisional patent application Ser. No. 60 / 287,115 filed on 27 Apr. 2001 entitled “Input Methods Using Planar Range Sensors”, from co-pending U.S. Provisional patent application Ser. No. 60 / 272,120 filed on 27 Feb. 2001 entitled “Vertical Triangulation System for a Virtual Touch-Sensitive Surface”, and from co-pending U.S. provisional patent application Ser. No. 60 / 231,184 filed on 7 Sep. 2000 entitled “Application of Image Processing Techniques for a Virtual Keyboard System”. Further, this application is a continuation-in-part from co-pending U.S. patent application Ser. No. 09 / 502,499 filed on 11 Feb. 2000 entitled “Method And Apparatus for Entering Data Using A Virtual Input Device”. Each of said applications is incorporated herein by reference.FIELD OF THE INVENTION [0002] The invention relates generally to sensing proximity of a stylus or user finger relative to a device to inpu...

AI Technical Summary

Benefits of technology

[0009] In a so-called structured-light embodiment, the invention includes a first optical system (OS1) that generates a plane of optical energy defining a fanbeam of beam angle φ parallel to and a small stand-off distance ΔY above the work surface whereon the virtual input device may be defined. In this embodiment, the plane of interest is the plane of light produced by OS1, typically a laser or LED light generator. The two parallel planes may typically be horizontal, but they may be disposed vertically or at any other angle that may be convenient. The invention further includes a second optical system (OS2) that is responsive to optical energy of the same wavelength as emitted by OS1. Preferably OS2 is disposed above OS1 and angled with offset θ, relative to the fan-beam plane, toward the region where the virtual input device is defined. OS2 is responsive to energy emitted by OS1, but the wavelength of the optical energy need not be visible to humans. The invention may also be implemented using non-structured-light configurations that may be active or passive. In a passive triangulation embodiment, OS1 is a camera rather than an active source of optical energy, and OS2 is a camera responsive to the same optical energy as OS1, and preferably disposed as described above. In such embodiment, the plane of interest is the projection plane of a scan line of the OS1 camera. In a non-structured-light embodiment such as an active triangulation embodiment, OS1 and OS2 are cameras and the invention further includes an active light source that emits optical energy having wavelengths to which OS1 and OS2 respond. Optionally in such embodiment, OS1 and OS2 can each include a shutter mechanism synchronized to output from the active light source, such that shutters in OS1 and OS2 are open when optical energy is emitted, and are otherwise closed. An advantage of a non-structured light configuration using two cameras is that bumps or irregularities in the work surface are better tolerated. The plane defined by OS1 may be selected by choosing an appropriate row of OS1 sensing pixel elements to conform to the highest y-dimension point (e.g., bump) of the work surface.

Problems solved by technology

For example, although computers can now be implemented in almost pocket-size, inputting data or commands on a mini-keyboard can be time consuming and error prone.

While many cellular telephones can today handle e-mail communication, actually inputting messages using the small telephone touch pad can be difficult.

For example, a PDA has much of the functionality of a computer but suffers from a tiny or non-existent keyboard.

The challenge is how to detect or sense where the user's fingers or a stylus are relative to the virtual device.

Such optical systems rely upon luminance data and require a stable source of ambient light, but unfortunately luminance data can confuse an imaging system.

Further, shadows and other image-blocking phenomena resulting from a user's hands obstructing the virtual device would seem to make implementing a Korth system somewhat imprecise in operation.

While the Bamji system worked and could be used to input commands and / or data to a computer system using three-dimensional imaging to analyze the interface of a user's fingers and a virtual input device, a less complex and perhaps less sophisticated system is desirable.

Images