Method and device for finding and tracking pairs of eyes

Abstract

A method for finding and subsequently tracking the 3-D coordinates of a pair of eyes in at least one face, including receiving image data, which contains a sequence of at least one digital video signal of at least one image sensor, finding eyes or tracking previously found eyes in the image data, ascertaining the 3-D coordinates of the found or tracked eyes, associating the found or tracked eyes with a pair of eyes and providing the 3-D coordinates of the pair of eyes.

Description

CROSS REFERENCE TO RELATED APPLICATIONS[0001]This application claims the priority of PCT / EP2008 / 065352, filed on Nov. 12, 2008, which claims priority to German Application No. 10 2007 056528.5, filed Nov. 16, 2007, the entire contents of which are hereby incorporated in total by reference.BACKGROUND OF THE INVENTION[0002]The present invention relates to a contactless method for finding and subsequent tracking of the 3D coordinates of a pair of eyes in at least one face in real time.[0003]In contrast for example to contact methods, contactless methods for finding and tracking faces do not require any additional means, such as head-mounted cameras or spots. The advantage of these contactless methods is that the freedom of movement of the subjects to be tracked is not restricted in any way by physical means and that the subjects are not bothered by the use of such means.[0004]Contactless detection and tracking methods are known in the prior art. U.S. Pat. No. 6,539,100 B1 and EP 0 350 ...

AI Technical Summary

Benefits of technology

[0012]It is thus the object of the present invention to provide a method that allows to find and track reliably, precisely and efficiently the eye positions of one or multiple faces in all three spatial directions in a sufficiently large detection and tracking range in real time, while only causing a low computational load. It is in particular the object of the present invention to make possible the efficient and precise finding of at least one eye.

[0016]In a further process step, the 3D coordinates of the found eyes are determined. In a preferred embodiment, another stereo analysis is applied to achieve this, but any other known method of determining 3D coordinates of an object can alternatively be applied. Thereafter, the found eyes are assigned to form pairs of eyes, where either the distance between two eyes, which can be computed based on the 3D coordinates of the eyes, is compared with the known eye separation of a model pair of eyes, or where any other suitable assignment method is applied, for example the classification which is preferred here. It is preferable for further processing of the found eyes in the form of pairs of eyes to determine the 3D coordinates of the pairs of eyes, because the positions of the eyes and of the face can be found based on those data using known face-eye models, thus reducing the amount of data which are required for subsequent tracking of a once identified face.

[0017]Tracking already found pairs of eyes generally includes the steps of defining at least one second search area on the identified face and tracking the eyes of the pair of eyes in this search area. According to one aspect, a second search area is defined for each eye. The pairs of eyes which are determined in the step of finding eyes are here used as a starting point for further tracking. The areas in which the eyes of the investigated pair of eyes are tracked are defined based on the 3D coordinates of the eyes. The thus defined areas which are preferably smaller than the first search areas, are understood and referred to as second search areas for eye tracking. The thus defined tracking of pairs of eyes allows the definition of sufficiently small second search areas even if the eyes or the face move at fast pace or in a large range, which in turn makes it possible to track the pairs of eyes in real time.

Problems solved by technology

A temporally non-coincident image recording in conjunction with a superposition of the images by the interlaced scanning method, which serves to reduce the amount of image data for transmission, does not allow a reliable detection and tracking of the eyes in real time.

On the other hand, this method only allows to detect and to track eyes which are spatially very close to the image sensor, because the effects caused by the active illumination diminish as the distance of the eyes to the illumination source grows, which leads to the effect that the eyes to be detected can no longer be distinguished from other objects or a noise in the differential image.

However, WO2007 / 019842 does not explain how the eyes are found and tracked.

The methods known in the prior art further exhibit the problem that the position information of the eyes cannot be delivered in real time, in particular not where more than one face is to be identified and tracked.

Images