Method for inferring metal states from eye movements

Abstract

A computer-implemented method infers mental states of a person from eye movements of the person. The method includes identifying elementary features of eye tracker data, such as fixations and saccades, and recognizing from the elementary features a plurality of eye-movement patterns. Each eye-movement pattern is recognized by comparing the elementary features with a predetermined eye-movement pattern template. A given eye-movement pattern is recognized if the elementary features satisfy a set of criteria associated with the template for that eye-movement pattern. The method further includes the step of recognizing from the eye-movement patterns a plurality of eye-behavior patterns corresponding to the mental states of the person. Because high level mental states of the user are determined in real time, the method provides the basis for reliably determining when a user intends to select a target.

Description

FIELD OF THE INVENTIONThe present invention relates generally to the field of eye tracking and methods for processing eye tracking data. In particular, the invention relates to a system and method for determining mental states or mental activities of a person from spatio-temporal eye-tracking data, independent of a priori knowledge of the objects in the person's visual field.BACKGROUNDIn recent years, eye-tracking devices have made it possible for machines to automatically observe and record detailed eye movements. One common type of eye tracker, for example, uses an infrared light-source, a camera, and a data processor to measure eye gaze positions, i.e., positions in the visual field at which the eye gaze is directed. The tracker generates a continuous stream of spatiotemporal data representative of eye gaze positions at sequential moments in time. Analysis of this raw data typically reveals a series of eye fixations separated by sudden jumps between fixations, called saccades.An ...

AI Technical Summary

Benefits of technology

In view of the above, it is an object of the present invention to overcome the disadvantages and limitations of existing methods for deriving useful information from eye tracker data. In particular, it is an object of the present invention to provide a method for accurately recognizing a variety of high-level mental states of a user from eye tracker data. It is another object of the invention to provide such a technique that does not require a priori information about objects in the user's visual field, and is not limited to situations where the user is looking at a computer screen. Yet another object of the invention is to provide a method for analyzing user mental states from detailed fixation-saccade data rather than from statistical data derived from eye movements. An additional object of the invention is to provide a technique for inferring mental states of a user without requiring a priori knowledge of the task the user is engaged in, or of the contents and locations of specific regions at which the user is looking.

Problems solved by technology

Fixation and saccade data alone, however, is still relatively low-level data that is of limited use, and Jacob fails to teach any specific methods for recognizing a user's conscious intentions or mental states.

These eye tracking methods, therefore, still fall short of the goal of providing useful information about any higher-level eye behavior or mental states.

One major disadvantage of this technique is that it requires a priori knowledge of the objects in the user's visual field, such as their positions, shapes and type information.

Consequently, the technique cannot be used in many computer software applications where information about what is displayed on a computer screen is not readily available.

In addition, it cannot be used in other situations where a priori knowledge is not available at all, such as when the user is not viewing virtual objects on a computer screen, but physical objects in the real world.

In addition, because the technique disclosed by Starker and Bolt identifies the attention of the user with single fixation points, it fails to accurately distinguish attentively looking at an object from "spacing out" while inattentively gazing at the object.

Thus, although the technique attempts to recognize the mental state of attentive interest, it actually fails to properly distinguish this state from non-attentiveness.

Takagi, however, does not disclose any general method for extracting a user's intention from eye movements.

As Takagi states, "Any general methods of analysis derived from known theories cannot be developed.

In other words, Takagi not only fails to teach a general method of extracting a user's intention from eye movement data, he also states that such a general method is impossible using known theories.

Takagi's techniques are also limited by the fact that they require a combination of eye movement data with information about the objects being viewed by the user.

This is a weak point of the method.

We cannot transform eye-movements data into fixation-saccade data because of some problems" (Takagi, p.

Thus, not only does Takagi require a priori knowledge of the content of specific regions in user's visual field, but Takagi's method only measures the region within which the user is gazing, and does not measure detailed fixation-saccade data.

Takagi's technique is also limited in other important respects.

Regarding the long-standing problem of correctly relating eye fixations with user attentions, Takagi acknowledges that his technique does "not deal with this problem" (Takagi, p.

It is clear, therefore, that the prior art techniques for interpreting eye tracker data suffer from one or more of the following disadvantages: they fail to properly identify user attention or intention, they do not identify a variety of mental states, they are limited to very specific and predetermined user tasks, and they require a priori knowledge of objects in the user's field of vision.

Images