A sight line tracking method combining bidirectional LSTM and Itacker

Abstract

The invention discloses a sight line tracking method combining bidirectional LSTM and Itacker. The method comprises the following steps of 1) data preprocessing, 1.1) enabling a face reference point to be at an image center at a fixed distance from a camera; 1.2) converting the human face into an image plane in a specific camera space through the transformation matrix; 2) performing sight line estimation by using a bidirectional LSTM network structure and combining time sequence information; 2.1) respectively inputting the face, the left eye and the right eye into one branch of the network, and finally extracting a combined feature from each branch and mapping the combined feature to a screen to obtain a final three-dimensional sight line direction; 2.2) filtering out required discarding information by each LSTM cell under the forgetting door, integrating effective information at the input door, acquiring a required hidden layer at the output door, and finally mapping the forward unitand the backward unit of the last frame through the full connection layer to obtain a sight vector to complete sight tracking. According to the method, the estimation precision under 3D sight line tracking is improved, and the influence of other interference factors is reduced.

Description

technical field [0001] The invention belongs to the technical field of image processing, in particular to a robust line-of-sight tracking method. Background technique [0002] The gaze tracking system mainly realizes the estimation of the gaze direction of the human eye or the gaze point of the subject. It is applied in many fields, such as medical assistance, entertainment games, market analysis, etc. However, existing gaze-tracking systems often suffer from the following drawbacks: redundant calibration process, complicated system setup, limitation of lighting conditions, non-universal calibration for different subjects, low tolerance to head motion, limiting gaze Estimated application. The current gaze tracking systems can be mainly divided into two categories, namely model-based gaze tracking systems and appearance-based gaze tracking systems. [0003] Model-based gaze tracking systems can be divided into corneal reflection methods and shape-based methods according to...

AI Technical Summary

Problems solved by technology

However, existing gaze-tracking systems often suffer from the following drawbacks: redundant calibration process, complicated system setup, limitation of lighting conditions, non-universal calibration for different subjects, low tolerance to head motion, limiting gaze estimated application

These traditional appearance-based methods have been able to achieve high accuracy, but they still have problems that cannot be ignored in terms of head pose changes, different lighting changes, and user-individual calibration complexity.

Images