A sight line tracking method combining bidirectional LSTM and Itacker

A gaze tracking and gaze estimation technology, applied in the field of image processing, can solve the problems of limited lighting conditions, complex system settings, low tolerance of head movement, etc., and achieve the effect of stable and accurate gaze tracking

Active Publication Date: 2019-04-02
ZHEJIANG UNIV OF TECH
View PDF5 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, existing gaze-tracking systems often suffer from the following drawbacks: redundant calibration process, complicated system setup, limitation of lighting conditions, non-universal calibration for different subjects, low tolerance to head motion, limiting

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A sight line tracking method combining bidirectional LSTM and Itacker
  • A sight line tracking method combining bidirectional LSTM and Itacker
  • A sight line tracking method combining bidirectional LSTM and Itacker

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] The present invention will be further described below in conjunction with the accompanying drawings.

[0043] refer to Figure 1 ~ Figure 1 0, a line of sight tracking method combining two-way LSTM and Itracker, said method comprising the following steps:

[0044] S1. Data preprocessing: In order to weaken the impact of different head poses and different camera parameters on the final line of sight estimation results, perform perspective transformation on the original image, and train the model to perform line of sight estimation in a specific virtual space. The steps are as follows:

[0045] S1.1. Make the face reference point at the center of the image at a fixed distance from the camera, the process is as follows:

[0046] First, assuming that a is the coordinates of the reference point of the face in the camera space, then the z-axis is obtained from the face to the reference point under the virtual camera as Then, suppose is the rotation matrix of the head pos...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sight line tracking method combining bidirectional LSTM and Itacker. The method comprises the following steps of 1) data preprocessing, 1.1) enabling a face reference point to be at an image center at a fixed distance from a camera; 1.2) converting the human face into an image plane in a specific camera space through the transformation matrix; 2) performing sight line estimation by using a bidirectional LSTM network structure and combining time sequence information; 2.1) respectively inputting the face, the left eye and the right eye into one branch of the network, and finally extracting a combined feature from each branch and mapping the combined feature to a screen to obtain a final three-dimensional sight line direction; 2.2) filtering out required discarding information by each LSTM cell under the forgetting door, integrating effective information at the input door, acquiring a required hidden layer at the output door, and finally mapping the forward unitand the backward unit of the last frame through the full connection layer to obtain a sight vector to complete sight tracking. According to the method, the estimation precision under 3D sight line tracking is improved, and the influence of other interference factors is reduced.

Description

technical field [0001] The invention belongs to the technical field of image processing, in particular to a robust line-of-sight tracking method. Background technique [0002] The gaze tracking system mainly realizes the estimation of the gaze direction of the human eye or the gaze point of the subject. It is applied in many fields, such as medical assistance, entertainment games, market analysis, etc. However, existing gaze-tracking systems often suffer from the following drawbacks: redundant calibration process, complicated system setup, limitation of lighting conditions, non-universal calibration for different subjects, low tolerance to head motion, limiting gaze Estimated application. The current gaze tracking systems can be mainly divided into two categories, namely model-based gaze tracking systems and appearance-based gaze tracking systems. [0003] Model-based gaze tracking systems can be divided into corneal reflection methods and shape-based methods according to...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T7/246G06N3/04G06N3/08G06F3/01
CPCG06F3/013G06N3/08G06T7/246G06T2207/20081G06T2207/20084G06T2207/10016G06T2207/30201G06T2207/30241G06N3/048G06N3/044G06N3/045
Inventor 周小龙姜嘉祺林家宁陈胜勇
Owner ZHEJIANG UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap