Line of sight estimation method based on generative adversarial network

Abstract

The invention puts forward a line of sight estimation method based on a generative adversarial network. The method comprises the following main contents of generating a texture, generating real data and refining eyes. The method comprises the following processes that: firstly, automatically aligning a face image with the texture space of the horizontal direction and the vertical direction of a 3Dmodel; then, mapping a synthesis image to a true domain by an unpaired pixel level domain adaptive technology; thirdly, using the annotation and synthesis data of a line of sight direction to pre-train a line of sight direction estimator; and finally, in a whole mapping process, executing a refined network to keep a line of sight direction, and using a pre-training network to serve as a conversioncirculation constraint from synthesis to truth to synthesis. By use of the method, a novel adversarial training method is used, the rendered synthesis image is mapped to a vivid domain, and accurateline of sight estimation can be obtained on a practical image without using any piece of additional flag data from a true user. For the situations of extreme head gesture, blur, long distance and thelike, the method can generate line of sight estimation with robustness.

Description

technical field [0001] The invention relates to the field of line of sight estimation, in particular to a line of sight estimation method based on a generation confrontation network. Background technique [0002] Estimation of human eye gaze direction is an important branch of human-computer interaction technology. It mainly studies the detection and recognition of human eye movement characteristics. In addition to inferring the target object or area that people pay attention to, it can further analyze and study people's psychological activities. Eye movement can be used to control external devices and systems. In the application of the traffic field, it can detect the driver's fatigue state and judge whether the driver is driving in fatigue; if the driver's eyes keep staring at a certain direction, it means that he is likely to be distracted or may cause distraction And a traffic accident happened. In the military and aviation fields, people can also control external equi...

AI Technical Summary

Problems solved by technology

[0004] Aiming at problems such as easy to be blocked by eyelids and eyelashes, and difficult to accurately judge the direction of sight, the purpose of the present invention is to provide a sight estimation method based on a generative confrontation network. Alignment, followed by an unpaired pixel-level domain adaptation technique to map the synthetic image to the real domain, pre-training a gaze direction estimator using gaze direction annotations and synthetic data, and finally performing a refinement network throughout the mapping process to preserve gaze direction, Using a pre-trained network as a constraint for the transformation cycle from synthetic to real to synthetic

Images