Method and device for 3D line-of-sight estimation for resource-constrained scenes

Abstract

The invention discloses a three-dimensional line of sight estimation method and device for resource-constrained scenes. The method includes: constructing an end-to-end line of sight estimation network, performing face detection and line of sight estimation simultaneously, and using multi-task learning to simultaneously process two data sets Sampling, training different branches with different data; fusion training of the collected face detection data set and line of sight estimation data set, so that the end-to-end line of sight estimation network can adapt to these two different data domains at the same time, and use multi-task learning to train The network obtains the trained model; the trained model is compressed and quantized, so that the trained model can be deployed on the edge device to realize real-time estimation of 3D realization. The invention uses an end-to-end method, avoids multiple feature extractions on images, improves the running speed and supports real-time line-of-sight estimation; the invention adopts a lightweight model and performs model compression, so that the model can run in resource-limited scenarios.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a method and device for estimating a three-dimensional line of sight oriented to scenes with limited resources. Background technique [0002] Eyes are an important way for a person to express his emotions and intentions. As an important nonverbal cue, sight has applications in many ways. Gaze estimation is the estimation of the gaze direction of the eyes. According to different scenarios and applications, research in this field can be roughly divided into three categories: gaze point estimation, gaze target estimation, and 3D gaze estimation. The goal of 3D line of sight estimation is to deduce the direction of sight of people from eye pictures or face pictures. Usually, this line of sight direction is represented by two angles, pitch (vertical direction) and yaw (horizontal direction). The existing 3D The input of the line-of-sight estimation algorithm ...

AI Technical Summary

Problems solved by technology

The goal of 3D line of sight estimation is to deduce the direction of sight of people from eye pictures or face pictures. Usually, this line of sight direction is represented by two angles, pitch (vertical direction) and yaw (horizontal direction). The existing 3D The input of the line-of-sight estimation algorithm is basically a human face or human eye image. The algorithm does not have the ability to detect the human face or human eye. It needs to use the detection algorithm as a pre-acquisition to obtain the corresponding image before performing line-of-sight estimation. This method has too many processes and is relatively slow. Slow, not conducive to real-time line of sight estimation

Images