Three-dimensional sight line estimation method and device oriented to resource-constrained scene

Abstract

The invention discloses a three-dimensional line-of-sight estimation method and device oriented to a resource-constrained scene, and the method comprises the steps: constructing an end-to-end line-of-sight estimation network, carrying out the face detection and line-of-sight estimation at the same time, carrying out the sampling of two data sets at the same time through multi-task learning, and training different branches through different data; performing fusion training on the collected face detection data set and the line-of-sight estimation data set to enable an end-to-end line-of-sight estimation network to adapt to the two different data fields at the same time, and training the network by adopting a multi-task learning mode to obtain a trained model; and compressing and quantifying the trained model, so that the trained model can be deployed on edge equipment, and real-time estimation of three-dimensional implementation is realized. According to the method, an end-to-end method is used, multiple times of feature extraction on the image is avoided, the running speed is increased, and real-time sight line estimation is supported; according to the method, the lightweight model is adopted and is compressed, so that the model can be operated in a resource-limited scene.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a method and device for estimating a three-dimensional line of sight oriented to scenes with limited resources. Background technique [0002] Eyes are an important way for a person to express his emotions and intentions. As an important nonverbal cue, sight has applications in many ways. Gaze estimation is the estimation of the gaze direction of the eyes. According to different scenarios and applications, research in this field can be roughly divided into three categories: gaze point estimation, gaze target estimation, and 3D gaze estimation. The goal of 3D line of sight estimation is to deduce the direction of sight of people from eye pictures or face pictures. Usually, this line of sight direction is represented by two angles, pitch (vertical direction) and yaw (horizontal direction). The existing 3D The input of the line-of-sight estimation algorithm ...

AI Technical Summary

Problems solved by technology

The goal of 3D line of sight estimation is to deduce the direction of sight of people from eye pictures or face pictures. Usually, this line of sight direction is represented by two angles, pitch (vertical direction) and yaw (horizontal direction). The existing 3D The input of the line-of-sight estimation algorithm is basically a human face or human eye image. The algorithm does not have the ability to detect the human face or human eye. It needs to use the detection algorithm as a pre-acquisition to obtain the corresponding image before performing line-of-sight estimation. This method has too many processes and is relatively slow. Slow, not conducive to real-time line of sight estimation

Images