Method and device for 3D line-of-sight estimation for resource-constrained scenes

A line-of-sight estimation, resource-oriented technology, applied in the field of artificial intelligence, can solve problems such as slow speed, many processes, and unfavorable line-of-sight estimation, and achieve the effect of improving speed, simplifying processes, and efficient line-of-sight estimation.

Active Publication Date: 2022-03-08
HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)
View PDF12 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The goal of 3D line of sight estimation is to deduce the direction of sight of people from eye pictures or face pictures. Usually, this line of sight direction is represented by two angles, pitch (vertical direction) and yaw (horizontal direction). The existing 3D The input of the line-of-sight estimation algorithm is basically a human face or human eye image. The algorithm does not have the ability to detect the human face or human eye. It needs to use the detection algorithm as a pre-acquisition to obtain the corresponding image before performing line-of-sight estimation. This method has too many processes and is relatively slow. Slow, not conducive to real-time line of sight estimation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for 3D line-of-sight estimation for resource-constrained scenes
  • Method and device for 3D line-of-sight estimation for resource-constrained scenes
  • Method and device for 3D line-of-sight estimation for resource-constrained scenes

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0068] Such as figure 1 As shown, this embodiment is a method for estimating a 3D line of sight with limited resource scenarios, and the method includes the following steps:

[0069] S1. Construct an end-to-end line of sight estimation network. The end-to-end line of sight estimation network performs face detection and line of sight estimation at the same time, and uses multi-task learning to sample two data sets at the same time, and trains different branches with different data;

[0070] Such as figure 2 As shown, the end-to-end line-of-sight estimation network includes a backbone network, a classification sub-network, a border regression sub-network and a line-of-sight estimation sub-network; the backbone network is used to convolute and calculate feature maps on the entire input image, and the classification sub-network The network is used to perform convolutional object classification on the output of the backbone network; the frame regression sub-network is used to pe...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a three-dimensional line of sight estimation method and device for resource-constrained scenes. The method includes: constructing an end-to-end line of sight estimation network, performing face detection and line of sight estimation simultaneously, and using multi-task learning to simultaneously process two data sets Sampling, training different branches with different data; fusion training of the collected face detection data set and line of sight estimation data set, so that the end-to-end line of sight estimation network can adapt to these two different data domains at the same time, and use multi-task learning to train The network obtains the trained model; the trained model is compressed and quantized, so that the trained model can be deployed on the edge device to realize real-time estimation of 3D realization. The invention uses an end-to-end method, avoids multiple feature extractions on images, improves the running speed and supports real-time line-of-sight estimation; the invention adopts a lightweight model and performs model compression, so that the model can run in resource-limited scenarios.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a method and device for estimating a three-dimensional line of sight oriented to scenes with limited resources. Background technique [0002] Eyes are an important way for a person to express his emotions and intentions. As an important nonverbal cue, sight has applications in many ways. Gaze estimation is the estimation of the gaze direction of the eyes. According to different scenarios and applications, research in this field can be roughly divided into three categories: gaze point estimation, gaze target estimation, and 3D gaze estimation. The goal of 3D line of sight estimation is to deduce the direction of sight of people from eye pictures or face pictures. Usually, this line of sight direction is represented by two angles, pitch (vertical direction) and yaw (horizontal direction). The existing 3D The input of the line-of-sight estimation algorithm ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06V40/16G06V10/80G06V10/774G06V10/764G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/045G06F18/25G06F18/24G06F18/214
Inventor 漆舒汉王轩张加佳蒋遇刘洋罗文坚高翠芸廖清蒋琳吴卓
Owner HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products