Multi-feature fusion sight line estimation method based on attention mechanism

A technology of multi-feature fusion and line-of-sight estimation, applied in neural learning methods, computing, computer components, etc., can solve problems such as low accuracy, and achieve the effect of improving accuracy, high precision, and high robustness

Pending Publication Date: 2021-11-12
CHONGQING UNIV OF POSTS & TELECOMM
View PDF8 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At the same time, variable head poses and low-quality eye images in natural scenes will lead to low accuracy of gaze estimation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-feature fusion sight line estimation method based on attention mechanism
  • Multi-feature fusion sight line estimation method based on attention mechanism
  • Multi-feature fusion sight line estimation method based on attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] The technical solutions in the embodiments of the present invention will be described clearly and in detail below with reference to the drawings in the embodiments of the present invention. The described embodiments are only some of the embodiments of the invention.

[0039] The technical scheme that the present invention solves the problems of the technologies described above is:

[0040]S1, first use the MTCNN algorithm to perform face detection and face key point positioning on the original picture, then cut the face picture according to the key points of the human eye to obtain the eye picture, and finally obtain the size required for the line of sight estimation task of 224×224 ×3 face and binocular image, where 224×224 represents the size of the face and binocular image, and 3 represents the channel number of the RGB image.

[0041] S2, using the face feature extractor based on the group convolution channel and the spatial attention mechanism to extract head pose...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a multi-feature fusion sight line estimation method based on an attention mechanism, and the method comprises the steps: carrying out the detection of a human face and human face key points through an MTCNN algorithm, and obtaining a human face image and an eye image needed by the sight line estimation; by utilizing a face and eye feature extractor based on a group convolution channel and a space attention mechanism, selecting and enhancing important features in the face and binocular images, and inhibiting information irrelevant to sight line estimation; and by utilizing the binocular feature fusion network and the pupil feature fusion network, fusing the center position features of the eyes and the pupil, and splicing the center position features of the eyes and the pupil with the face feature vector to realize multi-feature fusion, so that the influence of the asymmetry of the eyes and the inaccurate estimation of the head posture on the sight line estimation is avoided. Through verification on a public data set MPIIGaze and EyeDiap and compared with a current mainstream sight line estimation method, the sight line estimation method provided by the invention has a smaller average angle error, and the precision and robustness of sight line estimation in a natural scene are effectively improved.

Description

technical field [0001] The invention belongs to the field of image processing and pattern recognition, in particular to a multi-feature fusion line of sight estimation method based on an attention mechanism. Background technique [0002] Line of sight reflects human visual attention, which can be used to study people's intentions and understand social interactions. Therefore, accurate estimation of line of sight has become an important research topic in computer vision, in terms of human-computer interaction, saliency detection, and car driving. Wide range of applications. [0003] Gaze estimation is the process of detecting the direction of the gaze and locating the position of the gaze point. Gaze estimation methods are mainly divided into two categories: model-based and appearance-based methods. Model-based methods mainly estimate the gaze direction by extracting infrared reflection points on the corneal surface and the center of the pupil. This type of method can usua...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/048G06N3/045G06F18/253
Inventor 胡章芳夏艳玲罗元王兰何革
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap