Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A line-of-sight estimation method based on key point matching

A key point and face key point technology, applied in computing, computer components, instruments, etc., can solve problems such as line of sight estimation deviation and error, and achieve the effects of increasing robustness, improving accuracy, and increasing practicability

Active Publication Date: 2019-02-15
UNIV OF ELECTRONIC SCI & TECH OF CHINA
View PDF3 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, when the existing gaze estimation method based on the 3D face model calculates the pupil center position, due to the limitation of the database, it cannot cover all the real situations, and there is a big problem when the head posture or eye offset is large. error, leading to a large deviation in the final estimate of the line of sight

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A line-of-sight estimation method based on key point matching
  • A line-of-sight estimation method based on key point matching

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0034] see figure 1 , the present invention mainly includes the following steps: detection of the target face, detection of key points of the face, preliminary positioning of the pupil center, head pose estimation and eyeball positioning, correction of the position of the pupil center, and estimation of the line of sight direction.

[0035] Step 1: Detect the target face.

[0036] Input the video stream collected by the camera into the trained face detection network (MobileNet-SSD) for face detection, intercept the largest face as the target face for line of sight detection, and normalize it to a size of 300*300 as input to the keypoint detection network.

[0037] Step 2: Face key point detection and pupil center preliminary positioning.

[0038] SE-Net is used as the basic network for face key point and pupil center detection model training, and L1loss is used as the loss function during the training process to further improve the positioning accuracy. Pass the 300*300 fac...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a line-of-sight estimation method based on key point matching, belonging to the field of computer vision. After the pupil key points are initially located by the depth network,the invention adopts the SGBM template matching method to further modify the pupil center position. Compared with the existing line-of-sight estimation methods, it can locate pupil center more accurately, especially for the case of large head or eyeball bias. The implementation of the invention can effectively improve the accuracy of the line-of-sight estimation. Compared with the pupil-corneal reflection method, only a single network camera is adopted, thereby greatly reducing the equipment cost. Compared with the existing methods based on single image processing, the proposed algorithm doesnot need to restrict the pose of the head, and the robustness of the algorithm is greatly improved. By matching the 3D face model, the limitation that the existing database can not represent all poseis avoided, which increases the practicability of the method.

Description

technical field [0001] The invention proposes a line-of-sight estimation method based on key point matching, which is a new line-of-sight estimation technique in the field of computer vision. Background technique [0002] With the development of computer science, human-computer interaction has gradually become a hot field. The sight of the human eye can reflect the information that people pay attention to, and it is also an important source of information input in human-computer interaction. Human-computer interaction based on line-of-sight estimation has broad prospects for development in military, medical, entertainment and other fields. [0003] The current practical line-of-sight estimation technology is mainly based on pupil corneal reflection technology (PCCR), which uses near-infrared light sources to generate reflection images on the cornea and pupils of the user's eyes, and then uses image sensors to collect images of the eyes and reflections, and finally calculate...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00
CPCG06V40/161G06V40/168
Inventor 李宏亮颜海强尹康袁欢梁小娟邓志康
Owner UNIV OF ELECTRONIC SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com