Sight line detection and video processing method and device, equipment and storage medium

A line of sight detection and line-of-sight direction technology, applied in the field of image processing, can solve the problems of large amount of data processing, low accuracy of eye sight line, and low efficiency of line of sight judgment.

Active Publication Date: 2019-10-22
BIGO TECH PTE LTD
View PDF16 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] Among them, literature [1] and literature [2] are a similar data-driven method, and literature [3] uses a simpler and more intuitive bulletin board method, which is to select from a large number of eyeball textures that best match the current eyeball state. The eyeball texture is placed on the target eyeball to realize the migration of eyeball movement. It needs to be compared with a large number of historical eyeball textures

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sight line detection and video processing method and device, equipment and storage medium
  • Sight line detection and video processing method and device, equipment and storage medium
  • Sight line detection and video processing method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] Figure 1AIt is a flow chart of a gaze detection method provided by Embodiment 1 of the present invention. This embodiment can be applied to any situation where a user's gaze direction is detected by collecting face images. A line-of-sight detection method provided in this embodiment can be executed by the line-of-sight detection device provided in the embodiment of the present invention, and the device can be implemented by means of software and / or hardware, and integrated into a device that executes the method. The device may be any 3D model processing device capable of image processing.

[0048] Specifically, refer to Figure 1A , the method may include the following steps:

[0049] S110, according to the key feature points in the face image, determine the corresponding face posture and eye pupil rotation displacement.

[0050] Specifically, since in the display of video screens containing human facial expressions for entertainment applications or game websites, it ...

Embodiment 2

[0062] Figure 2A It is a flow chart of a line of sight detection method provided in Embodiment 2 of the present invention, Figure 2B It is a schematic diagram of the principle of the line of sight detection process provided by Embodiment 2 of the present invention. This embodiment is optimized on the basis of the foregoing embodiments. Specifically, in this embodiment, the process of determining the face pose and the rotational displacement of the pupils of the eyes in the face image will be described in detail.

[0063] optional, see Figure 2A , this embodiment may include the following steps:

[0064] S210, scanning the face image to obtain corresponding face data; using a three-dimensional face grid template to reconstruct the face data to obtain a reconstructed face grid model; extracting key feature points in the face grid model, as the key feature points in the face image.

[0065] S220. Determine the posture positions of the key feature points in the face image....

Embodiment 3

[0083] Figure 3A It is a flow chart of a line of sight detection method provided by Embodiment 3 of the present invention, Figure 3B It is a schematic diagram of the principle of the line of sight detection process provided by Embodiment 3 of the present invention. This embodiment is optimized on the basis of the foregoing embodiments. Specifically, in this embodiment, a detailed description is mainly given of the back-projection process in which the pupils of the eyes in the face image are rotated and displaced to the line-of-sight direction of the actual face in the three-dimensional space.

[0084] optional, see Figure 3A , this embodiment may include the following steps:

[0085] S310, according to the key feature points in the face image, determine the corresponding face posture and eye pupil rotation displacement.

[0086] S320. Construct a corresponding line-of-sight optimization function according to the preset projection function, face posture, and eye pupil ro...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a sight line detection and video processing method and device, equipment and a storage medium. The sight line detection method comprises the steps that according to key feature points in a human face image, a corresponding human face posture and eye pupil rotation displacement are determined, and the eye pupil rotation displacement is the displacement of the pupil center relative to the eyeball center in the human face image; and according to a preset projection function and the human face posture, the eye pupil rotation displacement is projected to athree-dimensional space where an actual human face is located to obtain a sight line direction of the actual human face. According to the technical scheme provided by the embodiment of the invention,the processing amount of the eye data is reduced, the detection efficiency of the sight line direction is improved, the sight line direction of the actual face is analyzed directly through eye pupil rotation in the face image, and the detection accuracy of the sight line direction is improved.

Description

technical field [0001] Embodiments of the present invention relate to the field of image processing, and in particular, to a method, device, device, and storage medium for line-of-sight detection and video processing. Background technique [0002] With the development of video technology, 3D virtual characters have been widely used in entertainment, movies, and virtual reality (VR) simulations. At this time, eyes are a very important part of virtual characters. There are much fewer body movements, but people are very sensitive to eye movement perception, unnatural eye angles and movements are easily noticed, and the eye movement capture device that collects the center point of the pupil of the eye is usually worn on the head It is very inconvenient to use. [0003] In response to the above problems, in the application scenario of realistic facial expression animation, the transformation from the captured pupil center point to eyeball movement is usually realized by using th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00
CPCG06V40/166G06V40/168G06V40/193G06V40/19G06T7/73G06T2207/10016G06T2207/20084G06T2207/30201
Inventor 刘更代
Owner BIGO TECH PTE LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap