Event camera-based gaze tracking using neural networks

A neural network and event technology, applied in the field of gaze tracking, can solve problems such as increasing equipment heat and power consumption

Pending Publication Date: 2020-10-30
APPLE INC
View PDF0 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Transmitting images at frame rates sufficient to enable gaze tracking requires communication links with considerable bandwidth, and the use of such communication links increases the generated heat and power consumption of the device

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Event camera-based gaze tracking using neural networks
  • Event camera-based gaze tracking using neural networks
  • Event camera-based gaze tracking using neural networks

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] Numerous details are described in order to provide a thorough understanding of the example implementations shown in the drawings. However, the drawings illustrate only some example aspects of the disclosure and are therefore not to be considered limiting. One of ordinary skill in the art will appreciate that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits have not been described in detail so as not to obscure more relevant aspects of the exemplary implementations described herein.

[0021] In various implementations, gaze tracking is used to enable user interaction, provide foveal rendering, or mitigate geometric distortion. The gaze tracking system includes a camera and a processor that performs gaze tracking on data received from the camera regarding light reflected from a user's eye from a light source. In various implementations, the c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

One implementation involves a device receiving a stream of pixel events output by an event camera. The device derives an input image by accumulating pixel events for multiple event camera pixels. Thedevice generates a gaze characteristic using the derived input image as input to a neural network trained to determine the gaze characteristic. The neural network is configured in multiple stages. Thefirst stage of the neural network is configured to determine an initial gaze characteristic, e.g., an initial pupil center, using reduced resolution input(s). The second stage of the neural network is configured to determine adjustments to the initial gaze characteristic using location-focused input(s), e.g., using only a small input image centered around the initial pupil center. The determinations at each stage are thus efficiently made using relatively compact neural network configurations. The device tracks a gaze of the eye based on the gaze characteristic.

Description

technical field [0001] The present disclosure relates generally to gaze tracking, and in particular to systems, methods, and devices for gaze tracking using event camera data. Background technique [0002] Existing gaze tracking systems determine the direction of a user's gaze based on shutter-based camera images of the user's eyes. Existing gaze tracking systems often include a camera that transmits images of the user's eyes to a processor that performs gaze tracking. Transmission of images at a frame rate sufficient to enable gaze tracking requires communication links with considerable bandwidth, and the use of such communication links increases the generated heat and power consumption of the device. Contents of the invention [0003] Various implementations disclosed herein include devices, systems, and methods using neural networks for event camera-based gaze tracking. One example implementation involves performing operations at an apparatus having one or more proces...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06K9/20G06K9/46G06V10/141G06V10/764
CPCG06V40/193G06V10/141G06V10/454G06V10/82G06V10/764G06F3/013G06T3/4046G06F18/214G06F18/2413
Inventor T·盖保尔R·贝迪基安
Owner APPLE INC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap