Gaze direction tracker and method for tracking a gaze direction

DE102015223891B8Active Publication Date: 2025-08-21HYUNDAI MOTOR CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
DE102015223891
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2015-09-21
Filing Date
2015-12-01
Publication Date
2025-08-21
Estimated Expiration
2035-12-01

AI Technical Summary

Technical Problem

Existing gaze trackers are unable to accurately track gaze direction when only one reflection point of illumination is generated on the iris, limiting the tracking range.

Method used

A gaze tracking system using a stereo camera with two cameras and two illuminations to capture and process images, calculating the gaze direction vector by detecting reflection points, converting two-dimensional coordinates to three-dimensional coordinates, and using distance data to determine the corneal center point even when only one reflection point is present.

Benefits of technology

Enables accurate gaze tracking even when only one reflection point is formed, expanding the tracking range and improving the system's ability to follow gaze movements.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The present disclosure provides a gaze tracker and a method for tracking a gaze direction. The gaze tracker comprises: an imaging device for obtaining images of a user's eye using two or more cameras and two or more illuminators; and a gaze direction calculator that detects a reflection point of an illumination from each image of an eye to extract a central point of a cornea, detects a central point of a pupil from each image of an eye, and calculates a gaze direction vector. Specifically, when illumination reflection points are captured from each image of an eye, the gaze calculator calculates vectors connecting each illumination to a corresponding illumination reflection point and detects an intersection point between the calculated vectors. When only one illumination reflection point is captured, the gaze calculator extracts the corneal central point using distance data between a previously recorded illumination reflection point and the corneal central point and the one illumination reflection point.
Need to check novelty before this filing date? Find Prior Art

Description

Reference to related application

[0001] This application claims priority to Korean Patent Application No. 10-2015-0133308, filed on September 21, 2015, which is incorporated by reference in its entirety. Technical field

[0002] The present disclosure relates to a gaze tracking device and a gaze tracking method for tracking the gaze direction of a user. TECHNICAL BACKGROUND

[0003] The statements in this section merely constitute background information related to the present disclosure and do not necessarily constitute prior art.

[0004] Gaze tracking is used in various fields, such as monitoring a driver's state, detecting intent, and controlling a vehicle device that uses gaze tracking. Recently, gaze tracking has been applied to monitor a driver's lapse of attention.

[0005] Such a gaze tracker captures an image of an eye using a stereo camera and detects two illumination reflection points from the captured image of the eye. Additionally, the gaze tracker calculates a corneal central point using the two detected illumination reflection points and calculates a user's gaze direction using the calculated corneal central point and a pupil central point.

[0006] However, we discovered that an ordinary gaze tracker is able to track a gaze direction only when two reflection points of illumination are generated in an iris, and therefore, when only one reflection point of illumination is generated in an iris due to face panning, it is not possible to track a gaze direction, which narrowly and excessively restricts a gaze tracking range. Description of the invention

[0007] The present disclosure provides a gaze tracker and a gaze tracking method for tracking a user's gaze even when only one reflection point is formed at a cornea during gaze tracking using a stereo camera having two cameras and two illuminations.

[0008] A gaze tracker comprises an imaging device configured to obtain images of a user's eye using two or more cameras and two or more illuminations, and a gaze calculator configured to detect a reflection point of the illumination from each image of an eye, to extract a central point of a cornea, to detect a central point of a pupil from each image of an eye, and to calculate a gaze direction vector using the central point of the cornea and the central point of the pupil, wherein, when two or more reflection points of the illumination are detected from each of the images of the eye, the gaze calculator calculates vectors connecting each illumination and a corresponding reflection point of the illumination and detects an intersection point between the calculated vectors as the central point of the cornea, and,if only one reflection point of the illumination is detected from one or more images of the eye among the images of the eye, the gaze direction calculator extracts the central point of the cornea using distance data between a previously stored reflection point of the illumination and the central point of the cornea and the one reflection point of the illumination.

[0009] The gaze direction calculator can calculate distances from the central point of the cornea to the two or more reflection points of the illumination and accumulate and record the calculated distance data.

[0010] The gaze direction calculator can convert two-dimensional coordinates of the reflection point of the illumination and the central point of the pupil into three-dimensional coordinates.

[0011] The gaze direction calculator may determine an illumination that forms the one reflection point of the illumination on a cornea among the two or more illuminations using a face direction vector.

[0012] The gaze direction calculator can calculate a similarity between reflection points of illumination in an image of an eye from which two or more reflection points of illumination are detected and an image of an eye from which one reflection point of illumination is detected, among the images of the eye, and determine an illumination which forms the one reflection point of illumination on a cornea.

[0013] The gaze direction calculator can calculate a vector connecting the one reflection point of the illumination and the specific illumination and extract the central point of the cornea using the calculated vector and the distance data.

[0014] The distance data may be obtained by calculating an average distance from distance data for a predetermined recent time period among data points of distance data from a previously recorded central point of the cornea to a reflection point of the illumination.

[0015] A method for tracking a gaze direction of a gaze tracker having two or more cameras and two or more illuminations comprises obtaining images of a user's eye using the two or more cameras, detecting a reflection point of illumination from each of the images of the eye, after detecting two or more reflection points of illumination from each of the images of the eye, calculating vectors connecting each illumination to a reflection point of illumination corresponding thereto, and detecting an intersection point between the calculated vectors as a central point of a cornea, wherein, if there is only one reflection point of illumination from one or more images of an eye among the images of the eye,the central point of the cornea is detected using distance data between a previously recorded reflection point of the illumination and the central point of the cornea and the one reflection point of the illumination, after detecting the central point of the cornea, extracting the central point of a pupil from the image of the eye and calculating a gaze direction vector using the central point of the cornea and the central point of the pupil.

[0016] Detecting the reflection point of the illumination may include converting two-dimensional coordinates of the reflection point of the illumination into three-dimensional coordinates.

[0017] Detecting the intersection point of the vectors as the central point of the cornea may comprise calculating distances from the central point of the cornea to the two or more reflection points of the illumination and accumulating and recording the calculated distance data.

[0018] Detecting the central point of the cornea using the one reflection point of the illumination may include determining an illumination corresponding to the one reflection point of the illumination, calculating a first vector for connecting the one reflection point of the illumination and the determined illumination, and extracting the central point of the cornea using the first vector and the distance data.

[0019] Determining the illumination corresponding to the one reflection point of the illumination may include determining an illumination constituting the one reflection point of the illumination on a cornea among the two or more illuminations using a vector of a face direction.

[0020] Determining the illumination corresponding to the one reflection point of the illumination may include calculating a similarity between reflection points of the illumination in an image of an eye from which two or more reflection points of the illumination are detected and an image of an eye from which one reflection point of the illumination is detected, and determining an illumination constituting the one reflection point of the illumination on a cornea.

[0021] The distance data can be obtained by calculating an average distance from distance data for a predetermined recent time period among the entries of distance data from a previously recorded central point of a cornea to a reflection point of the illumination.

[0022] Further areas of applicability will become apparent from the description provided herein. It should be understood that the description and specific examples are for purposes of illustration only and are not intended to limit the scope of the present disclosure. Short description of the drawings

[0023] In order that the content of the disclosure may be well understood, various forms of it will now be described, which are intended only as examples, with reference to the accompanying drawings in which

[0024] Fig.1 is a block diagram of a gaze tracker,

[0025] Fig. 2 is a diagram for explaining the extraction of a central point of a cornea and

[0026] Fig. 3A and Fig. 3B are flowcharts illustrating a method for gaze tracking.

[0027] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way. Detailed description

[0028] The following description is provided by way of example only and is not intended to limit the present disclosure, application, or uses. It should be noted that throughout the drawings, corresponding reference characters refer to like or corresponding parts and features.

[0029] It should also be noted that the terms “comprises” and / or “comprising”, when used in this specification, indicate the presence of the designated components but do not exclude the presence or addition of other components.

[0030] Also, terms such as "unit" or "module," etc., should be understood as a unit that performs at least one function or activity, and which may be implemented as hardware, software, or a combination of hardware and software. In this context, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly indicates otherwise.

[0031] A gaze tracker according to the present disclosure has two or more cameras and two or more illuminators, but an example in which the gaze tracker has two cameras and two illuminators will be described for understanding the present disclosure.

[0032] Fig. 1 is a block diagram of a gaze tracker.

[0033] As in Fig. 1, the gaze tracker may include an imaging device 110 and a viewing direction calculator 120 have.

[0034] The imaging device 110 can be a stereo camera and can capture images of a user's left and right eyes. The imaging device 110 can be a first camera 111 , a second camera 113 , a first illumination 115 and a second lighting 117 have.

[0035] The first camera111 and the second camera 113 can simultaneously photograph the same object (a user's eye) from different positions. In other words, the imaging device can 110 capture two images of each eye with respect to the left and right eyes of a user. When the images are captured by the first camera 111 and the second camera 113 generated, the first lighting 115 and the second lighting 117 Emit infrared rays to the user's eyes.

[0036] The first camera 111 and the second camera 113 can be implemented as an infrared camera with an infrared filter. In addition, the first illumination 115 and the second lighting 117 as a light-emitting diode (LED), a lamp or similar for emitting infrared rays or similar.

[0037] Although the imaging device 110 photographs a user's eye, the present disclosure is not limited thereto, and the imaging device 110 may be configured to photograph a face of a user and extract a region of an eye from the image of the face taken by an image processing device.

[0038] The viewing direction calculator 120 can calculate the gaze direction of a user from images of the eye acquired by the imaging device 110 were recorded. The gaze direction calculator 120 a recording module can 121 for the reflection point of a lighting, an extraction module 123 for the central point of a cornea, an extraction module 125 for the central point of a pupil and an extraction module 127 for one viewing direction.

[0039] The recording module 121for the reflection point of an illumination can detect a reflection point of an illumination which is caused by the first illumination 115 and / or the second lighting 117 is generated from each image of an eye which is captured by the first camera 111 and the second camera 113 In other words, the acquisition module 121 for the reflection point of the illumination the reflection point of the illumination which is formed by the first illumination 115 and / or the second lighting 117 generated by the image of the eye, which is captured by the first camera 111 In addition, the recording module 121 the reflection point of the illumination the reflection point of the illumination that is caused by the first illumination 115 and / or the second lighting 117 generated from the image of the eye, which is captured by the second camera 113 was recorded.

[0040] The recording module 121 For the illumination reflection point, the two-dimensional coordinates of the detected illumination reflection points can be converted into three-dimensional coordinates using 3D reconstruction techniques such as triangulation. For example, coordinates of a first illumination reflection point, which is determined by the first illumination 115 generated from a first image of an eye and a second image of an eye, which are obtained by the first camera 111 and the second camera 113 detected, and three-dimensional coordinates of the first reflection point of the illumination can be calculated using the coordinates of the first reflection point of the illumination extracted from the first image of the eye and the coordinates of the first reflection point of the illumination extracted from the second image of the eye.

[0041] In addition, the acquisition module 121 For the reflection point of the illumination, capture a reflection point of the illumination using a high-pass filter.

[0042] With reference to Fig. 2 can, if the current position coordinates of the first illumination 115 and the second lighting 117 L1 and L2 respectively, the acquisition module 121 for the reflection point of the illumination, calculate the three-dimensional coordinates L1' and L2' of the first reflection point of the illumination and the second reflection point of the illumination, which are reflected on the cornea by the first illumination 115 and the second lighting 117 be trained.

[0043] The extraction module 123for the central point of the cornea, a central point of the cornea can be determined using the detected reflection point of the illumination (the first reflection point of the illumination and / or the second reflection point of the illumination) and position coordinates of the illuminations 115 and / or 117 according to the detected reflection point of the illumination. In other words, the extraction module can 123 for the central point of the cornea, a first vector (straight line D3) connecting a three-dimensional coordinate L1' of the first reflection point of the illumination and a current position coordinate L1 of the first illumination 115 In addition, the extraction module can 123for the central point of the cornea, a second vector (straight line D4) connecting a three-dimensional coordinate L2' of the second reflection point of the illumination and a current position coordinate L2 of the second illumination 117 calculate. The extraction module 123 for the central point of the cornea, an intersection point between the first vector and the second vector can be extracted as a central point O of the cornea.

[0044] The extraction module 123 for the central point of the cornea can extract the central point O of the cornea and then calculate distances D1 and D2 to the first reflection point L1' of the illumination and to the second reflection point L2' of the illumination from the central point of the cornea O.

[0045] In addition, the extraction module 123For the central point of the cornea, the calculated distance data are collected and recorded in a recording device (not shown). The distance data can be recorded in the form shown in Table 1 below. (Table 1) Frame # Image of the right eye Image of the left eye D1 D2 D1 D2 Frame 1 22 mm 21 mm 20 mm 17 mm Frame 2 25 mm 25 mm 22 mm 24 mm . . . . . . . . . .

[0046] After detecting a reflection point of the illumination, the extraction module 123 For the central point of the cornea, calculate an average distance from distance data for a predetermined recent period among the distance data between the central point of the cornea and the first reflection point of the illumination captured from previous images of the eye previously recorded in a recording means (not shown). For example, the extraction module may 123For the central point of the cornea, calculate an average of the distances between the central point of the cornea and the first reflection point of the illumination of five recently passed frames.

[0047] In addition, the extraction module 123 calculate for the central point of the cornea an average distance from distance data for a predetermined recent period from distance data between the central point of the cornea and the second reflection point of the illumination obtained from previous images of the eye previously recorded in a recording means (not shown).

[0048] Although it has been described that an average distance is calculated and used from distance data for a predetermined recent period of time from previously recorded distance data, the present disclosure is not limited thereto, and the most recent distance data entry of the previously recorded distance data may be used.

[0049] If only one reflection point of the illumination is detected by the detection module 121 for the reflection point of the illumination, the extraction module can 123 for the central point of the cornea, check an illumination which covers the detected reflection point of the illumination on the cornea under the first illumination 115 and the second lighting 117 forms.

[0050] In this case, the extraction module 123for the central point of the cornea, determine whether the detected reflection point of the illumination is the first reflection point of the illumination that is generated on the cornea by the first illumination 115 is formed, or the second reflection point of the illumination, which is formed on the cornea by the second illumination 117 is formed using a face direction vector. The face direction vector may be calculated from an image of a user's face, which is generated by the imaging device 110 was received.

[0051] The extraction module 123for the central point of the cornea, a similarity between reflection points of illumination in an image of an eye from which two or more reflection points of illumination are detected and an image of an eye from which one reflection point of illumination is detected can be calculated and an illumination can be determined which represents the detected one reflection point of illumination on the cornea under the first illumination 115 and the second lighting 117 forms.

[0052] For example, when the first reflection point of illumination is positioned away from a central point of the pupil to the left by 3 pixels and the second reflection point of illumination is positioned away from the central point of the pupil to the right by 3 pixels in an image of an eye from which the two reflection points of illumination are detected, a distance between the one detected reflection point of illumination and the central point of the pupil is calculated, and whether the one detected reflection point of illumination is positioned to the left or to the right of the central point of the pupil can be determined based on the calculated distance.

[0053] The extraction module 123 for the central point of the cornea, a third vector can be used to connect the first reflection point of the illumination and the first illumination 115calculate if the detected reflection point of the illumination is the first reflection point of the illumination. In addition, the extraction module can 123 for the central point of the cornea, calculate a central point of the cornea using the third vector and an average distance between the first reflection point of the illumination and the central point of the cornea.

[0054] If the detected reflection point of the illumination is the second reflection point of the illumination, the extraction module 123 for the central point of the cornea the third vector connecting the second reflection point of the illumination and the second illumination 117 In addition, the extraction module can 123for the central point of the cornea, calculate the central point of the cornea using the third vector and an average distance between the second reflection point of the illumination and the central point of the cornea.

[0055] The extraction module 125 For the central point of the pupil, an image processing can be performed on an image of an eye to determine the central point of the pupil. In this case, the extraction module can 125 for the central point of the pupil, a region of an eye from the image of an eye produced by the imaging device 110 was captured and capture the central point of the pupil from the captured area of ​​the eye. The extraction module 125 for the central point of the pupil can reconstruct the two-dimensional central point of the pupil captured from the image of the eye into a three-dimensional central point of the pupil.

[0056] The calculation module 127 for the gaze direction can calculate a gaze direction vector along which a user looks, using the central point of the pupil and the central point of the cornea. That is, the calculation module 127 For the gaze direction, a straight line between the central point of the pupil and the central point of the cornea can be calculated as the gaze direction vector.

[0057] Fig. 3A and Fig. 3B are flowcharts illustrating a method for tracking a gaze direction.

[0058] As in Fig. 3A the viewing direction calculator can 120 of the gaze tracker an image of an eye of the user through the imaging device 110 (S101). In this case, the viewing direction calculator 120 the image of the user's eye through the first camera 111 and the second camera 113In other words, the gaze tracker can obtain two images, one related to the user's right and one related to the left eye.

[0059] The recording module 121 for the reflection point of the illumination of the gaze tracker can detect one or more reflection points of the illumination from the images of the eye taken by the first camera 111 and the second camera 113 output (S103). The acquisition module 121 for the reflection point of the illumination can convert the two-dimensional coordinates of the detected reflection points of the illumination into three-dimensional coordinates using 3D reconstruction techniques such as triangulation.

[0060] The extraction module 123 for the central point of the cornea of ​​the gaze tracker can check whether the two reflection points of the illumination by the detection module 121for the reflection point of the illumination (S105). In other words, the extraction module 123 For the central point of the cornea, check whether the first reflection point of the illumination and the second reflection point of the illumination applied to the cornea by the first illumination 115 and the second lighting 117 trained, are recorded.

[0061] When the two reflection points of the illumination are captured from the respective images of the eye, the extraction module 123 for the central point of the cornea vectors connecting the respective detected reflection points of the illumination and the corresponding illuminations 115 and 117 (S107). In other words, the extraction module 123for the central point of the cornea, a first vector for connecting a three-dimensional coordinate of the first reflection point of the illumination and an actual position coordinate of the first illumination 115 and a second vector for connecting a three-dimensional coordinate of the second reflection point of the illumination and an actual position coordinate of the second illumination 117 calculate.

[0062] The extraction module 123 For the central point of the cornea, the extraction module can extract a point where the calculated vectors intersect as the central point of the cornea (S109). For example, the extraction module 123 For the central point of the cornea, extract the intersection point between the previously calculated first vector and second vector as the central point of the cornea.

[0063] The extraction module 123for the central point of the cornea, distances from the central point of the cornea to the first reflection point of the illumination and the second reflection point of the illumination can be calculated and recorded (S110).

[0064] When the central point of the cornea is extracted, the gaze tracker can extract the central point of the pupil from the image of the eye through image processing (S111). In this case, the extraction module 125 for the central point of the pupil, capture a region of an eye from the image generated by the imaging device 110 and capture the central point of the pupil from the area of ​​the eye. In this case, the extraction module can 125For the pupil central point, reconstruct a two-dimensional pupil central point into a three-dimensional pupil central point. Although the pupil central point is extracted after the corneal central point is extracted, the present disclosure is not limited to this, and the pupil central point and the corneal central point can also be acquired simultaneously.

[0065] The calculation module 127 The gaze tracker's gaze direction can calculate a straight line connecting the central point of the cornea and the central point of the pupil as a gaze direction vector (S113).

[0066] If two illumination reflection points are not detected at step S105, the gaze tracker may check whether the number of detected illumination reflection points is one (S121). The gaze tracker may check whether only one illumination reflection point is detected from one or more images of the eye among the images of the eye acquired by the first camera. 111 and the second camera 113 were recorded.

[0067] If only one illumination reflection point is detected, the gaze tracker can calculate an average distance between a previously recorded illumination reflection point and the central point of the cornea (S123). The extraction module 123for the central point of the cornea can calculate an average distance between the first reflection point of the illumination and the central point of the cornea and an average distance between the second reflection point of the illumination and the central point of the cornea for a predetermined recent period among entries regarding the distance data to the first reflection point of the illumination and the second reflection point of the illumination from a previously recorded central point of the cornea.

[0068] The gaze tracker may determine an illumination corresponding to the detected reflection point of the illumination (S125). In this case, the gaze tracker may check an illumination that exceeds the detected reflection point of the illumination under the first illumination 115 and the second lighting 117 forms using a vector of the face direction.

[0069] The gaze tracker may calculate a third vector connecting the detected illumination reflection point to a corresponding illumination (S127). In other words, if the detected illumination reflection point is the first illumination reflection point, the extraction module 123 for the central point of the cornea a vector connecting the first reflection point of the illumination and the first illumination 115 If the detected reflection point of the illumination is the second reflection point of the illumination, the extraction module can 123 for the central point of the cornea a vector connecting the second illumination 117 and the second reflection point of the lighting.

[0070] The gaze tracker can calculate the central point of the cornea using the third vector and the average distance (S129).

[0071] Then, the gaze tracker can extract the central point of the pupil (S111) and can calculate a gaze direction vector using the central point of the cornea and the central point of the pupil (S113).

[0072] Although the case described so far is that S125 was performed after S123 was performed, S123 can also be performed after S125 was performed.

[0073] For example, if the one detected reflection point of the illumination is the first reflection point of the illumination, the extraction module 123calculate for the central point of the cornea an average distance from the distance data for a predetermined recent period among entries regarding the distance data between the central point of the cornea and the first reflection point of the illumination acquired from previous images of eyes previously stored in a recording means (not shown).

[0074] If one detected reflection point of the illumination is the second reflection point of the illumination, the extraction module 123calculate for the central point of the cornea an average distance from distance data for a predetermined recent period among entries regarding the data on distances between the central point of the cornea and the second reflection point of the illumination acquired from previous images of eyes previously stored in a recording device (not shown).

[0075] When only one reflection point of illumination is detected from an image of an eye, an illumination that generates the detected reflection point of illumination may be determined using a vector of a facial direction, but the present disclosure is not limited thereto.

[0076] For example, when one reflection point of illumination is detected from one image of one eye and two reflection points of illumination are detected from the other image of one eye among two images of one eye obtained by two cameras, the similarity between the reflection points of illumination in the two images of one eye can be calculated and an illumination that produces the one detected reflection point of illumination can be determined.

[0077] According to the present disclosure, when a user's gaze direction is tracked using a stereo camera with two cameras and two illuminations, even if only one illumination reflection point is formed on the cornea, a user's gaze direction can be tracked using only one illumination reflection point. Accordingly, a gaze tracking range of the gaze tracker can be enlarged.

[0078] The present disclosure is not limited thereto, but may be variously modified and altered by those skilled in the art to which this disclosure pertains without departing from the spirit and scope of the present disclosure as claimed in the following claims. List of reference symbols 110 Imaging device 111 first camera 113 second camera 115 first lighting 117 second lighting 120 viewing direction calculators 121 Detection module for the reflection point of the lighting 123 Extraction module for the central point of the cornea 125 Extraction module for the central point of the pupil 127 Calculation module for the viewing direction S101 Obtaining the image of a user's eye by two or more cameras S103 Detecting the reflection point of the illumination of each image of an eye S105 Are two reflection points of the lighting detected? S107 Calculating a first vector connecting a first reflection point of the illumination and the first illumination and a second vector connecting the second reflection point of the illumination and the second illumination S109 Extracting an intersection point between the first vector and the second vector as the central point of the cornea S110 Calculating and recording distances from the central point of the cornea to the first reflection point of the illumination and the second reflection point of the illumination S111 Extracting the central point of the pupil S113 Calculate the gaze direction vector using the central point of the cornea and the central point of the pupil S121 Is a reflection point of the lighting detected? S123 Calculate the average distance between the previously recorded reflection point of the illumination and the central point of the cornea S125 Determine the illumination according to the detected reflection point of the illumination S127 Calculating the third vector connecting the detected reflection point of the illumination and the corresponding illumination S129 Detecting the central point of the cornea using the third vector and the average distance QUOTES CONTAINED IN THE DESCRIPTION

[0079] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0080] KR 10-2015-0133308

[0001]

Claims

[1] Gaze tracker with: an imaging device configured to obtain images of a user's eye using at least two cameras and at least two illuminations, and a gaze direction calculator configured to detect a reflection point of the illumination from each of the images of the eye and extract a central point of a cornea, wherein the gaze direction calculator is configured to detect a central point of the pupil from each of the images of the eye and calculate a gaze direction vector using the central point of the cornea and the central point of the pupil, wherein, when at least two reflection points of the illumination from each of the images of the eye are detected, the gaze direction calculator is configured to calculate vectors connecting between each illumination and the corresponding reflection point of the illumination and to detect an intersection point between the calculated vectors as the central point of the cornea, and wherein, when only one reflection point of the illumination from one or more images of the eye among the images of the eye is detected, the gaze direction calculator is configured to extract the central point of the cornea using data regarding a distance between a previously recorded reflection point of the illumination and the central point of the cornea and the one reflection point of the illumination. [2] The gaze tracker of claim 1, wherein the gaze calculator is configured to calculate distances from the central point of the cornea to the at least two reflection points of the illumination and to accumulate and record the calculated data regarding the distance. [3] A gaze tracker according to claim 1 or 2, wherein the gaze calculator is configured to convert two-dimensional coordinates of the reflection point of the illumination and the central point of the pupil into three-dimensional coordinates. [4] A gaze tracker according to any one of the preceding claims, wherein the gaze direction calculator is configured to determine an illumination forming the one reflection point of the illumination on a cornea among the at least two illuminations using a vector of a facial direction. [5] A gaze tracker according to any one of the preceding claims, wherein the gaze calculator is configured to calculate a similarity between reflection points of illumination in an image of an eye from which the at least two reflection points of illumination are detected and an image of an eye from which one reflection point of illumination is detected, among the images of the eye, and to determine an illumination that forms the one reflection point of illumination on a cornea. [6] The gaze tracker according to claim 4, wherein the gaze calculator is configured to calculate a vector connecting the one reflection point of the illumination and the determined illumination and to extract the central point of the cornea using the calculated vector and the distance data. [7] A gaze tracker according to any one of the preceding claims, wherein the calculator of the gaze tracker is arranged to obtain the distance data by calculating an average distance from distance data for a predetermined recent period among entries relating to the data on distances from a previously recorded central point of the cornea to a reflection point of the illumination. [8] A method for tracking the line of sight of a line of sight tracker having at least two cameras and at least two illuminations, the method comprising: capturing images of a user’s eye using the at least two cameras, capturing a reflection point of the illumination from each of the images of the eyes, upon detecting at least two reflection points of the illumination from each of the images of the eyes, calculating vectors connecting each illumination to a corresponding reflection point of the illumination and detecting an intersection point between the calculated vectors as a central point of the cornea, and, if only one reflection point of the illumination is detected from one or more images of the eye among the images of the eyes, detecting the central point of the cornea using distance data between a previously recorded reflection point of the illumination and the central point of the cornea and the one reflection point of the illumination, when the central point of the cornea has been detected, extracting a central point of the pupil from the image of the eye and calculating a gaze direction vector using the central point of the cornea and the central point of the pupil. [9] The method of claim 8, wherein detecting the reflection point of the illumination comprises converting two-dimensional coordinates of the reflection point of the illumination into three-dimensional coordinates. [10] The method of claim 8 or 9, wherein detecting the intersection point of the vectors as the central point of the cornea comprises calculating distances from the central point of the cornea to the at least two reflection points of the illumination and accumulating and recording calculated distance data. [11] The method of claim 8, 9 or 10, wherein detecting the central point of the cornea using the one reflection point of the illumination comprises: determining an illumination corresponding to the one reflection point of the illumination, calculating a first vector connecting the one reflection point of the illumination and the determined illumination and extracting the central point of the cornea using the first vector and the distance data. [12] The method of claim 11, wherein determining the illumination corresponding to the one reflection point of the illumination comprises determining an illumination forming the one reflection point of the illumination at a cornea among the at least two illuminations using a vector of a facial direction. [13] The method of claim 11 or 12, wherein determining the illumination corresponding to the one reflection point of the illumination comprises calculating the similarity between reflection points of the illuminations in an image of an eye from which two or more reflection points of the illumination are detected and an image of an eye from which one illumination is detected, and determining an illumination which forms the one reflection point of the illumination on a cornea. [14] A method according to any one of claims 8 to 13, wherein the data on distance is obtained by calculating an average distance from data on distance for a predetermined recent period among entries on the data on a distance from a previously recorded central point of a cornea to a reflection point of illumination.

Citation Information

Patent Citations

  • Apparatus and method for tracking gaze based on camera array

    US20140313308A1