Corneal curvature radius calculation device, sight line detection apparatus, corneal curvature radius calculation method, and corneal curvature radius calculation program

Abstract

To accurately calculate the corneal curvature radius of a subject.SOLUTION: A corneal curvature radius calculation device includes: a light source which is disposed at a predetermined position and emits detection light to irradiate at least one eyeball of a subject with the detection light; a first imaging unit and a second imaging unit which capture images of the subject's eyeball irradiated with the detection light from different positions; a position calculation unit which calculates, on the basis of a first image that is an image of the subject's eyeball captured by the first imaging unit, a position of a first corneal reflection center caused by the detection light from the light source, and calculates, on the basis of a second image that is an image of the subject's eyeball captured by the second imaging unit, a position of a second corneal reflection center caused by the detection light; a distance calculation unit which calculates, on the basis of the positions of the first and second corneal reflection centers, an inter-corneal reflection distance; and a corneal curvature radius calculation unit which calculates a corneal curvature radius of the subject's eyeball on the basis of the inter-corneal reflection distance and a distance between the light source and the subject's eyeball.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a corneal radius of curvature calculation device, a gaze detection device, a corneal radius of curvature calculation method, and a corneal radius of curvature calculation program. [Background technology]

[0002] A gaze detection device is known that emits detection light from multiple light sources and irradiates the eyeball of a subject, acquires images of the eyeball irradiated with the detection light in a time-division manner using an imaging unit, and detects the gaze direction of the subject based on the acquired images and the distance between the imaging unit and the subject (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-38734 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-described gaze detection device, when performing gaze detection processing to detect the gaze direction of a subject, a process of calculating the corneal radius of curvature of the subject is performed. In this process, the corneal radius of curvature is calculated based on images of the eyeball acquired over multiple frames. Therefore, if the subject moves during measurement, an error may occur, which may reduce the calculation accuracy.

[0005] The present invention has been made in view of the above, and aims to provide a corneal radius of curvature calculation device, a gaze detection device, a corneal radius of curvature calculation method, and a corneal radius of curvature calculation program that are capable of accurately calculating the corneal radius of curvature of a subject. [Means for solving the problem]

[0006] The corneal radius of curvature calculation device according to the present invention includes a light source arranged at a predetermined position, which emits detection light and irradiates it onto at least one eyeball of a subject; first and second imaging units which capture images of the subject's eyeball irradiated with the detection light from different positions; a position calculation unit which calculates the position of a first corneal reflection center due to the detection light from the light source based on a first image which is an image of the subject's eyeball captured by the first imaging unit, and calculates the position of a second corneal reflection center due to the detection light based on a second image which is an image of the subject's eyeball captured by the second imaging unit; a distance calculation unit which calculates a distance between the corneal reflections based on the positions of the first and second corneal reflection centers; and a corneal radius of curvature calculation unit which calculates a corneal radius of curvature of the subject's eyeball based on the distance between the corneal reflections and the distance between the light source and the subject's eyeball.

[0007] The gaze detection device according to the present invention includes the above-described corneal radius of curvature calculation device, and a detection processing unit that detects the position of the pupil center of the subject based on an image of the eyeball of the subject captured by at least one of the first imaging unit and the second imaging unit of the corneal radius of curvature calculation device, and detects the gaze of the subject based on the detected position of the pupil center, the positions of the first imaging unit and the second imaging unit of the corneal radius of curvature calculation device, the positions of the first corneal reflection center and the second corneal reflection center calculated by the position calculation unit of the corneal radius of curvature calculation device, and the corneal radius of curvature calculated by the corneal radius of curvature calculation unit of the corneal radius of curvature calculation device.

[0008] A corneal radius of curvature calculation method according to the present invention includes emitting detection light from a light source arranged at a predetermined position and irradiating it onto at least one eyeball of a subject; capturing an image of the subject's eyeball irradiated with the detection light using a first imaging unit and a second imaging unit arranged at different positions; calculating a position of a first corneal reflection center due to the detection light from the light source based on a first image that is an image of the subject's eyeball captured by the first imaging unit; calculating a position of a second corneal reflection center due to the detection light based on a second image that is an image of the subject's eyeball captured by the second imaging unit; calculating a corneal reflection distance based on the positions of the first corneal reflection center and the second corneal reflection center; and calculating a corneal radius of curvature of the subject's eyeball based on the corneal reflection distance and the distance between the light source and the subject's eyeball.

[0009] The corneal curvature radius calculation program according to the present invention causes a computer to execute the following processes: emitting detection light from a light source arranged at a predetermined position and irradiating it onto at least one eyeball of a subject; capturing an image of the subject's eyeball irradiated with the detection light using a first imaging unit and a second imaging unit arranged at different positions; calculating the position of a first corneal reflection center due to the detection light from the light source based on a first image that is an image of the subject's eyeball captured from the first position; calculating the position of a second corneal reflection center due to the detection light based on a second image that is an image of the subject's eyeball captured from the second position; calculating a corneal reflection distance based on the positions of the first corneal reflection center and the second corneal reflection center; and calculating a corneal curvature radius of the subject's eyeball based on the corneal reflection distance and the distance between the light source and the subject's eyeball. [Effects of the Invention]

[0010] According to the present invention, it is possible to accurately calculate the corneal curvature radius of a subject. [Brief explanation of the drawings]

[0011] [Figure 1]FIG. 1 is a diagram schematically illustrating an example of a gaze detection device according to this embodiment. [Figure 2] FIG. 2 is a functional block diagram illustrating an example of a gaze detection device. [Figure 3] FIG. 3 is a diagram schematically illustrating a state in which detection light from a light source is irradiated onto the eyeball of a subject. [Figure 4] FIG. 4 is a diagram schematically illustrating a state in which detection light from a light source is irradiated onto the eyeball of a subject. [Figure 5] FIG. 5 is a diagram schematically illustrating a state in which detection light from a light source is irradiated onto the eyeball of a subject. [Figure 6] FIG. 6 is a diagram showing an example of the positional relationship between the light source, the image capturing unit, and the eyeball of the subject. [Figure 7] FIG. 7 is a diagram showing an example of image data of an eyeball captured by the first and second imaging units. [Figure 8] FIG. 8 is a schematic diagram for explaining the principle of the gaze detection process according to this embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of a corneal radius of curvature calculation method according to this embodiment. [Figure 10] FIG. 10 is a diagram schematically showing the flow of calculating the corneal radius of curvature in the corneal radius of curvature calculation method according to this embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of the gaze detection process in the gaze detection method according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, the components in the following embodiments include those that are easily replaceable by those skilled in the art, or those that are substantially identical.

[0013] In the following explanation, the positional relationship of each part will be explained using a three-dimensional global coordinate system. The direction parallel to the first axis of a predetermined plane is defined as the X-axis direction, the direction parallel to the second axis of the predetermined plane that is perpendicular to the first axis is defined as the Y-axis direction, and the direction parallel to the third axis that is perpendicular to both the first and second axes is defined as the Z-axis direction. The predetermined plane includes the XY plane.

[0014] [Corneal curvature radius calculation device, gaze detection device] 1 is a diagram schematically illustrating an example of an eye-gaze tracking device 100 according to this embodiment. The eye-gaze tracking device 100 according to this embodiment detects the eye gaze of a subject and outputs the detection result. The eye-gaze tracking device 100 detects the eye gaze based on, for example, the position of the subject's pupil and the position of the corneal reflection.

[0015] As shown in FIG. 1, the gaze detection device 100 includes a display device 10, an image capture device 20, a computer system 30, an output device 40, an input device 50, an input / output interface device 60, and a light-source-subject distance detection unit 70. The display device 10, the image capture device 20, the computer system 30, the output device 40, and the input device 50 communicate data via the input / output interface device 60. The display device 10 and the image capture device 20 each have a drive circuit (not shown). The light-source-subject distance detection unit 70 detects the distance between an imaging unit 22 (described below) of the image capture device 20 and the subject's eyeball. The light-source-subject distance detection unit 70 may be, for example, an ultrasonic sensor or a stereo camera, but is not limited thereto, and may have any other configuration as long as it can detect the distance between the imaging unit 22 and the subject's eyeball. The light-source-subject distance detection unit 70 may be installed in a location other than the top of the display device 10, for example, near the imaging unit 22.

[0016] The display device 10 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OLED). In this embodiment, the display device 10 has a display unit 11. The display unit 11 displays information such as images. The display unit 11 is substantially parallel to the XY plane. The X-axis direction is the left-right direction of the display unit 11, the Y-axis direction is the up-down direction of the display unit 11, and the Z-axis direction is the depth direction perpendicular to the display unit 11. The display device 10 may be a head-mounted display device. In the case of a head-mounted display, a configuration such as the image acquisition device 20 is disposed in a head-mounted module.

[0017] The image acquisition device 20 acquires image data of the left and right eyes EB of the subject, and transmits the acquired image data to the computer system 30. The image acquisition device 20 includes a light source 21 and an imaging unit 22.

[0018] The light source 21 emits detection light to the eye EB of the subject. The light source 21 includes an LED (light emitting diode) light source and can emit near-infrared light with a wavelength of, for example, 850 [nm]. The light source 21 may emit the detection light at a predetermined timing (for example, the period of a frame synchronization signal of the first imaging unit 22A and the second imaging unit 22B, which will be described later), or may emit the detection light continuously for a predetermined period.

[0019] The imaging unit 22 generates image data by capturing images of the subject's left and right eyes EB. In this embodiment, multiple imaging units 22 are provided. In this embodiment, the multiple imaging units 22 include a first imaging unit 22A and a second imaging unit 22B. Various cameras are used as the first imaging unit 22A and the second imaging unit 22B depending on the method for detecting the subject's gaze. In a method for detecting the gaze based on the position of the subject's pupil and the position of the corneal reflection, as in this embodiment, the first imaging unit 22A and the second imaging unit 22B include infrared cameras, each of which includes an optical system capable of transmitting near-infrared light with a wavelength of, for example, 850 nm, and an imaging element capable of receiving the near-infrared light. The imaging unit 22 outputs a frame synchronization signal. The period of the frame synchronization signal can be, for example, 20 msec, but is not limited to this. In this embodiment, the first imaging unit 22A and the second imaging unit 22B are each a single camera. The first imaging unit 22A and the second imaging unit 22B are disposed on both sides of the light source 21. In other words, the light source 21 is disposed at a position between the first imaging unit 22A and the second imaging unit 22B. The light source 21 may also be disposed at a position midway between the first imaging unit 22A and the second imaging unit 22B. The image data acquired by the imaging unit 22 is configured such that pixels having brightness set as, for example, an 8-bit value (0 to 255) are arranged two-dimensionally. The smaller the brightness value, the darker the image, and the larger the brightness value, the brighter the image. A pixel with a brightness value of 0 is displayed as black in the image data. A pixel with a brightness value of 255 is displayed as white in the image data.

[0020] The computer system 30 controls the overall operation of the gaze detection device 100. The computer system 30 includes an arithmetic processing unit 30A and a storage device 30B. The arithmetic processing unit 30A includes a microprocessor such as a CPU (central processing unit). The storage device 30B includes memory or storage such as a ROM (read only memory) and a RAM (random access memory). The arithmetic processing unit 30A performs arithmetic processing in accordance with a computer program 30C stored in the storage device 30B.

[0021] The output device 40 includes a display device such as a flat panel display. The output device 40 may also include a printing device. The input device 50 generates input data when operated. The input device 50 includes a keyboard or a mouse for a computer system. The input device 50 may also include a touch sensor provided on a display unit of the output device 40, which is a display device.

[0022] In the gaze detection device 100 according to this embodiment, the display device 10 and the computer system 30 are separate devices. The display device 10 and the computer system 30 may be integrated. For example, the gaze detection device 100 may include a tablet-type personal computer. In this case, the tablet-type personal computer may be equipped with a display device, an image acquisition device, a computer system, an input device, an output device, and the like.

[0023] FIG. 2 is a functional block diagram showing an example of the gaze detection device 100. As shown in FIG. 2, the computer system 30 includes an image capture control unit 31, a position calculation unit 32, a distance calculation unit 33, a corneal radius of curvature calculation unit 34, a detection processing unit 35, and a memory unit 36. The functions of the computer system 30 are performed by an arithmetic processing device 30A and a memory unit 30B (see FIG. 1). Note that some functions of the computer system 30 may be provided outside the gaze detection device 100. In this embodiment, a corneal radius of curvature calculation device 80 is configured by the light source 21 and the image capture unit 22 (first image capture unit 22A and second image capture unit 22B) of the image acquisition device 20, the image capture control unit 31, the position calculation unit 32, the distance calculation unit 33, and the corneal radius of curvature calculation unit 34. Note that the corneal radius of curvature calculation device 80 may include at least one of the detection processing unit 35 and the memory unit 36.

[0024] The imaging control unit 31 controls the light source 21 and the imaging unit 22. The imaging control unit 31 controls the timing and emission time of the detection light emitted by the light source 21. The imaging control unit 31 controls the timing of imaging for each of the first imaging unit 22A and the second imaging unit 22B of the imaging unit 22. The imaging control unit 31 can control the first imaging unit 22A and the second imaging unit 22B to capture images at the timing when the detection light is emitted from the light source 21. In this control, the imaging control unit 31 may cause the first imaging unit 22A and the second imaging unit 22B to capture images at any timing as long as the detection light is emitted from the light source 21. However, the imaging control unit 31 causes the first imaging unit 22A and the second imaging unit 22B to capture images at the same timing. For example, the imaging control unit 31 may synchronize the timing of emission of detection light from the light source 21 with the timing of imaging by the first imaging unit 22A and the second imaging unit 22B, or may cause the first imaging unit 22A and the second imaging unit 22B to capture images at any timing while the detection light is continuously emitted from the light source 21. The imaging control unit 31 acquires image data acquired by the image acquisition device 20. The imaging control unit 31 stores the acquired image data in the storage unit 36.

[0025] The position calculation unit 32 calculates the position of a first corneal reflection center 121 (see FIG. 3, etc.) which is the center of the corneal reflection image 120 captured by the first imaging unit 22A, and the position of a second corneal reflection center 122 (see FIG. 3, etc.) which is the center of the corneal reflection image 120 captured by the second imaging unit 22B, based on a first image IM1 and a second image IM2 which are image data of the eyeball of the subject captured by the first imaging unit 22A and the second imaging unit 22B. The position calculation unit 32 stores the calculated positions of the first corneal reflection center 121 and the second corneal reflection center 122 in the storage unit 36. In this embodiment, the position calculation unit 32 calculates the position of a first pupil center 112A of the subject based on the first image IM1, and calculates the position of a second pupil center 112B of the subject based on the second image IM2.

[0026] The distance calculation unit 33 calculates the inter-corneal reflection distance based on the position of the first corneal reflection center 121 and the position of the second corneal reflection center 122. In this embodiment, the distance calculation unit 33 calculates the inter-corneal reflection distance d based on a first distance d1 between the position of the first pupil center 112A and the position of the first corneal reflection center 121, and a second distance d2 between the position of the second pupil center 112B and the second corneal reflection center 122.

[0027] The corneal radius of curvature calculation unit 34 calculates the corneal radius of curvature of the subject's eyeball based on the corneal reflection distance and the distance between the light source 21 and the subject's eyeball EB (hereinafter referred to as the light-source-subject distance). The corneal radius of curvature calculation unit 34 stores the calculated light-source-subject distance in the memory unit 36. The corneal radius of curvature is the distance between the surface of the cornea of ​​the eyeball EB and the center of corneal curvature.

[0028] Here, the corneal curvature center will be described. In this embodiment, a case will be described in which the eyeball EB is illuminated by the light source 21 and images of the eyeball EB are captured by two imaging units 22 (a first imaging unit 22A and a second imaging unit 22B).

[0029] 3 is a diagram schematically illustrating a state in which detection light from the light source 21 is irradiated onto the eyeball EB of the subject. As shown in FIG. 3, in this embodiment, the first imaging unit 22A and the second imaging unit 22B are disposed at positions symmetrical to the left and right with respect to the light source 21. Therefore, the position of the light source 21 can be considered as a reference position.

[0030] The first corneal reflection center 121 indicates the corneal reflection center in an image obtained when the first imaging unit 22A captures a corneal reflection image 120 obtained by irradiating the eye EB with detection light from the light source 21. The second corneal reflection center 122 indicates the corneal reflection center in an image obtained when the second imaging unit 22B captures a corneal reflection image 120 obtained by irradiating the eye EB with detection light from the light source 21. In Fig. 3, the first corneal reflection center 121 and the second corneal reflection center 122 indicate virtual positions.

[0031] The detection processing unit 35 calculates the positions of the first corneal reflection center 121 and the second corneal reflection center 122 on the images captured by the first and second imaging units 22A and 22B. The detection processing unit 35 calculates the position of the corneal reflection center 124 from the positions of the first and second corneal reflection centers 121 and 122. Specifically, on the images captured by the first and second imaging units 22A and 22B, the midpoint between the positions of the first and second corneal reflection centers 121 and 122, relative to the position of the pupil center, is calculated as the position of the corneal reflection center 124 on the images, and this position is converted into a three-dimensional coordinate system based on the light-source-subject distance. Transformation parameters for converting the position into the three-dimensional coordinate system are stored in advance in the storage unit 36, for example.

[0032] Corneal curvature center 110 exists on a straight line 123 connecting light source 21 and corneal reflection center 124, which is the center of corneal reflection image 120. The position on straight line 123 at a predetermined distance from corneal reflection center 124 is the position of corneal curvature center 110. In this embodiment, this predetermined value is corneal radius of curvature r. In other words, corneal radius of curvature r is the distance between the corneal surface and corneal curvature center 110. Corneal radius of curvature calculation unit 34 calculates the value of corneal radius of curvature r.

[0033] The detection processing unit 35 detects the position of the pupil center 112 of the subject based on image data captured by the imaging unit 22. The pupil center is the center of the pupil. The detection processing unit 35 detects the gaze vector of the eyeball EB of the subject based on the calculated position of the pupil center 112, the positions of the first imaging unit 22A and the second imaging unit 22B, the positions of the first corneal reflection center 121 and the second corneal reflection center 122 calculated by the position calculation unit 32, and the value of the corneal curvature radius r calculated by the corneal curvature radius calculation unit 34.

[0034] Specifically, detection processing unit 35 determines a straight line 123 connecting light source 21 and corneal reflection center 124, and determines the position on straight line 123 where the distance from corneal reflection center 124 is the value of corneal radius of curvature r as the position of corneal curvature center 110. Detection processing unit 35 detects a straight line passing through corneal curvature center 110 and the center of the pupil as the subject's line of sight (gaze vector). After detecting the gaze vector, detection processing unit 35 also detects the position of the gaze point indicating the intersection of the gaze vector and display unit 11.

[0035] The storage unit 36 ​​stores various data and programs for performing processing in each unit of the computer system 30. The storage unit 36 ​​stores, for example, data regarding an image to be displayed on the display unit 11. The storage unit 36 ​​stores image data acquired by the imaging control unit 31, the positions of the first corneal reflection center 121 and the second corneal reflection center 122 calculated by the position calculation unit 32, and the corneal curvature radius r calculated by the corneal curvature radius calculation unit 34.

[0036] The memory unit 36 ​​also stores a corneal radius of curvature calculation program that causes the computer to execute the following processes: a process of emitting detection light from a light source 21 arranged at a predetermined position and irradiating it onto at least one eyeball EB of the subject; a process of capturing an image of the subject's eyeball EB irradiated with the detection light using a first imaging unit 22A and a second imaging unit 22B arranged at different positions; a process of calculating the position of a first corneal reflection center 121 due to the detection light from the light source 21 based on a first image IM1 that is an image of the subject's eyeball EB captured by the first imaging unit 22A, and calculating the position of a second corneal reflection center 122 due to the detection light based on a second image IM2 that is an image of the subject's eyeball EB captured by the second imaging unit 22B; a process of calculating a corneal reflection distance d based on the positions of the first corneal reflection center 121 and the second corneal reflection center 122; and a process of calculating a corneal curvature radius r of the subject's eyeball EB based on the corneal reflection distance d and a light-source-subject distance x between the light source 21 and the subject's eyeball EB.

[0037] [Corneal curvature radius calculation method] Next, a corneal radius of curvature calculation method according to this embodiment will be described. FIGS. 4 and 5 are diagrams schematically illustrating a state in which detection light from a light source 21 is irradiated onto the eye EB of a subject. Hereinafter, the right eye EB (right eye ER) of the subject will be described, but a similar description can be applied to the left eye EB (left eye EL). As shown in FIGS. 4 and 5, when detection light is irradiated onto the eye EB from the light source 21, a corneal reflection image 120 is formed on the cornea of ​​the eye EB. In FIG. 4, when the corneal reflection image 120 of the light source 21 is captured by the first imaging unit 22A, the center of the corneal reflection image 120 is a first corneal reflection center 121. Furthermore, when the corneal reflection image 120 of the light source 21 is captured by the second imaging unit 22B, the center of the corneal reflection image 120 is a second corneal reflection center 122.

[0038] 5, as the distance between imaging unit 22 and the subject changes, the corneal reflection distance d between first corneal reflection center 121 and second corneal reflection center 122 changes. For example, the relationship dA>dB holds between the corneal reflection distance dA when the light source-subject distance, which is the distance between light source 21 and the subject's eyeball EB, is x1 (position P1) and the corneal reflection distance dB when the light source-subject distance is x2, which is larger than x1 (position P2).

[0039] FIG. 6 is a diagram showing an example of the positional relationship among a light source unit, an imaging unit, and the eye of a subject. As shown in FIG. 6, let the distance between the light source 21 and the first imaging unit 22A be a, the distance between the light source 21 and the second imaging unit 22B be b, the distance between the light source 21 and the subject be x (the light source-subject distance), the distance between the first corneal reflection center 121 and the second corneal reflection center 122 detected on the eyeball EB be d, and the corneal radius of curvature be r. In this example, the light source-subject distance x is the distance between the light source 21 and the corneal curvature center 110 of the subject's eyeball EB. Note that, for example, the value detected by the light source-subject distance detection unit 70 can be used as the light source-subject distance x. Here, since r << x, it can be approximated that the angle formed by the straight line L1 passing through the corneal curvature center 110 and the first imaging unit 22A and the straight line L2 passing through the corneal curvature center 110 and the first corneal reflection center 121 is equal to the angle formed by the straight line L3 passing through the corneal curvature center 110 and the light source 21 and the straight line L2. Hereinafter, this angle is denoted as θ1. Similarly, it can be approximated that the angle formed by the straight line L4 passing through the corneal curvature center 110 and the second imaging unit 22B and the straight line L5 passing through the corneal curvature center 110 and the first corneal reflection center 121 is equal to the angle formed by the straight line L3 passing through the corneal curvature center 110 and the light source 21 and the straight line L5. Hereinafter, this angle is denoted as θ2.

[0040] In this case, the following equations 1 to 3 hold. θ1 = arctan(a / x) / 2 ···(Equation 1) θ2 = arctan(b / x) / 2 ···(Equation 2) d = r·sinθ1 + r·sinθ2 ···(Equation 3)

[0041] 7 is a diagram showing an example of image data of the eye EB captured by the first imaging unit 22A and the second imaging unit 22B. As shown in FIG. 7, in this embodiment, a first image IM1 captured by the first imaging unit 22A and a second image IM2 captured by the second imaging unit 22B are acquired as separate image data when the eye EB is irradiated with detection light from the light source 21. The first image IM1 shows a corneal reflection image (hereinafter referred to as a first corneal reflection image 120A) due to the detection light from the light source 21. The second image IM2 shows a corneal reflection image (hereinafter referred to as a second corneal reflection image 120B) due to the detection light from the light source 21. In this embodiment, the first imaging unit 22A and the second imaging unit 22B may simultaneously capture images at any timing during the period in which the detection light is emitted from the light source 21. In this case, the first image IM1 and the second image IM2, which are both image data, contain the first corneal reflection image 120A and the second corneal reflection image 120B. In this way, the first image IM1 and the second image IM2, in which the first corneal reflection image 120A and the second corneal reflection image 120B are captured in different ways, can be obtained in one frame.

[0042] Based on the acquired first image IM1, the position calculation unit 32 calculates the position of the first corneal reflection center 121 on the image and the position of the first pupil center 112A on the image. Based on the acquired second image IM2, the position calculation unit 32 also calculates the position of the second corneal reflection center 122 on the image and the position of the second pupil center 112B on the image. In the first image IM1 and the second image IM2, the pixels where the first corneal reflection image 120A and the second corneal reflection image 120B are formed have higher brightness than other pixels. The position calculation unit 32 searches for the pixel with the highest brightness in the first image IM1 and the second image IM2, determines a high-brightness region where pixels having brightness within a certain range exist based on the pixel, determines the brightness center of gravity of this region, and sets the coordinates of the brightness center of gravity on the image as the first corneal reflection center 121 and the second corneal reflection center 122. Furthermore, in the first image IM1 and the second image IM2, the pixels where the first pupil center 112A and the second pupil center 112B are formed have lower brightness than the other pixels. The position calculation unit 32 searches for the pixel with the lowest brightness in the first image IM1 and the second image IM2, determines a low-brightness region where pixels having brightness within a certain range exist using that pixel as a reference, determines the brightness center of gravity of this region, and can set the coordinates of the brightness center of gravity on the image as the first pupil center 112A and the second pupil center 112B. Note that while FIG. 7 schematically illustrates the high-brightness region and the low-brightness region as being uniform in brightness, there may be cases where, for example, the brightness decreases from one pixel in the high-brightness region to the surrounding pixels, or the brightness increases from one pixel in the low-brightness region to the surrounding pixels.

[0043] Distance calculation unit 33 calculates the inter-corneal reflection distance d based on the position of first corneal reflection center 121 and the position of second corneal reflection center 122. In this embodiment, the inter-corneal reflection distance d can be calculated using the relative positions from first pupil center 112A and second pupil center 112B in first image IM1 and second image IM2, respectively. That is, a first distance d1, which is the distance between first pupil center 112A and first corneal reflection center 121, is calculated in first image IM1, and a second distance d2, which is the distance between second pupil center 112B and second corneal reflection center 122, is calculated in second image IM2, and the sum of first distance d1 and second distance d2 can be calculated as the inter-corneal reflection distance d.

[0044] Specifically, the distance calculation unit 33 calculates the distance d1 (mm) between the first corneal reflection center 121 and the first pupil center 112A from the distance d1' (pixels) between the first corneal reflection center 121 and the first pupil center 112A (number of pixels of the imaging element) included in the first image IM1. The distance calculation unit 33 also calculates the distance d2 (mm) between the second corneal reflection center 122 and the second pupil center 112B from the distance d2' (pixels) between the second corneal reflection center 122 and the second pupil center 112B (number of pixels of the imaging element) included in the second image IM2. Note that in FIG. 7, the first corneal reflection center 121 and the second corneal reflection center 122 are shown as black dots to make their positions easier to distinguish, but in reality, no black dots exist.

[0045] Here, if the focal length of the lens that constitutes the imaging unit 22 is f and the distance (pitch) between pixels of the imaging element that constitutes the imaging unit 22 is p, then the following formulas 4 and 5 hold.

[0046] d1´=(f d1 / x) / p (Equation 4) d2´=(f d2 / x) / p (Equation 5)

[0047] The first distance d1 and the second distance d2 in the three-dimensional coordinate system can be calculated using the above equations 4 and 5. The distance calculation unit 33 calculates the sum of the calculated first distance d1 and second distance d2 as the distance d between corneal reflections, and can calculate the corneal radius of curvature r based on the distance d between corneal reflections.

[0048] [Gaze detection method] Next, an example of a gaze detection method using the gaze detection device 100 according to this embodiment will be described. As described above, the gaze detection method according to this embodiment can obtain the corneal reflection distance d between the first corneal reflection center 121 and the second corneal reflection center 122, and calculate the corneal curvature radius r based on this corneal reflection distance d. This eliminates the need for calibration processing as in conventional gaze detection methods, that is, calibration processing in which a target image is displayed on the display unit 11 and the subject gazes at it. Therefore, gaze detection processing can be performed without performing calibration processing.

[0049] 8 is a schematic diagram illustrating the principle of the gaze detection process according to this embodiment. In the gaze detection process, detection light is emitted from the light source 21 to illuminate the eye EB, and the first and second image capturing units 22A and 22B are synchronized to capture images of the eye EB during the period when the detection light from the light source 21 is emitted. The detection processing unit 35 detects the positions of the pupil center 112C and the corneal reflex center 113C based on the acquired image data of the eye EB. The detection processing unit 35 converts the positions of the pupil center 112C and the corneal reflex center 113C into a global coordinate system using, for example, the light-source-subject distance x detected by the light-source-subject distance detection unit 70.

[0050] The detection processing unit 35 calculates the position of the corneal center of curvature 110 based on the positions of the light source 21, the pupil center 112C, the corneal reflection center 113C, and the corneal curvature radius r calculated by the above method. Specifically, the detection processing unit 35 determines a straight line 173 connecting the light source 21 and the corneal reflection center 113C. The detection processing unit 35 also determines a position that is a distance equivalent to the corneal curvature radius r away from the corneal reflection center 113C toward the inside of the eyeball EB as the position of the corneal curvature center 110. The detection processing unit 35 determines a straight line 178 connecting the pupil center 112C and the corneal curvature center 110, and calculates the position of the intersection 166 between the straight line 178 and the display unit 11 as the position of the gaze point.

[0051] Next, an example of a corneal radius of curvature calculation method according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of a distance calculation method according to this embodiment. As shown in Fig. 9, under the control of the imaging control unit 31, the light source 21 is caused to emit light and irradiate the eyeball EB with detection light (step S101), and the first imaging unit 22A and the second imaging unit 22B capture images of the eyeball EB of the subject (step S102). The imaging control unit 31 acquires a first image IM1 and a second image IM2, which are image data of the left and right eyeballs EB captured by the first imaging unit 22A and the second imaging unit 22B.

[0052] Based on the acquired image data, the position calculation unit 32 calculates the first corneal reflection center 121 and the first pupil center 112A in the first corneal reflection image 120A and the second corneal reflection center 122 and the second pupil center 112B in the second corneal reflection image 120B for each of the left and right eyes EB, respectively, and calculates the first distance d1 and the second distance d2 based on the calculation results (step S103). The distance calculation unit 33 calculates the corneal reflection distance d based on the first distance d1 and the second distance d2 (step S104). The corneal curvature radius calculation unit 34 calculates the corneal curvature radius r based on the calculated corneal reflection distance d and the separately detected light-source-subject distance x (step S105). The light-source-subject distance x may be a constant value previously detected by, for example, the light-source-subject distance detection unit 70, or may be detected by the light-source-subject distance detection unit 70 for each gaze detection process.

[0053] FIG. 10 is a diagram schematically illustrating the flow of calculating the corneal radius of curvature r using the corneal radius of curvature calculation method according to this embodiment. As shown in FIG. 10, the corneal radius of curvature calculation method according to this embodiment can calculate the corneal radius of curvature r based on a first image IM1 and a second image IM2 acquired in a single frame. For example, when calculating the corneal radius of curvature using multiple images acquired over multiple frames, errors may occur if the subject moves between frames. In contrast, according to the corneal radius of curvature calculation method according to this embodiment, errors in the corneal radius of curvature r do not occur even if the subject moves during measurement. Therefore, the accuracy of calculating the corneal radius of curvature r is higher than when calculating the corneal radius of curvature using multiple images acquired over multiple frames.

[0054] Next, an example of the gaze detection method according to this embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the gaze detection process in the gaze detection method according to this embodiment.

[0055] 11, in the gaze detection process, detection light is emitted from the light source 21 and irradiated onto the eyeball EB under the control of the imaging control unit 31 (step S201), and the eyeball EB of the subject is imaged by the first imaging unit 22A and the second imaging unit 22B (step S202). The imaging control unit 31 acquires image data of the left and right eyeballs EB imaged by the first imaging unit 22A and the second imaging unit 22B.

[0056] The detection processing unit 35 detects the position of the pupil center 112C and the position of the corneal reflection center 124 (step S203). Note that the positions of the corneal reflection centers may be the positions (first corneal reflection center 121, second corneal reflection center 122) detected by the position calculation unit 32. The detection processing unit 35 converts each position into a global coordinate system using the above-mentioned light source-subject distance x (step S204).

[0057] The detection processing unit 35 obtains a straight line connecting the light source 21 and the corneal reflection center 124 (step S205). The detection processing unit 35 calculates, on the obtained straight line, a position on the back side of the eyeball EB from the position of the corneal reflection center 124 by the value of the corneal radius of curvature obtained by the calibration process, as the corneal curvature center 110 (step S206). The detection processing unit 35 calculates, as a gaze vector, the straight line connecting the pupil center 112C and the corneal curvature center 110 (step S207), and calculates the world coordinates of the intersection of the gaze vector and the display unit 11 (step S208). The detection processing unit 35 converts the calculated world coordinates into coordinates of the display unit 11, thereby obtaining the position of the gaze point (step S209).

[0058] As described above, the corneal radius of curvature calculation device 80 according to this embodiment includes the light source 21, which is disposed at a predetermined position and emits detection light to irradiate at least one eyeball EB of the subject, the first image capturing unit 22A and the second image capturing unit 22B, which capture images of the eyeball EB of the subject irradiated with the detection light from different positions, and the first image capturing unit 22A, which is an image of the eyeball EB of the subject captured by the first image capturing unit 22A, calculates the position of the first corneal reflection center 121 due to the detection light from the light source 21, and outputs the first image capturing unit 22A to the second image capturing unit 22B. The apparatus is provided with a position calculation unit 32 that calculates the position of the second corneal reflection center 122 by the detection light based on a second image IM2 that is an image of the subject's eyeball EB captured by 2B, a distance calculation unit 33 that calculates the corneal reflection distance d based on the position of the first corneal reflection center 121 and the position of the second corneal reflection center 122, and a corneal curvature radius calculation unit 34 that calculates the corneal curvature radius r of the subject's eyeball EB based on the corneal reflection distance d and the light source-subject distance x between the light source 21 and the subject's eyeball EB.

[0059] The corneal radius of curvature calculation method according to this embodiment includes emitting detection light from a light source 21 arranged at a predetermined position and irradiating it onto at least one eyeball EB of a subject; capturing an image of the subject's eyeball EB irradiated with the detection light using a first imaging unit 22A and a second imaging unit 22B arranged at different positions; calculating the position of a first corneal reflection center 121 due to the detection light from the light source 21 based on a first image IM1 which is an image of the subject's eyeball EB captured by the first imaging unit 22A; calculating the position of a second corneal reflection center 122 due to the detection light based on a second image IM2 which is an image of the subject's eyeball EB captured by the second imaging unit 22B; calculating a corneal reflection distance d based on the positions of the first corneal reflection center 121 and the second corneal reflection center 122; and calculating a corneal curvature radius r of the subject's eyeball EB based on the corneal reflection distance d and a light-source-subject distance x between the light source 21 and the subject's eyeball EB.

[0060] The corneal curvature radius calculation program of this embodiment causes a computer to execute the following processes: emitting detection light from a light source 21 arranged at a predetermined position and irradiating it onto at least one eyeball EB of the subject; capturing an image of the subject's eyeball EB irradiated with the detection light using a first imaging unit 22A and a second imaging unit 22B arranged at different positions; calculating the position of a first corneal reflection center 121 due to the detection light from the light source 21 based on a first image IM1 which is an image of the subject's eyeball EB captured from the first position; calculating the position of a second corneal reflection center 122 due to the detection light based on a second image IM2 which is an image of the subject's eyeball EB captured from the second position; calculating a corneal reflection distance d based on the positions of the first corneal reflection center 121 and the second corneal reflection center 122; and calculating a corneal curvature radius r of the subject's eyeball EB based on the corneal reflection distance d and a light source-subject distance x between the light source 21 and the subject's eyeball EB.

[0061] According to the above configuration, the corneal radius of curvature r can be calculated based on the first image IM1 and the second image IM2 acquired in one frame. For example, when calculating the corneal radius of curvature using multiple images acquired across multiple frames, errors may occur if the subject moves between frames. In contrast, according to the above configuration, errors in the corneal radius of curvature r do not occur even if the subject moves during measurement. Therefore, the accuracy of calculating the corneal radius of curvature r is higher than when calculating the corneal radius of curvature using multiple images acquired across multiple frames. Therefore, the corneal radius of curvature r of the subject can be calculated with high accuracy.

[0062] In the corneal curvature radius calculation device 80 according to this embodiment, the position calculation unit 32 calculates the position of the subject's first pupil center 112A based on the first image IM1, and calculates the position of the subject's second pupil center 112B based on the second image IM2, and the distance calculation unit 33 calculates the corneal reflection distance d based on the first distance d1 between the position of the first pupil center 112A and the position of the first corneal reflection center 121, and the second distance d2 between the position of the second pupil center 112B and the second corneal reflection center 122.

[0063] According to this configuration, the distance d between the corneal reflections can be calculated appropriately by using the relative positional relationship between the pupil center and the corneal reflection center in each of the first image IM1 and the second image IM2.

[0064] The gaze detection device 100 of this embodiment includes the above-mentioned corneal curvature radius calculation device 80, and a detection processing unit 35 that detects the position of the subject's pupil center based on an image of the subject's eyeball EB captured by at least one of the first imaging unit 22A and the second imaging unit 22B of the corneal curvature radius calculation device 80, and detects the subject's gaze based on the detected position of the pupil center, the positions of the first imaging unit 22A and the second imaging unit 22B of the corneal curvature radius calculation device 80, the positions of the first corneal reflection center 121 and the second corneal reflection center 122 calculated by the position calculation unit 32 of the corneal curvature radius calculation device 80, and the corneal radius of curvature r calculated by the corneal radius of curvature calculation unit 34 of the corneal curvature radius calculation device 80.

[0065] According to the above configuration, by using the corneal radius of curvature calculation device 80, the corneal radius of curvature r can be calculated with high accuracy even if the subject moves during measurement, and therefore the gaze detection process can be performed with high accuracy.

[0066] The technical scope of the present invention is not limited to the above-described embodiments, and appropriate modifications can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0067] d, dA, dB...distance between corneal reflections, d1...first distance, d2...second distance, EB...eyeball, IM1...first image, IM2...second image, L1, L2, L3, L4, L5, 123, 173, 178...straight line, r...corneal radius of curvature, x...light source-subject distance, 10...display device, 11...display unit, 20...image acquisition device, 21...light source, 22...imaging unit, 22A...first imaging unit, 22B...second imaging unit, 30...computer system, 30A...arithmetic processing unit, 30B...storage device, 30C...computer program, 31...imaging control unit, 32...position calculation unit, 33...distance calculation output unit, 34...corneal curvature radius calculation unit, 35...detection processing unit, 36...storage unit, 40...output device, 50...input device, 60...input / output interface device, 70...light source-subject distance detection unit, 80...corneal curvature radius calculation device, 100...gaze detection device, 110...corneal curvature center, 112, 112C...pupil center, 112A...first pupil center, 112B...second pupil center, 113C, 124...corneal reflection center, 120...corneal reflection image, 120A...first corneal reflection image, 120B...second corneal reflection image, 121...first corneal reflection center, 122...second corneal reflection center, 166...intersection point

Claims

1. a light source disposed at a predetermined position and emitting detection light to illuminate at least one eyeball of the subject; a first imaging unit and a second imaging unit configured to capture images of the subject's eyeball illuminated with the detection light from different positions; a position calculation unit that calculates a position of a first corneal reflection center due to the detection light from the light source based on a first image that is an image of the eyeball of the subject captured by the first image capturing unit, and calculates a position of a second corneal reflection center due to the detection light based on a second image that is an image of the eyeball of the subject captured by the second image capturing unit; a distance calculation unit that calculates a distance between the corneal reflections based on the position of the first corneal reflection center and the position of the second corneal reflection center; a corneal curvature radius calculation unit that calculates a corneal curvature radius of the eyeball of the subject based on the corneal reflection distance and the distance between the light source and the eyeball of the subject; A corneal curvature radius calculation device comprising:

2. the position calculation unit calculates a position of a first pupil center of the subject based on the first image, and calculates a position of a second pupil center of the subject based on the second image; The distance calculation unit calculates the corneal reflection distance based on a first distance between the position of the first pupil center and the position of the first corneal reflection center and a second distance between the position of the second pupil center and the second corneal reflection center. The corneal curvature radius calculation device according to claim 1 .

3. The corneal curvature radius calculation device according to claim 1 or 2, a detection processing unit that detects a position of a pupil center of the subject based on an image of the eyeball of the subject captured by at least one of the first imaging unit and the second imaging unit of the corneal radius of curvature calculation device, and detects the line of sight of the subject based on the detected position of the pupil center, the positions of the first imaging unit and the second imaging unit of the corneal radius of curvature calculation device, the positions of the first corneal reflection center and the second corneal reflection center calculated by the position calculation unit of the corneal radius of curvature calculation device, and the corneal radius of curvature calculated by the corneal radius of curvature calculation unit of the corneal radius of curvature calculation device; A gaze detection device comprising:

4. Emitting detection light from a light source disposed at a predetermined position and irradiating it onto at least one eyeball of the subject; capturing an image of the eyeball of the subject irradiated with the detection light by a first imaging unit and a second imaging unit disposed at different positions from each other; calculating a position of a first corneal reflection center due to the detection light from the light source based on a first image that is an image of the eyeball of the subject captured by the first imaging unit, and calculating a position of a second corneal reflection center due to the detection light based on a second image that is an image of the eyeball of the subject captured by the second imaging unit; calculating a corneal reflection distance based on the position of the first corneal reflection center and the position of the second corneal reflection center; calculating a corneal radius of curvature of the subject's eyeball based on the corneal reflection distance and the distance between the light source and the subject's eyeball; A method for calculating the corneal radius of curvature.

5. A process of emitting detection light from a light source arranged at a predetermined position and irradiating it onto at least one eyeball of the subject; capturing an image of the eyeball of the subject irradiated with the detection light by a first image capturing unit and a second image capturing unit disposed at different positions; calculating a position of a first corneal reflection center due to the detection light from the light source based on a first image that is an image of the eyeball of the subject captured by the first imaging unit, and calculating a position of a second corneal reflection center due to the detection light based on a second image that is an image of the eyeball of the subject captured by the second imaging unit; A process of calculating a distance between the corneal reflections based on the position of the first corneal reflection center and the position of the second corneal reflection center; calculating a corneal radius of curvature of the subject's eyeball based on the corneal reflection distance and the distance between the light source and the subject's eyeball; A corneal curvature radius calculation program that causes a computer to execute the above.

Images