Measuring apparatus, measuring method, and program

Abstract

To provide a measuring apparatus capable of highly accurately detecting a line of sight.SOLUTION: A measuring apparatus (1000) for measuring a line-of-sight direction of a user comprises: an illumination section (11) that emits light having a specific wavelength; a light detection section (21) that has a plurality of photoelectric conversion sections (201) arranged two-dimensionally and acquires light amount distribution information on reflected light emitted by the illumination section; and a calculation processing section (51) that calculates the line-of-sight direction on the basis of the light amount distribution information. The light amount distribution information includes: information about time required for the light detection section to detect the reflected light after the illumination section emits the light or distance corresponding to the time; and information about intensity of the reflected light.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a measurement device, a measurement method, and a program. [Background technology]

[0002] Conventionally, there is known a device (e.g., an eye camera) that detects the position where a user (observer) is observing, that is, detects the so-called line of sight (visual axis). Patent Document 1 discloses a method of projecting a parallel light beam from a light source onto the anterior segment of the user's eyeball, and determining the line of sight direction using a corneal reflection image due to light reflected from the cornea and the imaging position of the pupil. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 01-274736 Summary of the Invention [Problem to be solved by the invention]

[0004] The method disclosed in Patent Document 1 cannot distinguish between light reflected from the cornea and light reflected from the glasses when the user is wearing glasses. As a result, the light reflected from the glasses may be calculated as light reflected from the cornea, making it impossible to perform high-precision gaze detection.

[0005] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a measurement device capable of detecting the line of sight with high accuracy. [Means for solving the problem]

[0006] A measurement device as one aspect of the present invention is a measurement device for measuring a user's gaze direction, the measurement device having an illumination unit that illuminates light of a specific wavelength, a plurality of photoelectric conversion units arranged in a two-dimensional shape, a light detection unit that acquires light intensity distribution information of the reflected light of the light illuminated by the illumination unit, and a calculation processing unit that calculates the gaze direction based on the light intensity distribution information, the light intensity distribution information including information on the time required from when the illumination unit illuminates the light to when the light detection unit detects the reflected light, or a distance corresponding to said time, and information on the intensity of the reflected light.

[0007] Other objects and features of the present invention will be described in the following embodiments. Effect of the Invention

[0008] According to the present invention, a measurement device capable of detecting the line of sight with high accuracy can be provided. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram of a measurement device in a first embodiment and a second embodiment. [Diagram 2] 4 is a configuration diagram of a pixel region of a light detection unit in each embodiment. FIG. [Diagram 3] 5 is an explanatory diagram of a driving pulse for a light detection unit in each embodiment. FIG. [Figure 4] 4 is a flowchart showing the operation of the measurement device in the first embodiment. [Diagram 5] 10 is a flowchart showing the operation of the measurement device in the second embodiment. [Figure 6] FIG. 11 is a block diagram of a measurement device according to a third embodiment. [Figure 7] 13 is a flowchart showing the operation of the measurement device in the third embodiment. [Figure 8] FIG. 2 is an explanatory diagram of a measurement device in each embodiment. [Figure 9] 5A and 5B are diagrams illustrating the intensity and time of reflected light from the cornea and other light in each embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In each drawing, the same reference numerals are used to refer to the same components, and duplicated explanations will be omitted.

[0011] (First embodiment) First, a measurement device 1000 according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram of the measurement device 1000. The measurement device 1000 has an illumination unit 11, a light detection unit 21, a timing control unit 31, a signal processing unit 41, an arithmetic processing unit 51, a control unit 61, a display unit 71, and a storage unit 81.

[0012] The illumination unit 11 has a light source unit 12 in which a plurality of light sources are arranged. However, this embodiment is not limited to this, and a light source unit in which only one light source is arranged may be used. The light source unit 12 emits illumination light for illuminating a subject (such as a user's eye, eyeball, or glasses). The illumination light is light whose amount of light changes over time, and is, for example, pulsed light or light modulated by a sine wave. It may be pulsed laser light of a specific wavelength, or LED pulsed light. The illumination unit 11 illuminates the subject, and the light reflected from the subject is acquired by the light detection unit 21. The illumination unit 11 may be fixed to an HMD (head mounted display) or an automobile (mobile body). The illumination unit 11 receives a light emission control signal transmitted from a timing control unit 31, and illuminates light according to the light emission control signal. In this embodiment, the light source unit 12 illuminates light of a specific wavelength with a width of, for example, several p seconds to several tens n seconds.

[0013] The light detection unit 21 receives light reflected from the subject (reflected light). The light detection unit 21 acquires an optical image of the subject formed by the optical system 22. The sensor unit 23 counts the voltage pulses output from each pixel of the image sensor (photoelectric conversion element) each time light is incident on each pixel (photon count). The number of counted photons is output to the signal processing unit 41 together with the time when the photons were counted. In this embodiment, the light detection unit 21 uses a TOF (TIME OF FLIGHT) method. When performing one light receiving operation, the sensor unit 23 is exposed for a predetermined exposure time. The sensor unit 23 receives an exposure control signal transmitted from the timing control unit 31 and is exposed according to the exposure control signal. During that time, the light source 21 irradiates at least once with a predetermined pulse width, and the sensor unit 23 receives the light.

[0014] The timing control unit 31 controls the illumination unit 11 and the light detection unit 21 so that the light detection unit 21 receives reflected light at the timing of irradiation by the illumination unit 11. The signal processing unit 41 performs processing suitable for subsequent processing on the signal output from the sensor unit 23. The processing performed by the signal processing unit 41 is, for example, signal processing such as noise removal, but is not limited thereto. The signal processing unit 41 also maps the acquisition time by combining the voltage pulse output from the light detection unit 21 with the time when the photons were counted.

[0015] The calculation processing unit 51 has a corneal ball center calculation unit 52, a pupil center calculation unit 56, and a gaze direction calculation unit 59. The corneal ball center calculation unit 52 has an intensity memory unit 53, a timing memory unit 54, and a selection processing unit 55, and calculates the position coordinates where the reflection position of the reflected light is imaged on the sensor surface of the sensor unit 23. The intensity memory unit 53 stores the intensity (number of photons) of the reflected light from the light amount distribution information of the reflected light. The timing memory unit 54 stores the time or distance from the light amount distribution information. The selection processing unit 55 judges whether the reflected light is from the cornea based on the light receiving signal output from the sensor unit 23 via the signal processing unit 41, the light receiving time, the irradiation time and the photon count number from the timing control unit 31 and the intensity based on the count number. Then, the selection processing unit 55 selects only the reflected light from the cornea from the reflected light. The corneal ball center calculation unit 52 calculates the corneal center using the above-mentioned results, and outputs the calculation result to the gaze direction calculation unit 59.

[0016] Pupil center calculation unit 56 has an image generation unit 57 and a pupil edge coordinate acquisition unit 58. Image generation unit 57 acquires an image of the periphery of the pupil. Pupil edge coordinate acquisition unit 58 acquires pupil edge coordinates using the image acquired by image generation unit 57. Pupil center calculation unit 56 calculates the center coordinates of the pupil based on the acquired image and the pupil edge coordinates, and outputs the calculation result to gaze direction calculation unit 59. Gaze direction calculation unit 59 calculates the gaze direction based on the information acquired by corneal ball center calculation unit 52 and the information acquired by pupil center calculation unit 56.

[0017] The control unit 61 comprehensively controls the operation of the measuring device 1000. The control unit 61 includes a CPU and executes a program for controlling each unit of the measuring device 1000. The control unit 61 controls the calculation processing unit 51 according to a program for controlling the illumination unit 11 and the light detection unit 21. The control unit 61 may be realized by an FPGA together with at least one of the signal processing unit 41 or the calculation processing unit 51. The display unit 71 displays the measurement results. The memory unit 81 accumulates image intensity data, time record data, and gaze direction data.

[0018] Next, the pixel region of the sensor unit 23 will be described with reference to Fig. 2. Fig. 2 is a configuration diagram of the pixel region of the sensor unit 23. In the pixel region, a plurality of SPAD pixels 103 are repeatedly arranged two-dimensionally in the XY directions. Each SPAD pixel 103 has a photoelectric conversion unit (avalanche diode) 201, a quenching element 202, a control unit 210, a counter / memory 211, and a readout unit 212.

[0019] A potential based on a potential VH higher than the voltage VL supplied to the anode is supplied to the cathode of the photoelectric conversion unit 201. A potential is supplied to the anode and cathode of the photoelectric conversion unit 201 so that a reverse bias is applied so that photons incident on the photoelectric conversion unit 201 are avalanche multiplied. By performing photoelectric conversion in a state where such a reverse bias potential is supplied, charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0020] When a reverse bias potential is supplied, and the potential difference between the anode and the cathode is greater than the breakdown voltage, the photoelectric conversion unit (avalanche diode) 201 operates in the Geiger mode. An avalanche diode that uses the Geiger mode operation to quickly detect weak signals at the single photon level is called a SPAD (Single Photon Avalanche Diode).

[0021] The control unit 210 decides whether or not to count the output signal from the photoelectric conversion unit 201. For example, the control unit 210 is a switch (gate circuit) provided between the photoelectric conversion unit 201 and the counter / memory 211. The gate of the switch is connected to the pulse line 124, and the control unit 210 is switched on and off according to a signal input to the pulse line 124. A signal based on a control signal from the timing control unit 31 is input to the pulse line 124. The gates of the switches are controlled collectively for all columns. This allows the start and end of photodetection of all the SPAD pixels 103 to be controlled collectively.

[0022] Furthermore, the control unit 210 may be configured with a logic circuit instead of a switch. For example, if an AND circuit is provided as the logic circuit, and the first input of the AND circuit is the output from the photoelectric conversion unit 201 and the second input is the signal of the pulse line 124, it becomes possible to switch whether or not to count the output signal from the photoelectric conversion unit 201. Note that the control unit 210 does not need to be provided between the photoelectric conversion unit 201 and the counter / memory 211, and may be a circuit that inputs a signal that switches between operating and non-operating the counter of the counter / memory 211.

[0023] The counter / memory 211 counts the number of photons entering the photoelectric conversion unit 201 in accordance with a control signal from the control line 213, and holds the number as digital data. The readout unit 212 is connected to the counter / memory 211 and a readout signal line 123. A control pulse is supplied to the readout unit 212 from the vertical scanning circuit unit via a control line 214, and the readout unit 212 switches whether or not to output the count value of the counter / memory 211 to the readout signal line 123. The readout unit 212 includes, for example, a buffer circuit for outputting a signal.

[0024] The readout signal line 123 may be a signal line that outputs from the photodetector 21 to the arithmetic processing unit 51, or a signal line that outputs to the signal processing unit 41. In addition, the horizontal scanning circuit unit and the vertical scanning circuit unit may be provided on the substrate on which the SPAD array is provided, or may be provided on a substrate different from the substrate on which the SPAD array is provided.

[0025] However, this embodiment is not limited to the photodetector 21 having the SPAD pixels 103, but can also be applied to a photodetector having a photoelectric conversion unit other than a SPAD.

[0026] Next, the timing at which pulsed light is emitted from the illumination unit 11, and the timing at which the pulsed light is irradiated onto the subject, the reflected light reaches the photodetection unit 21, and light is detected (photon counted) will be described with reference to Fig. 3. Fig. 3 is an explanatory diagram of the drive pulse of the photodetection unit 21. The photodetection unit 21 has a plurality of SPAD pixels 103 arranged in an array, and the light detection timing of the plurality of SPADs arranged in each row is controlled collectively for all pixels. That is, the timing of the emitted light and the timing of the count period shown in Fig. 3 are the same for all SPAD pixels in one frame.

[0027] The light detection unit 21 is a SPAD array in which SPAD pixels 103 are arranged two-dimensionally. Therefore, by using the timing chart described above, a set of data (light intensity distribution information) including light intensity distribution information (x, y) in the XY plane and time information (t) indicating the time when this light intensity distribution information was acquired can be acquired for each frame. Therefore, information regarding x, y, and t (light intensity distribution information) can be acquired.

[0028] Next, the operation (measurement method) of the measuring device 1000 in this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the operation of the measuring device 1000. Each step in Fig. 4 is executed by each unit of the measuring device 1000 mainly based on a command from the control unit 61.

[0029] First, in step S101, the illumination unit 11 illuminates a subject, such as a user's eye (eyeball), with light of a specific wavelength (infrared light) emitted from the light source unit 12 having multiple light sources. Note that the light of the specific wavelength is not limited to infrared light, and may be light of another wavelength. Next, in step S102, the light detection unit 21 detects reflected light from the subject and obtains light quantity distribution information. Here, the light quantity distribution information refers to the number of photons counted at each pixel or the image luminance value calculated from the number of counts, and the position coordinates (pixel coordinates) of each pixel.

[0030] Next, in step S103, the calculation processing unit 51 acquires the light emission time from the light source unit 12 and the light reception time at the sensor unit 23, and acquires time information from when the light emitted from the light source unit 12 is reflected by the subject to when it reaches the sensor unit 23. Note that steps S102 and S103 may be acquired simultaneously by the sensor unit 23. Next, in step S104, the calculation processing unit 51 calculates (acquires) the reflection surface position coordinates of the reflected light (image formation position information of the reflected light) based on the information acquired in step S102 and the information acquired in step S103. Next, in step S105, it is determined whether the light amount of the reflected light (the intensity of the reflected light, for example, the number of photons that reach the light detection unit 21) is equal to or greater than a predetermined threshold (intensity threshold, second threshold) from the light amount distribution information. If the amount (intensity) of the reflected light is smaller than the second threshold, the arithmetic processing unit 51 determines that the reflected light is unnecessary light (second reflected light) different from the reflected light from the cornea (first reflected light), and returns to step S102, where different light amount distribution information is acquired. On the other hand, if the arithmetic processing unit 51 determines that the amount (intensity) of the reflected light is equal to or greater than the second threshold, the arithmetic processing unit 51 proceeds to step S106. In step S106, the arithmetic processing unit 51 selects the reflected light equal to or greater than the second threshold.

[0031] Next, in step S107, the arithmetic processing unit 51 refers to the time information (arrival time) acquired in step S103 and judges whether the time information is equal to or greater than a predetermined threshold (time threshold, first threshold). Here, the time information is information on the time required from when the illumination unit 11 emits light until the light detection unit 21 detects the reflected light, or the distance corresponding to that time. If the time or distance is smaller than the first threshold, the arithmetic processing unit 51 judges that the reflected light is unnecessary light (second reflected light) different from the reflected light from the cornea (first reflected light), and returns to step S102, where different light quantity distribution information is acquired. On the other hand, if the time or distance is equal to or greater than the first threshold, the arithmetic processing unit 51 proceeds to step S108.

[0032] In step S108, the calculation processing unit 51 (selection processing unit 55) selects the reflected light whose time information is determined to be equal to or greater than the first threshold value as the reflected light from the cornea. Then, in step S109, the calculation processing unit 51 (corneal ball center calculation unit 52) ​​selects a plurality of reflected lights from the cornea and calculates the corneal center coordinates. Then, in step S110, the calculation processing unit 51 (pupil center calculation unit 56) acquires a pupil image and acquires the pupil edge coordinates. Here, the pupil image may be acquired by the sensor unit 23, or may be captured and acquired by a separate sensor. Also, the image may be captured by a sensor equipped with a TOF function, or may be acquired by using a so-called CMOS camera not equipped with a TOF function.

[0033] Next, in step S111, the arithmetic processing unit 51 calculates the pupil center coordinate. That is, the arithmetic processing unit 51 calculates the rotation angle of the eyeball based on the cornea center coordinate acquired in step S109 and the pupil edge coordinate acquired in step S110. Next, in step S112, the arithmetic processing unit 51 calculates the viewpoint coordinate and detects the user's line of sight. Next, in step S113, the arithmetic processing unit 51 stores the data and communicates with the control unit 61, the display unit 71, the storage unit 81, or the like.

[0034] Second embodiment Next, the operation (measurement method) of the measuring device 1000 in the second embodiment of the present invention will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation of the measuring device 1000 in this embodiment. Each step in Fig. 5 is executed by each unit of the measuring device 1000 mainly based on a command from the control unit 61. Note that the basic configuration of the measuring device 1000 in this embodiment is similar to that of the measuring device 1000 in the first embodiment described with reference to Fig. 1, and therefore description thereof will be omitted.

[0035] First, in step S201, the illumination unit 11 illuminates a subject such as a user's eye (eyeball) with light (infrared light) of a specific wavelength emitted from the light source unit 12 having a plurality of light sources. Then, in step S202, the light detection unit 21 detects reflected light from the subject and acquires light quantity distribution information including luminance information (intensity information) of the reflected light. Then, in step S203, the calculation processing unit 51 acquires the light emission time from the light source unit 12 and the light reception time at the sensor unit 23, and acquires time information from when the light emitted from the light source unit 12 is reflected from the subject to when it reaches the sensor unit 23. Then, in step S204, the calculation processing unit 51 (pupil center calculation unit 56) acquires pupil edge coordinates and pupil center coordinates using the pupil image acquired by the light detection unit 21. Then, in step S205, the calculation processing unit 51 determines whether the reflected light detected in step S202 is reflected light from the pupil region based on the luminance information of the image (reflected light). If it is determined that the detected reflected light is not reflected light from the pupil region, the process returns to step S202, and different light quantity distribution information is obtained. On the other hand, if it is determined that the detected reflected light is reflected light from the pupil region, the process proceeds to step S206.

[0036] In step S206, the calculation processing unit 51 calculates (obtains) the reflection surface position coordinates of the reflected light (image formation position information of the reflected light) based on the information obtained in step S202 and the information obtained in step S203. Subsequently, in step S207, it is determined whether or not the light amount (intensity of the reflected light) of the light amount distribution information is equal to or greater than a predetermined threshold (intensity threshold, second threshold). When the light amount (intensity) of the reflected light is smaller than the second threshold, the calculation processing unit 51 determines that the reflected light is unnecessary light (second reflected light), and returns to step S202, where different light amount distribution information is obtained. On the other hand, when the calculation processing unit 51 determines that the light amount (intensity) of the reflected light is equal to or greater than the second threshold, the calculation processing unit 51 proceeds to step S208. In step S208, the calculation processing unit 51 selects the reflected light equal to or greater than the second threshold.

[0037] Next, in step S209, the calculation processing unit 51 refers to the time information (arrival time) acquired in step S203 and determines whether the time information is equal to or greater than a predetermined threshold (time threshold, first threshold). If the time information is smaller than the first threshold, the calculation processing unit 51 determines that the reflected light is unnecessary light (second reflected light) and returns to step S202, where different light quantity distribution information is acquired. On the other hand, if the time information is equal to or greater than the first threshold, the calculation processing unit 51 proceeds to step S210.

[0038] In step S210, the calculation processing unit 51 (selection processing unit 55) selects the reflected light whose time information is determined to be equal to or greater than the first threshold as the reflected light from the cornea. Then, in step S211, the calculation processing unit 51 (corneal sphere center calculation unit 52) ​​selects a plurality of reflected lights from the cornea and calculates the corneal center coordinates.

[0039] Next, in step S212, the arithmetic processing unit 51 calculates the pupil center coordinate. That is, the arithmetic processing unit 51 calculates the rotation angle of the eyeball based on the cornea center coordinate acquired in step S211 and the pupil edge coordinate acquired in step S204. Next, in step S213, the arithmetic processing unit 51 calculates the viewpoint coordinate and detects the user's line of sight. Next, in step S214, the arithmetic processing unit 51 stores the data and communicates with the control unit 61, the display unit 71, the storage unit 81, or the like.

[0040] Third embodiment Next, a measurement device 1000a according to a third embodiment of the present invention will be described with reference to Fig. 6. Fig. 6 is a block diagram of the measurement device 1000a. The measurement device 1000a differs from the measurement device 1000 in that it has a light detection unit 21a instead of the light detection unit 21. Other configurations of the measurement device 1000a are similar to those of the measurement device 1000, and therefore descriptions thereof will be omitted.

[0041] The light detection unit 21a receives light reflected from the subject (reflected light). The light detection unit 21a has a distance measurement optical system 22a, a sensor unit (distance measurement sensor unit) 23a, an imaging optical system 24a, and a sensor unit (imaging sensor unit) 25a. The light detection unit 21a collects the reflected light in the distance measurement optical system 22a and forms an image on the sensor unit 23a. The light detection unit 21a outputs a signal corresponding to the light quantity distribution information obtained by the sensor unit 23a, that is, the intensity of the reflected light and the acquisition time (time information), to the signal processing unit 41. In this embodiment, the distance measurement optical system 22a and the sensor unit 23a use a TOF (TIME OF FLIGHT) method.

[0042] The imaging optical system 24a and the sensor unit 25a capture an image of the user's eyeball (eye) and obtain an image for calculating the pupil center. The light amount distribution information obtained by the sensor unit 25a is output to the signal processing unit 41. The sensor unit 25a is an imaging element (photoelectric conversion element) such as a CCD sensor or a CMOS sensor.

[0043] Next, the operation (measurement method) of the measurement device 1000a in this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the operation of the measurement device 1000a. Each step in Fig. 7 is executed by each unit of the measurement device 1000a mainly based on a command from the control unit 61.

[0044] First, in step S301, the illumination unit 11 illuminates a subject, such as a user's eye (eyeball), with light of a specific wavelength (infrared light) emitted from the light source unit 12 having a plurality of light sources. Then, in step S302, the light detection unit 21a detects reflected light from the subject using the sensor unit 23a, and obtains light quantity distribution information including luminance information (intensity information) of the reflected light. Then, in step S303, the calculation processing unit 51 obtains the light emission time from the light source unit 12 and the light reception time at the sensor unit 23a, and obtains time information from when the light emitted from the light source unit 12 is reflected from the subject to when it reaches the sensor unit 23a.

[0045] Next, in step S304, the arithmetic processing unit 51 calculates (acquires) the reflection surface position coordinates of the reflected light (image formation position information of the reflected light) based on the information obtained in step S302 and the information obtained in step S303. Next, in step S305, it is determined whether or not the light amount (intensity of the reflected light) of the light amount distribution information is equal to or greater than a predetermined threshold (intensity threshold, second threshold). If the light amount (intensity) of the reflected light is smaller than the second threshold, the arithmetic processing unit 51 determines that the reflected light is unnecessary light (second reflected light), and returns to step S302, where different light amount distribution information is obtained. On the other hand, if the arithmetic processing unit 51 determines that the light amount (intensity) of the reflected light is equal to or greater than the second threshold, the process proceeds to step S306. In step S306, the arithmetic processing unit 51 selects the reflected light equal to or greater than the second threshold.

[0046] Next, in step S307, the calculation processing unit 51 refers to the time information (arrival time) acquired in step S303 and determines whether the time information is equal to or greater than a predetermined threshold (time threshold, first threshold). If the time or distance is smaller than the first threshold, the calculation processing unit 51 determines that the reflected light is unnecessary light (second reflected light) and returns to step S302, where different light quantity distribution information is acquired. On the other hand, if the time or distance is equal to or greater than the first threshold, the calculation processing unit 51 proceeds to step S308.

[0047] In step S308, the calculation processing unit 51 (selection processing unit 55) selects the reflected light whose time information is determined to be equal to or greater than the first threshold as the reflected light from the cornea. Then, in step S309, the calculation processing unit 51 (corneal sphere center calculation unit 52) ​​selects a plurality of reflected lights from the cornea and calculates the corneal center coordinates.

[0048] In step S310, the calculation processing unit 51 (pupil center calculation unit 56) acquires a pupil image from the sensor unit 25a and acquires pupil edge coordinates. Then, in step S311, the calculation processing unit 51 calculates pupil center coordinates based on the pupil edge coordinates.

[0049] Next, in step S312, the arithmetic processing unit 51 calculates the pupil center coordinates. That is, the arithmetic processing unit 51 calculates the rotation angle of the eyeball based on the cornea center coordinates acquired in step S309 and the pupil center coordinates acquired in step S311. Next, in step S313, the arithmetic processing unit 51 calculates the viewpoint coordinates and detects the user's line of sight. Next, in step S314, the arithmetic processing unit 51 stores the data and communicates with the control unit 61, the display unit 71, the storage unit 81, or the like. Note that the processes of steps S302 to S309 and the processes of steps S310 to S311 may be performed simultaneously.

[0050] Next, a process for separating the reflected light from the cornea (first reflected light) from other unnecessary light (second reflected light) in each embodiment will be described with reference to Fig. 8 and Fig. 9. Fig. 8 is an explanatory diagram of a measurement device in each embodiment. Fig. 9 is an explanatory diagram of the intensity and time of the reflected light from the cornea (first reflected light) and other reflected light (second reflected light).

[0051] In Fig. 8, O is the center of the eyeball. R is the radius of the cornea, which is about 7 to 8 mm for an average human eye. The pulsed light emitted from the light source unit 12 is reflected by the eyeball unit 103 and the glasses unit 104 to reach the light detection unit 21 (21a). In the glasses unit 104, there exists light that is reflected once by the surface B on the light source unit 12 side and the surface D on the eyeball side, and light that is reflected multiple times internally to reach the light detection unit 21 (21a).

[0052] Of these light beams, the light reflected by the corneal surface E of the eyeball portion 103 has a longer optical path length than the light reflected by the surface B on the light source side and the surface D on the eyeball side of the eyeglass portion 104, and therefore arrives at the light detection unit 21 (21a) later. On the other hand, the light that is reflected multiple times and finally reflects from the surface F of the eyeglass portion 104 to reach the detector 201 arrives at the same time as the light reflected by the corneal surface E of the eyeball portion 103, or arrives later. The light that is reflected multiple times by the eyeglass portion 104 reaches the light detection unit 21 (21a) in a smaller number of photons than the light reflected by the surface E of the eyeball portion 103, and the image intensity is weakened.

[0053] Generally, the reflection from the eyeball is about 2 to 3%, and the reflectance of ordinary glasses made of glass or plastic lenses is often 2% or less even in the near infrared region. Therefore, the more times the reflection occurs, the fewer the number of photons that reach the light detection unit 21 (21a), and the weaker the image intensity becomes. Note that in FIG. 8, the glasses unit 104 is shown as a reflection of unwanted light, but the cause of unwanted light is not limited to this. Another possible cause of unwanted light is, for example, the eyepiece or liquid crystal panel of an electronic viewfinder placed in front of the eyeball 103.

[0054] FIG. 9 shows a graph in which the light detection unit 21 (21a) detects the unwanted light (second reflected light) reflected by the glasses unit 104 or the eyepiece unit and the normal light (first reflected light) from the cornea for gaze detection. In FIG. 9, the vertical axis indicates the intensity or the number of photons, and the horizontal axis indicates the time or distance from the light source unit 12 to the light detection unit 21 (21a). As shown in FIG. 9, a threshold value (intensity threshold, second threshold value) for the intensity (brightness) of the reflected light and a threshold value (time threshold, first threshold value) for the arrival time are set. This makes it possible to determine whether the light detected by the light detection unit 21 (21a) is the reflected light (first reflected light) from the cornea or the unwanted reflected light (second reflected light).

[0055] In each embodiment, it is preferable that the light of a specific wavelength emitted by the light source unit 12 of the illumination unit 11 satisfies the following conditional expression (1), where the central wavelength of the light of the specific wavelength is λ (nm).

[0056] 600<λ<1500 (1) Conditional formula (1) defines the wavelength of the light from the light source unit 12. It is preferable that conditional formula (1) is satisfied, since the light is difficult for the user to recognize.

[0057] More preferably, the numerical range of conditional expression (1) is set as shown in the following conditional expression (1a).

[0058] 650<λ<1200 (1a) More preferably, the numerical range of conditional formula (1a) is set as in the following conditional formula (1b).

[0059] 680<λ<900 (1b) The measuring device of each embodiment is not limited to being installed in a gaze detection device. For example, it may be installed in a moving object such as an HMD (head mounted display) or an automobile. The measuring device of each embodiment can also be applied to an ophthalmic device such as a fundus camera or an ophthalmic examination system.

[0060] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0061] According to each embodiment, it is possible to provide a measurement device, a measurement method, and a program capable of detecting the gaze direction with high accuracy.

[0062] The disclosure of each embodiment includes the following configurations and methods.

[0063] (Configuration 1) A measurement device for measuring a user's gaze direction, An illumination unit that illuminates with light of a specific wavelength; a light detection unit having a plurality of photoelectric conversion units arranged two-dimensionally and acquiring light quantity distribution information of the reflected light of the light illuminated by the illumination unit; a calculation processing unit that calculates the line of sight direction based on the light quantity distribution information, a light intensity distribution information including information regarding a time required from when the illumination unit illuminates the light to when the light detection unit detects the reflected light, or a distance corresponding to said time, and information regarding an intensity of the reflected light. (Configuration 2) The arithmetic processing unit is storing the light quantity distribution information; determining whether the reflected light is a first reflected light or an unwanted second reflected light from a user's cornea based on the light amount distribution information; 2. The measuring device according to configuration 1, wherein the line of sight direction is calculated based on the first reflected light. (Configuration 3) 3. The measurement device according to configuration 2, wherein the calculation processing unit determines that the reflected light is the second reflected light when the time or the distance is smaller than a first threshold value. (Configuration 4) The measurement device according to configuration 2 or 3, wherein the arithmetic processing unit determines that the reflected light is the second reflected light when the intensity is smaller than a second threshold value. (Configuration 5) 5. The measuring device according to any one of configurations 1 to 4, further comprising a timing control unit that controls the irradiation timing of the illumination unit and the detection timing of the light detection unit. (Configuration 6) 5. The measuring device according to any one of configurations 1 to 4, wherein the illumination unit has a plurality of light sources. (Configuration 7) 7. The measuring device according to any one of configurations 1 to 6, further comprising a counter that counts light incident on each of the plurality of photoelectric conversion units. (Configuration 8) When the central wavelength of the specific wavelength is λ (nm), 600<λ<1500 8. The measuring apparatus according to any one of configurations 1 to 7, wherein the following conditional expression is satisfied: (Configuration 9) The measurement device according to any one of configurations 1 to 8, wherein the calculation processing unit calculates a corneal ball center position based on the light intensity distribution information, and calculates the line of sight direction based on the corneal ball center. (Configuration 10) 10. The measurement device according to any one of configurations 1 to 9, wherein the calculation processing unit calculates a pupil center position based on the light amount distribution information, and calculates the line of sight direction based on the pupil center position. (Configuration 11) 11. The measuring device according to any one of configurations 1 to 10, wherein the light emitted by the illumination unit is pulsed light. (Configuration 12) 12. The measuring device according to any one of configurations 1 to 11, wherein each of the plurality of photoelectric conversion units is an avalanche diode. (Configuration 13) The measurement device according to any one of configurations 1 to 12, characterized in that the operations of starting irradiation by the illumination unit and starting light detection by the light detection unit are each performed multiple times with a fixed period from the start of irradiation to the start of light detection. (Configuration 14) The measuring device according to any one of configurations 1 to 13, wherein the measuring device is provided in an HMD. (Configuration 15) The measuring device according to any one of configurations 1 to 13, wherein the measuring device is provided in a fundus camera. (Configuration 16) The measuring device according to any one of configurations 1 to 13, characterized in that the measuring device is provided on a moving body. (Method 1) A method for measuring a user's gaze direction, comprising: Illuminating with light of a specific wavelength using an illumination unit; acquiring light quantity distribution information of the reflected light of the light emitted by the illumination unit using a light detection unit having a plurality of photoelectric conversion units arranged two-dimensionally; calculating the line of sight direction based on the light quantity distribution information; A measurement method characterized in that the light quantity distribution information includes information regarding the time required from when the illumination unit emits the light to when the light detection unit detects the reflected light, or the distance corresponding to said time, and information regarding the intensity of the reflected light. (Configuration 17) A program for causing a computer to execute the measurement method according to Method 1.

[0064] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. In addition, at least a part of each embodiment may be combined. [Explanation of symbols]

[0065] 11 Lighting Section 21, 21a Light detection unit 51 Processing unit 201 Photoelectric conversion unit 1000, 1000a Measuring device

Claims

1. A measurement device for measuring a user's gaze direction, an illumination unit that emits light of a specific wavelength; a light detection unit having a plurality of photoelectric conversion units arranged two-dimensionally and acquiring light intensity distribution information of reflected light of the light emitted by the illumination unit; a calculation processing unit that calculates the line of sight direction based on the light quantity distribution information, the light quantity distribution information includes information on a time required from when the illumination unit emits the light until when the light detection unit detects the reflected light or a distance corresponding to the time, and information on an intensity of the reflected light, The arithmetic processing unit storing the light intensity distribution information; determining whether the reflected light is a first reflected light or an unwanted second reflected light from the user's cornea based on the light intensity distribution information; A measuring device that calculates the line of sight direction based on the first reflected light.

2. The measurement device according to claim 1 , wherein the calculation processing unit determines that the reflected light is the second reflected light when the time or the distance is smaller than a first threshold value.

3. The measuring device according to claim 1 , wherein the arithmetic processing unit determines that the reflected light is the second reflected light when the intensity is smaller than a second threshold value.

4. 2. The measuring device according to claim 1, further comprising a timing control unit that controls the irradiation timing of the illumination unit and the detection timing of the light detection unit.

5. 2. The measuring device according to claim 1, wherein the illumination unit has a plurality of light sources.

6. 2. The measuring device according to claim 1, further comprising a counter that counts the amount of light incident on each of the plurality of photoelectric conversion units.

7. When the center wavelength of the specific wavelength is λ (nm), 600<λ<1500 2. The measuring apparatus according to claim 1, wherein the following condition is satisfied:

8. 2. The measurement device according to claim 1, wherein the calculation processing unit calculates the position of the center of the cornea based on the light intensity distribution information, and calculates the line of sight based on the center of the cornea.

9. 2. The measurement device according to claim 1, wherein the calculation processing unit calculates a pupil center position based on the light intensity distribution information, and calculates the gaze direction based on the pupil center position.

10. 2. The measuring device according to claim 1, wherein the light emitted by the illumination unit is pulsed light.

11. 2. The measuring device according to claim 1, wherein each of the plurality of photoelectric conversion units is an avalanche diode.

12. 2. The measuring device according to claim 1, wherein the operations of starting irradiation of the illumination unit and starting light detection of the light detection unit are each performed multiple times with a fixed period from the start of irradiation to the start of light detection.

13. The measuring device according to any one of claims 1 to 12, wherein the measuring device is provided in an HMD.

14. The measuring device according to any one of claims 1 to 12, characterized in that the measuring device is provided in a fundus camera.

15. The measuring device according to any one of claims 1 to 12, characterized in that the measuring device is provided on a moving body.

16. A method for measuring a user's gaze direction, comprising: an illumination step of illuminating with light of a specific wavelength using an illumination unit; an acquisition step of acquiring light intensity distribution information of reflected light of the light emitted by the illumination unit using a light detection unit having a plurality of photoelectric conversion units arranged two-dimensionally; a calculation step of calculating the line of sight direction based on the light quantity distribution information, the light quantity distribution information includes information on a time required from when the illumination unit emits the light until when the light detection unit detects the reflected light or a distance corresponding to the time, and information on an intensity of the reflected light, The calculation step includes: storing the light intensity distribution information; determining whether the reflected light is a first reflected light or an unwanted second reflected light from the user's cornea based on the light intensity distribution information; and calculating the line of sight direction based on the first reflected light.

17. A program causing a computer to execute the measurement method according to claim 16.

Images