Optical device, line of sight detection device, retinal projection display device, head-mounted display device, optometry device, three-dimensional object inclination detection method, and line of sight detection method

Abstract

To reduce detection error when light reflected by an object is largely misaligned.SOLUTION: An optical device according to one aspect of the present invention includes: a light source for projecting light to an object; detection means for detecting the position of the light reflected by the object; output means for outputting inclination information of the object, which is obtained on the basis of the light position and a given parameter; and change means for changing the parameter on the basis of the light position.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an optical device, a gaze detection device, a retinal projection display device, a head-mounted display device, an eye examination device, a method for detecting the inclination of a three-dimensional object, and a method for detecting the gaze of a user. [Background technology]

[0002] 2. Description of the Related Art Optical devices, such as devices that optically detect the inclination of an object, such as an eyeball, are known.

[0003] As such an optical device, a configuration has been disclosed in which light is irradiated onto an eyeball and eye movement is detected based on the light reflected by the eyeball (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the configuration of Patent Document 1, there is a concern that detection errors will become large when the light reflected by an object such as an eyeball is significantly displaced, and there is room for improvement.

[0005] An object of the present invention is to reduce detection errors when light reflected by an object is significantly deviated. [Means for solving the problem]

[0006] An optical device according to one aspect of the present invention includes a light source for irradiating a target object with light, a detection unit for detecting a position of the light reflected by the target object, and a sensor for detecting the position of the light, information on the size or shape of an eyeball, or information on the position of the center of rotation of the eyeball. News the tilt information of the object obtained based on predetermined parameters including: an output means for outputting tilt information of the object obtained based on the position of the light; a modification means for modifying the parameters based on the position of the light; and a storage means for storing a detection value of the position of the light detected by the detection means, wherein the modification means modifies the parameters based on a center of gravity position of the detection values ​​constituting a cluster and a range of change of the detection values ​​constituting the cluster among a plurality of detection values ​​stored in the storage means. Effect of the Invention

[0007] According to the present invention, it is possible to reduce detection errors when the light reflected by the object is significantly shifted. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the configuration of a gaze detection device according to a first embodiment. [Diagram 2] 10A and 10B are diagrams showing an example of the relationship between the inclination of the eyeball and the incident position of the laser light on the PSD, where (a) is a diagram showing the case where the eyeball is not inclined, and (b) is a diagram showing the case where the eyeball is inclined. [Diagram 3] 3 is a block diagram of an example of a hardware configuration of a processing unit according to the first embodiment. FIG. [Figure 4] 3 is a block diagram illustrating an example of a functional configuration of a processing unit according to the first embodiment. FIG. [Diagram 5] 11 is a diagram showing an example of a method for identifying the inclination direction of the eyeball from the PSD incidence position. [Figure 6] FIG. 2 is a diagram showing an example of the relationship between the gaze direction of the eyeball and fixational eye movement. [Figure 7] FIG. 13 is a diagram showing an example of a change in the inclination of an eyeball. [Figure 8] FIG. 13 is a diagram showing an example of distribution of arrival positions of laser light on a PSD. [Figure 9] 13A to 13C are graphs showing an example of the correlation between the average and standard deviation of detection values, where (a) is a graph for 5 detection values, (b) is a graph for 10 detection values, and (c) is a graph for 15 detection values. [Figure 10] FIG. 13 is a diagram illustrating an example of a parameter change operation. [Figure 11] 4 is a flowchart of an operation example of the gaze detection device according to the first embodiment. [Figure 12] 4 is a flowchart of an example of a change operation by the gaze detection device according to the first embodiment. [Figure 13] FIG. 11 is a diagram showing an example of the configuration of a retinal projection display device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same components are denoted by the same reference numerals, and duplicated explanations may be omitted.

[0010] Furthermore, the embodiments shown below are illustrative of an optical device for embodying the technical ideas of the present invention, and the present invention is not limited to the embodiments shown below. Unless otherwise specified, the shapes of the components described below, their relative positions, parameter values, etc. are intended to be illustrative and not to limit the scope of the present invention. Furthermore, the sizes and positional relationships of the components shown in the drawings may be exaggerated to clarify the explanation.

[0011] The optical device according to the embodiment is a device for detecting the inclination of an object. The object is, for example, a human eyeball, and the optical device is a gaze detection device that detects the gaze direction, which is the direction of the human's gaze. Since the eyeball inclines in the direction in which the human gaze is directed, the gaze detection device detects the gaze direction by detecting the inclination of the eyeball. In other words, the inclination of the eyeball is the rotation angle of the rotating eyeball.

[0012] Information on the gaze direction detected by the gaze detection device is used, for example, in an eye tracking device or an eye examination device, etc. Alternatively, the information is used to correct the position of a projected image or the contents of an image according to the inclination of the eyeball when an image is projected onto the retina or the like in a head-mounted display device such as a retinal projection display device or a head mounted display (HMD).

[0013] The optical device according to the embodiment irradiates an object with light, detects the position of the light reflected by the object, and outputs tilt information of the object obtained based on the position of the light and a predetermined parameter. Here, the parameter refers to information used when calculating and obtaining the tilt information of the object from the detected value of the position of the light. The parameter includes, for example, information on the size or shape of the eyeball, or position information of the center of rotation of the eyeball.

[0014] It is conceivable to always use fixed parameters according to the general or average size, shape, position of the center of rotation, etc. of the eyeball as the parameters. However, due to individual differences, the size or shape of the eyeball may deviate significantly from the general parameters, or the mounting position of the optical device may deviate significantly from the specified position, causing the light reflected by the object to deviate significantly from the desired position. As a result, the use of fixed parameters may result in a large error in detecting the inclination of the object.

[0015] In the embodiment, the above parameters are changed based on the position of light reflected by an object such as an eyeball. For example, the above parameters are changed based on the center of gravity of detection values ​​constituting a cluster among a plurality of detection values ​​obtained by detecting the positions of light reflected by an object, and the variation range of the detection values ​​constituting the cluster. Note that a cluster means a set of detection values ​​constituted by a plurality of detection values.

[0016] For example, the human eyeball constantly performs an unconscious micro-movement called fixational eye movement. The position of the light reflected by the eyeball changes with a distribution according to the characteristics of fixational eye movement. The characteristics of fixational eye movement also differ according to individual differences in the size or shape of the eyeball, or the misalignment of the optical device.

[0017] Therefore, among the multiple detection values ​​obtained by detecting the position of the light, a feature quantity indicating the characteristics of fixational eye movement can be extracted using the center of gravity of the detection values ​​that make up a cluster and the range of change of the detection values ​​that make up the cluster.

[0018] By changing the parameters based on the extracted feature amount, it is possible to compensate for the effects of deviations in the size or shape of the eyeball, deviations in the mounting position of the optical device, etc. This makes it possible to reduce detection errors even when the light reflected by the target object is significantly deviated.

[0019] Hereinafter, an embodiment will be described using a gaze detection device mounted on a glasses-type support body and detecting the inclination of the eyeball of a person wearing the glasses-type support body as the gaze direction as an example of an optical device. The human eyeball is an example of an object. The gaze direction is also an example of the inclination of the object. Note that in the embodiment, a human right eyeball is exemplified, but the same applies to the left eyeball. Two gaze detection devices can also be applied to each of the eyeballs of both eyes.

[0020] In the figures shown below, for convenience of explanation, the horizontal direction perpendicular to the vertical direction is defined as the X-axis direction, the vertical direction is defined as the Y-axis direction, and the direction perpendicular to both the X-axis direction and the Y-axis direction is defined as the Z-axis direction. The Z-axis direction is approximately the same as the direction in which the eyeball is viewed normally. However, these do not limit the position and orientation of the gaze detection device, and the gaze detection device can be placed in any position and orientation.

[0021] [First embodiment] <Configuration example of gaze detection device 10> First, a configuration of the gaze detection device 10 according to the first embodiment will be described.

[0022] As shown in FIG. 1, the gaze detection device 10 includes a VCSEL (Vertical Cavity Surface Emitting Laser) 1, a plane mirror 2, a PSD (Position Sensitive Detector) 3, and a processing unit 100.

[0023] The eyeglass-type support 20 has an eyeglass frame 21 and an eyeglass lens 22, and the VCSEL 1, the plane mirror 2, and the PSD 3 are each disposed in the eyeglass frame 21. The processing unit 100 can be disposed in any position. Although Fig. 1 illustrates a configuration in which the processing unit 100 is disposed outside the eyeglass-type support 20, the processing unit 100 can also be disposed inside the eyeglass frame 21.

[0024] The VCSEL1 is an example of a light source that irradiates the eyeball 30 with laser light. The VCSEL1 has multiple light-emitting units arranged two-dimensionally in a plane. Each of the multiple light-emitting units emits laser light toward the negative Z-axis direction. The laser light emitted by each light-emitting unit of the VCSEL1 is an example of light irradiated by a light source. Note that "multiple light-emitting units" is synonymous with "multiple light-emitting points" or "multiple light-emitting elements."

[0025] However, the light source is not limited to a VCSEL as long as it can irradiate light from multiple light-emitting units to the eyeball 30. For example, the light source may be configured by arranging multiple LDs (semiconductor lasers; laser diodes) or multiple LEDs (light-emitting diodes; light emitting diodes) two-dimensionally in a plane. The light source may also be configured by combining multiple types of light sources. Furthermore, if the gaze detection range is not expanded using multiple light-emitting units (described later), the light source may have only one light-emitting unit such as an LD.

[0026] The light emitted by the light source may be coherent light such as laser light, or incoherent light. However, it is more preferable to use laser light having directionality, since this makes it easier to guide the light to the eyeball 30.

[0027] The light may be a continuous wave (CW) light or a pulsed light. There is no particular limitation on the wavelength of the light emitted by the light source, but it is more preferable to use a non-visible light wavelength such as near-infrared light, since this does not impede human visibility.

[0028] The plane mirror 2 is a light guiding means that guides the laser light emitted by the VCSEL 1 to the eyeball 30 by reflecting the laser light toward the eyeball 30. The laser light reflected by the plane mirror 2 is incident on the vicinity of the pupil 31 of the eyeball 30. The inclinations of the VCSEL 1 and the plane mirror 2 are adjusted so that the laser light is incident at a predetermined angle on the center of the pupil 31 of the eyeball 30 in a normal viewing state.

[0029] However, the light guiding means is not limited to the flat mirror 2. For example, the light guiding means may be composed of one or a combination of two or more of a convex lens, a microlens array, a concave curved mirror, a hologram diffraction element, a prism array, or a diffraction grating. By optimizing the configuration of the light guiding means, it is possible to obtain effects such as an expansion of the gaze detection range, a miniaturization of the gaze detection device 10, and a reduction in the assembly load.

[0030] Furthermore, the gaze detection device 10 may not be provided with a light guiding means and may allow the light emitted by the VCSEL 1 to be directly incident on the eyeball 30.

[0031] The pupil surface (corneal surface) of the eyeball 30 is a transparent body that contains water and generally has a reflectance of about 2 to 4%. The laser light incident on the vicinity of the pupil 31 of the eyeball 30 is reflected by a reflection point P on the pupil surface of the eyeball 30, and forms a beam spot on the light receiving surface of the PSD 3.

[0032] The PSD 3 is an example of a detection means for detecting the position of the laser light reflected by the eyeball 30. The PSD 3 is a two-dimensional light position detection element that has a light receiving surface and four output terminals, and outputs a detection signal indicating the position of the incident light in two orthogonal directions on the light receiving surface. The PSD 3 detects a current value according to the distance to an electrode of the laser light that forms a beam spot on the light receiving surface, and outputs a detection signal according to the ratio of the current values ​​in the two orthogonal directions.

[0033] The light-receiving surface is a continuous plane that is not divided into pixels, and includes a resistive film formed on the surface and electrode pairs in two perpendicular directions. The photocurrent generated at the beam spot position on the light-receiving surface is divided into four according to the distance to each output terminal. At this time, the electrical resistance of the resistive film acts so that the current becomes smaller as the distance between the beam spot position and the output terminal becomes longer.

[0034] The PSD3 detects the electrical signal that passes through the resistive film via four terminals, and outputs a detection signal that indicates the position on the light receiving surface obtained through electrical post-processing. The PSD3 also converts the current generated by photoelectric conversion into an analog voltage signal, which can be output as a detection signal from the four terminals. In other words, the PSD3 can detect the incident position by using the surface resistance to determine the distance to each terminal.

[0035] An image sensor (imaging element) including multiple pixels may be used as the detection means instead of the PSD 3. Also, the PSD may be not only a two-dimensional PSD that detects the two-dimensional position of the laser light on the light receiving surface, but also a one-dimensional PSD that detects the one-dimensional position.

[0036] However, in an image sensor, if the light intensity of the received laser light is smaller than the ambient light such as sunlight, the detection accuracy of the position of the laser light may decrease. If the output of the VCSEL1 is increased to suppress the decrease in the detection accuracy, the light intensity of the laser light incident on the eyeball 30 increases, which is not preferable from the viewpoint of safety.

[0037] Furthermore, in the case of an image sensor, image processing for position detection may cause detection errors due to calculations or increase the processing load.

[0038] By using the PSD 3 as the detection means, the position can be detected by the current ratio divided between the output terminals, so the effect of the intensity of the received laser light on the position detection accuracy is small. Therefore, it is not necessary to increase the intensity of the laser light incident on the eye 30 in order to suppress the effect of ambient light. Therefore, the intensity of the laser light incident on the eye 30 can be suppressed, which is advantageous from the viewpoint of safety.

[0039] Furthermore, since the position can be detected without performing image processing, it is advantageous in that the detection error and processing load associated with image processing can be reduced.

[0040] The position of the beam spot formed on the light receiving surface of the PSD 3 by the light reflected by the eyeball 30 changes depending on the inclination of the eyeball 30. The PSD 3 outputs a detection signal to the processing unit 100 according to the position of the beam spot.

[0041] The processing unit 100 converts the detection signal from the PSD 3 into coordinate information, and outputs information on the line of sight direction obtained by calculation based on the coordinate information indicating the position of the laser light and predetermined parameters.

[0042] In other words, the PSD 3 detects the direction of the normal vector of the reflection point on the eyeball 30, i.e., the three-dimensional shape. The processing unit 100 can output tilt information of the eyeball 30 acquired by estimation calculation based on the correspondence between the three-dimensional shape detected by the PSD 3 and a surface shape model of the eyeball 30. This surface shape model of the eyeball 30 is an example of a parameter, and includes information on the size or shape of the eyeball, information on the position of the center of rotation of the eyeball, etc.

[0043] 1 illustrates a configuration in which the eyeglass frame 21 is used as a holding member for the VCSEL 1 and other components, but the present invention is not limited to this. The VCSEL 1 and other components may also be held by a member that can be worn around the entire head, such as a hat or headgear.

[0044] <Example of relationship with eye movement> Here, the eyeball 30 performs eye movement such as rotation. The rotation causes the eyeball 30 to tilt. If the direction of the laser light reflected by the eyeball 30 changes significantly due to the tilt of the eyeball 30, the laser light will no longer be incident on the light receiving surface of the PSD 3, and as a result, the PSD 3 may not be able to detect the position of the reflected light.

[0045] In this embodiment, the light-emitting units in the VCSEL 1 are sequentially or selectively changed to prevent the laser light reflected by the eye 30 from failing to enter the light-receiving surface of the PSD 3. As a result, even if the eye 30 is significantly tilted, the laser light reflected by the eye 30 is maintained in a state where it enters the light-receiving surface of the PSD 3, thereby expanding the gaze detection range.

[0046] Fig. 2 is a diagram illustrating an example of the relationship between the inclination of the eyeball 30 and the incident position of the laser light on the PSD 3. Fig. 2(a) illustrates a case where the eyeball 30 is not inclined (when viewing normally), and Fig. 2(b) illustrates a case where the eyeball 30 is inclined.

[0047] Fig. 2 shows the propagation of laser light emitted from two light-emitting portions in the VCSEL 1. In Fig. 2, laser light 1a emitted from one light-emitting portion is shown by a dotted line, and laser light 1b emitted from the other light-emitting portion is shown by a dashed-dotted line.

[0048] 2(a), after being reflected by the eyeball 30, the laser beam 1a is incident on the vicinity of the center of the light receiving surface of the PSD 3. In this state, the PSD 3 can detect a change in the incident position of the laser beam 1a on the light receiving surface in accordance with the inclination of the eyeball 30, and the gaze detection device 10 can detect the inclination of the eyeball 30 based on the detection signal of the PSD 3, and detect the gaze direction.

[0049] On the other hand, after being reflected by the eyeball 30, the laser beam 1b is not incident on the light receiving surface of the PSD 3. In this state, the PSD 3 cannot detect the position of the laser beam 1b, and therefore the gaze detection device 10 cannot detect the inclination of the eyeball 30 based on the detection signal of the PSD 3, and therefore cannot detect the gaze direction.

[0050] 2(b), when the eyeball 30 is significantly tilted, the laser light 1a is not incident on the light receiving surface of the PSD 3 after being reflected by the eyeball 30. In this state, the PSD 3 cannot detect the position of the laser light 1a, and therefore the gaze detection device 10 cannot detect the tilt of the eyeball 30 and cannot detect the gaze direction.

[0051] Meanwhile, after being reflected by the eyeball 30, the laser beam 1b is incident on the vicinity of the center of the light receiving surface of the PSD 3. In this state, the PSD 3 can detect a change in the incident position of the laser beam 1b on the light receiving surface in accordance with the inclination of the eyeball 30, and the gaze detection device 10 can detect the inclination of the eyeball 30 and detect the gaze direction based on the detection signal of the PSD 3.

[0052] In this way, with the laser light emitted from only one light emitting portion in the VCSEL 1, the inclination of the eyeball 30 can only be detected within a limited angular range, and the gaze direction cannot be detected.

[0053] In contrast, in the embodiment, the light emitting section of the VCSEL 1 is changed in response to the inclination of the eyeball 30, and the incident angle of the laser light to the eyeball 30 is changed.

[0054] For example, one light emitting section in the VCSEL1 is made to emit light, and it is determined whether or not the PSD3 outputs a detection signal. If it is not being output, a different light emitting section in the VCSEL1 is made to emit light, and it is again determined whether or not the PSD3 outputs a detection signal. This operation is repeated until the PSD3 outputs a detection signal. This allows the laser light reflected by the eyeball 30 to be incident on the light receiving surface of the PSD3.

[0055] Even when the eyeball 30 is tilted significantly, the laser light reflected by the eyeball 30 is always kept incident on the light receiving surface of the PSD3, making it possible to detect the tilt of the eyeball 30 based on the detection signal of the PSD3, thereby expanding the gaze detection range.

[0056] The operation of changing the light emitting portion of the VCSEL 1 does not necessarily have to be performed in response to eye movement such as tilt. For example, the gaze detection device 10 may sequentially cause the light emitting portion of the VCSEL 1 to emit light at a predetermined time interval independent of eye movement, and may set the laser light reflected by the eye 30 to be incident on the light receiving surface of the PSD 3 based on the detection signal of the PSD 3 at that time.

[0057] 2 illustrates only laser light emitted from two light-emitting elements for ease of explanation, however, the gaze detection device 10 can utilize more light-emitting elements of the VCSEL 1 in accordance with the eye movement of the eye 30. In this case, the gaze detection device 10 appropriately selects the number and positions of the light-emitting elements of the VCSEL 1 in accordance with the size of the light-receiving surface of the PSD 3 and the size of the eye 30 so that the tilt of the eye 30 can be appropriately detected.

[0058] <Example of Hardware Configuration of Processing Unit 100> Next, a hardware configuration of the processing unit 100 will be described with reference to Fig. 3. Fig. 3 is a block diagram illustrating an example of a hardware configuration of the processing unit 100.

[0059] 3, the processing unit 100 has a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an SSD (Solid State Drive) 104. The processing unit 100 also has a light source driving circuit 105, a signal generating circuit 106, an A / D (Analog / Digital) conversion circuit 107, and an input / output I / F (Interface) 108. These are connected via a system bus B so as to be able to transmit and receive data or signals to and from each other.

[0060] The CPU 101 is a calculation device that reads out programs and data from a storage device such as the ROM 102 or the SSD 104 onto the RAM 103 and executes the programs to control the entire processing unit 100 and realize functions to be described later. Note that some or all of the functions of the CPU 101 may be realized by an electronic circuit such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0061] The ROM 102 is a non-volatile semiconductor memory (storage device) capable of retaining programs and data even when the power is turned off. The ROM 102 stores a BIOS (Basic Input / Output System) executed when the processing unit 100 is started, programs such as OS settings and network settings, and data. The RAM 103 is a volatile semiconductor memory (storage device) that temporarily retains programs and data.

[0062] The SSD 104 is a non-volatile memory that stores various data and programs for executing processes by the processing unit 100. The SSD may be a hard disk drive (HDD).

[0063] The light source drive circuit 105 is an electric circuit that is electrically connected to the VCSEL 1 and outputs a drive voltage to the VCSEL 1 in accordance with a control signal. The light source drive circuit 105 can drive a plurality of light emitting units included in the VCSEL 1 to emit light simultaneously or sequentially.

[0064] The driving voltage can have a voltage waveform of a square wave, a sine wave, or a predetermined waveform shape, and the light source driving circuit 105 can change the period (frequency) of these voltage waveforms to modulate the period of the driving voltage.

[0065] The signal generating circuit 106 is an electric circuit that generates an electric signal with a predetermined period. As the signal generating circuit 106, a multi-channel signal generator that can generate a plurality of electric signals with different periods and output the signals in parallel to a plurality of output destinations can be used.

[0066] The A / D conversion circuit 107 is an electric circuit that is electrically connected to the PSD 3 and outputs a digital voltage signal obtained by A / D converting an analog voltage signal output by the PSD 3. The detection value of the PSD 3 can be obtained from the digital voltage signal output by the A / D conversion circuit 107.

[0067] The input / output I / F 108 is an interface for connecting to external devices such as a PC (Personal Computer) and video equipment.

[0068] [First embodiment] <Example of functional configuration of processing unit 100> Next, the functional configuration of the processing unit 100 will be described with reference to Fig. 4. Fig. 4 is a block diagram illustrating an example of the functional configuration of the processing unit 100.

[0069] As shown in FIG. 4, the processing unit 100 includes a light emission control unit 111, a determination unit 112, an acquisition unit 113, a memory unit 114, an extraction unit 115, a change unit 116, a storage unit 117, an estimation unit 118, and an output unit 119.

[0070] Each of these units is a function or a means for performing a function that is realized when any of the components shown in Fig. 3 operates in response to an instruction from CPU 101 in accordance with a program loaded from ROM 102 onto RAM 103. Note that Fig. 4 shows the main components of processing unit 100, but processing unit 100 may have other components.

[0071] The light emission control unit 111 and the determination unit 112 cooperate to cause the laser light reflected by the eyeball 30 to be incident on the light receiving surface of the PSD 3. Specifically, the light emission control unit 111 selects one of the multiple light emitting units included in the VCSEL 1 and causes it to emit light.

[0072] The determination unit 112 receives a detection signal from the PSD 3 and determines whether or not the laser light reflected by the eyeball 30 has been incident on the light receiving surface of the PSD 3. For example, if the voltage or current of the detection signal from the PSD 3 is equal to or greater than a predetermined threshold, it determines that the laser light has been incident, and if the voltage or current of the detection signal from the PSD 3 is less than the predetermined threshold, it determines that the laser light has not been incident.

[0073] If it is determined that the laser light has not been incident, the light emission control unit 111 changes the light emitting unit of the VCSEL 1 to another light emitting unit. Then, the light emission control unit 111 repeats changing the light emitting unit of the VCSEL 1 until it is determined that the laser light has been incident. In this way, the light emission control unit 111 and the determination unit 112 can cause the laser light reflected by the eyeball 30 to be incident on the light receiving surface of the PSD 3.

[0074] After the laser light reflected by the eyeball 30 is incident on the light receiving surface of the PSD 3 , the determination unit 112 outputs a detection signal of the PSD 3 to the acquisition unit 113 .

[0075] The acquisition unit 113 A / D converts the analog voltage detection signal of the PSD 3 input via the determination unit 112, and acquires a digital voltage detection value that indicates the position of light on the light receiving surface of the PSD 3. The acquisition unit 113 stores the acquired detection value in the storage unit 114.

[0076] The storage unit 114 is an example of a storage means that sequentially stores the detection values ​​acquired by the acquisition unit 113. The function of the storage unit 114 is realized by the SSD 104 or the like.

[0077] The extraction unit 115 extracts a feature amount indicating the characteristics of fixational eye movement by calculating the center of gravity position of the detection values ​​constituting a cluster and the change width of the detection values ​​constituting the cluster from among the multiple detection values ​​stored in the storage unit 114. The extraction unit 115 outputs data of the extracted feature amount to the modification unit 116.

[0078] The modification unit 116 is an example of a modification means that modifies the parameters stored in the storage unit 117 based on the feature amount input from the extraction unit 115. The parameters are information indicating the size and shape of the eyeball, information indicating the position of the center of rotation of the eyeball, etc. The modification unit 116 can modify the parameters based on the feature amount by referring to a conversion formula or the like that indicates the relationship between the feature amount and each parameter.

[0079] The functions of the extraction unit 115 and the change unit 116 will be described in detail later with reference to FIGS.

[0080] The estimation unit 118 calculates the inclination of the eyeball 30 based on the position of the laser light reflected by the eyeball 30 and the parameters stored in the storage unit 117, and obtains information on the line of sight direction.

[0081] The inclination of the eyeball 30 is calculated using a formula of a linear function or a quadratic function. However, the formula for the calculation is not limited to this, and any formula may be used as long as it can calculate the inclination of the eyeball 30 from the incident angle of the laser light to the eyeball 30 and the beam spot position on the light receiving surface of the PSD 3. In this embodiment, a formula based on a quadratic function is used as a simple approximation formula.

[0082] A surface shape model of the eyeball 30 can be used to determine the angle at which the laser light enters the eyeball 30. For example, a simplified model eye (see, for example, "Optical Mechanism of the Eye", Precision Machinery 27-11, 1961), which has long been known as a general surface shape model of the eyeball, can be used.

[0083] The gaze detection device 10 determines in advance, as the angle of incidence of the laser light on the eyeball 30, an angle at which the incident position of the laser light on the PSD 3 becomes the center of the light receiving surface by ray tracing calculation or the like.

[0084] A more detailed description will now be given of how the estimation unit 118 calculates the tilt of the eyeball 30. The estimation unit 118 calculates the tilt of the eyeball 30, i.e., the gaze direction 34, based on parameters such as information on the size or shape of the eyeball 30 or position information on the center of rotation 58 of the eyeball 30.

[0085] Fig. 5 is a diagram for explaining an example of a method for identifying the tilt direction of the eyeball 30 from the incident position of the laser light on the PSD 3. In Fig. 5, the concave mirror reflection point Pm and the light vector sm are parameters given in advance by the gaze detection device 10. Assuming that the laser light reflection point Pc on the cornea 33 is located on an extension line (length A) of the light vector sm, the normal vector nc at the laser light reflection point Pc can be determined using the coordinates of the PSD landing point Pd, the concave mirror reflection point Pm, and the laser light reflection point Pc.

[0086] The radius of curvature R2 of the cornea 33 is given in advance as a parameter representing the shape of the eyeball 30, and the center of curvature 59 of the cornea 33 is uniquely determined using the normal vector nc. Since the center of rotation 58 of the eyeball 30 is given in advance, the distance Rd between the center of rotation 58 of the eyeball 30 and the center of curvature 59 of the cornea 33 is determined.

[0087] The estimation unit 118 selects the length A so that the distance Rd obtained when this length A is assumed is equal to the distance Δr between the center of rotation 58 of the eyeball 30 and the center of curvature 59 of the cornea, which is given in advance as a parameter representing the shape of the eyeball 30. For example, the estimation unit 118 calculates the straight line connecting the center of rotation 58 of the eyeball 30 and the center of curvature 59 of the cornea, i.e., the gaze direction 34 of the eyeball 30, by solving the following equation:

[0088] A = arg min(Rd-Δr) However, distance Rd is a function of length A of center of curvature 59 of cornea 33, and arg min in the above equation is a mathematical symbol representing selection of length A that minimizes the value in parentheses. In addition, in Fig. 5, the reflected light ray shown by the dashed dotted line indicates the optical path when length A is not appropriately selected, the distance between laser light reflection point Pc on cornea 33 and center of curvature 59 of cornea 33, and distance Rd between center of curvature 59 of cornea 33 and center of rotation 58 of eyeball 30, and indicates that distance Rd between center of curvature 59 of cornea 33 and center of rotation 58 of eyeball 30 does not coincide with Δr.

[0089] In the above example, the gaze direction 34 is identified by a procedure for optimizing the length A, but other methods are also applicable. For example, it is also possible to approximate the incident position of the laser light acquired by the PSD 3 by a polynomial approximation of the rotation angle of the eyeball 30, and calculate the gaze direction 34 of the eyeball 30 through an inverse calculation formula.

[0090] The storage unit 117 stores the incident angle of the laser light to the eyeball 30 and an inverse calculation formula for estimating the tilt of the eyeball 30. The estimation unit 118 acquires the inverse calculation formula including parameters by referring to the storage unit 117, and calculates the tilt of the eyeball 30 based on the position of the laser light reflected by the eyeball 30 and the inverse calculation formula to acquire information on the gaze direction.

[0091] The output unit 119 is an example of an output means that outputs the gaze direction information acquired by the estimation unit 118 to an external device, etc. The external device is an eye tracking device, an optometry device, a retinal projection display device, a head mounted display device such as a head mounted display (HMD), etc.

[0092] <Example of eyeball tilt 30> Next, the inclination of the eyeball 30 and the change in position of the laser light reflected by the eyeball 30 associated with the inclination of the eyeball 30 will be described with reference to FIGS.

[0093] The human eyeball is constantly performing an unconscious movement called fixational eye movement, and the gaze direction is constantly changing in response to the fixational eye movement. Fixational eye movement is classified into characteristic movements such as microsaccade, tremor, and drift, but here we will not go into the details of the classification and will regard fixational eye movement as a movement having a certain statistical fluctuation.

[0094] 6 is a diagram showing an example of the relationship between the gaze direction of eyeball 30 and fixational eye movement. Eyeball 30 rotates and tilts around rotation center 58. Eyeball 30 rotates and tilts in gaze direction 56 (shown by a dashed line in FIG. 6) toward which the person is directing their gaze, and with eyeball 30 tilted in gaze direction 56, eyeball 30 repeatedly performs a rotational movement of a small angle due to fixational eye movement around gaze direction 56 as the center.

[0095] 6 indicates the radius of curvature of the eyeball 30, the radius of curvature R2 indicates the radius of curvature of the cornea 33, and the amount of protrusion d indicates the amount of protrusion of the cornea 33 with respect to the eyeball 30.

[0096] The fixational eye movement direction 57 indicated by the dashed line in Fig. 6 indicates the direction in which the eyeball 30 tilts due to fixational eye movement. In the example shown in Fig. 6, the eyeball 30 tilts in the gaze direction 56 at an angle θ with respect to the Z axis in accordance with the human line of sight, and tilts in the fixational eye movement direction 57 at an angle θ' with respect to the gaze direction 56 in accordance with the fixational eye movement.

[0097] Next, FIG. 7 is a diagram showing an example of the simulation result of the inclination change of the eyeball 30. FIG. 7 shows the inclination of the eyeball 30 in each direction of the X-axis and Y-axis.

[0098] In the simulation, it was assumed that the line-of-sight direction of the eyeball 30 randomly changes 100 points within an angular range of ±3°. Also, it was assumed that the inclination of the eyeball 30 changes 100 points each probabilistically according to a normal distribution centered on the line-of-sight direction due to the fixation micro-vibration for each line-of-sight direction.

[0099] FIG. 7 shows the case where the line-of-sight direction changes 50 points, and the inclination of the eyeball 30 changes 100 points due to the fixation micro-vibration for each changed line-of-sight direction. The number of plotted points is 50×100 = 5000 (points) in total.

[0100] For example, in the region 61 indicated by the broken line, the angle θx, which is the X-axis direction component of the angle θ of the line-of-sight direction, is approximately -2.5 [degrees], and the angle θy, which is the Y-axis direction component, is approximately 2.5 [degrees]. Due to the fixation micro-vibration centered on the line-of-sight direction, the inclination changes slightly, and the inclination is distributed centered on the line-of-sight direction. In this way, the eyeball 30 changes its inclination as shown in FIG. 7 by changing the line-of-sight direction while performing fixation micro-vibration.

[0101] <Example of the distribution of the arrival position of the laser light on PSD3> Next, FIG. 8 is a diagram showing an example of the simulation result of the distribution of the position where the laser light reflected by the eyeball 30 reaches the light-receiving surface of PSD3 when the eyeball 30 changes the line-of-sight direction while performing fixation micro-vibration. The position Pd_y corresponding to the horizontal axis of FIG. 8 indicates the position on the light-receiving surface of PSD3 in the Y-axis direction, and the position Pd_z corresponding to the vertical axis indicates the position on the light-receiving surface of PSD3 in the Z-axis direction.

[0102] Fig. 8 shows how the position at which the laser light reflected by the eyeball 30 reaches the light receiving surface of the PSD 3 changes when the eyeball 30 specified by the parameters is tilted by rotational movement as shown in Fig. 7. The position at which the laser light reaches is calculated by a ray tracing simulation. Note that each component is arranged so that the laser light is incident on the cornea 33 of the eyeball 30 at an incident angle of 60[deg].

[0103] The following parameters were used: Radius R1 of eyeball 30 (see Figure 6): 13.12 [mm]. Radius of curvature R2 of cornea 33 in eyeball 30 (see Figure 6): 7.92 [mm]. Protrusion amount d of cornea 33 in eyeball 30 (see FIG. 6): 0.24 mm. Coordinates (Δx, Δy, Δz) of the center of rotation 58 of the eyeball 30 (see FIG. 6): (−21.46, 0.48, −17.37) [mm]. Note that the coordinates of the center of rotation 58 of the eyeball 30 are relative to the center of the light receiving surface of the PSD 3 as the origin.

[0104] 8, the positions of the laser beam are distributed over a wide range on the light receiving surface of the PSD 3. In addition, the distribution of the positions of the laser beam includes a plurality of clusters, which are collections of the positions of a plurality of laser beams.

[0105] 8 is composed of a set of plots showing the positions of a plurality of laser beams whose positions Pd_z are within a range of approximately -3.0 to -2.5 [mm] and whose positions Pd_y are within a range of approximately 0.2 to 0.4 [mm]. The distribution of the positions of the laser beams includes a plurality of clusters such as the cluster 71.

[0106] The change width of the position of the laser light in each cluster is different for each cluster. The change width of the position in each cluster corresponds to the change width of the inclination in the fixation eye movement of the eyeball 30. The change width can also be called the fluctuation width, and will be referred to as the fluctuation width hereinafter.

[0107] For example, in cluster 72 located on the positive side of the Z axis with respect to cluster 71, the fluctuation width of the position of the laser light along the Z axis direction is larger than that of cluster 71. The positive direction of the Z axis corresponds to the direction away from the eyeball 30.

[0108] The fluctuation width of the cluster in the Z-axis direction is minimum when the position Pd_z is near 0, and the fluctuation width of the cluster along the Z-axis direction increases toward both the positive and negative sides of the Z-axis.

[0109] The fluctuation width of the cluster in the Z-axis direction varies depending on the size or shape of the eyeball 30 or the position of the center of rotation 58 of the eyeball 30. Therefore, the above parameters can be extracted by extracting the feature amount of the cluster corresponding to the fixational eye movement in the distribution of the position of the laser light. A cluster analysis method or the like can be applied to extract the cluster from the multiple detection values.

[0110] The position of the center of rotation 58 of the eyeball 30 changes depending on the position of the gaze detection device 10 when worn by a person. If the misalignment of the wearing position is large, the position of the center of rotation 58 changes significantly.

[0111] <Example of correlation between average and standard deviation of detected values ​​of laser beam position> Next, with reference to FIG. 9, the correlation between the average and standard deviation of a plurality of detection values ​​constituting a cluster in the distribution of the position of the laser light will be described.

[0112] Figure 9 is a diagram for explaining an example of the correlation between the average and standard deviation of multiple detection values, where (a) shows a case where there are 5 detection values, (b) shows a case where there are 10 detection values, and (c) shows a case where there are 15 detection values. Note that Ngz shown in each graph represents the number of detection values.

[0113] The horizontal axis in each of Figures 9(a) to 9(c) indicates the average value of detection values ​​of multiple positions constituting a cluster among the detection values ​​of the position of the laser light reflected by the eyeball 30 by the PSD3. The vertical axis indicates the standard deviation of detection values ​​of multiple positions constituting a cluster among the detection values ​​of the position of the laser light reflected by the eyeball 30 by the PSD3. The average value corresponds to the center of gravity position of the cluster, and the standard deviation corresponds to the fluctuation width.

[0114] The solid line graphs 71a, 71b, and 71c shown in Figures 9(a) to 9(c) respectively show polynomial approximation curves in the correlation between the average value and the standard deviation, and the dashed line graphs 72a, 72b, and 72c show the generation error of the polynomial approximation curve. The generation error of the polynomial approximation curve indicates the reliability of the polynomial approximation curve, and the smaller the generation error, the higher the reliability of the polynomial approximation curve and the higher the correlation between the center of gravity position of the cluster and the fluctuation width.

[0115] As shown in FIG. 9, the more the number of detected values, the smaller the generation error in the polynomial approximation curve of the cluster center position and fluctuation width becomes, and the higher the reliability becomes.

[0116] The polynomial approximation curve and generation error in Figure 9 are calculated by applying Bayesian estimation. Here, Bayesian estimation refers to a method of probabilistically inferring an event to be estimated from an observed event based on the theory of Bayesian probability. By applying Bayesian estimation, the polynomial approximation curve and generation error can be calculated with higher reliability even with a small number of detected values.

[0117] In this embodiment, a polynomial approximation curve is used that is approximated to a sixth-order polynomial. By setting the number of detection values ​​to 10 or more, the generation error becomes approximately constant, and the polynomial approximation curve that shows the correlation between the center of gravity position of the cluster and the fluctuation width can be determined with high reliability.

[0118] If the size or shape of the eyeball 30 changes, or the position where the gaze detection device 10 is attached shifts, causing the parameters to change, the correlation curve between the center of gravity position of the cluster and the fluctuation width becomes unstable. However, by accumulating the detection values ​​as they are, the relationship between the center of gravity position of the cluster and the fluctuation width converges to a different polynomial approximation curve. As a result, the modification unit 116 creates a polynomial approximation curve that reflects the modified parameters.

[0119] In this way, a polynomial approximation curve of the cluster's center of gravity position and fluctuation width can be obtained that autonomously reflects parameters such as the size or shape of the eyeball 30 or the wearing position at the time the detection value by the PSD 3 is obtained.

[0120] In order to calculate the parameters based on the polynomial approximation curve of the cluster's center of gravity position and fluctuation width, for example, an approximation solution of the polynomial approximation curve of the cluster's center of gravity position and fluctuation width according to fixational eye movement is derived using geometric optics. Then, a conversion formula showing the correspondence between the coefficient of each term in the polynomial approximation curve and the parameters is derived. The parameters can be calculated using this conversion formula based on the cluster's center of gravity position and fluctuation width.

[0121] As a different method, a conversion formula between the coefficients of the polynomial approximation curve and the parameters is derived using canonical correlation analysis, which is a type of multivariate analysis. The parameters can be calculated using this conversion formula based on the center of gravity position and fluctuation width of the cluster.

[0122] Here, the coefficients and parameters of the polynomial of the correlation curve of the center of gravity position and the standard deviation can be expressed by canonical correlation analysis. In addition, publicly known techniques can be applied to the canonical correlation analysis (for example, see https: / / qiita.com / yoneda88 / items / 847cb99542538083b876).

[0123] The coefficients of the polynomial and the eyeball parameters are regarded as vectors and linearly transformed. A linear transformation matrix is ​​obtained that maximizes the correlation rate of each vector after linear transformation. In the embodiment, the coefficients of the sixth-order polynomial are regarded as vectors, and the eyeball parameters are also regarded as vectors. When each is linearly transformed, a linear transformation matrix is ​​obtained that maximizes the correlation coefficient of the transformed matrix. This linear transformation matrix is ​​used to estimate one from the other. When linear transformation is performed, the numbers of elements are converted to match each other.

[0124] In the embodiment, data on the center of gravity and fluctuation width of fixational eye movement are sequentially used as materials for estimation using Bayesian estimation, so that the estimation curve gradually converges even if the amount of data is not so large. The estimation accuracy of the sixth-order polynomial obtained by Bayesian estimation improves with each addition of data on the center of gravity and fluctuation width of fixational eye movement.

[0125] Data showing the relationship between the center of gravity and fluctuation width of fixation eye movement and eye parameters is collected in advance by simulation, and the data is used as learning data to derive a conversion formula between the fixation eye movement data and the eye parameters. Using the derived conversion formula, the eye parameters can be estimated from the fixation eye movement data when the eye parameters are unknown.

[0126] <Example of parameter change action> Next, the effect of parameter change will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the effect of parameter change.

[0127] Figure 10 shows the results of estimating, by simulation, the position of incidence of laser light reflected by eyeball 30 on the light receiving surface of PSD 3 when eyeball 30 is tilted around the X-axis and Y-axis from -5.0 degrees to 4.0 degrees in 1.0 degree increments.

[0128] 10, a plot 91 indicated by a black circle indicates the incident position of the laser light obtained when a predetermined parameter is used. The plot 91 can be said to be the true value of the incident position of the laser light.

[0129] Plot 92, indicated by squares, shows the incident position of the laser light obtained when general or average parameters are used. Plot 93, indicated by white circles, shows the incident position of the laser light obtained after the changing unit 116 changes the parameters based on the center of gravity position and fluctuation width of the cluster according to the fixational eye movement. Note that each of plots 91, 92, and 93 is a collective notation of a plurality of plots.

[0130] 10, plot 93 using the changed parameters is closer to the true values ​​of plot 91 than plot 92. The change in parameters reduces the estimation error of the incident position of the laser light reflected by the eyeball 30 onto the light receiving surface of the PSD 3. This indicates that the gaze direction detection error by the gaze detection device 10 is reduced. Specifically, the gaze direction detection error is reduced from 25.74 [mm] to 4.48 [mm] on average for each plot.

[0131] <Example of Operation of the Line-of-Sight Detection Device 10> Next, the operation of the gaze detection device 10 will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the operation of the gaze detection device 10. Fig. 11 shows the operation triggered by the point in time when the gaze detection device 10 starts the gaze detection operation.

[0132] First, in step S101, the estimation unit 118 refers to the storage unit 117 and acquires information on the parameters stored in the storage unit 117.

[0133] Next, in step S102, the light-emission control unit 111 selects one of the plurality of light-emission units included in the VCSEL1.

[0134] Next, in step S103, the light emission control unit 111 causes the selected light emitting unit to emit light.

[0135] Next, in step S104, the determination unit 112 receives a detection signal from the PSD 3 and determines whether the laser light reflected by the eye 30 has been incident on the light receiving surface of the PSD 3 or not.

[0136] If it is determined in step S104 that the light has not entered (Yes in step S104), the operations from step S102 onwards are performed again.

[0137] On the other hand, if it is determined in step S104 that light has been incident (step S104, Yes), in step S105, the acquisition unit 113 A / D converts the analog voltage detection signal of PSD3 input via the determination unit 112, and acquires a digital voltage detection value that indicates the position of light on the light receiving surface of PSD3.

[0138] Next, in step S106, the estimation unit 118 calculates the inclination of the eyeball 30 based on the position of the laser light reflected by the eyeball 30 and the parameters stored in the storage unit 117, and obtains information on the line of sight direction.

[0139] Next, in step S107, the output unit 119 outputs the information on the gaze direction acquired by the estimation unit 118 to an external device.

[0140] Next, in step S108, the extraction unit 115 and the change unit 116 cooperate to change the parameters. This operation will be described below with reference to FIG.

[0141] Next, in step S109, the processing unit 100 determines whether or not to end the gaze detection operation. If it is determined that the gaze detection operation is to be ended, the gaze detection device 10 ends the operation. On the other hand, if it is determined that the gaze detection operation is not to be ended, the gaze detection device 10 performs the operations from step S101 onwards again.

[0142] In this manner, the gaze detection device 10 can detect the gaze direction of a person wearing a glasses-type support on which the gaze detection device 10 is mounted.

[0143] 11, the gaze detection device 10 outputs the gaze direction information in step S107 and then performs the operation of changing the parameters in step S108, but the order is not limited to this. The gaze detection device 10 may perform the operation of changing the parameters in any order, or may perform the operation multiple times, as long as the laser light is incident on the light receiving surface of the PSD 3.

[0144] For example, the parameter change may be performed before the estimation unit 118 acquires information on the line of sight direction in step S106, or the parameter change may be performed multiple times during steps S105 to S109.

[0145] In this embodiment, the parameter change and the gaze direction detection are not performed in parallel, but the parameter change and the gaze direction detection are performed at different times. In other words, the output unit 119 outputs gaze direction information at a first time, and the change unit 116 changes the parameter at a second time different from the first time. The timing at which the operations of steps S105 to S107 are performed is an example of the first time, and the timing at which the operation of step S108 is performed is an example of the second time.

[0146] <Example of parameter change operation> Next, Fig. 12 is a diagram showing an example of a parameter change operation. Fig. 12 shows an operation triggered by the point in time when the gaze detection device 10 starts the parameter change operation.

[0147] First, in step S111, the acquisition unit 113 A / D converts the analog voltage detection signal of the PSD 3 input via the determination unit 112, and acquires a digital voltage detection value indicating the position of light on the light receiving surface of the PSD 3.

[0148] Next, in step S112, the extraction unit 115 calculates the center of gravity and the fluctuation width of the detection values ​​constituting the cluster among the multiple detection values ​​stored in the storage unit 114, and extracts them as features indicating the characteristics of fixational eye movement. The extraction unit 115 outputs the data of the extracted features to the modification unit 116.

[0149] Next, in step S 113 , the change unit 116 changes the parameters stored in the storage unit 117 based on the feature amount input from the extraction unit 115 .

[0150] In this manner, the gaze detection device 10 can change the parameters stored in the storage unit 117.

[0151] <Functions and Effects of the Line-of-Sight Detection Device 10> Next, the effects of the gaze detection device 10 will be described.

[0152] In recent years, technologies and products related to virtual reality (VR) and augmented reality (AR) have been attracting attention. In particular, AR technology is expected to be applied to industrial fields as a means of displaying digital information in real space. Considering that people who use AR technology obtain most of their cognitive information through their eyesight, optical devices such as eyeglass-type image display devices that can be used in the working environment have been developed.

[0153] As such a glasses-type image display device, a retinal projection display device using a retinal imaging method is known, which uses laser light to directly image a human retina. According to the retinal imaging method, a focus-free image is superimposed on visual information, so that digital information can be displayed on the retina with the viewpoint set on the outside world, allowing the human to recognize it.

[0154] However, in a retinal projection display device using laser light, due to limitations in the size of the cornea and pupil, in a behavioral (working) environment involving eye movement, vignetting of the laser light may occur around the outer periphery of the cornea or pupil, making it impossible to display a specified image at a specified position.

[0155] In response to this, a configuration is known in which a laser beam is scanned over the eyeball using a micro-electro-mechanical systems (MEMS) mirror, and the position of the cornea of ​​the eyeball is detected based on a detection signal of the laser beam reflected by the eyeball.

[0156] Furthermore, Patent Document 1 discloses a configuration in which a laser beam is irradiated onto an eyeball and the movement of the eyeball is detected based on the laser beam reflected by the eyeball.

[0157] However, in conventional configurations, when light reflected by an object such as an eyeball deviates significantly, parameters such as the size or shape of the eyeball used to detect the corneal position of the eyeball or the movement of the eyeball, or the position of the center of rotation of the eyeball, may change significantly from the specified values, resulting in large detection errors.

[0158] In this embodiment, a laser beam (light) is irradiated onto the eyeball 30 (target object), the position of the laser beam reflected by the eyeball 30 is detected, and information on the line of sight direction (tilt information of the target object) obtained based on the position of the laser beam and predetermined parameters is output. In addition, the above parameters are changed based on the position of the laser beam reflected by the eyeball 30.

[0159] For example, the above parameters are changed based on the center of gravity of the detection values ​​constituting the cluster according to the fixational eye movement of the eyeball 30 and the fluctuation width (change width) of the detection values ​​constituting the cluster. The position of the laser light reflected by the eyeball changes with a distribution according to the characteristics of the fixational eye movement, and the characteristics of the fixational eye movement differ according to individual differences in the size or shape of the eyeball, or a deviation in the mounting position of the gaze detection device (optical device), etc.

[0160] Therefore, by extracting feature amounts indicating the characteristics of fixational eye movement using the center of gravity position and fluctuation width of the detection values ​​constituting the cluster, and changing the parameters based on the extracted feature amounts, it is possible to compensate for the influence of deviations in size or shape of the eyeball 30, deviations in the mounting position of the gaze detection device 10, etc. This makes it possible to reduce detection errors even when the light reflected by the eyeball 30 is significantly deviated.

[0161] Furthermore, in this embodiment, output unit 119 outputs information on the gaze direction at a first time period, and change unit 116 changes the parameter at a second time period different from the first time period. This allows gaze direction detection and parameter change to be performed by a single PSD 3 (detection means), thereby simplifying the configuration of gaze detection device 10.

[0162] Note that gaze detection device 10 may be configured to include a plurality of PSDs 3, and to detect the gaze direction and change parameters in parallel using the plurality of PSDs 3.

[0163] Furthermore, the change unit 116 repeatedly changes the parameters. For example, when the eyeglass-type support including the gaze detection device 10 is worn and the position of the gaze detection device 10 is suddenly shifted due to the eyeglass-type support being shifted, a configuration in which the parameter change is performed only once will not be able to compensate for the position shift of the gaze detection device 10. By repeatedly changing the parameters, it becomes possible to deal with the sudden position shift of the gaze detection device 10. The repetition period may be a fixed period or a period in which the parameters are changed randomly.

[0164] In this embodiment, the change unit 116 changes the parameters based on the estimated values ​​by Bayesian estimation based on the detected values ​​of the laser light position. By applying Bayesian estimation, the polynomial approximation curve and the generation error can be calculated with high reliability even with a small number of detected values, and the detection error can be further reduced by changing to more accurate parameters.

[0165] [Second embodiment] Next, a retinal projection display device 50 according to a second embodiment will be described. Note that the same components as those described in the first embodiment are given the same part numbers, and duplicated descriptions will be omitted as appropriate.

[0166] Fig. 13 is a diagram illustrating an example of the configuration of a retinal projection display device 50. As shown in Fig. 13, the retinal projection display device 50 has an RGB (Red, Green, Blue) laser light source 51, a scanning mirror 52, a plane mirror 53, a half mirror 54, an image generating means 55, and the gaze detection device 10 described in the first embodiment.

[0167] The RGB laser light source 51 time-modulates and outputs laser light of three colors, RGB. The scanning mirror 52 two-dimensionally scans the light from the RGB laser light source 51. The scanning mirror 52 is, for example, a MEMS mirror.

[0168] However, the scanning mirror 52 is not limited to a MEMS mirror, and may be a polygon mirror, a galvanometer mirror, or any other mirror that has a reflecting portion that scans light. A MEMS mirror is advantageous in terms of size and weight reduction. The driving method of the MEMS mirror may be any of electrostatic, piezoelectric, electromagnetic, etc.

[0169] The plane mirror 53 reflects the scanning light from the scanning mirror 52 toward the half mirror 54. The half mirror 54 transmits a part of the incident light and reflects a part of it toward the eyeball 30. The half mirror 54 has a concave curved shape, and converges the reflected light near the pupil 31 of the eyeball 30 to form an approximate image at the position of the retina 32. In this way, the image formed by the scanning light is projected onto the retina 32.

[0170] Light 51a indicated by a dashed line in the figure represents light that forms an image on the retina 32. Note that the half mirror 54 does not necessarily have to have a one-to-one relationship between the amount of reflected light and the amount of transmitted light.

[0171] The gaze detection device 10 detects the gaze direction that changes in accordance with the movement of the eyeball, and transmits to the image generation means 55 a feedback signal indicating information on the gaze direction.

[0172] The image generating means 55 has a function of controlling the deflection angle of the scanning mirror 52 and a function of controlling the light emission of the RGB laser light source 51. The image generating means 55 receives a feedback signal indicating the gaze direction from the gaze detection device 10, and controls the deflection angle of the scanning mirror 52 and the light emission of the RGB laser light source 51 according to the gaze direction, and rewrites the projection angle of the image or the image content. This makes it possible to form an image on the retina 32 that tracks (eye-tracks) the change in the gaze direction accompanying eyeball movement.

[0173] In the above-described embodiment, the retinal projection display device 50 is a head-mounted display that is a wearable terminal. However, the retinal projection display device 50 as a head-mounted display may not only be directly mounted on the human head, but also be indirectly mounted on the human head via a member such as a fixing part (a head-mounted display device). Also, a binocular retinal projection display device having a pair of retinal projection display devices 50 for the left and right eyes may be used.

[0174] Although examples of embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims.

[0175] For example, in the above-described embodiment, a device for detecting the inclination of the eyeball 30 is shown as an example of an optical device, but the present invention is not limited to this. For example, an optical device may be mounted on a robot hand or the like to detect the inclination of the robot hand as an example of an object. In this case, a specific parameter is used to estimate the inclination of the robot hand. When the robot hand vibrates minutely for tracking purposes, the minute vibration is treated in the same way as fixational eye movement, and a feature amount of a cluster of the position of the laser light corresponding to the minute vibration is extracted and used to change the parameter.

[0176] For example, information on the gaze direction detected by the gaze detection device 10 can be used for eye tracking in an input device of an electronic device. This makes it possible to realize eye tracking that is robust against the size or shape of the eyeball, positional deviation of the gaze detection device 10, and the like.

[0177] The present invention can also be adopted in an optometry device having a function of detecting the inclination of the eyeball and the pupil position (cornea). An optometry device refers to a device capable of performing various tests such as visual acuity tests, eye refractive power tests, intraocular pressure tests, and axial length tests. An optometry device is a device capable of performing tests without contacting the eyeball, and includes a support section for supporting the face of the subject, an eye examination window, a display section for displaying the subject's eyeball (direction of gaze) during eye examination, a control section, and a measurement section. In order to increase the measurement accuracy of the measurement section, the subject is required to gaze at one point without moving the eyeball (gaze), and the subject fixes his / her face on the support section and stares at the display displayed on the display section through the eye examination window. At this time, the eyeball inclination position detection device of this embodiment can be used to detect the inclination position of the eyeball. The eyeball inclination position detection device is disposed to the side of the measurement section so as not to interfere with the measurement. The eyeball tilt position (gaze) information obtained by the eyeball tilt position detection device can be fed back to the control unit, making it possible to perform measurements according to the eyeball tilt position information.

[0178] The embodiment also includes a method for detecting the inclination of a three-dimensional object. For example, the method for detecting the inclination of a three-dimensional object includes a step of irradiating a target object with light, a step of detecting the position of the light reflected by the target object, a step of outputting inclination information of the target object obtained based on the position of the light and a predetermined parameter, and a step of changing the parameter based on the position of the light. Such a method for detecting the inclination of a three-dimensional object can achieve the same effect as the optical device described above.

[0179] The embodiment also includes a gaze detection method. For example, the gaze detection method includes a step of irradiating a target with light, a step of detecting a position of the light reflected by the target, a step of outputting tilt information of the target obtained based on the position of the light and a predetermined parameter, and a step of changing the parameter based on the position of the light. By using such a gaze detection method, it is possible to obtain the same effect as the optical device described above.

[0180] In addition, all the numbers such as ordinal numbers and quantities used above are merely examples for specifically explaining the technology of the present invention, and the present invention is not limited to the exemplified numbers. In addition, the connection relationships between the components are merely examples for specifically explaining the technology of the present invention, and the connection relationships for realizing the functions of the present invention are not limited to these.

[0181] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above. [Explanation of symbols]

[0182] 1 VCSEL (example of light source) 2 Plane mirror 3 PSD (an example of a detection method) 10. Line-of-sight detection device (an example of an optical device) 20 Spectacle-shaped support 21 Eyeglass frames 22 Eyeglass Lenses 30 Eyeball (example of object) 31 Pupil 32 Retina 33 Cornea 50 Retinal projection display device 56 View direction 57 Fixation direction 58 Center of rotation 101 CPU 102 ROM 103 RAM 104 SSD 105 Light source driving circuit 106 Signal Generator Circuit 107 A / D conversion circuit 108 Input / Output Interface 111 Light emission control unit 112 Judgment section 113 Acquisition Department 114 Memory unit (an example of a memory means) 115 Extraction part 116 Change unit (an example of a change means) 117 Storage Unit 118 Estimation Department 119 Output unit (an example of an output means) θ, θ' angles R1 Radius of curvature of the eyeball (an example of a parameter) R2 Corneal curvature radius (an example of a parameter) d Corneal protrusion (example of parameter) Pm Concave mirror reflection point (one example of a parameter) sm ray vector (an example of a parameter) Pc laser light reflection point A Length Pd PSD landing point nc normal vector [Prior art documents] [Patent documents]

[0183] [Patent Document 1] Special Publication No. 05-013659

Claims

1. A light source that irradiates light onto an object; a detection means for detecting a position of the light reflected by the object; an output unit that outputs tilt information of the object obtained based on a position of the light and predetermined parameters including information on a size or shape of the eyeball or position information of a center of rotation of the eyeball; A change means for changing the parameter based on the position of the light; a storage means for storing a detection value of the position of the light detected by the detection means, The change means changes the parameters based on the center of gravity of the detection values ​​that constitute a cluster among the multiple detection values ​​stored in the memory means and the range of change of the detection values ​​that constitute the cluster.

2. The output means outputs tilt information of the object at a first time period, The optical device according to claim 1 , wherein the change means changes the parameter at a second time point different from the first time point.

3. The optical device according to claim 2 , wherein the change means changes the parameter repeatedly.

4. The optical device according to claim 1 , wherein the change unit changes the parameter based on an estimate by Bayesian estimation based on a detected value of the position of the light.

5. A gaze detection device comprising the optical device according to claim 1 .

6. The gaze detection device according to claim 5 , wherein the parameter is at least one of a shape of an eyeball that is a target of gaze detection and a position of a center of rotation of the eyeball.

7. 7. A retinal projection display device comprising the line of sight detection device according to claim 5.

8. A head-mounted display device comprising the gaze detection device according to claim 5 or 6.

9. An eye examination apparatus comprising the gaze detection device according to claim 5 or 6.

10. Irradiating an object with light; detecting a position of the light reflected from the object; outputting tilt information of the object obtained based on a position of the light and predetermined parameters including information on a size or shape of the eyeball or position information of a center of rotation of the eyeball; and varying the parameter based on the position of the light; The method for detecting the inclination of a three-dimensional object, in which the step of changing the parameters changes the parameters based on the center of gravity position of the detection values ​​constituting a cluster and the range of change of the detection values ​​constituting the cluster, among a plurality of detection values ​​stored in a memory means that stores the detection values ​​of the position of the light.

11. Irradiating an object with light; detecting a position of the light reflected from the object; outputting tilt information of the object obtained based on a position of the light and predetermined parameters including information on a size or shape of the eyeball or position information of a center of rotation of the eyeball; and varying the parameter based on the position of the light; The step of changing the parameters is a gaze detection method in which the parameters are changed based on the center of gravity position of the detection values ​​that constitute a cluster and the range of change of the detection values ​​that constitute the cluster, among a plurality of detection values ​​stored in a memory means that stores detection values ​​of the position of the light.

Images