head-mounted display

Abstract

To provide a visual line detector capable of detecting a visual line with high accuracy regardless of situations.SOLUTION: A visual line detector includes: openings provided facing eyes of a user; visual line sensors provided at edges of the openings in a manner to face in a direction of the eyes facing the openings; and a plurality of light sources provided along the edges of the openings. The number of the light sources per one degree is the largest in a range on the opposite side to the visual line sensors across the centers of the openings among a range of 360 degrees around the centers of the openings.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a gaze detection device. [Background technology]

[0002] Various electronic devices capable of detecting the user's gaze (gaze position and gaze direction) have been put to practical use. For example, in the fields of virtual reality (VR) and augmented reality (AR), head-mounted displays that detect the gaze position and perform processing such as menu selection based on the gaze position have been put to practical use. In addition, cameras and video cameras that detect the gaze direction and select a focusing point based on the detected gaze direction have also been put to practical use.

[0003] In gaze detection, the gaze sensor captures an image of the user's eye by capturing an image of the eye. At this time, a light source arranged around the eyepiece optical system illuminates the user's eye. The light from the light source is specularly reflected by the surface of the cornea and appears as a corneal reflection image, known as a Purkinje image, in the image captured by the gaze sensor. The gaze is detected by calculating the direction in which the user's eye is facing from the coordinates of the eyeball and the corneal reflection image in the image captured by the gaze sensor.

[0004] In a conventional electronic device (gaze detection device), a plurality of light sources are arranged so as to surround the optical axis of a lens system (optical system). For example, Patent Document 1 discloses an electronic device in which a plurality of light sources are arranged at approximately equal intervals so as to surround the optical axis of a lens system. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] U.S. Pat. No. 1,113,8429 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in conventional gaze detection (gaze detection devices), the Purkinje image may not appear correctly in the image acquired by the gaze sensor, making it difficult to detect the gaze with high accuracy.

[0007] An object of the present invention is to provide a gaze detection device that is capable of detecting the gaze with high accuracy regardless of the situation. [Means for solving the problem]

[0008] The gaze detection device of the present invention has an opening arranged to face the user's eye, a gaze sensor arranged on the edge of the opening so as to face the direction of the eye facing the opening, and a plurality of light sources arranged along the edge of the opening, and is characterized in that the number of light sources per degree is greatest in the range on the opposite side of the center of the opening from the gaze sensor within a 360-degree range centered on the center of the opening. Effect of the Invention

[0009] According to the present invention, it is possible to provide a gaze detection device that is capable of detecting the gaze with high accuracy regardless of the situation. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a rear view of the display unit main body according to the first embodiment. [Diagram 2] FIG. 2 is a perspective view of a display unit main body according to the first embodiment. [Diagram 3] 1 is a cross-sectional view of a display unit main body according to a first embodiment. [Figure 4] 3 is a schematic diagram showing a reflection position of illumination light according to the first embodiment. FIG. [Diagram 5] FIG. 2 is a schematic diagram showing the internal structure of the right gaze sensor according to the first embodiment. [Figure 6] FIG. 11 is a rear view of the glasses-type device according to the second embodiment. [Figure 7]FIG. 11 is a perspective view of a glasses-type device according to a second embodiment. [Figure 8] FIG. 11 is a rear view of a display unit main body according to the third embodiment. [Figure 9] FIG. 11 is a cross-sectional view of a display unit main body according to a third embodiment. [Figure 10] FIG. 11 is a cross-sectional view of a display unit main body according to a third embodiment. [Figure 11] 13 is a flowchart of gaze detection according to a third embodiment. [Figure 12] 13 is a modified example of the flowchart of gaze detection according to the third embodiment. [Figure 13] FIG. 13 is a rear view of a display unit main body according to the fourth embodiment. [Figure 14] FIG. 13 is a cross-sectional view of a display unit main body according to a fourth embodiment. [Figure 15] FIG. 11 is a perspective view of a lens according to a fourth embodiment. [Figure 16] FIG. 13 is a cross-sectional view of a camera body according to a fifth embodiment. [Figure 17] FIG. 13 is a cross-sectional view of a camera body according to a fifth embodiment. [Figure 18] FIG. 13 is a schematic diagram showing an EVF unit according to a fifth embodiment. [Figure 19A] FIG. 13 is a perspective view showing a reflection position of illumination light according to the fifth embodiment. [Figure 19B] FIG. 13 is a front view showing a reflection position of illumination light according to the fifth embodiment. [Figure 19C] FIG. 11 is a perspective view showing a comparative example of the reflection position of illumination light. [Figure 19D] FIG. 13 is a front view showing a comparative example of the reflection position of illumination light. [Figure 20] FIG. 13 is a front view showing a reflection position of illumination light according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (First embodiment) A first embodiment of the present invention will be described below with reference to Figures 1 to 5. As the first embodiment, an example of a head-mounted display device (head-mounted display) equipped with a gaze detection device will be described.

[0012] Fig. 1 is a rear view of a display unit main body 101 of a head mounted display according to a first embodiment, and Fig. 2 is a perspective view of the display unit main body 101. Fig. 1 shows the display unit main body 101 as viewed from the eyeball side of a user (a user wearing a head mounted display).

[0013] The display unit body 101 has a right opening 145 that limits the field of view of the user's right eye, and a left opening 146 that limits the field of view of the user's left eye. A right lens system 102, which is a display optical system, is provided in the right opening 145 (inside the right opening), and a left lens system 103, which is a display optical system, is provided in the left opening 146 (inside the left opening). The right opening 145 and the right lens system 102 are disposed so as to face the right eye of the user (a user wearing a head mounted display). The left opening 146 and the left lens system 103 are disposed so as to face the left eye of the user (a user wearing a head mounted display).

[0014] As shown in FIG. 2, an infrared-transparent window 104 is provided around the right lens system 102 to hide the gaze detection unit described below from being seen from the outside. Similarly, an infrared-transparent window 105 is provided around the left lens system 103. Gaze detection is performed using infrared light (infrared rays). The infrared-transparent windows 104 and 105 are made of a material that does not transmit visible light but transmits infrared light. This makes it possible to realize a configuration that is aesthetically pleasing (the internal unit is not visible) and allows gaze detection. Note that even if some visible light is transmitted, if the transmittance of infrared light is higher than that of visible light, the internal unit becomes less visible, and a similar effect can be obtained; however, if the transmittance (blocking rate) of visible light and that of infrared light differ greatly, In this specification and the present invention, the term "transmitting infrared light" does not necessarily mean transmitting all of the incident infrared light, but may also block some of the infrared light. In FIG. 1, the display unit body 101 is in a state where the infrared-transmitting windows 104 and 105 have been removed so that the internal unit can be seen.

[0015] 1, right gaze sensor 106 is disposed on the edge of right opening 145 (right lens system 102) so as to face the direction of the right eye facing right opening 145. Similarly, left gaze sensor 107 is disposed on the edge of left opening 146 (left lens system 103) so as to face the direction of the left eye facing left opening 146.

[0016] FIG. 5 is a schematic diagram showing the internal structure of the right gaze sensor 106. The right gaze sensor 106 includes a gaze sensor lens 147 and a gaze sensor chip 148. A calculation for gaze detection is performed using an image formed on the gaze sensor chip 148 by the gaze sensor lens 147. The right gaze sensor 106 is disposed so that the optical axis 147a (gaze sensor lens optical axis) of the gaze sensor lens 147 of the right gaze sensor 106 faces the direction of the right eye facing the right opening 145. The internal structure of the left gaze sensor 107 is substantially the same as that of the right gaze sensor 106. The left gaze sensor 107 is disposed so that the optical axis 147a (gaze sensor lens optical axis) of the gaze sensor lens 147 of the left gaze sensor 107 faces the direction of the left eye facing the left opening 146.

[0017] Returning to the explanation of FIG. 1, right gaze sensor 106 is disposed at the 9 o'clock position (a position rotated 270 degrees clockwise from directly above) around the center of right opening 145 (the optical axis of right lens system 102). Left gaze sensor 107 is disposed at the 3 o'clock position (a position rotated 90 degrees clockwise from directly above) around the center of left opening 146 (the optical axis of left lens system 103). Right gaze sensor 106 and left gaze sensor 107 are disposed at approximately the same height (position in the vertical direction). Right gaze sensor 106 is disposed to the left of right opening 145 (near the right eye), and left gaze sensor 107 is disposed to the right of left opening 146 (near the left eye).

[0018] In this way, the right gaze sensor 106 and the left gaze sensor 107 are positioned on a horizontal line 185 passing through the optical axis of the right lens system 102 and the optical axis of the left lens system 103 (on a horizontal line passing through the center of the right opening 145 and the center of the left opening 146).

[0019] Normally, the eyelids of a user (a user wearing a head mounted display) open up and down. Therefore, by arranging the right gaze sensor 106 and the left gaze sensor 107 so that the eyeballs are observed from the direction of the horizon 185, the eyeballs are less likely to be eclipsed by the eyelids during gaze detection, and the success rate of gaze detection can be increased.

[0020] 1, a plurality of infrared light emitting diodes (right IREDs) are arranged along the edge of the right opening 145 (right lens system 102) as a plurality of light sources for illuminating the eyeball (right eye). Similarly, a plurality of infrared light emitting diodes (left IREDs) are arranged along the edge of the left opening 146 (left lens system 103) as a plurality of light sources for illuminating the eyeball (left eye).

[0021] The right IREDs 108, 109, 111 to 122 are disposed around the right opening 145 (right lens system 102). With the center of the right opening 145 (the optical axis of the right lens system 102) as the center, the right IRED 108 is disposed at the 10 o'clock position, the right IRED 109 at the 11 o'clock position, the right IRED 111 at the 1 o'clock position, the right IRED 112 at the 1:30 o'clock position, the right IRED 113 at the 2 o'clock position, and the right IRED 114 at the 2:30 o'clock position. The right IRED 115 is disposed at the 3 o'clock position, the right IRED 116 at the 3:30 o'clock position, the right IRED 117 at the 4 o'clock position, the right IRED 118 at the 4:30 o'clock position, the right IRED 119 at the 5 o'clock position, the right IRED 120 at the 6 o'clock position, the right IRED 121 at the 7 o'clock position, and the right IRED 122 at the 8 o'clock position.

[0022] The left IREDs 123, 124, 126 to 137 are disposed around the left opening 146 (left lens system 103). With the left opening 146 (the optical axis of the left lens system 103) as the center, the left IRED 123 is disposed at the 2 o'clock position, the left IRED 124 at the 1 o'clock position, the left IRED 126 at the 11 o'clock position, the left IRED 127 at the 10:30 o'clock position, the left IRED 128 at the 10 o'clock position, and the left IRED 129 at the 9:30 o'clock position. The left IRED 130 is disposed at the 9 o'clock position, the left IRED 131 at the 8:30 o'clock position, the left IRED 132 at the 8 o'clock position, the left IRED 133 at the 7:30 o'clock position, the left IRED 134 at the 7 o'clock position, the left IRED 135 at the 6 o'clock position, the left IRED 136 at the 5 o'clock position, and the left IRED 137 at the 4 o'clock position.

[0023] In the first embodiment, the number of light sources per degree (angular density) is maximum in the range on the opposite side of the right gaze sensor 106 across the center of the right opening 145 (optical axis of the right lens system 102) within a 360-degree range centered on the center of the right opening 145. In FIG. 1, nine right IREDs are arranged at a 15-degree pitch (angular density of 0.0667 pieces / degree) in a 120-degree range (±60 degrees) centered on the 3 o'clock position opposite the 9 o'clock position where the right gaze sensor 106 is arranged. The nine right IREDs are right IREDs 111 to 119 arranged at the 1 o'clock, 1:30, 2:00, 2:30, 3:00, 3:30, 4:00, 4:30, and 5 o'clock positions. In the range other than the 120-degree range, the right IREDs are arranged at a pitch larger than 15 degrees (angular density smaller than 0.0667 pieces / degree).

[0024] The number of right IREDs arranged in the range (the above-mentioned 120-degree range) on the opposite side of the center of right opening 145 from right gaze sensor 106 may be more than the number of IREDs arranged in the remaining range. In Fig. 1, the number of right IREDs arranged in the range (the above-mentioned 120-degree range) on the opposite side of the center of right opening 145 from right gaze sensor 106 is three or more more than the number of IREDs arranged in the remaining range.

[0025] Similarly, within a 360-degree range centered on the center of the left opening 146 (the optical axis of the left lens system 103), the number of light sources per degree (angular density) is maximum in the range opposite the left gaze sensor 107 across the center of the left opening 146. In FIG. 1, nine left IREDs are arranged at a 15-degree pitch (angular density of 0.0667 pieces / degree) in a 120-degree range (±60 degrees) centered on the 9 o'clock position opposite the 3 o'clock position where the left gaze sensor 107 is arranged. The nine left IREDs are left IREDs 126 to 134 arranged at the 11 o'clock, 10:30, 10:00, 9:30, 9:00, 8:30, 8:00, 7:30, and 7 o'clock positions. In a range other than the 120-degree range, the left IREDs are arranged at a pitch greater than 15 degrees (angular density less than 0.0667 pieces / degree).

[0026] The number of left IREDs arranged in the range (the above-mentioned 120-degree range) on the opposite side of the center of left opening 146 from left eye gaze sensor 107 may be more than the number of IREDs arranged in the remaining range. In Fig. 1, the number of left IREDs arranged in the range (the above-mentioned 120-degree range) on the opposite side of the center of left opening 146 from left eye gaze sensor 107 is three or more more than the number of IREDs arranged in the remaining range.

[0027] In the first embodiment, the number of right IREDs arranged below the right opening 145 is greater than the number of right IREDs arranged above the right opening 145. In FIG. 1, no right IRED is arranged at the 12 o'clock position, and the number of right IREDs arranged below the horizontal line 185 is greater than the number of right IREDs arranged above the horizontal line 185. Seven right IREDs, including right IRED 116, right IRED 117, right IRED 118, right IRED 119, right IRED 120, right IRED 121, and right IRED 122, are arranged below the horizontal line 185. Six right IREDs, including right IRED 108, right IRED 109, right IRED 111, right IRED 112, right IRED 113, and right IRED 114, are arranged above the horizontal line 185.

[0028] Similarly, the number of left IREDs disposed below the left opening 146 is greater than the number of left IREDs disposed above the left opening 146. In FIG. 1, no left IREDs are disposed at the 12 o'clock position, and the number of left IREDs disposed below the horizontal line 185 is greater than the number of left IREDs disposed above the horizontal line 185. Seven left IREDs are disposed below the horizontal line 185: left IRED 131, left IRED 132, left IRED 133, left IRED 134, left IRED 135, left IRED 136, and left IRED 137. Six left IREDs are disposed above the horizontal line 185: left IRED 123, left IRED 124, left IRED 126, left IRED 127, left IRED 128, and left IRED 129.

[0029] The movement of the upper and lower eyelids in humans is different, with the upper eyelids mainly moving downwards towards the lower eyelids. The position of the lower eyelid relative to the optical axis of the eye varies little from person to person, but the position of the upper eyelid relative to the optical axis of the eye varies greatly from person to person. For example, in people with narrow eyes, the position of the upper eyelids is lower and closer to the optical axis of the eye than in people with wide eyes.

[0030] To enable gaze detection for many users, it is necessary to enable gaze detection even when the position of the upper eyelid is close to the optical axis of the eye. However, if the position of the upper eyelid is close to the optical axis of the eye, the illumination light from the IRED placed at the 12 o'clock position will be vignetted by the eyelid and will not appear in the image as a Purkinje image (corneal reflection image). On the other hand, compared to the position of the upper eyelid, there is less individual variation in the position of the lower eyelid and it is farther from the optical axis of the eye, so the illumination light from the IRED placed at the 6 o'clock position is less likely to be vignetted by the eyelid and is more likely to appear in the image as a Purkinje image (corneal reflection image, Purkinje image).

[0031] For this reason, by arranging more IREDs below horizontal line 185 (below the opening) than above horizontal line 185 (above the opening), line-of-sight detection becomes possible for many users.

[0032] A right-eye gaze detection unit is configured by right gaze sensor 106, right IREDs 108, 109, 111-122, and a control circuit (control unit) (not shown). Similarly, a left-eye gaze detection unit is configured by left gaze sensor 107, left IREDs 123, 124, 126-137, and a control circuit (control unit) (not shown). The control circuit for the right-eye gaze detection unit and the control circuit for the left-eye gaze detection unit may be a single common control circuit, or may be separate control circuits.

[0033] Fig. 3 shows the arrangement of the eyeball of a user looking into the display unit main body 101 and the display unit main body 101. Fig. 3 shows a cross section obtained by a plane passing through the center of the right gaze sensor 106 and the center of the right lens system 102. Fig. 3 shows the right eye and its surroundings.

[0034] The right lens system 102, which is a display optical system, is held by a lens barrel 139. The right display panel 140 is a display panel (display unit) for displaying images to the user, and is an organic EL panel or the like. A cornea 142 is present inside the user's eyeball 141 (right eye). An image output from the right display panel 140 is input to the eyeball 141 through the right lens system 102. In other words, the user can view the image (video) displayed on the right display panel 140 with the eyeball 141 via the right lens system 102 (right opening 145).

[0035] The optical axis of the right gaze sensor 106 is not parallel to the optical axis of the right lens system 102 (display optical system), but is tilted toward the eyeball 141 (diagonally). This makes it possible to effectively utilize the limited angle of view of the right gaze sensor 106 and to detect the eyeball 141 with the right gaze sensor 106 even if the eyepoint distance changes. You can keep the ball 141 in play.

[0036] The configuration around the left eye is similar to the above configuration around the right eye.

[0037] Thus, in the first embodiment, right gaze sensor 106 is disposed around right opening 145 (right lens system 102), and gaze sensor lens optical axis 147a of right gaze sensor 106 is directed toward the right eye. Similarly, left gaze sensor 107 is disposed around left opening 146 (left lens system 103), and gaze sensor lens optical axis 147a of left gaze sensor 107 is directed toward the right eye.

[0038] The illumination light emitted from the right IRED 115 illuminates the cornea 142 and iris of the eyeball 141 (right eye). The right gaze sensor 106 acquires an iris image of the eyeball 141, and also acquires an image of light passing through optical paths 143 and 144 (a specular reflection image, a Purkinje image, of the illumination light from the right IRED 115 specularly reflected by the cornea 142). Although not shown in FIG. 3, light from other right IREDs arranged around the right lens system 102 also illuminates the iris and cornea. Similarly, the cornea and iris of the left eye are also illuminated by multiple left IREDs. Illuminating a single iris by multiple IREDs reduces illumination unevenness of the iris.

[0039] 4A and 4B are schematic diagrams showing the relationship between the reflection position of infrared light (illumination light) emitted from an IRED and the cornea. Figures 4A and 4B show eyeball 141 (right eye) of a user wearing display unit main body 101, as viewed from the side of display unit main body 101. The case of the right eye will be described, but the case of the left eye is similar to that of the right eye.

[0040] Infrared light emitted from right IREDs 108, 109, 111-122 arranged around right lens system 102 is reflected at reflection positions 108a, 109a, 111a-122a and enters right line of sight sensor 106. Reflection positions 108a, 109a, 111a-122a correspond to right IREDs 108, 109, 111-122, respectively.

[0041] The corneal boundary line 1422a in Fig. 4A and the corneal boundary line 1422b in Fig. 4B indicate the boundary between the cornea 142 and the sclera. The inside of the corneal boundary line 1422a (corneal boundary line 1422b) is a spherical surface. The corneal boundary line 1422a (corneal boundary line 1422b) is a gently curved surface that extends from the spherical surface (cornea) toward the white of the eye (sclera). The width of the cornea varies from person to person, and the corneal boundary line 1422a in Fig. 4A indicates the corneal boundary line of a user with a wide cornea, and the corneal boundary line 1422b in Fig. 4B indicates the corneal boundary line of a user with a narrow cornea.

[0042] 4A and 4B show a state when eyeball 141 faces forward. As shown in FIG. 4A, light emitted from right IREDs 111-119 arranged in a range of 120 degrees centered on the facing position of right gaze sensor 106 is specularly reflected inside corneal boundary line 1422a of a user with a wide cornea, that is, from the cornea. As shown in FIG. 4B, light emitted from right IREDs 112-118 arranged in a range of 90 degrees centered on the facing position of right gaze sensor 106 is specularly reflected inside corneal boundary line 1422b of a user with a narrow cornea, that is, from the cornea. Since reflection positions 112a-118a are near the optical axis of the eyeball, light emitted from right IREDs 112-118 is likely to be specularly reflected from the cornea even when the eye is rotated.

[0043] The reflection positions 108a, 109a, 120a to 122a of the infrared light emitted from the right IREDs 108, 109, 120 to 122 are located in the sclera. Therefore, when the user faces forward as shown in Figures 4A and 4B, the regular reflection images (Purkinje images) of the right IREDs 108, 109, 120 to 122 cannot be obtained. The right IREDs 108, 109, 120 to 122 are provided to illuminate the entire eyeball 141 and to obtain Purkinje images when the eyeball 141 rotates.

[0044] Therefore, by arranging many right IREDs in the portion facing right gaze sensor 106, the possibility of acquiring a specular reflection image (Purkinje image) for gaze detection increases, and the success rate of gaze detection can be increased.

[0045] In the first embodiment, many IREDs are arranged in a range of 120 degrees to accommodate people with wide corneas, but many IREDs may be arranged in a range of 90 degrees to accommodate people with narrow corneas. By narrowing the range in which many IREDs are arranged, costs can be reduced.

[0046] When an image of the eyeball 141 (right eye) is obtained by the right gaze sensor 106, the gaze is detected using a combination of the pupil image and the Purkinje image in the iris image. For example, the gaze detection is performed by the method disclosed in Japanese Patent No. 3186072.

[0047] As described above, according to the first embodiment, the number of light sources per degree is maximized in the range on the opposite side of the gaze sensor across the center of the opening facing the user's eyes (the optical axis of the display optical system). This makes it possible to provide a head-mounted display device (gaze detection device) that can detect the gaze with high accuracy regardless of the situation. For example, it is possible to provide a head-mounted display device that can detect the gaze even when the user moves their eyes, without increasing the size.

[0048] Second Embodiment A second embodiment of the present invention will be described below with reference to Figures 6 and 7. As the second embodiment, an example of a glasses-type device equipped with a gaze detection device will be described.

[0049] Fig. 6 is a rear view of the glasses-type device 249 according to the second embodiment, and Fig. 7 is a perspective view of the glasses-type device 249. Fig. 6 shows the glasses-type device 249 as viewed from the eyeball side of a user (a user wearing the glasses-type device 249).

[0050] The eyeglass-type device 249 has a frame 250, a right temple 250a, and a left temple 250b. Any lens, such as a lens for myopia, a lens for hyperopia, a dimming lens, or a non-prescription lens, may be fitted to the frame 250, or no lens may be fitted to the frame 250. The right temple 250a and the left temple 250b are fixing parts for fixing the eyeglass-type device 249 to the user's head. The user fixes the eyeglass-type device 249 to the head by hanging the right temple 250a on the right ear and the left temple 250b on the left ear. A camera 251 is attached to the right temple 250a, and the camera 251 is directed toward the front of the eyeglass-type device 249.

[0051] The frame 250 is provided with a right gaze sensor 252, a left gaze sensor 253, right IREDs 254, 255, 257 to 268, and left IREDs 269, 270, 272 to 283. The camera 251, the right gaze sensor 252, the left gaze sensor 253, the right IREDs 254, 255, 257 to 268, and left IREDs 269, 270, 272 to 283 are connected to a control circuit (not shown). The glasses-type device 249 has a function of detecting where in the imaging range of the camera 251 the gaze is directed.

[0052] As shown in FIG. 6, in the second embodiment, similar to the first embodiment, the number of light sources per degree is greatest in the range opposite the gaze sensor across the center of the opening facing the user's eye.

[0053] As described above, according to the second embodiment, similarly to the first embodiment, the user's eyes can be easily recognized. The number of light sources per degree is maximized in the range on the opposite side of the gaze sensor across the center of the opening (the optical axis of the display optical system) facing the eye. This makes it possible to provide a glasses-type device (gaze detection device) that can detect the gaze with high accuracy regardless of the situation.

[0054] (Third embodiment) A third embodiment of the present invention will be described below with reference to Figures 8 to 12. As a first embodiment, an example of a head-mounted display device (head-mounted display) equipped with a gaze detection device will be described.

[0055] FIG. 8 is a rear view of a display unit body 301 of a head mounted display according to the third embodiment, showing the display unit body 301 as viewed from the eyeball side of a user (a user wearing the head mounted display).

[0056] The display unit body 301 has a right opening 345 that limits the field of view of the user's right eye, and a left opening 346 that limits the field of view of the user's left eye. A right lens system 302, which is a display optical system, is provided in the right opening 345 (inside the right opening), and a left lens system 303, which is a display optical system, is provided in the left opening 346 (inside the left opening). The right opening 345 and the right lens system 302 are arranged to face the right eye of the user (user wearing a head mounted display). The left opening 346 and the left lens system 303 are arranged to face the left eye of the user (user wearing a head mounted display).

[0057] 8, right gaze sensor 306 is arranged on the edge of right opening 345 (right lens system 302) so as to face the direction of the right eye facing right opening 345. Similarly, left gaze sensor 307 is arranged on the edge of left opening 346 (left lens system 303) so as to face the direction of the left eye facing left opening 346.

[0058] Right gaze sensor 306 is disposed at the 9 o'clock position (a position rotated 270 degrees clockwise from directly above) around the center of right opening 345 (the optical axis of right lens system 302). Left gaze sensor 307 is disposed at the 3 o'clock position (a position rotated 90 degrees clockwise from directly above) around the center of left opening 346 (the optical axis of left lens system 303). Right gaze sensor 306 and left gaze sensor 307 are disposed at approximately the same height (position in the up-down direction). Right gaze sensor 306 is disposed to the left of right opening 345 (near the right eye), and left gaze sensor 307 is disposed to the right of left opening 346 (near the left eye).

[0059] In this way, the right gaze sensor 306 and the left gaze sensor 307 are positioned on a horizontal line 385 passing through the optical axis of the right lens system 302 and the optical axis of the left lens system 303 (on a horizontal line passing through the center of the right opening 345 and the center of the left opening 346).

[0060] Normally, the eyelids of a user (a user wearing a head mounted display) open up and down. Therefore, by arranging the right gaze sensor 306 and the left gaze sensor 307 so that the eyeballs are observed from the direction of the horizon 385, the eyeballs are less likely to be eclipsed by the eyelids during gaze detection, and the success rate of gaze detection can be increased.

[0061] 8, a plurality of infrared light emitting diodes (right IREDs) are arranged along the edge of the right opening 345 (right lens system 302) as a plurality of light sources for illuminating the eyeball (right eye). Similarly, a plurality of infrared light emitting diodes (left IREDs) are arranged along the edge of the left opening 346 (left lens system 303) as a plurality of light sources for illuminating the eyeball (left eye).

[0062] Around the right opening 345 (right lens system 302), right IREDs 308-311, 315, 317, 319-322 are arranged as a first light source group, and right IREDs 386 and 387 are arranged as a second light source group. With the center of the right opening 345 (the optical axis of the right lens system 302) as the center, the right IRED 308 is arranged at the 10 o'clock position, the right IRED 309 at the 11 o'clock position, the right IRED 310 at the 12 o'clock position, the right IRED 311 at the 1 o'clock position, the right IRED 313 at the 2 o'clock position, and the right IRED 315 at the 3 o'clock position. The right IRED 317 is arranged at the 4 o'clock position, the right IRED 319 at the 5 o'clock position, the right IRED 320 at the 6 o'clock position, the right IRED 321 at the 7 o'clock position, and the right IRED 322 at the 8 o'clock position. The right IRED 386 is disposed on a line passing through the center of the right opening 345 (the optical axis of the right lens system 302) and the right IRED 317. The right IRED 387 is disposed on a line passing through the center of the right opening 345 (the optical axis of the right lens system 302) and the right IRED 320.

[0063] Around the left opening 346 (left lens system 303), left IREDs 323-326, 328, 330, 332, 334-337 are arranged as a first light source group, and left IREDs 388 and 389 are arranged as a second light source group. With the center of the left opening 346 (the optical axis of the left lens system 303) as the center, the left IRED 323 is arranged at the 2 o'clock position, the left IRED 324 at the 1 o'clock position, the left IRED 325 at the 12 o'clock position, the left IRED 326 at the 11 o'clock position, the left IRED 328 at the 10 o'clock position, and the left IRED 330 at the 9 o'clock position. The left IRED 332 is arranged at the 8 o'clock position, the left IRED 334 at the 7 o'clock position, the left IRED 335 at the 6 o'clock position, the left IRED 336 at the 5 o'clock position, and the left IRED 337 at the 4 o'clock position. The left IRED 388 is disposed on a line passing through the center of the left opening 346 (the optical axis of the left lens system 303) and the left IRED 332. The left IRED 389 is disposed on a line passing through the center of the left opening 346 (the optical axis of the left lens system 303) and the left IRED 335.

[0064] The distance from each of the right IREDs 386 and 387 included in the second light source group to the right opening 345 is longer than the distance from each of the right IREDs 308-311, 315, 317, and 319-322 included in the first light source group to the right opening 345. Furthermore, the right IREDs 308-311, 313, 315, 317, and 319-322 included in the first light source group are disposed inside the right IREDs 386 and 387 included in the second light source group with respect to the right opening 345. Similarly, the distance from each of the left IREDs 388 and 389 included in the second light source group to the left opening 346 is longer than the distance from each of the left IREDs 323-326, 328, 330, 332, and 334-337 included in the first light source group to the left opening 346. Furthermore, with respect to the left opening 346, the left IREDs 323 to 326, 328, 330, 332, and 334 to 337 included in the first light source group are disposed inside the left IREDs 388 and 389 included in the second light source group.

[0065] For example, the IREDs (plurality of first light sources) included in the first light source group are distributed at a first distance r1 from the optical axis of the display optical system, and the IREDs (plurality of second light sources) included in the second light source group are distributed at a second distance r2 from the optical axis of the display optical system. The second distance r2 is greater than the first distance r1. Note that the distance from each of the IREDs included in the first light source group to the optical axis of the display optical system may not be constant, and the distance from each of the IREDs included in the second light source group to the optical axis of the display optical system may not be constant. For example, the distance from each of the IREDs included in the first light source group to the optical axis of the display optical system may vary within a range of distance r1a or more and distance r1b or less. The distance from each of the IREDs included in the second light source group to the optical axis of the display optical system may vary within a range of distance r2a or more and distance r2b or less. The distances r1, r1a, r1b, r2, r2a, and r2b satisfy the following formula. r1a <r1≦r1b<r2a≦r2<r2b

[0066] Although not shown, the display unit body 301 is equipped with a CPU (control unit) that controls each IRED. It controls the lighting of the RED. The CPU also detects the user's line of sight based on the output of the gaze sensor (Purkinje image, etc.).

[0067] Fig. 9 is a schematic diagram of a vertical section passing through the optical axis of the right lens system 302 in Fig. 8. Fig. 9 shows the right eye and its surroundings. Fig. 9 also shows the optical paths of light emitted from the right IREDs 310, 320, and 387. Although the right gaze sensor 306 does not exist on the vertical section, it is shown for ease of understanding.

[0068] In FIG. 9, light from the right IRED 310 arranged on the upper side of the right lens system 302 is vignetted by the user's upper eyelid 350 and is not specularly reflected on the surface of the cornea. Therefore, the light from the right IRED 310 is not reflected as a specularly reflected image (Purkinje image) in the image obtained by the right gaze sensor 306. In contrast, light from the right IRED 320 arranged on the lower side of the right lens system 302 is not vignetted by the upper eyelid 350 or the lower eyelid 351 and is specularly reflected on the surface of the cornea. Therefore, the light from the right IRED 320 is reflected as a specularly reflected image (Purkinje image) in the image obtained by the right gaze sensor 306. Similarly, light from the right IRED 387 arranged on the lower side of the right lens system 302 is not vignetted by the upper eyelid 350 or the lower eyelid 351 and is specularly reflected on the surface of the cornea. Therefore, the light from right IRED 387 appears as a specularly reflected image (Purkinje image) in the image obtained by right gaze sensor 306.

[0069] Although the shape of the eyelids differs from person to person, and the positional relationship between the display unit body 301 and the eyes also differs from person to person, there is a certain tendency for light to be vignetted. Consider a case where right gaze sensor 306 and left gaze sensor 307 are disposed on a horizontal line 385 passing through the optical axis of right lens system 302 and the optical axis of left lens system 303. In this case, light from an IRED disposed below horizontal line 385 is less likely to be vignetted by the eyelids than light from an IRED disposed above horizontal line 385.

[0070] 8, more IREDs of the second light source group are arranged below the horizontal line 385 (below the right opening 345 and the left opening 346) than above the horizontal line 385 (above the right opening 345 and the left opening 346). This makes it possible to avoid a situation in which light from all the IREDs is vignetted by the eyelid and no specular reflection image (Purkinje image) is obtained, thereby increasing the success rate of gaze detection (increasing the robustness of gaze detection). Note that, in FIG. 8, no IREDs of the second light source group are arranged above the horizontal line 385 (above the right opening 345 and the left opening 346), but IREDs of the second light source group may be arranged above the horizontal line 385.

[0071] 9, the right IREDs 310 and 320 belonging to the first light source group are disposed inside the right lens system 302, and the right IRED 387 belonging to the second light source group is disposed outside the right lens system 302. Therefore, the right IREDs 310 and 320 emit light to the outside via the right lens system 302 to illuminate the eyeball 341 (right eye). The right IRED 387 emits light to the outside without passing through the right lens system 302 to illuminate the eyeball 341. Note that the right IREDs are not limited to the right IREDs 310, 320, and 387, and the right IRED belonging to the first light source group emits light to the outside via the right lens system 302, and the right IRED belonging to the second light source group emits light to the outside without passing through the right lens system 302. Similarly, the left IRED belonging to the first light source group emits light to the outside via the left lens system 303 , and the left IRED belonging to the second light source group emits light to the outside without passing through the left lens system 303 .

[0072] As a result, when both the first light source group and the second light source group cannot be placed inside the display optical system (right lens system 302 or left lens system 303) due to space constraints, the device can be made smaller than when both the first light source group and the second light source group are placed outside the display optical system.

[0073] From the viewpoint of aesthetics, it is preferable that each IRED belonging to the second light source group is hidden by a material that does not transmit visible light but transmits infrared light. However, as in the first embodiment, the transmittance of visible light is not limited to 0, and some visible light may be transmitted. Although the same effect can be obtained if the transmittance of infrared light is higher than the transmittance of visible light, it is preferable that the transmittance (blocking rate) of visible light and that of infrared light are significantly different.

[0074] Here, when the light from the first light source group passes through the display optical system, the amount of the light is reduced. In particular, when the display optical system includes a polarized reflection optical system, the amount of light from the first light source group may be reduced to about half or less. For this reason, when all the IREDs are turned on with the same luminance, the illuminance illuminating the eyeball differs between the IREDs of the first light source group that illuminate the eyeball via the display optical system and the IREDs of the second light source group that illuminate the eyeball without the display optical system. As a result, the luminance of the specular reflection image (Purkinje image) differs between the first light source group and the second light source group. Such a difference in luminance has an adverse effect (such as a decrease in accuracy) on the detection of the Purkinje image, and thus has an adverse effect (such as a decrease in accuracy) on gaze detection.

[0075] Therefore, in the third embodiment, the CPU lights up each IRED of the first light source group and each IRED of the second light source group at different luminances. For example, the CPU increases the luminance of each IRED of the first light source group that illuminates the eyeball via the display optical system by the amount of light absorbed by the display optical system, and lights up each IRED of the first light source group at a luminance higher than each IRED of the second light source group that illuminates the eyeball without the display optical system.

[0076] This makes the luminance of the Purkinje image of the first light source group equal to the luminance of the Purkinje image of the second light source group, improving the accuracy of Purkinje image detection and, in turn, improving the accuracy of gaze detection.

[0077] 10A and 10B are schematic diagrams of a cross section passing through right IRED 315 and right gaze sensor 306. Illustrated in Figures 10A and 10B are the optical paths of light emitted from right IRED 315, 386. Note that right IRED 386 does not exist on the cross section, but is shown for clarity.

[0078] Fig. 10A shows a state where eyeball 341 (right eye) has rotated in the direction where right gaze sensor 306 is disposed (i.e., to the left). Fig. 10B shows a state where eyeball 341 has rotated in the direction opposite right gaze sensor 306 (i.e., to the right).

[0079] 10A, when the eyeball 341 rotates toward the position where the right gaze sensor 306 is disposed, the light emitted from the right IRED 386 is reflected by the outside (sclera) of the corneal boundary line 3422a. In contrast, the light emitted from the right IRED 315 is specularly reflected by the inside (cornea) of the corneal boundary line 3422a.

[0080] 10B, when the eyeball 341 rotates in a direction facing the right gaze sensor 306, the light emitted from the right IRED 315 is reflected by the outside (sclera) of the corneal boundary line 3422a. In contrast, the light emitted from the right IRED 386 is specularly reflected by the inside (cornea) of the corneal boundary line 3422a.

[0081] In this way, by arranging IREDs (first light source group) that are close to the optical axis of the display optical system and IREDs (second light source group) that are far from the optical axis of the display optical system, it is possible to obtain a specular reflection image (Purkinje image) with a high probability even when the user moves their eyes, which in turn increases the success rate of gaze detection (increases the robustness of gaze detection).

[0082] As shown in FIG. 10A, when the eyeball 341 rotates in the direction in which the right gaze sensor 306 is located, the gaze can be detected using the specular reflection image of the right IRED 315 (first light source group), which is closer to the right opening 345.

[0083] As shown in FIG. 10B, when the eyeball 341 rotates in a direction facing the right gaze sensor 306, the gaze can be detected using the specular reflection image of the right IRED 386 (second light source group), which is located at a longer distance from the right opening 345.

[0084] 11 is a flowchart of gaze detection. In the following, gaze detection for the right eye will be described, but gaze detection for the left eye is performed in the same way.

[0085] In step S3001, the CPU turns on all the right IREDs (first light source group and second light source group) and captures an image of the eyeball 341 (right eye) using the right gaze sensor 306. Note that although all of the right IREDs are turned on in the above embodiment, only some of the right IREDs may be turned on.

[0086] In step S3002, the CPU performs a first gaze detection (provisional detection of the gaze) using all the reflected images that appear in the image of the eyeball 341 acquired in step S3001. The reflected images that appear in the image of the eyeball 341 can also be regarded as reflected images formed on the right gaze sensor 306. Here, because the reflected images (inaccurate Purkinje images) reflected from the sclera are also used, the gaze cannot be detected with high accuracy, but the approximate rotation direction and rotation angle of the eyeball 341 can be determined.

[0087] In step S3003, the CPU determines whether eyeball 341 has rotated in the direction in which right gaze sensor 306 is disposed, or in a direction opposite right gaze sensor 306. If the CPU determines that eyeball 341 has rotated in the direction in which right gaze sensor 306 is disposed, the CPU advances the process to step S3004, and if the CPU determines that eyeball 341 has rotated in a direction opposite right gaze sensor 306, the CPU advances the process to step S3005.

[0088] In step S3004, the CPU selects a reflected image (specular reflected image) of the first light source group that is close to right opening 345, and performs second gaze detection (main gaze detection) using the selected reflected image.

[0089] In step S3005, the CPU selects a reflected image (specular reflected image) of the second light source group that is farther away from right opening 345, and performs second gaze detection (main gaze detection) using the selected reflected image.

[0090] In this way, the reflected image used for gaze detection is changed based on the rotation angle of the eyeball 341. This allows gaze detection to be performed without using the inaccurate Purkinje image reflected from the sclera, improving the accuracy of gaze detection compared to a configuration in which gaze detection is always performed using all reflected images.

[0091] Although an example in which only one of the reflected image of the first light source group and the reflected image of the second light source group is finally used has been described, the present invention is not limited to this. For example, only one of the reflected image of the first light source group and the reflected image of the second light source group may be used in a certain portion, and only the other of the reflected image of the first light source group and the reflected image of the second light source group may be used in another portion. In a scene in which both the reflected image of the first light source group and the reflected image of the second light source group are regular reflected images, the result of gaze detection using both the reflected image of the first light source group and the reflected image of the second light source group (all reflected images) may be the final result.

[0092] 12 is a modified flowchart of gaze detection. In the following, gaze detection for the right eye will be described, but gaze detection for the left eye is performed in the same manner.

[0093] In step S3006, the CPU turns on all of the right IREDs (first light source group and second light source group), and uses the right line of sight sensor 306 to capture an image of eyeball 341 (right eye).

[0094] In step S3007, the CPU performs a first gaze detection (provisional gaze detection) using all the reflected images that appear in the image of the eyeball 341 acquired in step S3001. Here, because the reflected images (inaccurate Purkinje images) reflected from the sclera are also used, the gaze cannot be detected with high accuracy, but the approximate rotation direction and rotation angle of the eyeball 341 can be determined.

[0095] In step S3008, the CPU determines whether eyeball 341 has rotated in the direction in which right gaze sensor 306 is disposed, or in a direction opposite right gaze sensor 306. If the CPU determines that eyeball 341 has rotated in the direction in which right gaze sensor 306 is disposed, the CPU advances the process to step S3009, and if the CPU determines that eyeball 341 has rotated in a direction opposite right gaze sensor 306, the CPU advances the process to step S3010.

[0096] In step S3009, the CPU turns off the second light source group, which is located farther from the right opening 345, and turns on only the first light source group, which is located closer to the right opening 345.

[0097] In step S3010, the CPU turns off the first light source group, which is closer to the right opening 345, and turns on only the second light source group, which is farther from the right opening 345.

[0098] In step S3011, the CPU performs second gaze detection (main gaze detection) using an image of the eyeball 341 obtained with some of the right IREDs turned off.

[0099] In this way, the right IRED to be used for gaze detection (the right IRED to be turned on) is changed based on the rotation angle of the eyeball 341. This makes it possible to turn off the right IRED not used for gaze detection, thereby saving power.

[0100] Although an example in which only one of the first light source group and the second light source group is finally turned on has been described, this is not limiting. For example, only one of the IRDEs of the first light source group and the IREDs of the second light source group may be turned on in a certain portion, and only the other of the IREDs of the first light source group and the IREDs of the second light source group may be turned on in another portion. In a scene in which both the reflected image of the first light source group and the reflected image of the second light source group are regular reflected images, the result of gaze detection obtained by turning on both the first light source group and the second light source group (all IREDs) may be the final result.

[0101] As described above, according to the third embodiment, IREDs (first light source group) that are close to the optical axis of the display optical system and IREDs (second light source group) that are long from the optical axis of the display optical system are arranged. This makes it possible to provide a head-mounted display device (gaze detection device) that enables highly accurate gaze detection regardless of the situation. For example, even if the user moves their eyes, a specular reflection image (Purkinje image) can be acquired with a high probability. This in turn makes it possible to increase the success rate of gaze detection (increase the robustness of gaze detection).

[0102] The third embodiment may be combined with the first embodiment. For example, when focusing on at least one of the first light source group and the second light source group, the number of light sources per degree may be maximized in the range on the opposite side of the gaze sensor across the center of the opening facing the user's eye (the optical axis of the display optical system). This can further increase the success rate of gaze detection.

[0103] <Fourth embodiment> Hereinafter, a fourth embodiment of the present invention will be described with reference to FIGS. As an embodiment, an example of a head-mounted display device equipped with a gaze detection device will be described.

[0104] Fig. 13 is a rear view of a display unit body 401 of a head mounted display according to a fourth embodiment. Fig. 14 is a cross-sectional view taken along line AA in Fig. 13. Fig. 13 shows the display unit body 101 as viewed from the eyeball side of a user (a user wearing a head mounted display).

[0105] The display unit body 401 has a right opening 445 that limits the field of view of the user's right eye, and a left opening 446 that limits the field of view of the user's left eye. A right lens system 402, which is a display optical system, is provided in the right opening 445 (in the right opening), and a left lens system 403, which is a display optical system, is provided in the left opening 446 (in the left opening). The right opening 445 and the right lens system 402 are arranged to face the right eye of the user (a user wearing a head mounted display). The left opening 446 and the left lens system 403 are arranged to face the left eye of the user (a user wearing a head mounted display).

[0106] As shown in FIG. 13, right gaze sensor 406 is disposed on the edge of right opening 445 (right lens system 402) so as to face the direction of the right eye facing right opening 445. Right gaze sensor 406 is disposed on the back side of right lens system 402, and light is incident on right gaze sensor 406 from the outside via right lens system 402. Similarly, left gaze sensor 407 is disposed on the edge of left opening 446 (left lens system 403) so as to face the direction of the left eye facing left opening 446. Left gaze sensor 407 is disposed on the back side of left lens system 403, and light is incident on left gaze sensor 407 from the outside via left lens system 403. Right gaze sensor 406 and left gaze sensor 407 are sensitive only to infrared light, and are sensitive only to a range of, for example, 900 nm±20 nm.

[0107] Right gaze sensor 406 is disposed at the 9 o'clock position (a position rotated 270 degrees clockwise from directly above) around the center of right opening 445 (the optical axis of right lens system 402). Left gaze sensor 407 is disposed at the 3 o'clock position (a position rotated 90 degrees clockwise from directly above) around the center of left opening 446 (the optical axis of left lens system 403). Right gaze sensor 406 and left gaze sensor 407 are disposed at approximately the same height (position in the up-down direction). Right gaze sensor 406 is disposed to the left of right opening 445 (near the right eye), and left gaze sensor 407 is disposed to the right of left opening 446 (near the left eye).

[0108] In this way, the right gaze sensor 406 and the left gaze sensor 407 are arranged on a horizontal line 485 passing through the optical axis of the right lens system 402 and the optical axis of the left lens system 403 (on a horizontal line passing through the center of the right opening 445 and the center of the left opening 446).

[0109] Normally, the eyelids of a user (a user wearing a head mounted display) open up and down. Therefore, by arranging the right gaze sensor 406 and the left gaze sensor 407 so that the eyeballs are observed from the direction of the horizon 485, the eyeballs are less likely to be eclipsed by the eyelids during gaze detection, and the success rate of gaze detection can be increased.

[0110] In FIG. 13, a plurality of infrared light emitting diodes (right IREDs) are arranged along the edge of the right opening 445 (right lens system 402) as a plurality of light sources that illuminate the eyeball (right eye). The right IRED is arranged on the rear side of the right lens system 402, and emits light (infrared light) to the outside via the right lens system 402 to illuminate the eyeball (right eye). Similarly, a plurality of infrared light emitting diodes (left IREDs) are arranged along the edge of the left opening 446 (left lens system 403) as a plurality of light sources that illuminate the eyeball (left eye). The left IRED is arranged on the rear side of the left lens system 403. The left lens system 403 emits light (infrared light) to the outside, illuminating the eyeball (left eye).

[0111] Right IREDs 408, 409, 411 to 422 are disposed around the right opening 445 (right lens system 402). With the center of the right opening 445 (the optical axis of the right lens system 402) as the center, the right IRED 408 is disposed at the 10 o'clock position, the right IRED 409 at the 11 o'clock position, the right IRED 411 at the 1 o'clock position, the right IRED 412 at the 1:30 o'clock position, the right IRED 413 at the 2 o'clock position, and the right IRED 414 at the 2:30 o'clock position. The right IRED 415 is disposed at the 3 o'clock position, the right IRED 416 at the 3:30 o'clock position, the right IRED 417 at the 4 o'clock position, the right IRED 418 at the 4:30 o'clock position, the right IRED 419 at the 5 o'clock position, the right IRED 420 at the 6 o'clock position, the right IRED 421 at the 7 o'clock position, and the right IRED 422 at the 8 o'clock position.

[0112] The left IREDs 423, 424, 426 to 437 are disposed around the left opening 446 (left lens system 403). With the center of the left opening 446 (the optical axis of the left lens system 403) as the center, the left IRED 423 is disposed at the 2 o'clock position, the left IRED 424 at the 1 o'clock position, the left IRED 426 at the 11 o'clock position, the left IRED 427 at the 10:30 o'clock position, the left IRED 428 at the 10 o'clock position, and the left IRED 429 at the 9:30 o'clock position. The left IRED 430 is disposed at the 9 o'clock position, the left IRED 431 at the 8:30 o'clock position, the left IRED 432 at the 8 o'clock position, the left IRED 433 at the 7:30 o'clock position, the left IRED 434 at the 7 o'clock position, the left IRED 435 at the 6 o'clock position, the left IRED 436 at the 5 o'clock position, and the left IRED 437 at the 4 o'clock position.

[0113] In the fourth embodiment, similarly to the first embodiment, the number of light sources per degree (angular density) is maximum in the range on the opposite side of right gaze sensor 406 across the center of right opening 445 (optical axis of right lens system 402). Similarly, the number of light sources per degree (angular density) is maximum in the range on the opposite side of left gaze sensor 407 across the center of left opening 446 (optical axis of left lens system 403). This makes it possible to obtain the same effect as the first embodiment.

[0114] A right-eye gaze detection unit is configured by right gaze sensor 406, right IREDs 408, 409, 411-422, and a control circuit (not shown). Similarly, a left-eye gaze detection unit is configured by left gaze sensor 407, left IREDs 423, 424, 426-437, and a control circuit (not shown). The control circuit for the right-eye gaze detection unit and the control circuit for the left-eye gaze detection unit may be a single common control circuit, or may be separate control circuits.

[0115] FIG. 14 shows the arrangement of the display unit main body 401 and the eyeballs (right eye 441R and left eye 441L) of a user looking into the display unit main body 401.

[0116] The right lens system 402 and the left lens system 403 are configured with a polarizing reflection optical system. The polarizing reflection optical system can be realized, for example, by using the technology disclosed in JP 2020-95205 A.

[0117] Right display panel 450R is a display panel such as an organic EL panel, and is disposed on the rear side of right lens system 402. The user's right eye 441R views right display panel 450R via right lens system 402. Similarly, left display panel 450L is a display panel such as an organic EL panel, and is disposed on the rear side of left lens system 403. The user's left eye 441L views left display panel 450L via left lens system 403.

[0118] The right lens system 402 is composed of a right polarizing panel 452, a right first lens 453, a right second lens 454 having a right polarizing half mirror 454a on one surface (the surface on the right first lens 453 side), and a right polarizing reflector 455. Similarly, the left lens system 403 is composed of a left polarizing panel 462, a left first lens 463, and a left polarizing half mirror 464 on one surface (the surface on the left first lens 463 side). a), and a left polarizing reflector 465. Note that the surfaces of the right second lens 454 and the left second lens 464 facing the user are assumed to be flat.

[0119] The optical path of the right display panel 450R becomes a display optical path 456 that is folded back inside the right second lens 454. Reflection and transmission are controlled by the right polarizing panel 452, the right polarizing half mirror 454a, and the right polarizing reflector 455, thereby realizing the display optical path 456. Similarly, the optical path of the left display panel 450L becomes a display optical path 466 that is folded back inside the left second lens 464. Reflection and transmission are controlled by the left polarizing panel 462, the left polarizing half mirror 464a, and the left polarizing reflector 465, thereby realizing the display optical path 466.

[0120] A right mask 457 having a higher transmittance for infrared light than for visible light is disposed in front of the right IRED 415 (on the user side). Here, the right mask 457 is a mask that does not transmit visible light but transmits infrared light, but if the transmittance (blocking rate) of visible light and infrared light is significantly different, the right mask 457 may transmit some visible light. In FIG. 14, the right mask 457 is depicted in front of the right IRED 415, but in reality, it is disposed between the right IREDs 408, 409, 411 to 422 and the right first lens 453. Similarly, a left mask 467 having a higher transmittance for infrared light than for visible light is disposed in front of the left IRED 430 (on the user side). Here, the left mask 467 is also a mask that does not transmit visible light but transmits infrared light, but like the right mask 457, if the transmittance (blocking rate) of visible light and infrared light is significantly different, the left mask 467 may transmit some visible light. Although the left mask 467 is depicted in front of the left IRED 430 in FIG. 14, in reality it is disposed between the left IREDs 423, 424, 426-437 and the left first lens 463.

[0121] Light emitted from right IRED 415 passes through right lens system 402, reflects off right eye 441R, passes through right lens system 402 again, and enters right gaze sensor 406. This optical path is shown as right gaze detection optical path 460. Similarly, light emitted from left IRED 430 passes through left lens system 403, reflects off left eye 441L, passes through left lens system 403 again, and enters left gaze sensor 407. This optical path is shown as left gaze detection optical path 470.

[0122] In Fig. 14, light emitted from right IRED 415 illuminates the cornea 442R and iris (not shown) of right eye 441R. Right gaze sensor 406 acquires an iris image of right eye 441R and also acquires a specular reflection image (Purkinje image) of right IRED 415 passing through right gaze detection optical path 460. Although not shown in Fig. 14, light from other right IREDs similarly illuminates the cornea 442R and iris of right eye 441R. Because a single iris is illuminated by multiple right IREDs, uneven illumination of the iris is reduced.

[0123] Similarly, light emitted from the left IRED 430 illuminates the cornea 442L and iris (not shown) of the left eye 441L. The left gaze sensor 407 acquires an iris image of the left eye 441L, as well as a specular reflection image (Purkinje image) of the left IRED 430 that passes through the left gaze detection optical path 470. Although not shown in Fig. 14, light from other left IREDs similarly illuminates the cornea 442L and iris of the left eye 441L. Because a single iris is illuminated by multiple left IREDs, uneven illumination of the iris is reduced.

[0124] 14, of the surface of right first lens 453 (surface on the right display panel 450R side), a portion (entrance surface 453a) facing right IRED 415 (plural right IREDs) and a portion (exit surface 453b) facing right eye gaze sensor 406 have different shapes. Similarly, of the surface of left first lens 463 (surface on the left display panel 450L side), a portion (entrance surface 463a) facing left IRED 430 (plural left IREDs) and a portion (exit surface 463b) facing left eye gaze sensor 407 have different shapes.

[0125] 15 is a perspective view of right first lens 453 and right second lens 454. Right first lens 453 has the basic shape of a body of revolution, and incident surface 453a is a conical surface. In the portion of exit surface 453b, optical correction unit 470a similar to a triangular prism is integrally formed on the conical surface. And exit surface 453b (surface facing right gaze sensor 406, surface of optical correction unit 470a) is a plane (flat).

[0126] A triangular prism generally has the property that the refraction angle changes for each wavelength. However, the right IRED only emits light in a limited wavelength range (infrared light). In addition, the right gaze sensor 406 also only receives light in a limited wavelength range (infrared light). Therefore, the above-mentioned property of the triangular prism (the property that the refraction angle changes for each wavelength) can be ignored.

[0127] Here, consider a case where exit surface 453b is not flat but has a curvature. In that case, optical aberration occurs in the light from exit surface 453b, and the image received by right gaze sensor 406 is deteriorated. By making exit surface 453b flat, the optical aberration can be improved. The light emitted from the IRED is divided into diffuse reflected light that illuminates and reflects a wide range of the iris, and specular reflected light that is specularly reflected by the cornea. By making exit surface 453b flat, a pupil image with improved optical aberration can be detected from the diffuse reflected light. The specular reflected light can be treated as reflected light from a point light source. Therefore, by making exit surface 453b flat, a Purkinje image with improved optical aberration can also be detected from the specular reflected light. As a result, the gaze can be detected with high accuracy from the pupil image and the Purkinje image.

[0128] In the fourth embodiment, since the portion of the incident surface 453a does not have a prism shape, it is possible to place the right mask 457. By placing the right mask 457, the right IRED is not visible when the user looks into the right lens system 402, and an excellent aesthetic appearance can be realized.

[0129] The left eye side is similar to the right eye side. Left first lens 463 has the basic shape of a body of revolution, and incident surface 463a is a conical surface. At exit surface 463b, an optical correction unit similar to a triangular prism is integrally formed on the conical surface. Exit surface 463b (the surface facing left gaze sensor 407, the surface of the optical correction unit) is a flat surface (flat). This allows the same effect to be obtained on the left eye side as on the right eye side.

[0130] As described above, according to the fourth embodiment, the portion of the surface of the display optical system facing the gaze sensor is made flat, which can improve the optical aberration of the image obtained by the gaze sensor and improve the accuracy of gaze detection.

[0131] <Fifth embodiment> Hereinafter, a fifth embodiment of the present invention will be described with reference to Figs. 16 to 20. As the fifth embodiment, an example of an imaging device (camera body) equipped with a gaze detection device will be described. The camera body according to the fifth embodiment detects the gaze of a user looking into a finder (EVF (Electronic ViewFinder) unit) and captures an image of a subject that is in front of the camera body and the user. The front direction of the camera body parallel to the optical axis of the photographing optical system is defined as the Z axis (Z axis direction). In the most basic standard position of the camera body, that is, in the normal position, the vertical upward direction perpendicular to the Z axis is defined as the Y axis (Y axis direction). Then, according to the right-handed system, the direction perpendicular to the Y axis and the Z axis is defined as the X axis (X axis direction).

[0132] Fig. 16 is a schematic diagram showing a cross section of a camera body 500 according to the fifth embodiment. Fig. 16 shows a so-called central vertical cross section of the camera body 500 cut along a plane parallel to the Y axis and the Z axis. In the camera body 500, a shutter 590 and an image sensor are arranged on the optical axis of a photographing optical system (not shown). The EVF unit 501, shutter 590, imaging sensor 591, and rear monitor 592 are lined up. The imaging sensor captures an image of a subject that is in front of the camera body 500. A rear monitor 592 is provided on the rear of the camera body 500. The rear monitor 592 displays menus and images for accepting operations of the camera body 500 and for viewing and editing images obtained by the camera body 500. The rear monitor 592 is composed of a backlit LCD panel or an organic EL panel. As with a normal general camera, the EVF unit 501, shutter 590, imaging sensor 591, and rear monitor 592 are controlled by a CPU 593, which performs input and output processing of various necessary information.

[0133] The EVF unit 501 includes an EVF panel 540 as a display panel, an EVF lens system 502 as a display optical system, and a line-of-sight sensor 506 as an eyeball imaging unit. The EVF unit 501 is built-in or attached so that a user of the camera body 500 can view the display screen of the EVF panel 540. In Figure 16, the user is looking into the EVF unit 501 with his eyeball 541.

[0134] The EVF panel 540 is a display panel, and is composed of an organic EL panel or a liquid crystal display panel with a backlight. The EVF panel 540 is arranged so that the display screen faces the negative Z-axis direction. The EVF lens system 502 is arranged in front of the display screen of the EVF panel 540, and is composed of one or more lenses arranged along a display optical system optical axis 583 (the optical axis of the display optical system (EVF lens system 502)) extending in the negative Z-axis direction. The lenses of the EVF lens system 502 are optical glass or transparent optical plastic lenses manufactured by cutting and grinding or molding. In FIG. 16, the EVF lens system 502 is composed of three lenses, a G1 lens 562, a G2 lens 563, and a G3 eyepiece lens 564, which are optical lenses that transmit visible light. The number of lenses included in the EVF lens system 502 is not limited to three, and may be, for example, four or five. In order to enlarge the display screen of the EVF panel 540, the EVF lens system 502 can be configured by combining an appropriate number of lenses.

[0135] The gaze sensor 506 forms an image of the eyeball 541 looking into the EVF unit 501 on the gaze sensor chip 548. The gaze sensor 506 has a gaze sensor lens 547 and a gaze sensor chip 548 inside a gaze sensor housing 546. The gaze sensor chip 548 is arranged on a gaze sensor lens optical axis 547a (optical axis of the gaze sensor lens). The gaze sensor lens 547 is an optical system required to form an image of the eyeball 541 on the gaze sensor chip 548, and is composed of an appropriate optical lens. Although one lens is shown as the gaze sensor lens 547 in FIG. 16, the gaze sensor lens 547 may include multiple lenses. The gaze sensor chip 548 is an image sensor that A / D converts an image including an infrared light component of the formed eyeball 541 and inputs the result to the CPU 593. For example, a so-called CMOS imaging sensor or a CCD matrix sensor is used as the gaze sensor chip 548. Although the line of sight sensor 506 is a small camera in which the above components are integrated into a package, the above components do not have to be integrated.

[0136] EVF unit 501 will be described in more detail with reference to Figures 17 and 18. Figure 17 is a schematic diagram showing a portion of a cross section (a portion close to eyeball 541) of camera body 500. Figure 18 is a schematic diagram of EVF unit 501 as viewed from the eyeball 541 side, that is, as viewed in the positive direction of the Z axis.

[0137] When a user looks into the display screen of the EVF panel 540, the eyeball 541 is positioned near the optical axis 583 of the display optical system of the G3 eyepiece 564 as shown in Fig. 17. The eyeball 541 is covered by the upper eyelid 550 and the lower eyelid 551, and the cornea 542 is exposed between the upper eyelid 550 and the lower eyelid 551. At this time, infrared light is emitted from the IREDs 552 to 561 arranged around the G3 eyepiece 564, and the eyeball 541 is illuminated by the infrared light. .

[0138] Around the G3 eyepiece 564, an infrared-transmitting cover 504 is arranged, which is made of a material (e.g., resin) that does not transmit (absorbs) visible light and transmits infrared light, and the infrared-transmitting cover 504 hides the IREDs 552 to 561 so that they cannot be seen from the outside. The infrared-transmitting cover 504 is provided with an opening 545 that passes an effective light beam of visible light that has passed through the G3 eyepiece 564, so that the display screen of the EVF panel 540 can be seen. The opening 545 may or may not be a physical opening as long as it can transmit visible light. For example, the infrared-transmitting cover 504 may be an infrared-transmitting paint (a paint that does not transmit visible light and transmits infrared light) applied to the G3 eyepiece 564. In that case, a portion where the infrared-transmitting paint is not applied may be provided in an opening shape, and the portion may be the opening 545. As in the first embodiment, the visible light transmittance of the infrared transparent cover 504 is not limited to 0, and if the infrared light transmittance is higher than the visible light transmittance, it is possible to obtain the effect of making the IREDs 552 to 561 less visible to the user. As in the first embodiment, it is preferable that the difference between the visible light transmittance and the infrared light transmittance is large.

[0139] A proximity sensor 549 for detecting the proximity of the eyeball 541 is disposed around the G3 eyepiece 564. The proximity sensor 549 is a unit having an infrared light emitting section and an infrared light receiving section, and measures the distance between the G3 eyepiece 564 and the eyeball 541 using, for example, the reflection angle, time difference, or frequency of the irradiated infrared light. The distance information measured by the proximity sensor 549 is sent to the CPU 593, and is used, for example, as information necessary for controlling the lighting of the EVF panel 540 and the IREDs 552 to 561. The proximity sensor 549 is also hidden by the infrared transparent cover 504.

[0140] Display optical system optical axis 583 and gaze sensor lens optical axis 547a are not parallel, but form an angle T510. For example, in the normal position of camera body 500, gaze sensor 506 is located below display optical system optical axis 583, and gaze sensor lens optical axis 547a faces upward. The normal position can also be considered as a posture in which the left-right direction of eyeball 541 approximately matches the left-right direction of opening 545.

[0141] Upper eyelid 550 and lower eyelid 551 cover eyeball 541. As described in the first embodiment, in most cases, upper eyelid 550 is larger and thicker than lower eyelid 551. Camera body 500 is often used in the normal position. Therefore, gaze sensor 506 is arranged so as to look up at eyeball 541 from the lower eyelid 551 side in the normal position. This makes it possible to suppress a situation in which the image of eyeball 541 formed on gaze sensor chip 548 is vignetted by the eyelids, compared to a case in which gaze sensor 506 is arranged so as to look down at eyeball 541 from the upper eyelid 550 side in the normal position.

[0142] As shown in FIG. 18, ten IREDs 552 to 561 are arranged so as to surround the periphery of a G3 eyepiece 564 (or an opening 545). With respect to the display optical system optical axis 583 as the center, IRED 555 is arranged at the 1 o'clock position, IRED 561 at the 1:30 position, IRED 556 at the 2 o'clock position, IRED 557 at the 3 o'clock position, and IRED 558 at the 5 o'clock position. IRED 559 is arranged at the 7 o'clock position, IRED 552 at the 9 o'clock position, IRED 553 at the 10 o'clock position, IRED 560 at the 10:30 position, and IRED 554 at the 11 o'clock position. The line of sight sensor 506 is arranged at the 6 o'clock position. These positions are merely approximate positions, and do not need to completely match the positions of the above-mentioned times.

[0143] In the fifth embodiment, similarly to the first embodiment, the number of light sources (IREDs) per degree is maximum in the range on the opposite side of the line of sight sensor 506 across the center of the opening 545 facing the user's eye. Focus on line 585. In Fig. 18, more IREDs are arranged on the side (upper side) where gaze sensor 506 is not arranged with respect to horizontal line 585 than on the side (lower side) where gaze sensor 506 is arranged with respect to horizontal line 585. Two IREDs, IRED 558 and IRED 559, are arranged on the side (below horizontal line 585) where gaze sensor 506 is arranged. On the other hand, six IREDs, IRED 553, IRED 554, IRED 555, IRED 556, IRED 560, and IRED 561, are arranged on the side (above horizontal line 585) where gaze sensor 506 is not arranged.

[0144] 19A to 19D are schematic diagrams showing the relationship between a plurality of IREDs, an eyeball 541, and a gaze sensor 506. FIGS. 19A and 19B show a case where IREDs 552 to 559 shown in FIG. 18 are used as a plurality of IREDs (fifth embodiment). FIG. 19A is a perspective view of the eyeball 541 seen from the G3 eyepiece 564 side, and FIG. 19B is a front view of the eyeball 541 seen from the G3 eyepiece 564 side. FIGS. 19C and 19D show a case where IREDs 552b to 559b arranged evenly are used as a plurality of IREDs (comparative example). FIG. 19C is a perspective view of the eyeball 541 seen from the G3 eyepiece 564 side, and FIG. 19D is a front view of the eyeball 541 seen from the G3 eyepiece 564 side. Note that IREDs 560 and 561 are omitted.

[0145] 19A to 19D, eyeball 541 is located on display optical system optical axis 583, and views EVF panel 540 in close proximity to G3 eyepiece lens 564. Chief rays emitted from IREDs 552 to 559 (552b to 559b) are reflected at reflection positions R552 to R559 (R552b to R559b) and enter gaze sensor 506. Of reflection positions R552 to R559 (R552b to R559b), reflection positions on cornea 542 appear as Purkinje images in an image captured by gaze sensor 506. Therefore, by arranging many reflection positions on cornea 542, the accuracy of gaze detection can be improved.

[0146] 19A and 19B (fifth embodiment), four reflection positions R552, R553, R556, and R557 are located on the cornea 542. Reflection positions R553 and R554 are located on the upper eyelid 550, and reflection positions R558 and R559 are located on the lower eyelid 551. Therefore, reflection positions R553, R554, R558, and R559 are not captured as Purkinje images in the image captured by the line-of-sight sensor 506.

[0147] 19C and 19D (comparative example), reflection positions R552b and R557b are also located on the lower eyelid 551. Therefore, the only reflection positions located on the cornea 542 are reflection positions R553b and R556b.

[0148] Eye-gaze sensor 506 is disposed so as to look up from below at spherical cornea 542. As a result, as shown in Fig. 19D, the reflection positions of the chief rays emitted from the IRED are concentrated on the lower side of cornea 542. This tendency becomes more noticeable when the user's eyes are close to EVF unit 501.

[0149] For this reason, in the fifth embodiment, many IREDs are arranged on the opposite side of display optical system optical axis 583 from eye gaze sensor 506. In this way, the reflection positions that would otherwise be concentrated on the lower side of cornea 542 can be brought closer to the center of cornea 542, and a situation in which the chief ray is vignetted by lower eyelid 551 can be suppressed.

[0150] In the fifth embodiment, multiple IREDs are used, but lighting many IREDs at high output may not be desirable from the viewpoint of user safety. Also, constantly lighting many IREDs increases the power consumption of the camera body 500. Therefore, it is better not to light up too many IREDs.

[0151] FIG. 20 is a schematic diagram showing the relationship between the eyeball 541 and the EVF unit 501 when the user looks into the EVF unit 501 in a state where the camera body 500 is rotated approximately 90 degrees counterclockwise, that is, in a so-called vertical position. The vertical position can also be regarded as a posture in which the left-right direction of the eyeball 541 approximately coincides with the up-down direction of the opening 545. FIG. 20 is a front view of the eyeball 541 seen from the EVF panel 540 side. The eyeball 541 is located on the optical axis 583 of the display optical system. In FIG. 20, the camera body 500 is shown in the same manner as in the normal position, and the eyeball 541 is rotated 90 degrees counterclockwise. Note that the rotation direction of the camera body 500 is not limited to counterclockwise, and the same explanation as the following explanation is also possible when the camera body 500 is rotated clockwise.

[0152] 20, reflection positions R555, R556, R557, R558, and R560 corresponding to IREDs 555, 556, 557, 558, and 560 arranged on the upper eyelid 550 side, respectively, are located on the upper eyelid 550. Therefore, the reflection positions R555, R556, R557, R558, and R560 are not reflected as Purkinje images in the image captured by the line-of-sight sensor 506. On the other hand, reflection positions R552, R553, R554, R559, and R561 corresponding to IREDs 552, 553, 554, 559, and 561 arranged on the lower eyelid 551 side, respectively, are located on the cornea 542. Therefore, the reflection positions R552, R553, R554, R559, and R561 are reflected as Purkinje images in the image captured by the line-of-sight sensor 506. Although the shape of the eyelids varies from person to person, as described above, the upper eyelid 550 tends to be larger than the lower eyelid 551 and cover the eyeball 541 .

[0153] 20, it can be seen that in the vertical position, it is preferable to light up the IRED on the side of lower eyelid 551. However, whether to rotate camera body 500 clockwise or counterclockwise when shooting in the vertical position is up to the user. Also, the IRED is used not only to obtain a Purkinje image, but also to ensure that gaze sensor 506 captures eyeball 541 and provides the necessary overall light amount. Considering various users and usage situations, it is preferable to select IREDs to be lighted symmetrically rather than asymmetrically.

[0154] 19B and 20 are compared. In the normal position, as shown in FIG. 19B, cornea 542 sandwiched between upper eyelid 550 and lower eyelid 551 is exposed horizontally (cornea 542 is exposed with its width in the X-axis direction wide). On the other hand, in the vertical position, as shown in FIG. 20, cornea 542 is exposed vertically regardless of the rotation direction of camera body 500 (cornea 542 is exposed with its width in the Y-axis direction wide).

[0155] For this reason, in the case of the normal position, light emitted from an IRED located far away on the positive or negative side of the Y axis (e.g., IREDs 554, 555, 558, 559) is unlikely to appear as a Purkinje image in the image captured by the line-of-sight sensor 506. Similarly, in the case of the vertical position, light emitted from an IRED located far away on the positive or negative side of the X axis (e.g., IREDs 552, 553, 556, 557) is unlikely to appear as a Purkinje image in the image captured by the line-of-sight sensor 506.

[0156] Therefore, in the fifth embodiment, the CPU 593 controls each of the multiple IREDs so that the IREDs that are turned on are switched between the normal position and the vertical position. In the normal position, the CPU 593 turns on the first light source group including only the IREDs arranged above the center of the opening 545, and in the vertical position, turns on the second light source group different from the first light source group. Some of the multiple IREDs are included in both the first light source group and the second light source group. For example, in the normal position, the IREDs 552, 553, 556, 557, 560, and 561 are turned on, and the other IREDs are turned off. In the vertical position, the IREDs 554, 555, 558, 559, 560, and 561 are turned on, and the other IREDs are turned off. Note that in the fifth embodiment, The definitions of the first light source group and the second light source group in this embodiment are different from those of the first light source group and the second light source group in the third embodiment.

[0157] Here, the light emitted from IREDs 560 and 561 is likely to appear as a Purkinje image in the image captured by gaze sensor 506 regardless of the user's shooting posture. Therefore, IREDs 560 and 561 are turned on both in the normal position and in the vertical position. It is preferable to always turn on the IRED disposed in an oblique direction on the side opposite gaze sensor 506 across display optical system optical axis 583 (an IRED disposed in a direction that passes through display optical system optical axis 583 and is tilted at 45 degrees ±10 degrees with respect to the Y axis).

[0158] However, when the eyeball 541 is far from the display optical system optical axis 583 or when the eyeball 541 is too close to the EVF unit 501, there are cases where a sufficient number of Purkinje images cannot be acquired even if the turned-on IREDs are controlled according to the user's posture. Therefore, when the number of acquired Purkinje images (Purkinje images formed on the gaze sensor 506) is less than a predetermined number, the CPU 593 may temporarily turn on one or more of the turned-off IREDs according to the state of the acquired Purkinje images. The predetermined number is a number sufficient for gaze detection, for example, two. This makes it possible to increase the number of acquired Purkinje images and reduce the frequency of gaze detection errors.

[0159] As described above, according to the fifth embodiment, similarly to the first embodiment, the number of light sources per degree is maximized in the range on the opposite side of the gaze sensor across the center of the opening facing the user's eye (the optical axis of the display optical system). This makes it possible to provide an imaging device (gaze detection device) that enables the gaze to be detected with high accuracy regardless of the situation.

[0160] The above embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above embodiment. For example, the above embodiment may be combined with image processing (focus peaking) that emphasizes the contours of the area in focus.

[0161] The disclosure of this embodiment includes the following configuration. (Configuration 1) An opening disposed opposite the user's eye; A gaze sensor disposed on an edge of the opening so as to face the eye facing the opening; a plurality of light sources arranged along an edge of the opening; having Within a 360-degree range centered on the center of the opening, the number of light sources per degree is greatest in the range on the opposite side of the center of the opening from the line of sight sensor. A gaze detection device comprising: (Configuration 2) The range on the opposite side of the center of the opening to the line of sight sensor is a range of 120 degrees centered in the direction opposite to the line of sight sensor from the center of the opening. 2. The gaze detection device according to configuration 1, (Configuration 3) The range on the opposite side of the center of the opening to the line of sight sensor is a range of 90 degrees centered in the direction opposite to the line of sight sensor from the center of the opening. 2. The gaze detection device according to configuration 1, (Configuration 4) The line of sight sensor is disposed to the left or right of the opening. 4. The gaze detection device according to any one of configurations 1 to 3. (Configuration 5) The line of sight sensor is disposed on a horizontal line passing through the center of the opening. 5. The gaze detection device according to configuration 4. (Configuration 6) The number of light sources arranged below the opening is greater than the number of light sources arranged above the opening. 6. The gaze detection device according to any one of configurations 1 to 5. (Configuration 7) The number of light sources arranged below a horizontal line passing through the center of the opening is greater than the number of light sources arranged above the horizontal line. 7. The gaze detection device according to configuration 6, (Configuration 8) The number of light sources arranged in the area on the opposite side of the center of the opening to the line of sight sensor is greater than the number of light sources arranged in the remaining area. 8. The gaze detection device according to any one of configurations 1 to 7. (Configuration 9) The number of light sources arranged in the area on the opposite side of the center of the opening to the line of sight sensor is three or more than the number of light sources arranged in the remaining area. 9. The gaze detection device according to configuration 8, (Configuration 10) A control unit that controls each of the plurality of light sources, When the left-right direction of the eye arranged to face the opening substantially coincides with the left-right direction of the opening, the control unit turns on a first light source group including only light sources arranged above the center of the opening. 10. The gaze detection device according to any one of configurations 1 to 9. (Configuration 11) When a left-right direction of the eye arranged to face the opening substantially coincides with a top-bottom direction of the opening, the control unit turns on a second light source group different from the first light source group, Some of the plurality of light sources are included in both the first light source group and the second light source group. 11. The gaze detection device according to configuration 10. (Configuration 12) When the number of Purkinje images formed on the line-of-sight sensor is less than a predetermined number, the control unit temporarily turns on one or more light sources that have been turned off. 12. The gaze detection device according to configuration 10 or 11. (Configuration 13) A gaze detection device according to any one of configurations 1 to 10 and 12, a display unit that displays an image viewable through the opening; A head-mounted display device comprising: (Configuration 14) A gaze detection device according to any one of configurations 1 to 10 and 12, a fixing part for fixing the gaze detection device to a user's head; A glasses-type device comprising: (Configuration 15) A gaze detection device according to any one of configurations 1 to 12, An imaging unit that captures an image of a subject; An imaging device comprising: [Explanation of symbols]

[0162] 101: Display unit body 145: Right opening 146: Left opening 106: Right gaze sensor 107: Left gaze sensor 108, 109, 111~122: Right IRED 123, 124, 126~137: Left IRED

Claims

1. A right image sensor is disposed on the edge of a right opening and captures an image of the right eye facing the right lens disposed in the right opening; a left image sensor disposed on an edge of the left opening and configured to capture an image of the left eye facing the left lens disposed in the left opening; a plurality of right light sources arranged along an edge of the right opening and arranged to illuminate the right eye; a plurality of left light sources arranged along an edge of the left opening and arranged to illuminate the left eye; and the right image sensor is disposed near the inner corner of the right eye, the left image sensor is disposed near the inner corner of the left eye, the number of right light sources arranged on the opposite side of the right image sensor with respect to the optical axis of the right lens is greater than the number of right light sources arranged on the same side as the right image sensor with respect to the optical axis of the right lens; The number of left light sources arranged on the opposite side of the left image sensor with respect to the optical axis of the left lens is greater than the number of left light sources arranged on the same side as the left image sensor with respect to the optical axis of the left lens. A head-mounted display characterized by:

2. The right image sensor and the left image sensor are disposed on a horizontal line passing through the optical axis of the right lens and the optical axis of the left lens.

2. The head-mounted display according to claim 1.

3. The right image sensor is disposed at a position rotated 270 degrees clockwise from directly above the optical axis of the right lens, with the optical axis as the center; The left image sensor is disposed at a position rotated 90 degrees clockwise from directly above the optical axis of the left lens.

2. The head-mounted display according to claim 1.

4. The number of right light sources arranged in a range of 120 degrees on the opposite side of the right image sensor with respect to the optical axis of the right lens is greater than the number of right light sources arranged in a range other than the 120 degree range on the opposite side of the right image sensor with respect to the optical axis of the right lens, The number of left light sources arranged within a 120-degree range on the opposite side of the left image sensor with respect to the optical axis of the left lens is greater than the number of left light sources arranged within a range other than the 120-degree range on the opposite side of the left image sensor with respect to the optical axis of the left lens.

2. The head-mounted display according to claim 1.

5. The interval between right light sources arranged within a range of 120 degrees on the opposite side of the right image sensor with respect to the optical axis of the right lens is smaller than the interval between right light sources arranged within a range other than the 120 degree range on the opposite side of the right image sensor with respect to the optical axis of the right lens, The interval between left light sources arranged within a 120-degree range on the opposite side of the left image sensor with respect to the optical axis of the left lens is smaller than the interval between left light sources arranged within a range other than the 120-degree range on the opposite side of the left image sensor with respect to the optical axis of the left lens.

2. The head-mounted display according to claim 1.

6. The number of right light sources arranged below a horizontal line passing through the optical axis of the right lens and the optical axis of the left lens is greater than the number of right light sources arranged above the horizontal line, The number of left light sources arranged below the horizontal line is greater than the number of left light sources arranged above the horizontal line.

2. The head-mounted display according to claim 1.

Images