Eye-gaze detection system

Abstract

Provided is a view point detection system with which it is possible to stably detect the view point position of a subject even when outside light is incident thereon. In a head-up display device to which the view point detection system according to the present invention is applied, a control unit 10 is capable of detecting that outside light is approaching an iris I of one eye E of a driver D in a captured image P. When the approach of the outside light is detected, a view point position is calculated on the basis of the position of an iris I of the other eye E of the driver D in the captured image P (for example, the position of the center C of the iris I).

Description

[Technical field]

[0001] The present invention relates to a viewpoint detection system that detects the viewpoint position of a subject based on a captured image of the subject. [Background technology]

[0002] Head-up display devices generate and display a virtual image by display light reflected by a reflective translucent member, such as the vehicle's front windshield or combiner, and superimpose the virtual image on a real scene (scenery in front of the vehicle) that passes through the reflective translucent member. Some head-up display devices include a viewpoint detection system that irradiates the vehicle user (subject) with infrared light to capture an image of the user, detects the centers of the user's left and right pupils (irises) as viewpoints based on the captured image, and detects the midpoint between the left and right viewpoints as the viewpoint position.

[0003] For example, the head-up display device described in Patent Document 1 is configured to form a display image by reflecting visible light emitted from a display means toward a user by a combiner member, and includes an infrared irradiation means for irradiating infrared rays toward the user, a mirror member for reflecting the visible light emitted from the display means toward the combiner member and transmitting the infrared rays reflected by the user and the combiner member, a plurality of imaging means for sensing the infrared rays transmitted through the mirror member and capturing images of the user from different directions, and an image processing means for calculating the position of the user's eyes based on the images captured by the imaging means. This makes it possible to accurately calculate the position of the user's eyes, and the user's viewpoint position can be obtained from the eye positions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-126984 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when strong external light such as sunlight or street light is incident on a viewpoint detection system, the external light adds noise to the captured image, making it difficult to detect the viewpoint position. In the head-up display device described in Patent Document 1, the infrared component of the external light passes through a mirror member and enters the imaging means, and when strong external light approaches the iris of one of the left and right eyes in the captured image, the viewpoint of that eye (the eye closer to the external light) cannot be detected accurately, and the detection result fluctuates irregularly, and the viewpoint position detected as the midpoint between the left and right viewpoints also fluctuates irregularly and becomes unstable.

[0006] The present invention has been made in consideration of the above circumstances, and has an object to provide a viewpoint detection system that can stably detect the viewpoint position of a subject even when external light is incident. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the present invention provides a viewpoint detection system that detects a viewpoint position of a subject based on a captured image of the subject, comprising: an external light detection means that detects that external light is approaching an iris of one eye of the subject in the captured image; and a viewpoint position calculation means that calculates the viewpoint position based on the position of the iris of the other eye of the subject in the captured image when the external light detection means detects the approach of the external light. death, Here, the viewpoint position refers to a midpoint between the left and right irises of the subject, and the midpoint includes a virtual or hypothetical midpoint calculated based on the position of the iris of the other eye of the subject in the captured image. .

[0008] The external light detection means detects an iris region of the one eye in the captured image. White of the eye The approach of the external light may be detected when the contrast with the area is less than a threshold value.

[0009] In addition, the external light detection means may detect the approach of the external light when the distance between the iris area of ​​one of the eyes in the captured image and a blown-out area (an area where the brightness of the captured image exceeds the upper limit and gradation is lost) is less than a threshold value.

[0010] Alternatively, the external light detection means detects the iris of the one eye in the captured image. Territory When the brightness of the area exceeds a threshold value, the approach of the external light may be detected. Effect of the Invention

[0011] According to the viewpoint detection system of the present invention, the viewpoint position of the subject can be stably detected even when external light is incident. [Brief description of the drawings]

[0012] [Figure 1] 1 is an explanatory diagram showing a head-up display device to which a viewpoint detection system according to an embodiment of the present invention is applied; [Diagram 2] FIG. 2 is an explanatory diagram showing a captured image of a driver of a vehicle. [Diagram 3] 2 is an explanatory diagram showing a state in which external light enters the head-up display device of FIG. 1. [Figure 4] 1 is an explanatory diagram showing an example in which external light approaches the iris of the right eye of a vehicle driver; [Diagram 5] 5 is an explanatory diagram showing an enlarged view of the right eye portion of FIG. 4. [Figure 6] 11 is an explanatory diagram showing another example in which external light approaches the iris of the right eye of a vehicle driver; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to the drawings.

[0014] 1, a head-up display device (HUD) 1 to which the viewpoint detection system according to the present embodiment is applied is provided below a front windshield 2 of a vehicle, and projects display light L1, which is visible light, onto a part of the front windshield 2. The display light L1 is reflected by the front windshield 2 to generate a virtual image V, which is visually recognized by a driver D of the vehicle, superimposed on an actual scene transmitted through the front windshield 2.

[0015] The HUD 1 also projects infrared light L2 onto the driver D to capture an image of the driver D, and detects the midpoint between the viewpoints of the left and right eyes E of the driver D as the viewpoint position based on the captured image.

[0016] In detail, the HUD 1 is covered by a housing 3 molded from a black ABS resin or the like to prevent external light from entering, and is partitioned from the outside, and the housing 3 has a light-transmitting section 4 covered with a transparent resin such as polycarbonate (not shown). A display unit 5, a folding mirror 6, a concave mirror 7, an infrared light emitting unit 8, a camera 9, and a control unit 10 are held and housed inside the housing 3 (the viewpoint detection system according to this embodiment is composed of the concave mirror 7, the infrared light emitting unit 8, the camera 9, and the control unit 10).

[0017] The display unit 5 is provided with a light source consisting of a chip-type light-emitting diode and a liquid crystal panel, and the liquid crystal panel two-dimensionally modulates the light emitted from the light source to project and display image light (display light L1) which is visible light. The folding mirror 6 is made by evaporating a metal such as aluminum onto a resin such as polycarbonate molded to have a flat portion, and simply reflects light. The concave mirror 7 is made by evaporating a metal such as aluminum onto a glass plate or a resin such as polycarbonate molded to have a concave portion, and has the property of magnifying and reflecting visible light while transmitting infrared light.

[0018] The infrared light irradiation unit 8 is provided on the rear side of the concave mirror 7 (the side opposite the light-transmitting section 4 and the folding mirror 6 with respect to the concave mirror 7), and irradiates infrared light (near-infrared light) emitted by a light source consisting of a light-emitting diode toward the concave mirror 7. The camera 9 includes an imaging element sensitive to the infrared light in the wavelength band irradiated from the infrared irradiation unit 8, and a lens capable of transmitting the infrared light and forming an image on the imaging element, and captures a near-infrared image.

[0019] The control unit 10 is composed of a microprocessor, a memory, and various electronic components, a board, and a case for operating them, and controls the display unit 5 so as to appropriately display the image of the HUD 1 based on vehicle information and input information from the driver D.

[0020] In addition, as shown in FIG. 2, the control unit 10 detects characteristic points such as the facial outline, corners of the eyes, and outline of the nose of the driver D, as well as the irises I of the left and right eyes E, based on the contrast of the image P captured by the camera 9, and detects the coordinates of the midpoint Q between the left and right viewpoints as the viewpoint position, after setting the centers C of the left and right irises I as the left and right viewpoints, respectively.

[0021] Furthermore, when the control unit 10 detects that external light is approaching the iris I of one eye E of the driver D in the captured image P, it calculates the viewpoint position based on the position of the iris I of the other eye E of the driver D in the captured image P (here, the position of the center C of the iris I).

[0022] For example, as shown in Fig. 3, when the infrared component of strong external light L3, which is sunlight, passes through the front windshield 2, the light-transmitting portion 4, and the concave mirror 7 and enters the camera 9, the sun is reflected in the captured image P close to the iris I of the right eye E of the driver D, and the area S1 (the area where the sun is reflected) becomes overexposed due to overflow of the image sensor of the camera 9, and a bright area S2 around the sun around the area S1 may come close to the iris I of the right eye E of the driver D, as shown in Fig. 4. When the approach of the external light L3 is detected, the control unit 10 detects the coordinates of a hypothetical or hypothetical midpoint Q' calculated by inferring from the viewpoint of the left eye E as the viewpoint position. Any method may be used for the estimation. For example, if the distance between the centers C of the left and right irises I of the driver D's eyes E is known from an image captured when no external light is approaching the iris I of either eye E, the coordinates of the midpoint Q' (viewpoint position) can be estimated from that distance and the coordinates of the center C (viewpoint) of the iris I of the left eye E (where external light L3 is not approaching in Figure 4).

[0023] Regarding the approach of the external light L3 to the iris I, as shown in FIG. 5, when the contrast between a region S3 (at least a part of the region S3) near the boundary between the iris I and the sclera of one eye E in the captured image P and a region S4 of the sclera outside the adjacent iris I is less than a predetermined threshold value, it can be detected that the external light L3 (regions S1 and S2 in FIG. 4) has approached the iris I. For example, when the brightness of a region S5, which is a part of the region S3 in the captured image P, is high, the gradation value of its luminance is X5, the gradation value of the luminance of the region S4 is X4, and the gradation value representing the predetermined threshold value is X0, the control unit 10 determines that the external light L3 has approached the iris I if X4 - X5 < X0.

[0024] In the HUD1, the display light L1 from the display unit 5 is reflected by the folding mirror 6, then magnified and reflected by the concave mirror 7, passes through the light-transmitting portion 4, and is projected onto the front windshield 2. The display light L1 projected onto the front windshield 2 is magnified and reflected toward the driver D's side, generates a virtual image V, and is displayed to the driver D by overlapping it with the real scene passing through the front windshield 2.

[0025] On the other hand, the infrared light L2 from the infrared light irradiation unit 8 passes through the concave mirror 7, passes through the light-transmitting portion 4, is projected onto the front windshield 2, and is reflected toward the driver D's side by the front windshield 2 to irradiate the driver D. When reflected by the driver D, a part of the infrared light L2 passes through the concave mirror 7 and enters the camera 9, the driver D is imaged by the infrared light L2, and the captured image P is input to the control unit 10.

[0026] When the captured image P is input, the control unit 10 detects the center C of the iris I of the left and right eyes E of the driver D in the captured image P as the viewpoints, detects the midpoint Q between the two viewpoints as the viewpoint position, and then outputs information regarding the viewpoint position outside the HUD1.

[0027] However, if the control unit 10 is unable to detect the center C of either the left or right iris I, or if the contrast between area S3 and area S4 for either the left or right eye E is less than a predetermined threshold, it stops detecting the viewpoint for the eye E that could not be detected or whose contrast was less than the threshold, estimates and calculates the position of the midpoint Q' (viewpoint position) based on the position of the center C of the iris I of the other eye E, and outputs information regarding the viewpoint position to the outside of HUD1.

[0028] In the HUD 1 of this embodiment, the control unit 10 can detect that external light is approaching the iris I of one eye E of the driver D in the captured image P, and when the approach of external light is detected, the control unit 10 calculates the viewpoint position based on the position of the iris I of the other eye E of the driver D in the captured image P. This prevents irregular fluctuations in the viewpoint position and unstable detection that occur when the detection result of the viewpoint of one eye E fluctuates irregularly due to the approach of external light, and makes it possible to stably detect the viewpoint position.

[0029] Although the embodiment of the present invention has been described above, the embodiment of the present invention is not limited to the above, and may be appropriately modified without departing from the spirit of the invention.

[0030] For example, in the above embodiment, the display light L1 is projected onto the front windshield 2 of the vehicle, but a combiner may be used instead of the front windshield.

[0031] Furthermore, as shown in FIG. 6, the control unit 10 may detect that external light L3 has approached the iris I when the distance L1 between the area S3 of the iris I of one eye E in the captured image P and the blown-out area S1 (or the distance L2 between the area S3 and the area S2) is less than a predetermined threshold (including the case where the area S3 overlaps with the area S1 (or the area S2)). In this case, the contrast between the area S3 and the area S1 (or the area S2) is not important, and the threshold may be determined based on the angle of view of the camera 9 and the optical distance from the camera 9 to the driver D (iris I), taking into account the vibration of the camera 9 and the movement of the driver D, and the determined threshold and distance L1 (or distance L2) may be specified by the number of pixels, etc., instead of a unit of length.

[0032] Alternatively, the control unit 10 may detect that the external light L3 has approached the iris I when the brightness of a region outside the pupil R of the iris I of one eye E in the captured image P exceeds a threshold. In FIG. 5, a region S5 that is a part of the region S3 and outside the pupil R overlaps with the region S2 and has a high brightness (in the vicinity of the center of the iris I, light (especially infrared light) passes through the pupil R and is reflected by the retina, and tends to appear bright, so here we consider the brightness of the region S5 outside the pupil R near the center of the iris I). If this brightness exceeds a predetermined threshold, the control unit 10 can determine that the external light L3 has approached the iris I. In this case, the threshold can be set to the brightness of the white region of the eye E, the average brightness of the skin around the eye E, or the brightness of the black eye (iris I) of the opposite eye E to which the external light L3 has not approached. With this configuration, even when the brightness and contrast around the iris is low, a gaze point detection system that can identify the iris area has difficulty determining the approach of external light depending on the contrast threshold.However, if there is a bright area in the iris area that exceeds the brightness of the white of the eye or skin, it can determine that strong external light is overlapping regardless of contrast. [Explanation of symbols]

[0033] 8 Infrared light irradiation unit 9. Camera 10 Control unit (external light detection means, viewpoint position calculation means) D Driver (subject) E eyes I Iris L3 External Light P Captured image Q midpoint Q' (hypothetical / hypothetical) midpoint R Pupil S1 area (whiteout area) S3 region (iris region) S4 region (region outside the iris) S5 region (area of ​​the iris outside the pupil)

Claims

1. 1. A viewpoint detection system that detects a viewpoint position of a subject based on a captured image of the subject, an external light detection means for detecting that external light is approaching an iris of one eye of the subject in the captured image; a viewpoint position calculation means for calculating the viewpoint position based on a position of the iris of the other eye of the subject in the captured image when the external light detection means detects the approach of the external light; The present invention is characterized by comprising: Here, the viewpoint position refers to the midpoint between the left and right irises of the subject, and the midpoint includes a virtual or hypothetical midpoint calculated based on the position of the iris of the other eye of the subject in the captured image.

2. 2. The viewpoint detection system according to claim 1, wherein the external light detection means detects the approach of the external light when a contrast between an iris area and a white area of ​​the eye of the one eye in the captured image is less than a threshold value.

3. The viewpoint detection system according to claim 1, wherein the external light detection means detects the approach of the external light when the distance between the iris area of ​​the one eye and a blown-out area in the captured image is less than a threshold value.

4. 2. The viewpoint detection system according to claim 1, wherein the external light detection means detects the approach of the external light when the brightness of the iris area of ​​the one eye in the captured image exceeds a threshold value.

Images