Projection device

Abstract

A projection device according to an embodiment of the present disclosure comprises: a video display device that includes an illumination optical system; a retroreflective plate that reflects video light, which is input from the video display device, at a predetermined offset angle with respect to an incident direction; a rotation drive unit that rotates the retroreflective plate; and a detection camera that captures a real image of an illumination region of the video light reflected by the retroreflective plate. The optical axis of the illumination optical system and the optical axis of the detection camera are substantially coaxially set.

Description

Projection device

[0001] The present disclosure relates to, for example, a projection device used as a head-up display.

[0002] For example, Patent Document 1 discloses a projection device that can display a virtual image with a wide viewing angle by including a display device, a reflection mirror that reflects a light beam emitted from the display device in a predetermined direction, and a retroreflective plate that reflects the incident light beam via the reflection mirror in a substantially incident direction. In this projection device, a detection camera arranged in front of an observer detects the position of the observer's eye to track an eye box.

[0003] International Publication No. 2024 / 219307

[0004] As described above, a projection device that displays driving assistance information to a vehicle driver is required to realize eye tracking that allows a virtual image to be visually recognized at all times.

[0005] It is desirable to provide a projection device that allows a virtual image to be visually recognized at all times.

[0006] A projection device according to an embodiment of the present disclosure includes: a video display device including an illumination optical system; a retroreflective plate that reflects video light incident from the video display device by imparting a predetermined offset angle with respect to the incident direction; a rotation drive unit that rotates the retroreflective plate; and a camera that captures a real image near a focal point of the video light reflected by the retroreflective plate, wherein the optical axis of the illumination optical system and the optical axis of the camera are arranged substantially coaxially.

[0007] In the projection device according to an embodiment of the present disclosure, the optical axis of the illumination optical system included in the video display device and the optical axis of the camera are arranged substantially coaxially. This enables acquisition of information near the focal point of the video light to realize an effective closed-loop eye tracking system.

[0008] Figure 1 is a schematic diagram showing an example configuration of a projection device according to one embodiment of the present disclosure. Figure 2 is a functional block diagram showing the configuration of the projection device shown in Figure 1. Figure 3 is a schematic plan view of the special retroreflector shown in Figure 1. Figure 4 is a diagram illustrating the retroreflector elements constituting the special retroreflector shown in Figure 3. Figure 5 is a schematic cross-sectional view showing an example of the configuration of the special retroreflector corresponding to the I-I' line shown in Figure 3. Figure 6 is a schematic diagram showing an example of the configuration of the image display device shown in Figure 1. Figure 7 is a schematic diagram showing an example of the configuration of the pupil replication device shown in Figure 1. Figure 8 is a diagram showing the pattern of image light incident on the pupil replication device shown in Figure 7. Figure 9 is a diagram illustrating the relationship between each optical member and the driver constituting the projection device shown in Figure 1. Figure 10 is a diagram showing an example of the optical configuration of the projection device shown in Figure 1. Figure 11 is a flowchart illustrating the operation of the projection device shown in Figure 1. Figure 12 is a diagram illustrating the flow of the eye-tracking system and the tracking of the eyebox of the projection device shown in Figure 1. Figure 13 is a diagram illustrating the flow of the eye-tracking system and the tracking of the eyebox following Figure 12. Figure 14 is a diagram illustrating the flow of the eye-tracking system and the tracking of the eye box following Figure 13. Figure 15 is a diagram showing an example of the optical configuration of a projection device according to Modification 1 of the present disclosure. Figure 16 is a diagram showing an example of the optical configuration of a projection device according to Modification 2 of the present disclosure.

[0009] The embodiments described below will be explained in detail with reference to the drawings. The following description is one specific example of the present disclosure, and the present disclosure is not limited to the following embodiments. Furthermore, the present disclosure is not limited to the arrangement, dimensions, dimensional ratios, etc., of each component shown in each figure. The order of explanation is as follows: 1. Embodiment (An example of a projection device that realizes a closed-loop eye-tracking system by coaxially arranging the optical axis of the illumination optical system included in the image display device and the optical axis of the camera, and acquiring positional information of the position where the eye box is formed instead of detecting the eye box) 2. Modifications 2-1. Modification 1 (Another example of the configuration of the projection device) 2-2. Modification 2 (Another example of the configuration of the projection device)

[0010] <1. Embodiments> Figure 1 shows an example of a schematic configuration of a projection device (projection device 1) according to one embodiment of the present disclosure. The projection device 1 is used in a head-up display (HUD) system that displays speed, navigation, etc., to the driver in the front seat of a vehicle, and displays a virtual image including driving support information and warning information using light reflection from the windshield.

[0011] The projection device 1 comprises an image display device 10, a special retroreflector 20, a rotation drive unit 30, and a detection camera 40. The special retroreflector 20 reflects the image light L incident from the image display device 10 with a predetermined offset angle relative to the incident direction. The rotation drive unit 30 rotates the special retroreflector 20 by a predetermined angle, for example, in the in-plane direction. The detection camera 40 is positioned so that its optical axis is coaxial with the optical axis of the projection optical system 19 included in the image display device 10, and captures a real image near the focal point of the image light L reflected by the special retroreflector 20.

[0012] Here, the video display device 10 corresponds to a specific example of the "video display device" as one embodiment of the present disclosure. The projection optical system 19 corresponds to a specific example of the "illumination optical system" as one embodiment of the present disclosure. The special retroreflector 20 corresponds to a specific example of the "retroreflector" as one embodiment of the present disclosure. The rotation drive unit 30 corresponds to a specific example of the "rotation drive unit" as one embodiment of the present disclosure. The detection camera 40 corresponds to a specific example of the "camera" as one embodiment of the present disclosure.

[0013] [Outline Configuration of Projection Device] As described above, the projection device 1 comprises, for example, an image display device 10, a special retroreflective plate 20, a rotary drive unit 30, and a detection camera 40. The projection device 1 further includes a pupil replicating device 50 positioned between the image display device 10 and the special retroreflective plate 20, a lens 41 positioned between the detection camera 40 and the pupil replicating device 50, and a reflective mirror 60. The image light L emitted from the image display device 10 enters the special retroreflective plate 20 via the pupil replicating device 50. The image light L that enters the special retroreflective plate 20 is reflected in a direction different from the direction of incidence with a predetermined offset angle. The image light L emitted from the special retroreflective plate 20 is partially reflected by the reflective mirror 60 and enters the area around the eyes of the driver 100. Here, the driver 100 corresponds to one specific example of a "viewer" as one embodiment of the present disclosure. As described above, the detection camera 40 is positioned so that its optical axis is coaxial with the optical axis of the projection optical system 19. This allows the detection camera 40 to capture a real image near the region (eyebox EB) where the image light L emitted from the special retroreflector 20 and partially reflected by the reflection mirror 60 is focused.

[0014] (Image display device) Figure 2 is a functional block diagram showing the configuration of the projection device 1.

[0015] The video display device 10 projects a virtual image in front of the driver 100. The video display device 10 is connected to an external image supply device, such as a PC (not shown) or various image players, via an I / F (interface), and projects a virtual image based on the image signal input to this interface.

[0016] The video display device 10 includes, for example, a light source device 11, a control unit 12, a light source drive unit 13, a light modulation device 14, an image processing unit 15, a frame memory 16, a panel drive unit 17, a projection optical system drive unit 18, and a projection optical system 19.

[0017] The light source device 11, although not shown in the diagram, includes a visible light source, a light source driver for driving the light source, and a current value setting unit for setting the current value when driving the light source. The light source driver generates a current with the current value set by the current value setting unit, synchronized with the signal input from the light source drive unit 13, based on the power supply from a power supply circuit (not shown). The generated current is supplied to the respective light sources.

[0018] The control unit 12 controls the rotation drive unit 30 based on positional information of the real image near the region where the image light L emitted from the special retroreflective plate 20 and partially reflected by the reflective mirror 60 is focused, as captured by the light source drive unit 13, image processing unit 15, panel drive unit 17, projection optical system drive unit 18, and detection camera 40.

[0019] The light source drive unit 13 outputs signals to control the light emission timing of the light source located in the light source device 11. This light source drive unit 13 includes, for example, a PWM setting unit, a PWM signal generation unit, and a limiter (not shown), and controls the light source driver of the light source device 11 based on the control of the control unit 12, and controls the light source using PWM to turn the light source on and off, or adjust the brightness.

[0020] The optical modulator 14 generates image light by modulating the light (illumination light) output from the light source 11 based on the image signal. The optical modulator 14 is composed of, for example, three light bulbs corresponding to each of the RGB colors described later. Examples of the optical modulator 14 include a liquid crystal display panel that modulates blue light (B) (panel (B)), a liquid crystal display panel that modulates red light (R) (panel (R)), and a liquid crystal display panel that modulates green light (G) (panel (G)). The RGB color lights modulated by the optical modulator 14 are combined by a cross dichroic prism or the like (not shown) and guided to the projection optical system 19.

[0021] The image processing unit 15 acquires an image signal input from an external source and performs functions such as determining the image size, resolution, and whether it is a still image or a moving image. If it is a moving image, it also determines the attributes of the image data, such as the frame rate. Furthermore, if the resolution of the acquired image signal differs from the display resolution of the optical modulator 14, it performs a resolution conversion process. The image processing unit 15 then expands the image after each of these processes into the frame memory 16 frame by frame, and outputs the image of each frame expanded in the frame memory 16 as a display signal to the panel drive unit 17.

[0022] The panel drive unit 17 drives the optical modulator 14. The drive of this panel drive unit 17 changes the light transmittance at each pixel arranged in the optical modulator 14, thereby forming an image.

[0023] The projection optical system drive unit 18 includes a motor that drives the lenses arranged in the projection optical system 19. This projection optical system drive unit 18 drives the projection optical system 19 according to the control of the control unit 12, and performs operations such as zoom adjustment, focus adjustment, and aperture adjustment.

[0024] The projection optical system 19 includes a group of lenses and the like for forming an image of the light modulated by the light modulation device 14.

[0025] In addition to the three-panel system using the three liquid crystal display panels described above, the video display device 10 may also be configured as a single-panel time-division projector using one liquid crystal display panel as the optical modulation device 14.

[0026] (Special Retroreflective Plate) Figure 3 schematically shows the planar configuration of the special retroreflective plate 20.

[0027] First, let's explain a typical retroreflector. A typical retroreflector has a mechanism that returns reflected light directly to the optical axis of the incident light. A retroreflector consists of multiple retroreflective elements arranged periodically in a two-dimensional array. A retroreflective element is generally made up of, for example, three plane mirrors arranged perpendicular to each other with their reflective surfaces facing inward, like the vertices of a cube, forming a triangular pyramidal recess. When viewed from directly above, a retroreflector with such retroreflective elements arranged in a two-dimensional array has equilateral triangles arranged in the densest possible order, as shown in Figure 3, for example. Light that reaches any of the reflective surfaces is reflected by the three corner deflectors and returns in the direction from which it was incident (retroreflection).

[0028] The special retroreflector 20 of this embodiment, like the general retroreflector described above, has multiple retroreflector elements 21, each consisting of three reflective surfaces, arranged periodically in a two-dimensional array. However, the special retroreflector 20 differs from a general retroreflector in that one of the three surfaces constituting the retroreflector element 21 is a tilted surface that is inclined by an angle θ from a state where the angle it makes with the other adjacent surfaces is a right angle (90°). Specifically, for example, as shown in Figure 4, when the three mutually orthogonal surfaces are designated as surface A 21S1, surface B 21S2, and surface C 21S3, the retroreflector element 21 consists of three surfaces: surface A 21S1 and surface B 21S2, which are mutually orthogonal, and surface C' 21S3', which is inclined by an angle θ from surface C 21S3 inward into a triangular pyramidal recess. The angle of inclination of this surface C' 21S3' determines the offset angle of the emitted light relative to the incident light.

[0029] Note that the angle θ may be greater than or less than 0°. In other words, the C' surface 21S3' may be inclined outward from the triangular pyramidal recess.

[0030] Figure 5 schematically shows an example of the cross-sectional configuration of the special retroreflector 20 corresponding to the line I-I' shown in Figure 3. The special retroreflector 20 has a pair of opposing surfaces 20S1 and 20S2, and on the surface 20S1 side, the multiple triangular pyramidal recesses of the multiple retroreflective elements 21 that constitute the special retroreflector 20 are arranged in the closest possible proximity. As described above, one of the three surfaces (surface A 21S1, surface B 21S2, and surface C' 21S3') that constitute the retroreflective element 21 (surface C' 21S3') is a tilt surface that is inclined by an angle θ from a state where the angle it makes with the other adjacent surfaces is a right angle (90°). The tilt surfaces (surface C' 21S3') of the multiple retroreflective elements 21 are adjacent in the YZ plane, for example, in the Z-axis direction which is the direction of travel of the vehicle, as shown in Figures 3 and 5.

[0031] (Rotational drive unit) The rotational drive unit 30 rotates the special retroreflective plate 20 by a predetermined angle in the in-plane direction. The rotational drive unit 30 has a rotary motor 31 and a shaft 32 that connects the special retroreflective plate 20 and the rotary motor 31.

[0032] (Detection Camera) The detection camera 40 is for capturing a real image near the focal point of the image light L reflected by the special retroreflector 20, and is positioned so that its optical axis is coaxial with the optical axis of the projection optical system 19 included in the image display device 10. The detection camera 40 has, for example, an infrared (IR) light source 410 as a sensing light source and an image sensor 420 (see, for example, Figure 9). The image sensor 420 takes in incident light (image light) from a subject and converts the incident light formed on the imaging surface into an electrical signal on a pixel-by-pixel basis and outputs it as a pixel signal. The subject here corresponds to the real image near the focal point of the image light L reflected by the special retroreflector 20. The real image here refers to one of the eyes of the driver 100 that views the virtual image of the image emitted from the image display device 10 and its vicinity. In other words, the detection camera 40 captures the eye of the driver 100 and its surroundings where the image light L reflected by the special retroreflector 20 is focused.

[0033] The detection camera 40 may be placed outside the video display device 10 (see Modification 2 described later, Figure 16) so as to be coaxial with the optical axis of the projection optical system 19 of the video display device 10, as shown in Figure 1, or it may be placed inside the video display device 10, as shown in Figure 6. In the video display device 10 shown in Figure 6, for example, a light guide 111, a Fresnel lens 112, a polarizing plate 113, a light modulator 14, a Fresnel lens 114, and a reflective mirror 115 are arranged on the optical path of the output light (illumination light) emitted from the light source device 11, and the detection camera 40 is placed behind the reflective mirror 115 that reflects the illumination light toward the projection optical system 19.

[0034] [Pupil duplication device] The pupil duplication device 50 duplicates the pupil conjugate point of the image light L emitted from the image display device 10, and is intended to enlarge the point (eye box EB) from which the driver 100 can view the entire virtual image of the image emitted from the image display device 10. Figure 7 schematically shows the planar configuration (A) and cross-sectional configurations (B), (C) of the pupil duplication device 50. The pupil duplication device 50 includes, for example, an incident section 51 and a light guide plate 52. The light guide plate 52 has a light guide section 52A that reflects the image light L incident via the incident section 51 into the interior of the light guide plate 52, and a light guide section 52B that emits the image light L reflected by the light guide section 52A to the special retroreflective plate 20.

[0035] The incident portion 51 is a component to which the image light L emitted from the image display device 10 is incident. The incident portion 51 is composed of a triangular prism, such as a right-angle prism. As shown in Figure 7(B), for example, the incident portion 51 is arranged on one of a pair of opposing surfaces of the light guide portion 52A (for example, surface 52AS1), and one surface of the triangular prism becomes the incident surface 51S1 for the image light L emitted from the image display device 10.

[0036] The light guide section 52A is a plate-shaped member having a pair of opposing surfaces 52AS1 and 52AS2, and the incident section 51 is arranged on surface 52AS1 as described above. The light guide section 52A has a total reflection region 53 formed by a plurality of half mirrors 531 (half mirrors 531H1, ..., 531HM) arranged in the X-axis direction, each having a predetermined inclination (for example, about 45° with respect to the body axis direction (X-axis direction) of the driver 100), as shown in Figures 7(A) and (B). Hereinafter, M is an integer of 2 or more.

[0037] The light guide portion 52B is a plate-shaped member having a pair of opposing surfaces 52BS1 and 52BS2. The light guide portion 52B has an emission region 54 formed by a plurality of half mirrors 541 (half mirrors 541V1, ..., 541VN) extending in the X-axis direction, arranged in the Y-axis direction, as shown in Figures 7(A) and (C), for example. Hereinafter, N is an integer of 2 or more.

[0038] The reflectivity of each of the multiple half mirrors 531 (half mirrors 531H1, ..., 531HM) and multiple half mirrors 541 (half mirrors 541V1, ..., 541VN) may be adjusted in stages. The reflectivity of each half mirror will be explained below using the multiple half mirrors 531 that constitute the light guide section 52A as an example. For example, if the reflectivity of the first-stage half mirror 531H1 among the half mirrors 531H1, ..., 531HM arranged in the X-axis direction is set to (M-1) / M, and the reflectivity of the second-stage and subsequent half mirrors 531 is set to 1 / (M-m+1), the amount of light reflected by each half mirror 531H1, ..., 531HM will be equal. For example, if the amount of light incident on the light guide section 52A is Q, and M stages of half mirrors 531 are arranged, the amount of light reflected from the m-th stage half mirror 531Hm is expressed by the following formula (1).

[0039]

[0040] The same applies to the multiple half-mirrors 541 (half-mirrors 541V1, ..., 541VN) that constitute the light guide section 52B.

[0041] The light guide portion 52A further has a side surface 52AS3 perpendicular to a pair of opposing surfaces 52AS1 and 52AS2. The light guide portion 52B further has a side surface 52B3 perpendicular to a pair of opposing surfaces 52BS1 and 52BS2. Surface 52AS3 of the light guide portion 52A and surface 52BS3 of the light guide portion 52B face each other directly. The light guide portion 52A and the light guide portion 52B are integrated by bonding surfaces 52AS3 and 52BS3 together, for example, as shown in Figures 7(A) and 7(C). Furthermore, surface 52AS1 of the light guide portion 52A and surface 52BS1 of the light guide portion 52B form a continuous, identical surface (surface 52S1), for example, as shown in Figure 8. Similarly, the surface 52AS2 of the light guide portion 52A and the surface 52BS2 of the light guide portion 52B form a continuous, identical surface (surface 52S2), as shown in Figure 8, for example.

[0042] The image light L that enters the light guide section 52A via the incident section 51 propagates through the total reflection region 53 and is reflected in the Y-axis direction by a plurality of half mirrors 531 (half mirrors 531H1, ..., 531HM) arranged in the X-axis direction, and is emitted from the surface 52AS3 of the light guide section 52A. The image light L emitted from the surface 52AS3 of the light guide section 52A enters from the surface 52BS3 of the opposite light guide section 52B and is reflected in the Z-axis direction by a plurality of half mirrors 541 (half mirrors 541V1, ..., 541VN) arranged in the Y-axis direction, and is emitted from the emission region 54 of the surface 52S2 of the light guide plate 52, for example, as shown in Figure 8.

[0043] If the pupil duplication device 50 has a light guide section 52A in which, for example, 10 half-mirrors 531 are arranged in the X-axis direction, and a light guide section 52B in which, for example, 5 half-mirrors 541 are arranged in the Y-axis direction, then the image light L incident on the incident section 51 is expanded (duplicated) into 10 rays in the total reflection region 53, and further expanded (duplicated) into 50 rays in the exit region 54 before being emitted. In other words, 50 pupil conjugate points are duplicated in the X-axis and Y-axis directions.

[0044] (Reflective Mirror) The reflective mirror 60 reflects, for example, light emitted from the special retroreflective plate 20 in a predetermined direction. Here, the reflective mirror 60 reflects the light emitted from the special retroreflective plate 20 toward the periphery of the driver 100's eyes via partial reflection, and is, for example, the windshield of a vehicle. Note that the reflective mirror 60 can also be used as needed, for example, as a mirror that reflects light emitted from the video display device 10 toward the special retroreflective plate 20.

[0045] [Optical Configuration of Projection Device] Figure 9 shows the relationship between each optical member constituting the projection device 1 shown in Figure 1 and the driver 100.

[0046] In the projection device 1 using the special retroreflective plate 20, replicated pupils of the light source (light source device 11) of the video display device 10 are generated on the emission surface (surface 52S2) of the light guide plate 52 of the pupil replication device 50 and around the eyes of the driver 100, and the light guide plate 52 of the pupil replication device 50 and the periphery of the eyes of the driver 100 have a conjugate (conjugate 1) relationship with the special retroreflective plate 20 interposed therebetween. That is, an eye box EB is formed around the eyes of the driver 100, and the driver 100 can visually recognize the video emitted from the video display device 10 as a virtual image.

[0047] Furthermore, the light guide plate 52 of the pupil replication device 50 and the periphery of the eyes of the driver 100 also have a relationship that forms an image of parts such as the driver 100's eyes and nose, and the real image of the driver 100's eye where the eye box EB is formed has a conjugate (conjugate 2) relationship with the light guide plate 52. By arranging the lens 41 on the side of the surface (surface 52S1) opposite to the emission surface (surface 52S2) of the light guide plate 52, and arranging the detection camera 40 at the focal length position of the lens 41 coaxially with the optical axis of the light guide plate 52, the image sensor 420 in the detection camera 40 and the emission surface (surface 52S2) of the light guide plate 52 have a conjugate (conjugate 3) relationship. This allows the detection camera 40 to capture, as a real image, the periphery of the eyes of the driver 100 who visually recognizes the video emitted from the video display device 10 as a virtual image.

[0048] The area that can be imaged by the detection camera 40 can be moved, for example, in the direction of both eyes (Y-axis direction) of the driver 100 by rotating the special retroreflective plate 20, as shown in Figure 10. Furthermore, in the projection device 1 of this embodiment, the optical axis of the projection optical system 19 and the optical axis of the detection camera 40 are arranged coaxially, so that the area that can be imaged by the detection camera 40 is in a positional relationship with the area (eye box EB) where the image light L emitted from the light guide plate 52 is focused. In this way, by arranging the optical axis of the projection optical system 19 and the optical axis of the detection camera 40 coaxially, the detection camera 40 is configured to always focus on the real image of the area where the eye box EB is formed, instead of imaging the eye box EB where the image light L is focused.

[0049] [Eyebox tracking operation] Figure 10 shows an example of the optical configuration of the projection device 1 shown in Figure 1, as well as the optical axis (Laxis) of the coaxially arranged projection optical system 19 and detection camera 40, and the optical axis Lv of the virtual image of the image light L emitted from the image display device 10.

[0050] In the projection device 1, as described above, the optical axis of the illumination optical system included in the image display device 10 (here, the light guide plate 52 of the pupil replication device 50 which is substantially coaxial with the projection optical system 19 (specifically, the emission area 54 of the light guide plate 52)) and the optical axis of the detection camera 40 are arranged coaxially. The projection device 1 further has a lens 41 between the detection camera 40 and the pupil replication device 50 that narrows the field of view of the detection camera 40. The lens 41 corresponds to a specific example of the "first lens" as one embodiment of the present disclosure, and the detection camera 40 is positioned at the focal length of the lens 41. The detection camera 40 is, for example, an infrared camera, and can acquire an image of the illuminated object by projecting onto the object (here, a real image of the area in which the eye box EB is formed) and receiving the IR reflected from the surface of the object. The projection device 1 further has an optical filter 42 between the detection camera 40 and the lens 41 that selectively transmits IR and removes visible light. The optical filter 42 corresponds to a specific example of the "first optical element" as one embodiment of the present disclosure. This makes it possible to separate the detection signal (IR) of the detection camera 40 from the virtual image recognition signal (visible light).

[0051] In the projection device 1, the image light L emitted from the image display device 10 positioned above the driver 100 (for example, on the vehicle ceiling) is incident toward a special retroreflector 20 positioned below the driver 100 (for example, on the dashboard). The image light L from each image height is incident toward the special retroreflector 20 before it can be formed as convergent light, and after repeated specular reflection at each retroreflector element 21, it is given a predetermined offset angle and reflected as divergent light in a direction different from the incident direction. The divergent light emitted from the special retroreflector 20 is partially reflected by the reflection mirror 60 and incident toward the area around the driver 100's eyes. As a result, a wide field of view (wide FOV) distant virtual image is displayed in front of the driver 100.

[0052] In this embodiment, as shown in Figure 10, the optical axis of the light guide plate 52 of the pupil duplication device 50 (specifically, the emission area 54 of the light guide plate 52) and the optical axis of the detection camera 40 are arranged coaxially (Laxis). Instead of imaging the eye box EB where the image light L is focused, the detection camera 40 can constantly monitor the real image of the area where the eye box EB is formed. The image captured by the detection camera 40 is viewed on the monitor 70, for example, as shown in Figure 10.

[0053] Figure 11 is a flowchart illustrating the eye-tracking system of the projection device 1 shown in Figure 1.

[0054] First, the detection camera 40 images the area corresponding to the position where the eye box EB is formed (step S101).

[0055] Next, the control unit 12 detects and processes the relative difference between the center position of the driver 100's eye and the position of the eye box EB (specifically, the center of the imaging area X (see, for example, Figure 13(B))) (step S102).

[0056] Next, the control unit 12 calculates an appropriate rotation angle for the retroreflective element 21 in order to eliminate the difference (step S103).

[0057] Next, the control unit 12 generates a rotation signal corresponding to the calculated rotation angle and outputs it to the rotation drive unit 30 (step S104).

[0058] The rotary drive unit 30 drives the rotary motor 31 based on the input rotation signal to rotate the shaft 32 and rotate the special retroreflective plate 20 by a predetermined angle (step S105).

[0059] As a result, the eye box EB moves to the position of the driver 100's eye (step S106).

[0060] Figures 12 to 14 illustrate the flow of the eye-tracking system of the projection device 1 described above and the tracking of the eye box EB. For example, Figure 12 shows the imaging area X of the detection camera 40 (B) and the state of the special retroreflective plate 20 (C) displayed on the monitor 70 when the eye box EB is formed at the center of the driver 100's eye (A). Figure 13 shows the imaging area X of the detection camera 40 (B) and the state of the special retroreflective plate 20 (C) displayed on the monitor 70 when the driver 100's eye is shifted from the eye box EB (A). Figure 14 shows the imaging area X of the detection camera 40 (B) and the state of the special retroreflective plate 20 (C) displayed on the monitor 70 when the eye box EB tracks the center of the driver 100's eye (A).

[0061] In projection device 1, as described above, the optical axis of the projection optical system 19 and the optical axis of the detection camera 40 are arranged coaxially (Laxis), so the formation position of the eye box EB = the recurring pupil position = the imaging area X of the detection camera 40. In other words, the formation position of the eye box EB and the imaging area X of the detection camera 40 always coincide.

[0062] For example, as shown in Figure 13(A), when the driver 100 moves and the position of the driver 100's eye shifts from the position of the eyebox EB, the monitor 70 displays an image where the center of the driver 100's eye is shifted from the center of the imaging area X of the detection camera 40, as shown in Figure 13(B). In the projection device 1, as described above, after detecting the relative difference between the center position of the driver 100's eye and the center of the imaging area X in step S102, the special retroreflective plate 20 is rotated in step 105 after going through steps S103 and S104. As a result, for example, as shown in Figure 14(B), the center of the imaging area X of the detection camera 40 moves to the center of the driver 100's eye. That is, the eyebox EB follows the movement of the driver 100, and the image light L is focused on the driver 100's eye again (the eyebox EB is formed), allowing the driver 100 to see a virtual image.

[0063] In other words, by repeating steps 101 to 106 above, the projection device 1 can always make the eye box EB follow the driver 100's eye, making it possible to set up a closed-loop eye-tracking system in real time.

[0064] [Function and Effects] In the projection device 1 of this embodiment, the optical axis of the projection optical system 19 and the optical axis of the detection camera 40 are arranged coaxially. As a result, the detection camera 40 can capture a real image near the focal point of the image light L reflected by the special retroreflective plate 20. This will be explained below.

[0065] In recent years, development has been progressing on HUD (Head-Up Display) systems that display speed, navigation, and other information to the driver in the front seats of a vehicle, and use light reflection on the windshield to display virtual images including driving support information and warning information.

[0066] In HUD systems using retroreflective technology, light emitted from each image height of the display device overlaps again at the driver's pupil position, forming an image on the retina, allowing the driver to see a virtual image projected in front of them. In other words, the point where the light overlaps is the only point (eyebox) where the entire image can be seen, and if the eyebox moves away from the driver's pupil position, the image becomes invisible. Therefore, there is a need to develop an eye-tracking system that keeps the eyebox constantly following the driver's eye position so that the virtual image can always be seen.

[0067] For example, the aforementioned projection device proposes a method in which a detection camera is placed in front of the observer to determine the position of the observer's eyes and track them with the eye box. However, because the visible light focused on the eye box for viewing the virtual image is weak, it is difficult to detect the eye box in bright environments.

[0068] Therefore, in typical HUD systems using retroreflective technology, the position of the eyebox, which returns to the vicinity of the driver's eyes, is calibrated each time before the vehicle leaves the factory or when it starts driving. After that, a detection camera monitors the driver's eye position, and the eyebox is tracked in an open-loop system according to the amount of movement. However, because this method is not a closed-loop system, it is prone to the accumulation of deviations and deviations due to signals and temperature, making it difficult to build a stable tracking system.

[0069] In contrast, in this embodiment, as described above, the optical axis of the projection optical system 19 and the optical axis of the detection camera 40 are arranged coaxially so that the image and the pupil are in an appropriate positional relationship. As a result, without detecting the focal point (eye box EB) of the image light L reflected by the special retroreflective plate 20, a real image near the eye box EB is captured and its positional information is acquired, thereby realizing an effective closed-loop eye-tracking system with excellent accuracy and tracking capabilities.

[0070] As described above, the projection device 1 of this embodiment makes it possible to provide a head-up display that allows the virtual image to be viewed at all times.

[0071] Next, modified examples 1 and 2 of the present disclosure will be described. In the following, components similar to those in the above embodiments will be denoted by the same reference numerals, and their descriptions will be omitted as appropriate.

[0072] <2. Modifications> (2-1. Modification 1) Figure 15 shows an example of the optical configuration of a projection device (projection device 1A) according to Modification 1 of the present disclosure. Similar to the projection device 1 of the above embodiment, projection device 1A is used in a HUD system that displays speed, navigation, etc., to the driver in the front seat of a vehicle, and displays a virtual image including driving support information and warning information using light reflection from the windshield.

[0073] In the above embodiment, the optical axis of the projection optical system 19 and the optical axis of the detection camera 40 are arranged coaxially, and an example is shown in which a real image near the focal point of the image light L reflected by the special retroreflector 20 is captured. However, the invention is not limited to this. In this modified projection device 1A, as shown in Figure 15, the optical axis Laxis1 of the emission region 54 of the light guide plate 52 and the optical axis Laxis2 of the detection camera 40A are arranged with a slight angular offset. Except for this point, the projection device 1A in this modified embodiment has substantially the same configuration as the projection device 1 of the above embodiment.

[0074] The detection camera 40A, like the detection camera 40 in the above embodiment, is for capturing a real image near the focal point of the image light L reflected by the special retroreflective plate 20, and has, for example, a visible light source as a sensing light source.

[0075] As described above, the detection camera 40A is positioned such that the optical axis Laxis1 of the light guide plate 52's emission area 54 and the optical axis Laxis2 of the detection camera 40A are slightly offset in angle. Specifically, the detection camera 40A is configured to capture a real image not of the area where the eye box EB is formed (the driver 100's eye), but rather a real image within a range of approximately ±7 cm from the driver 100's eye. The detection camera 40A configured in this way captures images of the driver 100's glabella, nose, mouth, or ears, for example, as shown in Figure 15.

[0076] The above "±7 cm" is merely an example of the range around the human eye where characteristic shapes exist (such as the nose, mouth, and ears), and is not limited to this range.

[0077] In projection device 1A, the lens 41 is positioned on the optical axis of the detection camera 40A, and the detection camera 40A is positioned at the focal length of the lens 41, as in the above embodiment. Furthermore, the optical filter 42 used in projection device 1 of the above embodiment is not required.

[0078] As described above, in the projection device 1A of this modified example, the optical axis Laxis1 of the light guide plate 52 of the pupil replication device 50 and the optical axis Laxis2 of the detection camera 40A are positioned with a slight angular offset, so that a real image is captured within a range of approximately ±7 cm from the eye of the driver 100. In the projection device 1A, the displacement between the formation position of the eye box EB and the center of the eye of the driver 100 can be tracked, for example, based on the relative positional relationship with the ear. Even with this configuration, the projection device 1A of this modified example can obtain the same effects as the projection device 1 of the above embodiment.

[0079] Furthermore, in the projection device 1A of this modified example, as described above, the optical axis Laxis1 of the light guide plate 52 of the pupil replication device 50 and the optical axis Laxis2 of the detection camera 40A are positioned with a slight angular offset, so a visible light source can be used as the light source for sensing.

[0080] (2-2. Modification 2) Figure 16 shows an example of the optical configuration of a projection device (projection device 1B) according to Modification 2 of the present disclosure. Similar to the projection device 1 of the above embodiment, the projection device 1B is used in a HUD system that displays speed, navigation, etc., to the driver in the front seat of a vehicle, and displays a virtual image including driving support information and warning information using light reflection from the windshield.

[0081] In the above embodiment, the video display device 10 and the detection camera 40 are arranged in the same direction such that the optical axis of the projection optical system 19 and the optical axis of the detection camera 40 are coaxial. For example, an example was shown in which the detection camera 40 is located inside the video display device 10. However, the embodiment is not limited to this. In this modified projection device 1B, for example as shown in Figure 16, the detection camera 40B is located outside the video display device 10, and the optical axis Laxis 1 of the projection optical system 19 and the optical axis Laxis 3 of the detection camera 40B are combined and made coaxial via an optical filter 43. Except for this point, the projection device 1B in this modified embodiment has substantially the same configuration as the projection device 1 of the above embodiment.

[0082] The projection device 1B comprises, for example, an image display device 10, a special retroreflective plate 20, a rotation drive unit 30, and a detection camera 40B. The projection device 1B further includes an optical filter 43 positioned at the intersection of the optical axis Laxis 1 of the projection optical system 19 and the optical axis Laxis 3 of the detection camera 40B, a lens 44 positioned between the detection camera 40B and the optical filter 43, and a reflective mirror (reflective mirror 60), although not shown.

[0083] The detection camera 40B, like the detection camera 40 in the above embodiment, is for capturing a real image near the focal point of the image light L reflected by the special retroreflective plate 20, and is located outside the image display device 10, for example, as shown in Figure 16. The detection camera 40B has, for example, an IR light source as a light source for sensing.

[0084] The optical filter 43 corresponds to a specific example of the "second optical element" as one embodiment of the present disclosure. The optical filter 33 selectively transmits visible light and reflects IR, and as described above, is positioned at the intersection of the optical axis Laxis 1 of the projection optical system 19 and the optical axis Laxis 3 of the detection camera 40B. The lens 44 corresponds to a specific example of the "second lens" as one embodiment of the present disclosure. The lens 44 narrows the field of view of the detection camera 40B, and the detection camera 40B is positioned at the focal length of the lens 44.

[0085] The image light emitted from the image display device 10 enters the special retroreflector 20 via the optical filter 43. The image light that enters the special retroreflector 20 is reflected in a direction different from the direction of incidence with a predetermined offset angle. The image light emitted from the special retroreflector 20 is partially reflected by the reflection mirror 60 and enters the area around the eyes of the driver 100. The detection camera 40B is positioned outside the image display device 10 as described above, and the optical axis Laxis1 of the projection optical system 19 and the optical axis Laxis3 of the detection camera 40B are combined and become coaxial via the optical filter 43. As a result, the detection camera 40B can always focus on the real image of the area in which the eye box EB is formed, instead of imaging the eye box EB where the image light is focused, as in the above embodiment.

[0086] Thus, in the projection device 1B of this modified example, the detection camera 40B is placed outside the image display device 10, and the optical axis Laxis1 of the projection optical system 19 and the optical axis Laxis3 of the detection camera 40B are combined and made coaxial via the optical filter 43. Even with this configuration, the projection device 1B of this modified example can obtain the same effects as the projection device 1 of the above embodiment.

[0087] Furthermore, in the modified projection device 1B, the pupil replication device 50 can be omitted. In other words, since the number of parts is reduced in the modified projection device 1B, manufacturing costs can be reduced.

[0088] Although embodiments and modifications 1 and 2 have been described above, this disclosure is not limited to the above embodiments, and various modifications are possible. For example, the arrangement and number of optical system components illustrated in the above embodiments are merely examples, and it is not necessary to include all components, and other components may also be included.

[0089] For example, the above embodiments show an example in which an eyebox EB is formed on one of the two eyes of the driver 100, but the invention is not limited to this. For example, by providing light sources for the right eye and the left eye in the image display device 10, eyeboxes EB for the right eye and the left eye are formed, respectively.

[0090] Furthermore, while the above embodiments show an example using a special retroreflector 20 that reflects the video light L incident from the video display device 10 with a predetermined offset angle relative to the incident direction, the invention is not limited to this. For example, if a general retroreflector is used that returns the reflected light directly to the optical axis of the incident light, it will not be possible to track the eye box EB, but it is possible to configure a projection device that warns that the position of the driver 100's eye is deviating from the position of the eye box EB.

[0091] Furthermore, the effects described herein are merely examples and are not limited to those described; other effects may also occur.

[0092] The technology can also take the following configurations. According to the technology with the following configuration, the optical axis of the illumination optical system included in the image display device and the optical axis of the camera are arranged substantially coaxially, so that information on the position where the image light is focused or near thereon can be acquired to realize an effective closed-loop eye-tracking system. Therefore, it is possible to provide a projection device that can always see a virtual image. (1) A projection device comprising: an image display device including an illumination optical system; a retroreflector that reflects the image light incident from the image display device with a predetermined offset angle with respect to the incident direction; a rotation drive unit that rotates the retroreflector; and a detection camera that captures a real image near the point of focus of the image light reflected by the retroreflector, wherein the optical axis of the illumination optical system and the optical axis of the detection camera are arranged substantially coaxially. (2) The projection device according to (1), wherein the detection camera captures, as the real image, one of the eyes of a viewer who is viewing a virtual image of an image projected from the image display device, or the vicinity of both eyes. (3) The projection device according to (1) or (2), wherein the detection camera is located inside the image display device. (4) The projection device according to (2) or (3), further comprising a pupil replication device for replicating the pupil conjugate point of the image light emitted from the image display device, wherein the pupil replication device is located between the image display device and the retroreflective plate. (5) The projection device according to (4), wherein, when the direction of the viewer's body axis is the X-axis direction and the direction of the viewer's eyes is the Y-axis direction, the pupil replication device replicates the pupil conjugate point in at least one of the X-axis direction and the Y-axis direction. (6) The projection device according to (4) or (5), wherein the pupil replication device includes a light guide plate, and the projection surface of the light guide plate from which the image light is emitted and one of the viewer's eyes or the vicinity of both eyes captured by the detection camera are conjugate with respect to the retroreflective plate. (7) The projection apparatus according to (6), further comprising a first lens between the detection camera and the pupil replicating device, wherein the detection camera is positioned at the focal length of the first lens.(8) The projection apparatus according to (7), wherein the detection camera includes an image sensor, and the emission surface of the light guide plate and the image sensor are in a conjugate relationship. (9) The projection apparatus according to (7) or (8), wherein the detection camera includes an infrared light source as a sensing light source, and further comprises a first optical element between the detection camera and the first lens that selectively transmits infrared light and removes visible light. (10) The projection apparatus according to any one of (1) to (9), wherein the detection camera includes a visible light source as a sensing light source, and the detection camera captures, as the real image, one of the glabella, nose, mouth, and ears of a viewer viewing a virtual image of an image projected from the image display device. (11) The projection apparatus according to any one of (1) to (10), wherein the detection camera is located outside the image display device. (12) The projection apparatus according to (11), wherein the detection camera includes an infrared light source as a sensing light source, and further comprises a second optical element between the image display device and the retroreflector that transmits visible light and reflects infrared light, and the optical axis of the detection camera is coaxial with the optical axis of the illumination optical system via the second optical element. (13) The projection apparatus according to (12), wherein the detection camera further comprises a second lens between the detection camera and the second optical element, and the detection camera is positioned at the focal length of the second lens. (14) The projection apparatus according to any one of (1) to (13), wherein the retroreflector has a main surface formed by a plurality of retroreflectors arranged in a two-dimensional array, and the plurality of retroreflectors consist of one adjacent surface and a plurality of other surfaces, the plurality of other surfaces are in contact with each other at an angle of 90°, and the one surface and the plurality of other surfaces are in contact with each other at an angle greater than or less than 90°. (15) The projection apparatus according to any one of (1) to (14), wherein the retroreflector has a main surface formed by a plurality of retroreflectors arranged in a two-dimensional array, and each of the plurality of retroreflectors has a triangular pyramidal shape consisting of three faces. (16) The projection apparatus according to any one of (1) to (15), wherein the rotation drive unit rotates the retroreflector in the in-plane direction.(17) The projection device according to any one of (1) to (16), further comprising a reflective mirror that reflects the image light reflected by the retroreflector in a predetermined direction. (18) The projection device according to (17), wherein the reflective mirror causes the image light reflected by the retroreflector to be incident around the eyes of a viewer who views a virtual image of an image projected from the image display device. (19) The projection device according to (17) or (18), wherein the reflective mirror is the windshield of a vehicle.

[0093] This application claims priority based on Japanese Patent Application No. 2025-041264, filed with the Japan Patent Office on 14 March 2025, and all contents of that application are incorporated herein by reference.

[0094] Those skilled in the art will understand that various modifications, combinations, subcombinations, and changes can be conceived depending on design requirements and other factors, and that these fall within the scope of the attached claims and their equivalents.

Claims

1. A projection device comprising: an image display device including an illumination optical system; a retroreflector that reflects image light incident from the image display device with a predetermined offset angle with respect to the direction of incidence; a rotation drive unit that rotates the retroreflector; and a detection camera that captures a real image near the focal point of the image light reflected by the retroreflector, wherein the optical axis of the illumination optical system and the optical axis of the detection camera are arranged substantially coaxially.

2. The projection apparatus according to claim 1, wherein the detection camera captures, as the real image, one of the eyes of a viewer who views a virtual image of the image projected from the image display device, or the vicinity of said eyes.

3. The projection device according to claim 1, wherein the detection camera is located inside the image display device.

4. The projection apparatus according to claim 2, further comprising a pupil replicating device for replicating the pupil conjugate point of the image light emitted from the image display device, wherein the pupil replicating device is disposed between the image display device and the retroreflective plate.

5. The projection device according to claim 4, wherein, when the direction of the viewer's body axis is the X-axis direction and the direction of both eyes of the viewer is the Y-axis direction, the pupil duplication device duplicates the pupil conjugate point in at least one of the X-axis direction and the Y-axis direction.

6. The projection device according to claim 4, wherein the pupil duplication device includes a light guide plate, and the emission surface of the light guide plate from which the image light is emitted and one of the viewer's eyes or the vicinity of both eyes captured by the detection camera are conjugate with respect to the retroreflective plate.

7. The projection apparatus according to claim 6, further comprising a first lens between the detection camera and the pupil replicating device, wherein the detection camera is positioned at the focal length of the first lens.

8. The projection apparatus according to claim 7, wherein the detection camera includes an image sensor, and the light guide plate's emission surface and the image sensor are in a conjugate relationship.

9. The projection apparatus according to claim 7, wherein the detection camera includes an infrared light source as a sensing light source, and further comprises a first optical element between the detection camera and the first lens that selectively transmits infrared light and removes visible light.

10. The projection apparatus according to claim 1, wherein the detection camera includes a visible light source as a sensing light source, and the detection camera captures, as a real image, one of the glabella, nose, mouth, and ears of a viewer who views a virtual image of an image projected from the image display device.

11. The projection device according to claim 1, wherein the detection camera is located outside the image display device.

12. The projection apparatus according to claim 11, wherein the detection camera includes an infrared light source as a sensing light source, and further comprises a second optical element between the image display device and the retroreflector that transmits visible light and reflects infrared light, and the optical axis of the detection camera is coaxial with the optical axis of the illumination optical system via the second optical element.

13. The projection device according to claim 12, further comprising a second lens between the detection camera and the second optical element, wherein the detection camera is positioned at the focal length of the second lens.

14. The projection device according to claim 1, wherein the retroreflector has a main surface on which a plurality of retroreflecting elements are arranged in a two-dimensional array, the plurality of retroreflecting elements consist of one adjacent surface and a plurality of other surfaces, the plurality of other surfaces are in contact with each other at an angle of 90°, and the one surface and the plurality of other surfaces are in contact with each other at an angle greater than or less than 90°.

15. The projection device according to claim 1, wherein the retroreflector has a main surface formed by a plurality of retroreflecting elements arranged in a two-dimensional array, and each of the plurality of retroreflecting elements has a triangular pyramidal shape consisting of three faces.

16. The projection apparatus according to claim 1, wherein the rotational drive unit rotates the retroreflective plate in the in-plane direction.

17. The projection apparatus according to claim 1, further comprising a reflective mirror that reflects the image light reflected by the retroreflector in a predetermined direction.

18. The projection apparatus according to claim 17, wherein the reflective mirror causes the image light reflected by the retroreflective plate to be incident on the area around the eyes of a viewer who views a virtual image of an image projected from the image display device.

19. The projection device according to claim 17, wherein the reflective mirror is the windshield of a vehicle.

Images