Image projection device

Abstract

To enable for a pupil to be easily adjusted to a prescribed position.SOLUTION: There is provided an image projection device 100 comprising: a projection unit 10 including a light source 11, a scanning unit 13 for scanning a beam 50 emitted from the light source 11, and a projection optical system 20 for converging a plurality of beams 50 emitted in different directions from the scanning unit 13 by being scanned by the scanning unit 13 to the convergent point 70 in the user's eye 60 and then projecting the converged beam to the retina 61 to project an image to the retina 61; an imaging unit 40 for capturing an image of the user's eye 60; a generation unit 34 for generating a superposed image 82 in which the captured image 80 of the user's eye 60 having been captured by the imaging unit 40 and a reference image 81 in which is shown an initial position 83 to which the user's pupil 62 must be adjusted are superposed one on another; and a projection control unit 31 for controlling the projection unit 10 to project the superposed image 82 to the pupil 61.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image projection device.

Background Art

[0002] There is known an image projection device using Maxwell vision that directly projects an image onto a user's retina using light rays emitted from a light source. In Maxwell vision, an image is projected onto the retina by converging a plurality of light rays forming the image inside the user's eye and then irradiating the retina. In an image projection device using Maxwell vision, since an image is not projected onto the retina unless the light rays pass through the pupil, a method has been proposed to keep the light rays passing through the pupil even when the pupil moves (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a user wears an image projection device, it is desirable that the pupil be at a predetermined position corresponding to the convergence point where a plurality of light rays converge inside the eye, but it may deviate from this predetermined position. To adjust the position of the pupil to the predetermined position, for example, a method of emitting a sound with a magnitude corresponding to the amount of deviation from the predetermined position to guide the user can be considered. However, since it is difficult to determine the direction of deviation based only on the magnitude of the sound, it is difficult to adjust the position. As another method, a method of making an announcement with voice guidance can be considered, but in this method, a delay may occur in the voice guidance because the position of the pupil changes during the announcement.

[0005] The present invention has been made in view of the above problems, and an object thereof is to enable the position of the pupil to be easily adjusted to a predetermined position.

Means for Solving the Problem

[0006] The present invention includes a scanning unit that scans a light beam emitted from a light source, and a projection optical system that converges a plurality of the light beams emitted from the scanning unit in different directions onto a convergence point in the user's eye and then projects the light beams onto the retina to project an image onto the retina. The projection unit includes a projection unit, an imaging unit that images the user's eye, a generation unit that generates a superimposed image by superimposing the captured image of the user's eye captured by the imaging unit and a reference image on which a position to align with the user's pupil is displayed, and a projection control unit that controls the projection unit to project the superimposed image onto the retina.

[0007] In the above configuration, the generation unit generates a first guiding mark for guiding the center position of the user's pupil to coincide with the position based on the center position of the user's pupil shown in the captured image and the position in the reference image, and the projection control unit can be configured to control the projection unit to project the first guiding mark onto the retina.

[0008] In the above configuration, the first guiding mark can be configured as an arrow.

[0009] In the above configuration, the projection unit and the imaging unit can be configured to be attached to a frame worn by the user.

[0010] In the above configuration, an illumination light source that irradiates a plurality of illumination lights to the user's eye is attached to the frame worn by the user together with the projection unit and the imaging unit. The generation unit generates a second guiding mark for guiding the interval between the plurality of illumination lights shown in the captured image to a predetermined value, and the projection control unit can be configured to control the projection unit to project the second guiding mark onto the retina.

[0011] In the above configuration, the second guiding mark can be configured as a color that changes according to the interval between the plurality of illumination lights.

[0012] In the above configuration, the illumination light source irradiates the user's eyes with four illumination lights that respectively face each other at least in a first direction and a second direction orthogonal to the first direction as the plurality of illumination lights, and the generation unit guides so that the interval between two illumination lights facing each other in the first direction and the interval between two illumination lights facing each other in the second direction become the predetermined value. It can be set as the structure which produces | generates the said 2nd induction | guidance indication.

[0013] In the above configuration, it is attached to a frame worn by the user together with the projection unit and the imaging unit, and includes an illumination light source that irradiates the user's eyes with a plurality of illumination lights, and the generation unit is based on the positions of the plurality of illumination lights shown in the captured image. And a superimposed image in which the center position of the pupil shown in the captured image is displayed.

[0014] In the above configuration, the illumination light source irradiates the user's eyes with four illumination lights that respectively face each other at least in a first direction and a second direction orthogonal to the first direction as the plurality of illumination lights, and the generation unit is a straight line passing through the centers of two illumination lights facing each other in the first direction. And a straight line passing through the centers of two illumination lights facing each other in the second direction, and the intersection point where they intersect is set as the position based on the plurality of illumination lights.

Advantages of the Invention

[0015] According to the present invention, it becomes possible to easily align the position of the pupil with a predetermined position.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Best Mode for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment

[0018] Figure 1 is a diagram showing an image projection apparatus 100 according to Example 1. The broken line in Figure 1 indicates an electrical connection. The image projection apparatus 100 according to Example 1 is a retina projection type head-mounted display that directly projects a light beam (laser light) for allowing a user to visually recognize an image onto the retina of the user, and uses Maxwell vision.

[0019] As shown in FIG. 1, the image projection apparatus 100 according to Embodiment 1 includes a projection unit 10, a control unit 30, an imaging unit 40, a drive unit 41, and a storage unit 42. The projection unit 10 includes a light source 11, a lens 12, a scanning unit 13, and a projection optical system 20. The projection optical system 20 includes a reflection mirror 21, a projection mirror 22, and a lens 23. The control unit 30 includes a projection control unit 31, a position control unit 32, an imaging control unit 33, and a generation unit 34. The control unit 30 is a processor such as a CPU (Central Processing Unit). The projection control unit 31, the position control unit 32, the imaging control unit 33, and the generation unit 34 may function by the same processor or may function by different processors. The storage unit 42 is a non-volatile semiconductor memory such as a flash memory.

[0020] Based on an instruction from the projection control unit 31, the light source 11 emits a light beam 50 (laser light). The light source 11 emits a light beam 50 that is visible light, such as red laser light (wavelength: about 610 nm to 660 nm), green laser light (wavelength: about 515 nm to 540 nm), and blue laser light (wavelength: about 440 nm to 480 nm). Examples of the light source 11 that emits red, green, and blue laser light include a light source in which RGB (red, green, blue) laser diode chips and a three-color synthesis device are integrated. Note that the light source 11 may emit a light beam 50 having a single wavelength.

[0021] The projection control unit 31 receives image data from a camera and / or a recording device (not shown). Further, the projection control unit 31 receives the image data generated by the generation unit 34 from the generation unit 34. Based on the input image data, the projection control unit 31 controls the emission of the light beam 50 from the light source 11. Also, the projection control unit 31 controls the driving of the scanning unit 13. Based on the instruction of the projection control unit 31, the scanning unit 13 rapidly scans the light beam 50 from the upper left to the lower right of the image (e.g., raster scan). The projection control unit 31 controls the projection of the light beam 50 onto the user's retina 61 by controlling the light source 11 and the scanning unit 13. If a camera is installed at an appropriate position facing the user's line of sight, the image in the line of sight captured by this camera can be projected onto the retina 61. Also, it is possible to project an image input from a recording device or the like, or to super-impose a camera image and an image from a recording device or the like to project a so-called augmented reality (AR) image.

[0022] The light beam 50 emitted by the light source 11 passes through the lens 12. The lens 12 is a condenser lens that converts the light beam 50 from diffused light to convergent light. The light beam 50 that has passed through the lens 12 is incident on the scanning unit 13. The scanning unit 13 (scanner) scans the light beam 50 in two-dimensional directions of the horizontal and vertical directions. The scanning unit 13 is, for example, a MEMS (Micro Electric Mechanical System) mirror. Note that the scanning unit 13 may be other than a MEMS mirror, and may be, for example, a scanner using potassium niobate tantalate (KTN) or the like.

[0023] Scanned in two-dimensional directions by the scanning unit 13, a plurality of light beams 50 emitted from the scanning unit 13 in different directions at different times enter the projection optical system 20. The projection optical system 20 converges the plurality of light beams 50 emitted from the scanning unit 13 at the convergence point 70 inside the user's eye 60 and then projects them onto the retina 61 to project an image onto the retina 61. For example, the plurality of light beams 50 emitted from the scanning unit 13 enter the reflection mirror 21. The reflection mirror 21 is a concave mirror having a reflecting surface formed of a curved surface such as a free-form surface and has positive focusing power. The plurality of light beams 50 reflected by the reflection mirror 21 converge at the convergence point 71 in front of the projection mirror 22. A lens 23 is provided at the convergence point 71 between the projection mirror 22 and the reflection mirror 21. The lens 23 is, for example, a biconvex lens. The plurality of light beams 50 pass through the lens 23 and enter the projection mirror 22. The projection mirror 22 is disposed in front of the user's eye 60. The projection mirror 22 reflects the plurality of light beams 50 toward the user's eye 60. The projection mirror 22 is a concave mirror having a reflecting surface formed of a curved surface such as a free-form surface and has positive focusing power. The plurality of light beams 50 reflected by the projection mirror 22 pass through the user's pupil 62, converge at the convergence point 70 near the crystalline lens 63 or the crystalline lens 63, and then are projected onto the retina 61. Thereby, the user can visually recognize the image formed by the light beams 50. The projection mirror 22 may be a half mirror. In this case, the user can visually recognize the external image in a see-through manner.

[0024] FIG. 2 is a perspective view showing a configuration in which a housing 91 incorporating the projection unit 10 and the imaging unit 40 in the first embodiment is attached to a glasses-type frame 90. As shown in FIG. 2, a housing 91 incorporating the projection unit 10 and the imaging unit 40 is attached to a glasses-type frame 90 worn on the user's face. The glasses-type frame 90 is provided with an adjustment unit 92 (for example, an adjustment knob) capable of adjusting the position of the housing 91.

[0025] As shown in FIG. 1, the imaging unit 40 is disposed in front of the user's eye 60 and images the user's eye 60 based on an instruction from the imaging control unit 33. The imaging unit 40 is, for example, an infrared camera. Considering that the user can visually recognize the external image through, the imaging unit 40 may be disposed at a position slightly deviated from the front of the eye 60 within a range where the eye 60 can be imaged. For example, when the projection unit 10 is attached to the glasses-type frame 90, the imaging unit 40 may be provided near the temple or near the nose pad of the glasses-type frame 90.

[0026] The generation unit 34 acquires the captured image of the eye 60 captured by the imaging unit 40 from the imaging unit 40, and generates a superimposed image by superimposing the acquired captured image and the reference image stored in the storage unit 42. The reference image is an image in which an initial position for aligning the center position of the pupil 62 is displayed when the user wears the glasses-type frame 90. Further, the generation unit 34 generates a first guiding mark for guiding the center position of the pupil 62 to coincide with the initial position based on the center position of the pupil 62 shown in the captured image of the eye 60 and the initial position in the reference image.

[0027] FIGS. 3(a) to 3(c) are diagrams showing a captured image 80, a reference image 81, and a superimposed image 82 in the first embodiment. As shown in FIG. 3(a), at least the pupil 62 and the iris 64 of the user's eye 60 are shown in the captured image 80 captured by the imaging unit 40. As shown in FIG. 3(b), an initial position 83 for aligning the center position of the pupil 62 is displayed in the reference image 81 stored in the storage unit 42. A straight line extending in the horizontal and vertical directions through the initial position 83 may be displayed. The superimposed image 82 obtained by superimposing the captured image 80 and the reference image 81 is as shown in FIG. 3(c). In the superimposed image 82, a center position 65 of the pupil 62 and a straight line extending in the horizontal and vertical directions through the center position 65 may be displayed. The center position 65 of the pupil 62 may be specified, for example, by analyzing the black eye range in the captured image 80 or the superimposed image 82, and specifying the center portion of the specified range as the center position 65 of the pupil 62, or may be specified by other methods.

[0028] As shown in FIG. 1, the drive unit 41 is, for example, a biaxial actuator, and moves the reflection mirror 21 based on an instruction from the position control unit 32. The reflection mirror 21 is a concave mirror as described above. When the reflection mirror 21 has the apex of the concave curved surface at its approximate center, it moves so as to swing about the vicinity of the apex. Among the plurality of light rays 50, the light ray 50a corresponding to the center of the projected image projected onto the retina 61 enters the swing center of the reflection mirror 21 even when the reflection mirror 21 moves.

[0029] FIGS. 4(a) to 4(c) are diagrams showing the state until the light ray 50 in the first embodiment reaches the retina 61. FIGS. 4(b) and 4(c) show the case where the reflection mirror 21 has moved with respect to FIG. 4(a).

[0030] As shown in FIGS. 4(a) to 4(c), the light ray 50 emitted from the light source 11 passes through the lens 12. The lens 12 is a condenser lens that converts the light ray 50 from diffused light to convergent light. The light ray 50 that has passed through the lens 12 enters the scanning unit 13 in a state of convergent light. The lens 12 is provided between the light source 11 and the scanning unit 13 in order to make the light ray 50 reflected by the reflection mirror 21 into substantially parallel light. The plurality of light rays 50 scanned in the two-dimensional direction by the scanning unit 13 and emitted in different directions at different times from the scanning unit 13 enter the reflection mirror 21. Each of the plurality of light rays 50 enters the reflection mirror 21 as diffused light after being condensed in front of the reflection mirror 21. Since the reflection mirror 21 has positive focusing power, each of the plurality of light rays 50 is converted from diffused light to substantially parallel light by being reflected by the reflection mirror 21.

[0031] The plurality of light rays 50 reflected by the reflection mirror 21 converge at a convergence point 71 in front of the projection mirror 22. A lens 23 is provided at the convergence point 71. The lens 23 is a condenser lens that converts each of the plurality of light rays 50 from substantially parallel light into converging light. The lens 23 is provided at the convergence point 71 in order to make each of the plurality of light rays 50 reflected by the projection mirror 22 into substantially parallel light. Each of the plurality of light rays 50 that has passed through the lens 23 converges at a condensing point 72 in front of the projection mirror 22 and then becomes diffused light and enters the projection mirror 22. Since the projection mirror 22 has a positive condensing power, each of the plurality of light rays 50 is converted from diffused light into substantially parallel light by being reflected by the projection mirror 22. The plurality of light rays 50 converge at a convergence point 70 within the user's eye 60. The convergence point 70 is located, for example, in or near the crystalline lens 63. The light rays 50 are converted from substantially parallel light into converging light by the crystalline lens 63 and are focused near the retina 61. As a result, the user can visually recognize an image.

[0032] As shown in FIG. 4(a), when the light ray 50a corresponding to the center of the projected image projected onto the retina 61 enters the eye 60 substantially from the front, the convergence point 71 at which the plurality of light rays 50 reflected by the reflection mirror 21 converge is near the center of the lens 23.

[0033] As shown in FIG. 4(b), when the user tries to look at the right side of the image and turns the line of sight to the right, the pupil 62 moves to the right with respect to the user's face. In this case, the position control unit 32 gives an instruction to the drive unit 41 to move the reflection mirror 21 so that the plurality of light rays 50 pass through the left region of the lens 23 with respect to the traveling direction of the light rays 50. As a result, the convergence point 71 where the plurality of light rays 50 converge is located in the left region of the lens 23 with respect to the traveling direction of the light rays 50. The plurality of light rays 50 pass through the left region of the lens 23 with respect to the traveling direction of the light rays 50, are refracted to the right side of the traveling direction by the lens 23, and then enter the projection mirror 22. By being refracted to the right side of the traveling direction by the lens 23, the position and the incident angle at which the plurality of light rays 50 enter the projection mirror 22 change from the state of FIG. 4(a). As a result, the position of the convergence point 70 of the plurality of light rays 50 moves to the right, and even when the pupil 62 moves to the right, the plurality of light rays 50 enter the pupil 62.

[0034] As shown in FIG. 4(c), when the user tries to look at the left side of the image and turns the line of sight to the left, the pupil 62 moves to the left with respect to the user's face. In this case, the position control unit 32 gives an instruction to the drive unit 41 to move the reflection mirror 21 so that the plurality of light rays 50 pass through the right region of the lens 23 with respect to the traveling direction of the light rays 50. As a result, the convergence point 71 where the plurality of light rays 50 converge is located in the right region of the lens 23 with respect to the traveling direction of the light rays 50. The plurality of light rays 50 pass through the right region of the lens 23 with respect to the traveling direction of the light rays 50, are refracted to the left side of the traveling direction by the lens 23, and then enter the projection mirror 22. By being refracted to the left side of the traveling direction by the lens 23, the position and the incident angle at which the plurality of light rays 50 enter the projection mirror 22 change from the state of FIG. 4(a). As a result, the position of the convergence point 70 of the plurality of light rays 50 moves to the left, and even when the pupil 62 moves to the left, the plurality of light rays 50 enter the pupil 62.

[0035] Note that, in FIGS. 4(b) and 4(c), the case where the pupil 62 moves in the horizontal direction is shown as an example, but the same applies when the pupil 62 moves in other directions such as the vertical direction. Thus, by moving the reflecting mirror 21 with the driving unit 41, the position of the convergence point 70 can be moved, and thereby, even when the line-of-sight direction of the eye 60 moves, the light beam 50 can be made to enter the pupil 62. From this, if the position of the pupil 62 is specified from the captured image 80 captured by the imaging unit 40, and the reflecting mirror 21 is moved so that the convergence point 70 is positioned at the specified position of the pupil 62, a plurality of light beams 50 can be continuously made to enter the pupil 62. To achieve this, control information associating a plurality of points in the captured image 80 captured by the imaging unit 40 and the control values for driving the driving unit 41 to position the convergence point 70 at these plurality of points is acquired in advance. Then, the position control unit 32 specifies the position of the pupil 62 imaged in the captured image 80 captured by the imaging unit 40, and controls the driving unit 41 based on the specified position of the pupil 62 and the control information to move the reflecting mirror 21, thereby positioning the convergence point 70 at the pupil 62. The control information is, for example, stored in advance in the storage unit 42.

[0036] Table 1 is an example of the control information stored in the storage unit 42. Table 1 is an example when the horizontal viewing angle of the image projected onto the retina 61 is 28° and the vertical viewing angle is 14°. As shown in Table 1, the control information describes a plurality of points in the captured image 80 captured by the imaging unit 40 and the current values (control values) for positioning the convergence point 70 at these plurality of points in association with each other. Here, the plurality of points are defined by the angle in the horizontal direction and the angle in the vertical direction. Since the driving unit 41 for moving the reflecting mirror 21 is, for example, a biaxial actuator, when the normal direction at the vertex of the concave surface of the reflecting mirror 21 is the roll axis, the horizontal direction of the reflecting mirror 21 is the pitch axis, and the vertical direction is the yaw axis, the current value for yaw rotation (the left numerical values in Table 1) and the current value for pitch rotation (the right numerical values in Table 1) are described. For example, to position the convergence point 70 at the point defined by the horizontal direction angle and the vertical direction angle of (10.5°, 3.5°), the driving unit 41 may be driven with the current values of (36.44 mA, 9.75 mA) to move the reflecting mirror 21.

Table 1

[0037] Here, an example of the method for acquiring the reference image 81 shown in FIG. 3(b) will be described. FIG. 5 is a diagram showing an example of the method for acquiring the reference image 81 in the first embodiment. As shown in FIG. 5, an image projection device 100 is prepared, and the light source 11 is driven to emit a light beam 50 from the image projection device 100 to the outside. At this time, the driving unit 41 is driven or not driven, and the reflecting mirror 21 is in the initial position. The light beam 50 emitted from the image projection device 100 converges at the convergence point 70. A screen 84 is arranged at the convergence point 70. The light beam 50 irradiated on the screen 84 is imaged by the imaging unit 40. The image imaged by the imaging unit 40 becomes the reference image 81 shown in FIG. 3(b), and the position of the convergence point 70 of the light beam 50 when the reflecting mirror 21 is in the initial position becomes the initial position 83.

[0038] Therefore, when the user wears the glasses-type frame 90 (image projection device 100), by aligning the center position 65 of the pupil 62 shown in the captured image 80 with the initial position 83 displayed on the reference image 81, the center position 65 of the pupil 62 can be aligned with the position of the convergence point 70 where the light beam 50 converges. Thus, by moving the reflecting mirror 21 using the control information as shown in Table 1, even when the line-of-sight direction of the eye 60 moves, the light beam 50 can continue to be incident on the pupil 62.

[0039] [Control Method] FIG. 6 is a flowchart showing an example of the control of the control unit 30 in the first embodiment. As shown in FIG. 6, when the glasses-type frame 90 (image projection device 100) is worn on the user's face, the imaging control unit 33 controls the imaging unit 40 to start imaging the user's eye 60 (step S10). For example, when an imaging start instruction is input from a user wearing the glasses-type frame 90, the imaging control unit 33 controls the imaging unit 40 to start imaging the user's eye 60.

[0040] Next, the generation unit 34 acquires a captured image 80 obtained by imaging the user's eye 60 from the imaging unit 40 (step S12). For example, the generation unit 34 acquires a captured image 80 as shown in FIG. 3(a). Next, the generation unit 34 generates a superimposed image 82 obtained by superimposing the acquired captured image 80 and a reference image 81 indicating the initial position 83 stored in the storage unit 42 (step S14). For example, the generation unit 34 superimposes a captured image 80 as shown in FIG. 3(a) and a reference image 81 as shown in FIG. 3(b) to generate a superimposed image 82 as shown in FIG. 3(c).

[0041] Further, the generation unit 34 generates a first guidance mark for guiding the center position 65 of the pupil 62 and the initial position 83 to coincide with each other based on the center position 65 of the pupil 62 shown in the captured image 80 and the initial position 83 in the reference image 81 (step S16). For example, the generation unit 34 generates an arrow indicating the direction to move in order to make the center position 65 of the pupil 62 and the initial position 83 coincide with each other from the center position 65 of the pupil 62 and the initial position 83 as the first guidance mark.

[0042] Next, the projection control unit 31 controls the light source 11 and the scanning unit 13 to project the superimposed image 82 and the first guidance index generated by the generation unit 34 onto the user's retina 61 (step S18). FIGS. 7(a) and 7(b) are diagrams showing an example of a projection image 86 in the first embodiment. As shown in FIG. 7(a), a projection image 86 including the superimposed image 82 and, for example, a first guidance mark 87 in the form of an arrow is projected onto the retina 61. The projection image 86 may include an image (for example, mountains, rivers, and forests in FIGS. 7(a) and 7(b)) based on image data input from, for example, a camera and / or a recording device. By projecting the projection image 86 including the superimposed image 82 and the first guidance mark 87, the user can visually grasp the deviation between the position of his or her own pupil 62 and the initial position 83. Therefore, for example, the user can adjust the position of the adjustment unit 92 (see FIG. 2) that moves the position of the projection unit 10, the position of the nose pad of the glasses-type frame 90, etc., so that the center position 65 of the pupil 62 and the initial position 83 coincide with each other. FIG. 7(b) shows an example of the superimposed image 82 after alignment.

[0043] Returning to FIG. 6, the generation unit 34 determines whether the position adjustment for adjusting so that the center position 65 of the pupil 62 and the initial position 83 coincide is completed (step S20). If not completed (No), the process returns to step S12. If completed (Yes), this process ends. The determination of whether the position adjustment is completed may be made, for example, based on whether an instruction indicating completion is input from the user, or may be made based on whether the deviation amount between the center position 65 of the pupil 62 and the initial position 83 becomes equal to or less than a predetermined value.

[0044] According to the first embodiment, as shown in FIGS. 3(a) to 3(c), the generation unit 34 generates a superimposed image 82 by superimposing the captured image 80 of the user's eye 60 captured by the imaging unit 40 and the reference image 81 on which the initial position 83 to align the user's pupil 62 is displayed. The projection control unit 31 controls the projection unit 10 to project the superimposed image 82 generated by the generation unit 34 onto the user's retina 61. Thereby, the user can visually grasp the positional relationship between the position of his or her own pupil 62 and the initial position 83 to which the pupil 62 should be aligned. Therefore, since the user can adjust the position of the pupil 62 and the initial position 83 while viewing the superimposed image 82, alignment can be easily performed.

[0045] Further, in the first embodiment, as shown in FIG. 7(a), the generation unit 34 generates a first guiding mark 87 for guiding the center position 65 of the pupil 62 and the initial position 83 to coincide with each other based on the center position 65 of the pupil 62 shown in the captured image 80 and the initial position 83 in the reference image 81. The projection control unit 31 projects the first guiding mark 87 onto the retina 61. Thereby, the user can visually grasp the direction to adjust in order to make the center position 65 of the pupil 62 and the initial position 83 coincide with each other, and thus alignment can be easily performed.

[0046] Note that, in the first embodiment, the case where the first guiding mark 87 is an arrow is shown as an example, but as long as it is possible to guide the center position 65 of the pupil 62 and the initial position 83 to coincide with each other, a mark other than an arrow may also be used.

Embodiment

[0047] FIG. 8(a) is a diagram showing the image projection apparatus 200 according to the second embodiment, and FIG. 8(b) is a diagram showing the irradiation of the illumination light 17 emitted from the illumination light source 15 onto the cornea 66. As shown in FIG. 8(a), the image projection apparatus 200 according to the second embodiment includes a plurality of illumination light sources 15 in front of the user's eye 60. The illumination light source 15 is built into the housing 91 in FIG. 2 together with the projection unit 10 and the imaging unit 40, and is attached to the glasses-type frame 90. The illumination light source 15 is, for example, an infrared LED (Light Emitting Diode), and emits illumination light 17 which is infrared light based on an instruction from the imaging control unit 33. At least four illumination light sources 15 are provided so as to face the pupil 62 in the vertical direction and the horizontal direction, respectively. For this reason, as shown in FIG. 8(b), four illumination lights 17 facing the pupil 62 in the vertical direction and the horizontal direction are irradiated onto the user's cornea 66. Since the other configurations are the same as those in the first embodiment, the description thereof is omitted.

[0048] FIGS. 9(a) to 9(e) are diagrams showing the relationship between the distance d between the illumination light source 15 and the cornea 66 in the second embodiment, and the illumination light 17 shown in the captured image 80 captured by the imaging unit 40. As shown in FIG. 9(a), the distance from the plurality of illumination light sources 15 to the apex 67 of the cornea 66 is defined as d. Since the illumination light source 15 emits the illumination light 17 in an oblique direction inclined toward the apex 67 side of the cornea 66, as shown in FIGS. 9(b) to 9(e), the intervals D1 and D2 of the illumination light 17 shown in the captured image 80 change depending on the magnitude of the distance d from the illumination light source 15 to the apex 67 of the cornea 66. FIG. 9(b) shows an example when the distance d is 3 mm, FIG. 9(c) shows an example when the distance d is 5 mm, FIG. 9(d) shows an example when the distance d is 7 mm, and FIG. 9(e) shows an example when the distance d is 9 mm. As the distance d increases, the intervals D1 and D2 decrease. The interval D1 is the distance between the centers of two illumination lights 17 facing each other in the vertical direction with respect to the pupil 62. The interval D2 is the distance between the centers of two illumination lights 17 facing each other in the horizontal direction with respect to the pupil 62.

[0049] As shown in FIG. 8(a), it is desirable that the convergence point 70 where a plurality of light rays 50 emitted from the image projection device 100 converge is at an appropriate position on or near the crystalline lens 63. However, when the user wears the glasses-type frame 90 (image projection device 100), depending on the position of the glasses-type frame 90 with respect to the user's face, the position of the convergence point 70 may deviate from an appropriate position in the traveling direction of the light rays 50. As shown in FIGS. 9(a) to 9(e), since there is a correlation between the interval d and the intervals D1 and D2, the interval d can be specified by obtaining the intervals D1 and D2. Since the illumination light source 15 is built into the housing 91 together with the projection unit 10 and the imaging unit 40 and attached to the glasses-type frame 90, the value of the interval d when the convergence point 70 is at an appropriate position is specified. Therefore, by adjusting the intervals D1 and D2 to be appropriate sizes, the convergence point 70 can be set at an appropriate position.

[0050] Therefore, in the second embodiment, the generation unit 34 detects the intervals D1 and D2 between the plurality of illumination lights 17 shown in the captured image 80, and generates a second guiding mark for guiding the intervals D1 and D2 to predetermined values. The projection control unit 31 projects the second guiding mark generated by the generation unit 34 onto the retina 61.

[0051] FIGS. 10(a) and 10(b) are diagrams for explaining the case where the line of sight direction of the eye 60 in the second embodiment is inclined with respect to the optical axis 44 of the imaging unit 40. In FIGS. 10(a) and 10(b), the upper figure is a cross-sectional view of the eye 60, and the lower figure is a front view of the eye 60. As shown in the upper figure of FIG. 10(a), when there is no inclination between the optical axis 44 of the imaging unit 40 and the line of sight direction of the eye 60, the vertex 67 of the cornea 66 and the center position 65 of the pupil 62 are located on the optical axis 44 of the imaging unit 40. At this time, as shown in the lower figure, the irradiation position of the illumination light 17 on the cornea 66 is such that the intersection point 18 of the straight line passing through the center of the illumination light 17 facing the pupil 62 in the vertical direction and the straight line passing through the center of the illumination light 17 facing the pupil 62 in the horizontal direction is located on the optical axis 44.

[0052] As shown in the upper figure of Fig. 10(b), when the user wears the glasses-type frame 90 (image projection device 100), the optical axis 44 of the imaging unit 40 and the line of sight direction of the eye 60 may be inclined by an inclination θ. In this case, as shown in the upper and lower figures, the center position 65 of the pupil 62 is displaced by a displacement amount δ with respect to the optical axis 44 of the imaging unit 40. In other words, the center position 65 of the pupil 62 is displaced by a displacement amount δ with respect to the intersection point 18 of the straight line passing through the centers of the plurality of illumination lights 17. By eliminating this displacement amount δ, the inclination displacement can be suppressed.

[0053] Therefore, in the second embodiment, in order to guide the displacement amount δ to be reduced, the generation unit 34 generates a superimposed image 82 in which the intersection point 18 of the straight line passing through the centers of the plurality of illumination lights 17 shown in the captured image 80 and the center position 65 of the pupil 62 shown in the captured image 80 are displayed.

[0054] Figs. 11(a) to 11(c) are diagrams showing examples of the superimposed image 82 and the second guiding mark 88 generated by the generation unit 34 in the second embodiment. As shown in Fig. 11(a), the generation unit 34 may generate a superimposed image 82 in which a second guiding mark 88 (hatched portion), which is a color that changes according to the displacement amount from appropriate sizes of the intervals D1 and D2, is displayed at a location overlapping the illumination light 17. As shown in Fig. 11(b), the generation unit 34 may generate a superimposed image 82 in which a second guiding mark 88 (hatched portion), which is a color that changes according to the displacement amount from appropriate sizes of the intervals D1 and D2, is displayed in a region that does not overlap the eye 60. As shown in Fig. 11(c), the generation unit 34 may generate a second guiding mark 88 (hatched portion), which is a color that changes according to the displacement amount from appropriate sizes of the intervals D1 and D2, in a region other than the superimposed image 82 in the projection image 86. For example, the second guiding mark 88 may be red when the displacement amount is large, and may be a color that changes like orange, yellow, green, or blue as the displacement amount decreases.

[0055] Also, as shown in Figs. 11(a) to 11(c), the generation unit 34 generates a superimposed image 82 in which the intersection point 18 of the straight line passing through the centers of the plurality of illumination lights 17 shown in the captured image 80 and the center position 65 of the pupil 62 shown in the captured image 80 are displayed.

[0056] [Control Method] FIG. 12 is a flowchart showing an example of the control of the control unit 30 in the second embodiment. As shown in FIG. 12, when the glasses-type frame 90 (image projection device 100) is worn on the user's face, the imaging control unit 33 controls the illumination light source 15 to irradiate the illumination light 17 onto the user's eyes 60 (step S30). For example, when an instruction to irradiate the illumination light 17 is input from the user wearing the glasses-type frame 90, the imaging control unit 33 controls the illumination light source 15 to irradiate the illumination light 17 onto the user's eyes 60. Next, the imaging control unit 33 controls the imaging unit 40 to start imaging the user's eyes 60 (step S32).

[0057] Next, the generation unit 34 acquires the captured image 80 of the user's eyes 60 captured by the imaging unit 40 (step S34). Next, the generation unit 34 detects the intervals D1 and D2 of the illumination light 17 shown in the captured image 80, and generates a second guiding mark 88 for guiding the intervals D1 and D2 to a predetermined value according to the detection result (step S36). For example, the second guiding mark 88 displayed in the superimposed image 82 or the second guiding mark 88 displayed in a region other than the superimposed image 82 in the projected image 86 as shown in FIGS. 11(a) to 11(c) is generated.

[0058] Next, the generation unit 34 superimposes the captured image 80 and the reference image 81 indicating the initial position 83, and generates a superimposed image 82 in which the intersection point 18 of the straight line passing through the centers of the plurality of illumination lights 17 shown in the captured image 80 and the center position 65 of the pupil 62 shown in the captured image 80 are displayed (step S38). For example, the superimposed image 82 in which the intersection point 18 and the center position 65 are displayed as shown in FIGS. 11(a) to 11(c) is generated.

[0059] In addition, the generation unit 34 generates a first guiding mark 87 for guiding the center position 65 of the pupil 62 and the initial position 83 to coincide with each other based on the center position 65 of the pupil 62 shown in the captured image 80 and the initial position 83 in the reference image 81 (step S40).

[0060] Next, the projection control unit 31 controls the light source 11 and the scanning unit 13 to project the superimposed image 82, the first guiding mark 87, and the second guiding mark 88 generated by the generation unit 34 onto the user's retina 61 (step S42). Thus, similar to the first embodiment, since the user can visually grasp the deviation between the position of his / her pupil 62 and the initial position 83 by the superimposed image 82, the user can adjust the position so that the position of the pupil 62 and the initial position 83 match while referring to the first guiding mark 87 (see FIG. 7(a)). Also, the user can adjust the position of the convergence point 70 where the plurality of light beams 50 converge to an appropriate position by the second guiding mark 88. Further, the user can adjust the inclination deviation between the optical axis 44 of the imaging unit 40 and the line-of-sight direction of the eye 60 by the display of the intersection point 18 of the straight line passing through the centers of the plurality of illumination lights 17 and the center position 65 of the pupil 62. For example, the user can make the above various adjustments by moving the position of the projection unit 10 with the adjustment unit 92 (see FIG. 2) or adjusting the position of the nose pad of the glasses-type frame 90 or the like.

[0061] Next, the generation unit 34 determines whether or not the position adjustment using the superimposed image 82 is completed (step S44). If not completed (No), the process returns to step S34. If completed (Yes), this process ends. The determination of whether or not the position adjustment is completed may be made, for example, based on whether or not an instruction indicating completion is input from the user, or may be made by other methods.

[0062] According to Example 2, as shown in Fig. 8(a), by being built into the housing 91 together with the projection unit 10 and the imaging unit 40, it is attached to the glasses-type frame 90 and includes an illumination light source 15 that irradiates a plurality of illumination lights 17 onto the user's eye 60. As shown in Figs. 9(a) to 9(e) and Figs. 11(a) to 11(c), the generation unit 34 generates a second guiding mark 88 that guides the intervals D1 and D2 between the plurality of illumination lights 17 shown in the captured image 80 to a predetermined value, and the projection control unit 31 controls the projection unit 10 to project the second guiding mark 88 onto the retina 61. As a result, the user can adjust the intervals D1 and D2 of the plurality of illumination lights 17 to an appropriate size based on the second guiding mark 88, and as a result, the position of the convergence point 70 can be adjusted to an appropriate position.

[0063] Also, in Example 2, as shown in Fig. 8(b), the illumination light source 15 irradiates four illumination lights 17 that face the user's eye 60 in the vertical direction (first direction) and the horizontal direction (second direction), respectively. The generation unit 34 generates a second guiding mark 88 that guides the intervals D1 between the two illumination lights 17 facing each other in the vertical direction and the intervals D2 between the two illumination lights 17 facing each other in the horizontal direction to a predetermined value, as shown in Figs. 9(b) to 9(e). By adjusting such intervals D1 and D2 to an appropriate size, the position of the convergence point 70 can be adjusted to an appropriate position.

[0064] In addition, in Example 2, the case where the second guiding mark 88 is a color that changes according to the intervals D1 and D2 of the plurality of illumination lights 17 is shown as an example, but as long as the intervals D1 and D2 can be guided to a predetermined value, other cases than color may be used.

[0065] Also, according to Example 2, as shown in Figs. 11(a) to 11(c), the generation unit 34 generates a superimposed image 82 that displays the intersection point 18, which is the position based on the plurality of illumination lights 17 shown in the captured image 80, and the center position 65 of the pupil 62 shown in the captured image 80. As a result, the user can adjust the position so that the intersection point 18 and the center position 65 of the pupil 62 coincide, and as a result, the inclination deviation between the optical axis 44 of the imaging unit 40 and the line-of-sight direction of the eye 60 can be reduced.

[0066] Also, in the second embodiment, as shown in FIGS. 10(a) and 10(b), the generation unit 34 sets the intersection point 18 where the straight line passing through the centers of the two illumination lights 17 facing each other in the vertical direction and the straight line passing through the centers of the two illumination lights 17 facing each other in the horizontal direction intersect as the position based on the plurality of illumination lights 17. By displaying such an intersection point 18 on the superimposed image 82 and performing position adjustment so that the intersection point 18 and the center position 65 of the pupil 62 coincide, it is possible to reduce the inclination deviation between the optical axis 44 of the imaging unit 40 and the line of sight direction of the eye 60.

[0067] In the first and second embodiments, the reflection mirror 21 and the projection mirror 22 are not limited to the case of a concave mirror. As long as they have positive condensing power, in addition to a curved mirror, other optical components such as a combination of lenses and mirrors, or the use of a diffractive element may be used. The lens 23 is not limited to the case of a convex lens. As long as it is possible to make the light beam 50 enter the projection mirror 22 in a diffused light state, other optical members such as a mirror or a diffractive element may be used. The lens 23 may have a function of suppressing chromatic aberration.

[0068] In the first and second embodiments, the case where the image projection device is attached to the glasses-type frame 90 is shown, but it may also be attached to frames such as a goggle type, an eye patch type, an ear hook type, or a helmet-mounted type.

[0069] As described above in detail with respect to the embodiments of the present invention, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

Explanation of reference numerals

[0070] 10…Projection unit, 11…Light source, 12…Lens, 13…Scanning unit, 15…Illumination light source, 17…Illumination light, 18…Intersection point, 20…Projection optical system, 21…Reflection mirror, 22…Projection mirror, 23…Lens, 30…Control unit, 31…Projection control unit, 32…Position control unit, 33…Imaging control unit, 34…Generation unit, 40…Imaging unit, 41…Drive unit, 42…Memory unit, 44…Optical axis, 50, 50a…Light rays, 60…Eye, 61…Retina, 62…Pupil, 63…Lens, 64…Iris, 65…Central position, 66…Cornea, 67…Vertex of the cornea, 70…Convergence point, 71…Convergence point, 80…Captured image, 81…Reference image, 82…Superimposed image, 83…Initial position, 84…Screen, 86…Projection image, 87…First guiding marker, 88…Second guiding marker, 90…Glasses-type frame, 91…Housing, 92…Adjustment unit, 100, 200…Image projection device

Claims

1. A scanning unit that scans light rays emitted from a light source, and a projection optical system that converges a plurality of the light rays emitted from the scanning unit in different directions onto a convergence point in the user's eye and then projects the light rays onto the retina to project an image onto the retina, including a projection unit; An imaging unit that images the user's eye; A generation unit that generates a superimposed image by superimposing an imaging image of the user's eye captured by the imaging unit and a reference image on which a position to align the user's pupil is displayed; An image projection device comprising a projection control unit that controls the projection unit to project the superimposed image onto the retina.

2. The generation unit generates a first guiding mark that guides the center position of the user's pupil to coincide with the position based on the center position of the user's pupil shown in the imaging image and the position in the reference image; The projection control unit controls the projection unit to project the first guiding mark onto the retina. The image projection device according to claim 1.

3. The first guiding mark is an arrow. The image projection device according to claim 2.

4. The projection unit and the imaging unit are attached to a frame worn by the user. The image projection device according to claim 1 or 2.

5. An illumination light source that is attached to a frame worn by the user together with the projection unit and the imaging unit and irradiates a plurality of illumination lights onto the user's eye; The generation unit generates a second guiding mark that guides the interval between the plurality of illumination lights shown in the imaging image to a predetermined value; The projection control unit controls the projection unit to project the second guiding mark onto the retina. The image projection device according to claim 1.

6. The second guiding mark is a color that changes according to the interval between the plurality of illumination lights. The image projection device according to claim 5.

7. The illumination light source irradiates the user's eye with at least four illumination lights that face each other in a first direction and a second direction orthogonal to the first direction as the plurality of illumination lights; The generation unit generates the second guiding mark that guides the interval between two illumination lights facing each other in the first direction and the interval between two illumination lights facing each other in the second direction to the predetermined value. The image projection device according to claim 5 or 6.

8. An illumination light source that is attached to a frame worn by the user together with the projection unit and the imaging unit and irradiates a plurality of illumination lights onto the user's eye; The image projection device according to claim 1, wherein the generation unit generates the superimposed image that displays a position based on the plurality of illumination lights shown in the captured image and a center position of the pupil shown in the captured image.

9. The illumination light source irradiates the user's eyes with four illumination lights that face each other at least in a first direction and a second direction orthogonal to the first direction as the plurality of illumination lights. The image projection device according to claim 8, wherein the generation unit sets, as a position based on the plurality of illumination lights, an intersection point at which a straight line passing through centers of two illumination lights facing each other in the first direction and a straight line passing through centers of two illumination lights facing each other in the second direction intersect.

Images