Image projection device and control method of image projection device

JP2023078881A5Active Publication Date: 2024-11-29CANON KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2021192193
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-11-26
Publication Date
2024-11-29
Estimated Expiration
2041-11-26

AI Technical Summary

Technical Problem

Existing image projection devices that project images onto the retina struggle with users being unable to gaze at peripheral areas due to the projected image following the movement of the pupil, making it difficult to view the entire image when the user looks away from the center.

Method used

An image projection device that includes a light source, projection unit, detection means, and control section to control the projection image based on the position of the pupil, allowing the device to adjust the projected image to maintain visibility when the pupil moves.

Benefits of technology

Enables easy gazing at peripheral areas of the projected image by adjusting the projection to match the user's line of sight, reducing the need for significant pupil movement and enhancing the overall viewing experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

To provide an image projection device which allows a user to easily gaze at a peripheral region of an image.SOLUTION: An image projection device 300 comprises: a light source 303 which outputs a light flux based on image data 301; a projection unit which projects a light flux 314 from the light source 303 to a pupil 306 of a user; detection means 311 which detects information about the position of the pupil 306; and a control unit 302 which controls a projection image 401 projected to a retina 308 by the light flux 314 based on the image data 301. The control unit 302 controls the light source 303 so as to change the projection image according to the position of the pupil 306.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an image projection device, and particularly to an image projection device that projects an image onto the retina.

Background Art

[0002] Conventionally, an image projection device that converges laser light for forming an image near the pupil of a user and projects the image onto the retina has been known. For example, Patent Document 1 discloses an image projection device that forms an image on the retina by scanning laser light modulated based on an input image. In such an image projection device, if the pupil position of the user and the convergence point of the laser light do not substantially coincide, the laser light is blocked by the pupil and the laser light does not reach the retina. Therefore, there has been a problem that when the user moves the pupil, the image cannot be visually recognized.

[0003] Therefore, in Patent Document 2, an image projection device has been proposed that can visually recognize an image even when the pupil moves by detecting and tracking the position of the pupil and projecting an image toward the pupil.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Thus, when an image projection device projects an image onto the retina, making it visible to the user, if the user wants to focus on a peripheral area (especially the edges), they are likely to move their pupil in the direction of that area. The image projection device disclosed in Patent Document 2 is configured to detect the pupil position and track the laser beam, so an image can be projected onto the retina even if the user moves their pupil. However, even though the pupil moves to focus on a peripheral area, the projected image also tracks it, making it difficult to focus on the peripheral area.

[0006] Therefore, the object of the present invention is to provide an image projection device that makes it easy to focus on the peripheral areas of an image. [Means for solving the problem]

[0007] To achieve the above objective, an image display device as one aspect of the present invention comprises a light source that outputs a light beam based on image data, a projection unit that projects the light beam from the light source onto the user's pupil, a detection means for detecting information on the position of the pupil, and a control unit that controls the projected image projected onto the retina by the light beam based on the image data, wherein the control unit controls the light source so that the projected image is changed according to the position of the pupil.

[0008] Other aspects of the present invention will be revealed in the embodiments described below. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an image projection device that makes it easy to focus on the peripheral areas of an image. [Brief explanation of the drawing]

[0010] [Figure 1] Top view of the image projection device according to Embodiments 1 and 2 of the present invention. [Figure 2] Flowchart showing the operation of the output control unit in response to the results of the detection means of the image projection device according to Embodiment 1 of the present invention. [Figure 3]Schematic diagram illustrating the position of the laser light and pupil according to Embodiment 1 of the present invention. [Figure 4] A diagram illustrating the display image and user's line of sight according to Embodiment 1 of the present invention. [Figure 5] Flowchart showing the operation of the output control unit in response to the results of the detection means of the image projection device according to Embodiment 2 of the present invention. [Figure 6] Schematic diagram illustrating the position of the laser light and pupil according to Embodiment 2 of the present invention. [Figure 7] A diagram illustrating the display image and user's line of sight according to Embodiment 2 of the present invention. [Modes for carrying out the invention]

[0011] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In each drawing, the same reference numeral is used for identical components, and redundant descriptions are omitted. [Examples]

[0012] This embodiment will be described with reference to Figures 1 to 4. Figure 1 is a top view of the image projection device 300 according to this embodiment. The image projection device 300 comprises an output control unit 302, a light source 303, a scanning unit 304, a projection unit 305, a pupil light projection unit 309, a light division means 310, a pupil detection unit 311, an eyepiece unit 312, and an eyepiece detection unit 313.

[0013] The output control unit 302 is composed of one or more processors and circuits and is generated by an imaging device or computer (not shown). The output control unit 302 is a control means that controls the output of a bundled RGB parallel light beam (laser light 314) from the light source 303 based on the image data 301 input to the image projection device 300. It also controls the entire image projection device 300.

[0014] The light source 303 is a light-emitting means including a light-emitting portion, and outputs a light beam (hereinafter referred to as laser light 314) obtained by bundling RGB parallel lights. The laser light 314 emitted from the light source 303 enters the scanning means 304 and is scanned two-dimensionally, thereby projecting an image. In the present invention and this specification, the laser light obtained by converting the input image data 301 may be referred to as image light rays.

[0015] The scanning unit 304 is a scanning means for two-dimensionally scanning the laser light 314 emitted from the light source 303, and can be configured by, for example, a MEMS mirror. By changing the angle of the mirror, it is possible to scan the laser light 314 at a predetermined angle. The laser light 304 scanned by the scanning unit 305 enters the projection unit 306.

[0016] The projection unit 305 is a means for projecting the laser light 314 scanned by the scanning unit 304 onto the eyeball 307, and can be configured by, for example, a free-form surface mirror. Further, the projection unit 305 converges the scanned laser light (image light rays) near the pupil 306 of the user. The laser light 314 whose position of the pupil 306 and the convergence point substantially coincide passes through the pupil 306 of the user and enters the eyeball. As the laser light 314 is scanned by the scanning unit 304, the image light rays are scanned on the retina 308 on a thin film that spreads deep in the eyeball 307, and an image corresponding to the input image is projected on the retina 308. The user recognizes the image light rays scanned by the afterimage effect of the image light rays irradiated on the retina 308 as an image.

[0017] Near the finder (not shown), an eyepiece portion 312 that abuts when the user looks through is provided on the outer appearance side of the image projection device 300, and an eyepiece detection unit 313 (eyepiece detection sensor) used to determine whether the user has looked through is provided near the eyepiece portion 312.

[0018] Inside the image projection device 300, an optical splitting means 310, a pair of pupil projection units 309, and a pupil detection unit 311 are arranged, and the position of the pupil 306 is detected by these. When the user looks into the finder of the image projection device 300 to view an image, the position of the user's pupil 306 is detected by the optical splitting means 310, the pair of pupil projection units 309, and the pupil detection unit 311. The pupil projection unit 309 is composed of an infrared light-emitting diode, and infrared light indicated by a dashed line in the figure is projected from the pupil projection unit 309 toward the user's eyeball 307. The optical splitting means 310 is composed of, for example, a dichroic mirror that transmits visible light and reflects infrared light, and reflects the infrared light reflected by the user's eyeball 307 toward the pupil detection means 311. The pupil detection means 311 monitors the eyeball 307 via the optical splitting means 310 and detects the position (for example, central coordinates) and the pupil diameter of the pupil 306. Although the optical splitting means 310 in the present embodiment has a reflection surface of 45 degrees, it is not necessary to limit this angle as long as the infrared light reaches the pupil detection means 311.

[0019] Details of the pupil detection method are described, for example, in Japanese Patent Laid-Open No. 6-34873, and thus the description is omitted here. With such a configuration, it becomes possible to detect the position and the diameter of the pupil 306. In the present embodiment, the position of the user's pupil 306 is detected by monitoring the eyeball 307 via the optical splitting means 310, but the method for detecting the position of the pupil 306 is not particularly limited. For example, the optical splitting means 310 may be omitted and the position of the user's pupil 306 may be determined directly by the pupil detection means 311.

[0020] As described above, the information on the position of the pupil 306 detected by the pupil detection means 311 is input to the output control unit 302. The output control unit 302 can calculate the distance of the laser light 314 from the edge of the pupil 306 from the information on the position and the diameter of the pupil 306, the direction of the user's line of sight, and the direction of the laser light 314 output from the light source 303. FIG. 2 is a flowchart showing the operation of the output control unit 302 according to the result of the pupil detection means 311 of the image projection device 300 shown in FIG. 1.

[0021] The flowchart shown in Figure 2 will be explained with reference to Figures 3 and 4. When the power switch is operated in step S101, the image projection device 300 is powered ON, and the output control unit 302 detects that the power has been turned ON and proceeds to step S102.

[0022] In step S102, the user is determined to be looking at the image projection device 300 based on the detection result of the eyepiece detection unit 313, which is provided near the eyepiece of the image projection device 300. The method of eyepiece detection is not particularly limited, and instead of the eyepiece detection unit 313, the user may be determined to be looking at the image projection device 300 based on the detection result of, for example, the pupil detection means 311.

[0023] If it is determined in step S102 that the user is in the eyepiece position, the process proceeds to step S103, in which the light source 303 of the image projection device 300 is controlled to output a laser beam 314 based on the input image data 301.

[0024] On the other hand, if it is determined in step S102 that the user is not in the eyepiece position, the process proceeds to S104, the laser beam 314 is not emitted, and the process returns to the determination in step S102, waiting until it is determined that the user is in the eyepiece position. Thus, in this embodiment, when it is determined that the user is not in the eyepiece position, the laser beam 314 is not emitted for the purpose of saving power.

[0025] When the irradiation of the laser light 314 is started in step S103, the process proceeds to step S105. In step S105, the output control unit 302 controls the pupil detection means 311 to detect the position of the user's pupil and the diameter of the pupil 306, and obtains the detection results. Based on the detected position and diameter of the pupil 306, the output control unit 302 calculates the position and size of the projection region (first region) 351, which is the area through which the laser light passes, capable of projecting an image onto the user's retina 308.

[0026] The projection region 351 will be explained using Figure 3. Figure 3 is a schematic diagram illustrating the positions of the laser beam 314 and the pupil 306. The projection region 351 is the region that the laser beam must pass through in order to reach the user's retina 308 and project an image. Laser beams that pass through the projection region 351 can project an image onto the retina 308, but laser beams that do not pass through the projection region 351 are vignetted by the iris (outer edge of the pupil) and cannot project an image onto the retina. Figures 3(a) and (b) show the state in which the laser beam 314 is passing through the projection region 351. As shown in Figure 3(a), the laser beam 314 passes through the projection region 351 and reaches the user's retina 308 to project an image. Figure 3(b) shows the state in which the user moves the pupil 306 to the right from the state in Figure 3(a). As the pupil 306 moves to the right, the projection area 351 also moves to the right, changing the relative position of the laser beam 314 and the projection area 351. However, the laser beam can still pass through the projection area 351 and reach the retina 308 to project an image. On the other hand, Figure 3(c) shows a state where the laser beam 314 does not pass through the projection area 351. Figure 3(c) shows a state where the user moves the pupil 306 further to the right from the state in Figure 3(b). As the pupil 306 moves to the right, the projection area 351 also moves to the right, changing the relative position of the laser beam 314 and the pupil 306, and the laser beam 314 does not pass through the projection area 351. Therefore, the laser beam 314 is vignetted by the iris, cannot pass through the pupil, and cannot project an image onto the retina 308.

[0027] In this embodiment, the position of the projection region 351 coincides with the pupil 306, and the size of the projection region 351 coincides with the size of the pupil 306. Furthermore, when determining whether the laser beam 314 passes through the projection region 351, the determination is made based on whether the convergence point of the laser beam 314 is within the projection region 351.

[0028] Next, in step S106, based on the position and size of the projection region 351 calculated in step S105 and the known direction of the laser beam 314, it is determined whether the first distance, which is the distance between the laser beam 314 and the outer edge of the projection region 351, is less than or equal to a predetermined threshold. Here, the distance between the laser beam 314 and the outer edge of the projection region 351 refers to the shortest distance between the convergence point of the laser beam 314 and the outer edge of the projection region 351. Note that if the direction of laser beam irradiation (up and down direction in Figure 3) is the z direction, the shortest distance is the distance on the xy plane.

[0029] In this embodiment, the determination of whether the first distance is below a threshold is made by whether the laser beam 314 passes through a second region 352 set inside the projection region 351. As shown in Figure 3(a), if the laser beam 314 passes inside the second region 352, it is determined that the first distance is greater than the threshold (NO in S106). On the other hand, as shown in Figure 3(b), if the laser beam 314 passes outside the second region 352, it is determined that the first distance is below the threshold (YES in S106). Whether the laser beam 314 passes inside the second region 352 is determined by the positional relationship between the second region 352 and the convergence point of the laser beam. If the convergence point of the laser beam 314 is inside the second region 352, it is determined that it has passed inside the second region; if it is outside the second region, it is determined that it has not passed inside the second region (i.e., it has passed outside). Furthermore, the method for determining whether the first distance is below the threshold is not limited to this method and can be modified in various ways. For example, this step may be performed by calculating the first distance based on the coordinates of the laser beam's convergence point, the coordinates of the center of the projection region 351, and the diameter of the projection region 351, and then comparing the calculated distance with the threshold.

[0030] If it is determined in step S106 that the first distance is below the threshold, the process proceeds to step S107, where the position of the projected image 401 is shifted based on the direction that shows the shortest distance between the laser beam 314 and the outer edge of the optical projection area 351. If it is determined in step S106 that the first distance is not below the threshold, the process proceeds to step S108, where the image 401 is projected onto its normal, unshifted position.

[0031] The shift of the projected image 401 will be explained using Figures 3 and 4. Figures 4(a) and 4(b) show the projected image 401 projected onto the user's retina 308 in the state shown in Figures 3(a) and 3(b), respectively, with the position of the user's pupil 306. The gaze position P101 indicates the position where the user's gaze is focused, and in Figures 3(a) and 4(a), the user's gaze is focused on the center of the image 401 (looking straight ahead). The third region 402 enclosed by the dotted line is the region on the projected image 401 that corresponds to the projection region 351 calculated in step S105. When the gaze position P101 moves outside the third region 402, as shown in Figure 3(c), the laser light 314 is vignetted by the iris, and the laser light 314 does not irradiate the user's retina 308.

[0032] Furthermore, the information indicating the gaze position P101 and the information indicating the third region 402 may be superimposed on the projected image 401 and projected to the user. By superimposing the gaze position P101 and the third region 402 onto the projected image 401 in Figure 4, the user can use it as a guide to prevent unintentionally moving their gaze outside the third region 402 when moving their gaze. Furthermore, either the gaze position P101 or the third region 402 may be superimposed, and it is preferable to have a configuration that allows the user to switch whether or not to superimpose them.

[0033] In the state shown in Figure 4(a), point P102 (which corresponds to the face of the person in the upper right of the projected image 401), located near the edge of the peripheral region of the projected image 401, is projected outside the third region 402, which corresponds to the projection region 351. Therefore, when the user tries to fixate on point P102 and shifts their gaze to P102, the laser beam 314 does not pass through the projection region 351, as shown in Figure 3(c), and cannot enter the pupil 306, so the user is unable to see the projected image 401.

[0034] When a user shifts their gaze from point P101 to P102, there is a moment during the movement when the laser light passes through projection area 351 but not through second area 352 (a moment when it passes between the outer edge of projection area 351 and the outer edge of second area 352). Therefore, in this embodiment, if it is determined in step S106, which is performed during the user's movement of shifting their gaze from point P101 to P102 in the projected image 401, that the first distance is below a threshold, a projected image 401 is projected, as shown in Figure 4(b), which is a shifted version of the projected image 401 in Figure 4(a). That is, the projected image 401 is an image in which the projection position of the projected image 401 in the state of Figure 4(a) is shifted downwards to the left. To shift the position of the projected image in this way, the position of the area to be projected image 401 in the image corresponding to the input image data 301 is shifted downwards to the left. As shown in Figures 3(b) and 4(b), the position on which point P102 is projected on the retina also shifts accordingly, moving into the interior of the third region 402. Therefore, even if the user aligns their gaze position P101 with point P102, the laser light 314 is projected onto the retina 308, allowing them to focus on the face of the person in the upper right.

[0035] Figures 3(b) and 4(b) show point P103 as the position where point P102 was projected before the projection image 401 was shifted (i.e., in the state shown in Figures 3(a) and 4(a)). Thus, in this embodiment, by shifting the position of the region that will be the projection image 401 as the pupil 306 moves, the projection position of the person's face in the upper right moves from the position of point P103 in Figure 4(b) to point P102. Note that the fourth region 404 shown in Figure 4(b) is the region of the image corresponding to the image data 301 that has been removed from the region that will be the projection image 401 as described above, and the image data in this part is not converted into laser light.

[0036] As described above, in this embodiment, when the first distance is below a threshold, an image is projected that is like a shifted version of the original projection image (projection image 401 in Figure 4(a)). However, this is not limited to this, as long as the position of at least a portion of the peripheral region of the image corresponding to the input image data 301 (hereinafter sometimes referred to as the input image) differs between when the first distance is below a threshold and when it exceeds a threshold. If the area on the retina onto which the image is projected by the laser light 314 is called the projected area 315, then when the first distance is below a threshold, the position where at least a portion of the peripheral region of the input image is projected should be closer to the center of the projected area 315 compared to when the first distance exceeds a threshold. In the case of Figure 3, the position where point P102, which is part of the peripheral region of the input image, is projected is closer to the center of the projected area 315 in the situation of Figure 3(b) than in the situation of Figure 3(a), where the first distance is greater than the threshold. Alternatively, by reducing the size of the projected image 401, for example by scaling down the input image, the entire surrounding area including point P102 can be brought closer to the center of the projection area 315, thereby causing point P102 to fall inside the third area 402. When scaling down the image in this way, a large image cannot be projected if the first distance is below a threshold, but if the first distance exceeds the threshold, an image larger than the third area 402 can be projected. Therefore, it is possible to project larger images in more scenes than when the image is always projected so that the projected image fits within the third area 402.

[0037] Furthermore, the peripheral region of the input image can be any region other than the center of the input image. However, as shown in Figure 4, if only a portion of the peripheral region is to be brought closer to the center of the projection region 315, the part of the peripheral region (the upper right portion in Figure 4(b)) to be brought closer to the center of the projection region 315 is determined according to the position of the detected pupil 306.

[0038] As in this embodiment, when shifting the projected image 401, the amount of shift may be determined based on a first distance. For example, instead of determining whether or not to shift the projected image based solely on whether the first distance is below a threshold, the shift amount may be set to a first amount if the first distance is below the first threshold but exceeds the second threshold, and to a second amount greater than the first amount if it is below the second threshold. When the state changes from the first distance being below the first threshold but exceeding the second threshold to being below the second threshold, the difference between the first and second amounts is shifted further. Whether or not the first distance is below the second threshold can be determined by setting an additional region inside the second region. Alternatively, the first distance may be calculated, and the shift amount may be increased as the first distance decreases. In this way, by changing the shift amount based on the first distance, it is possible to control the projection image 401 to prevent abrupt shifts, thereby reducing the discomfort experienced by the user.

[0039] Furthermore, the modification of the projected image 401 according to the first distance only needs to be performed when the laser beam 314 passes through the projection area 315. If the laser beam 314 does not pass through the projection area 315, it is not necessary to modify the projected image 401 according to the first distance (such as shifting or shrinking the image).

[0040] The process then proceeds to step S109, where the eyepiece detection unit 313 determines whether or not the user is looking at the image projection device 300. If it is determined in step S109 that the user is not looking at the device, the process proceeds to S110, where the power is turned off and this flow is terminated.

[0041] If the system determines in step S109 that the user is using the eyepiece, the process returns to step S105 and repeats the above steps until the user is not using the eyepiece or the power is turned off by the user.

[0042] In this embodiment, a large projected image 401 can be displayed while the user's gaze is directed to the edges of the projected image 401. This also allows the user to reduce the amount of pupil movement required when viewing the edges of a large projected image. [Examples]

[0043] This embodiment will be described with reference to Figures 5-7. This embodiment is similar to Embodiment 1 in that the projected image differs depending on whether the first distance, which is the distance between the laser beam 314 and the outer edge of the projection area 351, is below a predetermined threshold or exceeds a threshold. However, this embodiment differs from Embodiment 1 in that if the state in which the first distance is below a predetermined threshold continues, the change in the projection area of ​​the input image also continues. In other words, in this embodiment, if the first distance is below a predetermined threshold, the input image is projected onto the user's retina in a scrolling manner.

[0044] The configuration of the image projection device 300 according to this embodiment is as shown in Figure 1 and is the same as in Embodiment 1, so no explanation is provided.

[0045] Figure 5 is a flowchart showing the operation of the output control unit 302 in response to the results of the pupil detection means 311 of the image projection device 300 shown in Figure 1.

[0046] The flowchart shown in Figure 5 will be explained with reference to Figures 6 and 7.

[0047] When the power switch is operated in step S201, the image projection device 300 is powered ON, and the output control unit 302 detects that the power has been turned ON and proceeds to step S102.

[0048] In step S202, the system determines whether the user is looking at the image projection device 300 based on the detection result of the eyepiece detection unit 313, which is located near the eyepiece of the image projection device 300. The method of eyepiece detection is not particularly limited, and instead of the eyepiece detection unit 313, the system may determine whether the user is looking at the image projection device 300 based on the detection result of, for example, a pupil detection sensor 311.

[0049] If it is determined in step S202 that the user is in the eyepiece position, the process proceeds to step S203, in which the light source 303 of the image projection device 300 is controlled to output a laser beam 314 based on the input image data 301.

[0050] On the other hand, if it is determined in step S202 that the user is not in the eyepiece position, the process proceeds to S204, the laser beam 314 is not emitted, and the process returns to step S202 to wait until it is determined that the user is in the eyepiece position.

[0051] Thus, in this embodiment as well, when it is determined that the user is not in the eyepiece position, the laser beam 314 is not emitted for the purpose of saving power.

[0052] When the irradiation of the laser beam 314 is started in step S203, the process proceeds to step S205. In step S205, the output control unit 302 controls the pupil detection means 311 to detect the position of the user's pupil and the diameter of the pupil 306, and obtains the detection result. Then, based on the detected position of the pupil 306 and the diameter of the pupil 306, the position and size of the projection area 351 are calculated in the same manner as in Embodiment 1. In this embodiment as well, the position of the projection area 351 is assumed to coincide with the pupil 306, and the size of the projection area 351 is assumed to coincide with the size of the pupil 306. Furthermore, when determining whether the laser beam 314 passes through the projection area 351, the determination is made based on whether the convergence point of the laser beam 314 is within the projection area 351.

[0053] Next, in step S206, based on the position and size of the projection region 351 calculated in step 205 and the known direction of the laser beam 314, it is determined whether the first distance, which is the distance between the laser beam 314 and the outer edge of the projection region 351, is less than or equal to a predetermined threshold. As described above, the distance between the laser beam 314 and the outer edge of the projection region 351 refers to the shortest distance between the convergence point of the laser beam 314 and the outer edge of the projection region 351.

[0054] In this embodiment as well, the determination of whether the first distance is below the threshold is made by whether the laser beam 314 passes through the second region 352, which is located inside the projection region 351. As shown in Figure 6(a), if the laser beam 314 passes inside the second region 352, it is determined that the first distance is greater than the threshold (NO in S206). On the other hand, for example, as shown in Figure 6(b), if the laser beam 314 passes outside the second region 352, it is determined that the first distance is below the threshold (YES in S206). Whether the laser beam 314 passes inside the second region 352 is determined by the positional relationship between the second region 352 and the convergence point of the laser beam. Similar to Embodiment 1, the determination of whether the first distance is below the threshold is not limited to this method and can be modified in various ways.

[0055] If it is determined in step S206 that the first distance is below a threshold, the process proceeds to step S207, where the projected image 501 is projected while being fed based on the direction in which the shortest distance between the laser beam 314 and the outer edge of the projection area 351 is shown. This image feeding is achieved by changing the range of the image 502 corresponding to the input image that will be used as the projected image 501 to the direction in which the first distance is shown as the shortest distance, that is, the direction of the user's viewpoint.

[0056] If it is determined in step S206 that the first distance is not below the threshold, the process proceeds to step S209, and the projection is performed without sending the projection image 501.

[0057] Image scrolling will be explained using Figures 6 and 7.

[0058] Figures 7(a) and 7(b) show the projected image 501 projected onto the user's retina 308 in the state shown in Figures 6(a) and 6(b), respectively, with the position of the user's pupil 306. In this embodiment, the input image 502 is a panoramic image as shown in Figure 7, and only a trimmed portion of it becomes the projected image 501 projected by the laser light 314 and is projected onto the user. The gaze position P201 indicates the position where the user's gaze is focused, and in Figures 6(a) and 7(a), the user's gaze is focused on the center of the projected image 501 (looking straight ahead). The third region 402 enclosed by the dotted line corresponds to the projection region 351 calculated in step S205, similar to Embodiment 1. That is, if the gaze position P201 is outside the third region 402, the laser light 314 is vignetted by the iris, and the laser light 314 does not irradiate the user's retina 308. Similar to Example 1, this line of sight position P201 and the third region 402 may actually be superimposed on the projected image 501 and projected to the user.

[0059] Here, point P202, located near the edge of the peripheral region of the projected image 501, is outside the third region 402 that represents the projection region 351. Therefore, when the user attempts to align their gaze with point P202, the laser beam 314 will no longer pass through the projection region 351, and the user will no longer be able to see the projected image 501.

[0060] When the user shifts their gaze from point P201 to P202, there is a moment during the movement when the laser light passes through the projection area 351 but not through the second area 352 (a moment when it passes between the outer edge of the projection area 351 and the outer edge of the second area 352). Therefore, in this embodiment, in step S206, which is performed during the user's movement of shifting their gaze from point P201 to P202, if it is determined that the first distance is below a threshold, the output control unit 302 moves the position of the area to be projected as the projection image 501 to the right. As a result, the area shown in Figure 7(b) becomes the projection image 501. That is, the image obtained by scrolling the input image to the right from the state in Figure 7(a) becomes the projection image 501. The position on the retina where point P202 is projected also shifts accordingly, moving closer to the center of the projection area 315 and into the inside of the third area 402 which represents the projection area 351. Therefore, even if the user aligns their gaze position P201 with point P202, the laser beam 314 can still be projected onto the retina 308. In Figures 6(b) and 7(b), the gaze position P201 is shown overlapping with point P202.

[0061] Subsequently, in step S208, it is determined whether the area to be projected as image 501 has reached the edge of the input image 502 by image advancement. If it is determined that the edge has been reached, the process proceeds to step S209 and image advancement is stopped. If it is determined that the edge has not been reached, the process returns to step S205.

[0062] Figure 7(c) shows the region designated as the projected image 501 being moved further to the right from Figure 7(b), reaching the right edge of the input image 502. In this state, even if the first distance is determined to be below the threshold, the image is not moved to the right of the region designated as the projected image 501 (or to the left of the input image 502, relative to the projected image 501). However, in the up, down, and left directions, the region designated as the projected image 501 has not reached the edge of the input image 502, so depending on the determination, the image may be moved up, down, or left.

[0063] In this way, by creating a projected image in which the input image is scrolled according to the first distance, it becomes possible to reduce the likelihood of situations where the laser light 314 does not illuminate the user's retina 308, even when the user attempts to align their gaze outside the third region 402.

[0064] Furthermore, it is preferable that the feed amount (scroll amount) of the projected image 501 is determined by the first distance. Similar to Example 1, there is no particular method as long as a shorter first distance results in a larger feed amount. This makes it possible to control the image so that abrupt image feeds do not occur, thereby reducing the discomfort given to the user.

[0065] The process then proceeds to step S210, where the eyepiece detection unit 313 determines whether the user is not using an eyepiece on the image projection device 300. If it is determined in step S210 that the user is not using an eyepiece, the power is turned off and this flow is terminated.

[0066] If the system determines in step S210 that the user is using the eyepiece, it proceeds back to step S205 and repeats the above process until the user is not using the eyepiece or the power is turned off by the user.

[0067] In this embodiment, a portion of the large panoramic input image 502 is projected onto the retina 308 as a projected image 501. Even when the gaze is directed towards the edge of this projection, the position where the edge is projected can be moved toward the center of the projected area 315. Therefore, even when the gaze is directed towards point P202, located near the edge of the initial projected image 501 shown in Figure 7(a), it is less likely that the projected image 501 will become invisible due to the laser light 314 not passing through the pupil 306. Furthermore, even when a large image is projected, the amount of pupil 306 movement required to view its edge can be reduced. In addition, scrolling of the projected image 501 can be performed without cumbersome operations.

[0068] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist. [Explanation of symbols]

[0069] 300 Image Projection Device 301 Image Data 302 Output Control Unit 303 Light source 304 Scanning Unit 305 Projection section 306 Pupil 307 Eyeball 308 Retina 309 Pupil Illuminator 310 Light splitting means 311 Pupil detection unit 312 Eyepiece 313 Eyepiece detection unit 314 Luminous flux 351 Projection area Image displayed in 401 and 501.

Claims

1. a light source that outputs a light beam based on image data; a projection unit that projects a light beam from the light source onto a pupil of a user; A detection means for detecting information on the position of the pupil; a control unit that controls a projection image projected onto a retina by the light beam based on image data; Equipped with The control unit is controlling the light source so that the projected image is changed in response to the position of the pupil; An image projection device, characterized in that at least a portion of a peripheral area of ​​an image corresponding to the image data is projected onto the retina at a different position.

2. The control unit is Depending on the position of the pupil, The image projection device according to claim 1 , wherein an area of ​​the image data that corresponds to the projection image is changed.

3. The control unit is The projection image projected onto the retina when the user is looking to the right as viewed from the user, The image projection device according to claim 2, wherein the projected image is controlled so that the projected image projected onto the retina when the user is looking straight ahead corresponds to an image shifted to the left.

4. The control unit is the projected image corresponds to a portion of the image data; and, 4. The image projection device according to claim 1, wherein the projection image is controlled so that a position of a region of the image data corresponding to the projection image is changed according to a position of the pupil.

5. The control unit is 2. The image projection device according to claim 1, wherein the size of the projected image is changed in accordance with the position of the pupil.

6. The control unit is acquiring a first area through which a light beam passing through the pupil passes, based on the position of the pupil detected by the detection means; At least when the light beam passes through the projection area, 6. The image projection device according to claim 1, wherein the light source is controlled so that the projected image is changed based on a distance between the light beam and an outer edge of the projection area.

7. The control unit is when the distance between the light beam and the outer edge of the first region is a first value and when the distance between the light beam and the outer edge of the first region is a second value that is greater than the first value, 7. The image projection device according to claim 6, wherein the position where at least a part of the peripheral area of ​​the image corresponding to the image data is projected onto the retina is different.

8. The control unit is Depending on the distance between the light beam and the outer edge of the first region, The image projection device according to claim 7 , wherein an area of ​​the image data that corresponds to the projection image is changed.

9. The control unit is the projected image corresponds to a portion of the image data; and, 9. The image projection device according to claim 8, wherein the projection image is controlled so that an area of ​​the image data corresponding to the projection image is changed according to a distance between the light beam and an outer edge of the first area.

10. The control unit is 10. The image projection device according to claim 9, wherein when the distance between the light beam and the first region is equal to or smaller than a threshold, the projected image is controlled so as to scroll an image corresponding to the image data.

11. The control unit is A second area is set inside the first area; and image data for controlling the light source so that the projected image is changed depending on whether the light beam passes outside the second region or inside the second region.

11. The image projection device according to claim 6, wherein:

12. The control means The image projection device according to any one of claims 1 to 11, characterized in that the light source is controlled so that the projected image is an image in which at least one of information indicating the position of the user's gaze and information indicating the outer edge of the first area is superimposed on an image corresponding to the image data.

13. The image projection device according to claim 12, further comprising a selection means for selecting whether or not to superimpose and project at least one of information indicating the position of the user's gaze and information indicating the outer edge of the first area onto an image corresponding to the image data.

14. 1. A method for controlling an image projection device that projects an image based on image data by projecting a light beam based on the image data onto a pupil of a user, the method comprising: outputting a light beam based on image data from the light source; detecting a pupil position of the user; A light beam based on the image data is projected onto the retina in accordance with the detected pupil position. and controlling the light source such that a projected image is altered; A method for controlling an image projection device, wherein the position at which at least a part of a peripheral area of ​​an image corresponding to the image data is projected onto the retina is different in the controlling step.