Image display device, head-mounted display, and head-up display

Abstract

Provided is an image display device that displays a frame image (20) in a display region (40) through which the rear is visible, said image display device comprising: a light source that emits a beam modulated by a video signal; a scanning unit that scans a beam by raster scanning and that generates the frame image (20); a camera that captures an image of scenery of a region corresponding to the display region (40); and a control unit that controls the light source and the scanning unit. The control unit identifies, from the image captured by the camera, a high luminance region (R2) in which the illuminance is greater than or equal to a prescribed threshold value in the display region (40), and executes scanning in the vertical direction in raster scanning such that a second scanning range (SR2) in the vertical direction corresponding to the high luminance region (R2) has a higher luminance due to the approach of a beam further in the vertical direction in comparison to a first scanning range (SR1).

Description

Image display device, head-mounted display, and head-up display

[0001] The present invention relates to an image display device, a head-mounted display, and a head-up display that display an image by scanning light.

[0002] Conventionally, as an image display device that displays an image by scanning light, for example, a head-mounted display such as goggles or glasses that realize AR (Augmented Reality) has been known. In these devices, for example, light based on a video signal is irradiated onto a translucent display, and the reflected light is irradiated onto a user's eyes. Alternatively, light based on the video signal is irradiated directly onto a user's eyes.

[0003] Japanese Patent Application Laid-Open No. 2006-124494 (Patent Document 1) describes an apparatus that realizes a first line density in a first portion of an image and a second line density lower than the first line density in a second portion of the image by controlling the rotation of the fast axis and the slow axis of a MEMS mirror, and determines the position of the first portion of the image based on the line of sight of the eye. As a result, the resolution of the image in the second portion that does not correspond to the line of sight is lower than the resolution of the image in the first portion that corresponds to the line of sight, thereby reducing eye fatigue for the user.

[0004] U.S. Patent No. 9,986,215

[0005] In the image display device described above, the user can see the image superimposed on the scenery ahead in a see-through manner. However, when the scenery ahead is highly illuminant, it becomes difficult to see the image displayed by the image display device.

[0006] In view of the above problem, the present invention aims to provide an image display device, a head-mounted display, and a head-up display that enable a user to smoothly view an image displayed in a see-through manner even when the illumination of the scenery is high.

[0007] A first aspect of the present invention relates to an image display device that displays an image in a display area that allows a user to see through to the background. The image display device according to this aspect includes a light source that emits a beam modulated by a video signal, a scanning unit that generates one frame of an image by raster scanning the beam, a camera that captures an image of a scene in an area corresponding to the display area, and a control unit that controls the light source and the scanning unit. The control unit identifies, from the image captured by the camera, a high-brightness area in the display area where the illuminance is equal to or greater than a predetermined threshold, and performs vertical scanning in the raster scanning so that the beam is closer to the vertical scanning range corresponding to the high-brightness area and is therefore brighter than the other scanning ranges.

[0008] According to the image display device of this aspect, the image in the high-brightness area is made brighter, so that the image displayed in a see-through manner can be viewed smoothly even when the illumination of the scenery is high.

[0009] A second aspect of the present invention relates to an image display device that displays an image in a display area that allows a user to see through to the background. The image display device according to this aspect includes a light source that emits a beam modulated by a video signal, a scanning unit that generates one frame of an image by raster scanning the beam, a camera that captures a scene of an area corresponding to the display area, a viewpoint detection unit that detects a user's viewpoint in the display area, and a control unit that controls the light source and the scanning unit. The control unit identifies a high-brightness area in the display area from the image captured by the camera, where the illuminance is equal to or greater than a predetermined threshold, and divides the display area into a first area that includes the viewpoint and a second area that does not include the viewpoint, in the vertical direction of the raster scan. The control unit performs vertical scanning in the raster scan so that the beam is closer to the first area in the vertical direction and is brighter than the second area in the vertical direction.

[0010] According to the image display device of this aspect, when the image of the area viewed by the user overlaps with a highly illuminating scene, at least the scanning range of the image that overlaps with the highly illuminating scene is made brighter, thereby enabling the image displayed in a see-through manner to be viewed smoothly even when the scene is highly illuminating.

[0011] A third aspect of the present invention relates to an image display device. The image display device according to this aspect includes a scanning unit that displays an image by raster scanning a beam modulated by a video signal, and a control unit that controls the scanning unit. The control unit has a first mode in which the entire image is displayed at a uniform resolution, and a second mode in which the image is displayed at a higher brightness and a lower resolution than in the first mode in a predetermined scanning range perpendicular to the scanning lines, and in the second mode, the control unit reduces the pitch between scanning lines in the predetermined scanning range compared to the pitch between scanning lines in the first mode.

[0012] According to the image display device of this embodiment, when a display range occurs in the displayed image where it is desired to increase the brightness even at the expense of resolution, the second mode can be executed to effectively increase the brightness of the image in a specified scanning range corresponding to this display range.

[0013] A fourth aspect of the present invention relates to a head-mounted display, which includes the image display device according to the first or second aspect and a frame that holds the image display device.

[0014] The head-mounted display according to this aspect has the same effects as those of the first or second aspect. In addition, by wearing the head-mounted display on the head, the user can view the image displayed by the image display device together with the scenery ahead, without having to hold the image display device.

[0015] A fifth aspect of the present invention relates to a head-up display that projects light onto a display area of ​​a windshield to display a virtual image of an image in front of the windshield. The head-up display according to this aspect includes the image display device according to the first or second aspect, and an optical system that displays the virtual image of the image in front of the windshield based on the light emitted from the image display device.

[0016] The head-up display according to this aspect has the same effects as those of the first or second aspect. In addition, since the driver can see a virtual image of an image in front of the windshield, the driver can check the image displayed by the image display device while looking at the scenery in front of the windshield.

[0017] As described above, the present invention can provide an image display device, a head-mounted display, and a head-up display that allow a user to smoothly view an image displayed in a see-through manner even when the illuminance of the scenery is high.

[0018] The effects and significance of the present invention will become more apparent from the following description of the embodiments, however, the embodiments shown below are merely examples of how the present invention can be implemented, and the present invention is not limited to the embodiments described below.

[0019] FIG. 1 is a perspective view schematically illustrating a configuration of a head-mounted display according to the first embodiment. FIG. 2 is a diagram schematically illustrating a configuration of a projection unit according to the first embodiment. FIG. 3 is a block diagram illustrating the configuration of the projection unit according to the first embodiment. FIGS. 4(a) and 4(b) are diagrams schematically illustrating captured images captured by a camera according to the first embodiment. FIGS. 5(a) and 5(b) are diagrams schematically illustrating frame images displayed in a display area when the captured images of FIGS. 4(a) and 4(b) are acquired, respectively, according to the first embodiment. FIGS. 6(a) and 6(b) are graphs schematically illustrating the rotation speed of a second mirror when the frame images of FIGS. 5(a) and 5(b) are displayed, respectively, according to the first embodiment. FIGS. 7(a) and 7(b) are diagrams schematically illustrating scanning lines in a display area when the second mirror is rotated as shown in FIGS. 6(a) and 6(b), respectively, according to the first embodiment. FIG. 8 is a diagram schematically illustrating beam overlaps on each scanning line of FIG. 7(a) according to the first embodiment. FIG. 9 is a diagram schematically illustrating beam overlap on each scanning line in FIG. 7B according to the first embodiment. FIG. 10 is a flowchart illustrating a drive process of the image display device according to the first embodiment. FIG. 11 is a flowchart illustrating a drive process of the second mirror when a frame image corresponding to one frame is displayed according to the first embodiment. FIG. 12 is a flowchart illustrating a mirror drive process related to the second mirror according to the first embodiment. FIG. 13 is a perspective view schematically illustrating a configuration of a head-mounted display according to the second embodiment. FIG. 14 is a block diagram illustrating configurations of a projection unit and a viewpoint detection unit according to the second embodiment. FIGS. 15A and 15B are diagrams schematically illustrating captured images captured by a camera according to the second embodiment. FIGS. 16A and 16B are diagrams schematically illustrating frame images displayed in the display area when the captured images of FIGS. 15A and 15B are acquired, respectively, according to the second embodiment. FIGS. 17A and 17B are graphs schematically illustrating the rotation speed of the second mirror when the frame images of FIGS. 16A and 16B are displayed, respectively, according to the second embodiment. 18A and 18B are diagrams each showing a schematic diagram of scanning lines in a display area when the second mirror is rotated as shown in FIGS. 17A and 17B according to the second embodiment.FIG. 19 is a flowchart showing a process for driving the second mirror when a frame image corresponding to one frame is displayed according to the second embodiment. FIG. 20 is a flowchart showing a process for driving the second mirror when a frame image corresponding to one frame is displayed according to the second embodiment. FIG. 21(a) is a diagram schematically showing a frame image displayed in the display area when the captured image of FIG. 15(b) is acquired according to a modified example of the second embodiment. FIG. 21(b) is a graph schematically showing the rotation speed of the second mirror when the frame image of FIG. 21(a) is displayed according to a modified example of the second embodiment. FIG. 22 is a flowchart showing a process for driving the second mirror when a frame image corresponding to one frame is displayed according to a modified example of the second embodiment. FIG. 23(a) is a diagram schematically showing a captured image captured by the camera according to the third embodiment. FIG. 23(b) is a diagram schematically showing a frame image displayed in the display area when the captured image of FIG. 23(a) is acquired according to the third embodiment. FIG. 24 is a flowchart showing a process for driving the second mirror when a frame image corresponding to one frame is displayed according to the third embodiment. FIG. 25( a) is a diagram schematically illustrating a captured image captured by a camera according to a modified example of the third embodiment. FIG. 25( b) is a diagram schematically illustrating a frame image displayed in a display area when the captured image of FIG. 25( a) is acquired according to a modified example of the third embodiment. FIG. 26 is a flowchart illustrating a drive process of a second mirror when a frame image corresponding to one frame is displayed according to a modified example of the third embodiment. FIG. 27 is a block diagram illustrating a configuration of a signal processing unit according to the fourth embodiment. FIGS. 28( a) and 28( b) are diagrams conceptually illustrating a first video signal in a first buffer and a second video signal in a second buffer according to the fourth embodiment. FIGS. 29( a) and 29( b) are diagrams conceptually illustrating a first video signal in a first buffer and a second video signal in a second buffer according to the fifth embodiment. FIGS. 30( a) and 30( b) are diagrams conceptually illustrating scan lines in a display area when the first video signal and the second video signal are used according to the fifth embodiment. 31(a) to 31(c) are diagrams schematically showing usage patterns of a head-up display according to a sixth embodiment.

[0020] However, the drawings are for illustrative purposes only and do not limit the scope of the present invention.

[0021] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments 1 to 5, examples in which the present invention is applied to an image display device for a head-mounted display are shown. Examples of head-mounted displays include AR glasses and AR goggles. The head-mounted displays in the following embodiments 1 to 5 are AR glasses. Furthermore, in the following embodiment 6, an example in which the present invention is applied to an image display device for an in-vehicle head-up display is shown. However, the following embodiments are one embodiment of the present invention, and the present invention is not limited to the following embodiments in any way.

[0022] First Embodiment FIG. 1 is a perspective view schematically illustrating the configuration of a head-mounted display 1. As shown in FIG.

[0023] 1 also shows mutually orthogonal X, Y, and Z axes in addition to the front-rear, left-right, up-down directions of the head-mounted display 1. The positive X-axis direction, positive Y-axis direction, and positive Z-axis direction correspond to the rightward, backward, and upward directions of the head-mounted display 1, respectively.

[0024] The head-mounted display 1 is a device capable of displaying an image superimposed on a landscape. The head-mounted display 1 is worn on the user's head in the same manner as ordinary eyeglasses. The user can see what is in front of the front portion 2 a of the frame 2 (the area behind the front portion 2 a as seen from the user's eyes) via the mirror 4 and the front portion 2 a.

[0025] The head-mounted display 1 includes a frame 2 , a pair of image display devices 3 , a pair of mirrors 4 , a communication unit 5 , and a direction sensor 6 .

[0026] The frame 2 holds a pair of image display devices 3 and a pair of mirrors 4. The frame 2 is composed of a front portion 2a and a pair of support portions 2b. The pair of support portions 2b extend rearward from the right and left ends of the front portion 2a. When the frame 2 is worn by a user, the front portion 2a is positioned in front of a pair of the user's eyes E. The front portion 2a of the frame 2 is made of a transparent material.

[0027] The pair of image display devices 3 are arranged symmetrically with respect to the YZ plane passing through the center of the head-mounted display 1. The image display devices 3 generate images for the eyes E of a user wearing the head-mounted display 1 on their head.

[0028] The mirror 4 is a half mirror with a concave reflective surface, and is installed on the inner surface of the front surface 2a of the frame 2. The mirror 4 reflects light projected from the corresponding projection unit 11 and guides it to the user's eye E, while transmitting light from the front of the frame 2 (behind the front surface 2a as seen from the eye E). In other words, the mirror 4 constitutes an optical system for guiding light from the image display device 3 and light from the front of the frame 2 to the user's eye E. Note that if the light from the image display device 3 is directly irradiated onto the user's eye E, the mirror 4 may be omitted.

[0029] The communication unit 5 is installed on the inner surface of one of the support parts 2b and is connected to the pair of image display devices 3. The communication unit 5 receives time, temperature, navigation information, etc. from an external information terminal device. The orientation sensor 6 is installed on the inner surface of one of the support parts 2b and is connected to the pair of image display devices 3. The orientation sensor 6 detects the orientation of the head mounted display 1.

[0030] The image display device 3 includes a pair of projection units 11 and a pair of cameras 12 .

[0031] The projection unit 11 is installed on the inner surface of the support unit 2b. The projection unit 11 projects light modulated by a video signal onto the corresponding mirror 4. The light from the projection unit 11 reflected by the mirror 4 is irradiated onto the fovea centralis, which is located at the center of the retina in the eye E. This allows the user to visually grasp the frame image 20 (see FIG. 2) generated by the image display device 3.

[0032] The pair of cameras 12 are installed on the outer surface of the front part 2a in front of the pair of mirrors 4. The cameras 12 capture an image within the field of view of the cameras 12. In this embodiment, the field of view of the cameras 12 is in front of the head-mounted display 1.

[0033] FIG. 2 is a diagram schematically illustrating the configuration of the projection unit 11. As shown in FIG.

[0034] The projection unit 11 includes light-emitting elements 101, 102, and 103, collimator lenses 111, 112, and 113, apertures 121, 122, and 123, a mirror 131, dichroic mirrors 132 and 133, a first scanning unit 140, a relay optical system 150, and a second scanning unit 160. The light-emitting elements 101, 102, and 103, the collimator lenses 111, 112, and 113, the apertures 121, 122, and 123, the mirror 131, and the dichroic mirrors 132 and 133 configure a light source 170.

[0035] Light-emitting elements 101, 102, and 103 are, for example, semiconductor laser elements. Light-emitting element 101 emits laser light with a red wavelength in the range of 635 nm to 645 nm, light-emitting element 102 emits laser light with a green wavelength in the range of 510 nm to 530 nm, and light-emitting element 103 emits laser light with a blue wavelength in the range of 440 nm to 460 nm.

[0036] In this embodiment, a color image is generated as the frame image 20 described below, and therefore the projection unit 11 is equipped with light-emitting elements 101, 102, and 103 capable of emitting red, green, and blue laser light. When a monochromatic image is displayed as the frame image 20, the projection unit 11 may be equipped with only one light-emitting element corresponding to the color of the image. Alternatively, the projection unit 11 may be configured to include two light-emitting elements with different emission wavelengths. These light-emitting elements may be LEDs.

[0037] The light emitted from the light emitting elements 101, 102, and 103 is converted into parallel light by collimator lenses 111, 112, and 113. The light transmitted through the collimator lenses 111, 112, and 113 is shaped into a substantially circular beam by apertures 121, 122, and 123, respectively.

[0038] Mirror 131 almost totally reflects the red light that has passed through aperture 121. Dichroic mirror 132 reflects the green light that has passed through aperture 122 and transmits the red light reflected by mirror 131. Dichroic mirror 133 reflects the blue light that has passed through aperture 123 and transmits the red light and green light that have passed through dichroic mirror 132. Mirror 131 and the two dichroic mirrors 132 and 133 are arranged to align the optical axes of the light of each color emitted from light-emitting elements 101, 102, and 103.

[0039] The first scanning unit 140 reflects the light that has passed through the dichroic mirror 133. The first scanning unit 140 is, for example, a MEMS (Micro Electro Mechanical System) mirror. The first scanning unit 140 is configured to rotate a first mirror 141, onto which the light that has passed through the dichroic mirror 133 is incident, around an axis 141a parallel to the Z-axis direction in response to a drive signal. The rotation of the first mirror 141 changes the reflection direction of the light. As a result, the light reflected by the first mirror 141 scans the retina of the eye E along a scanning line that extends in the X-axis direction (horizontal direction), as described below.

[0040] The relay optical system 150 directs light reflected by the first scanning unit 140 toward the center of the second mirror 161 of the second scanning unit 160. That is, light incident on the first scanning unit 140 is deflected by the first mirror 141 at a predetermined deflection angle. The relay optical system 150 directs light at each deflection angle toward the center of the second mirror 161. The relay optical system 150 also has multiple mirrors, and reflects the light reflected by the first scanning unit 140 by the multiple mirrors to direct the light toward the second scanning unit 160. This makes it possible to achieve a long optical path length inside the relay optical system 150 and reduce the deflection angle of the light when viewed from the second mirror 161.

[0041] The second scanning unit 160 reflects the light that has passed through the relay optical system 150. The second scanning unit 160 is, for example, a MEMS mirror. The second scanning unit 160 has a configuration that rotates a second mirror 161, onto which the light that has passed through the relay optical system 150 is incident, around an axis 161a parallel to the X-Y plane in response to a drive signal. The rotation of the second mirror 161 changes the reflection direction of the light. As a result, the scanning line on the retina of the eye E, along which the light is scanned by the first scanning unit 140, is changed to the Z-axis direction (vertical direction) as described below.

[0042] The light reflected by the second scanning unit 160, that is, the light emitted from the projection unit 11, is reflected by the mirror 4 and forms a frame image 20 on the retina of the eye E.

[0043] FIG. 3 is a block diagram showing the configuration of the projection unit 11.

[0044] The projection unit 11 includes a control unit 201, a first mirror driving circuit 211, a second mirror driving circuit 212, a first mirror monitor sensor 213, a second mirror monitor sensor 214, a signal processing unit 300, a line memory 221, and a laser driving circuit 222.

[0045] The communication unit 5 is capable of communicating with an external information terminal device (for example, a smartphone) and is configured by a communication unit (for example, a communication unit based on the Bluetooth (registered trademark) standard) that can communicate with the information terminal device. The control unit 201 acquires the time, temperature, navigation information, etc. from the external information terminal device via the communication unit 5. The direction sensor 6 is configured to be able to detect the orientation of the head mounted display 1 and outputs the detected orientation of the head mounted display 1 to the control unit 201.

[0046] In response to instructions from the control unit 201, the camera 12 captures an image of the scenery in an area corresponding to the display area in which the frame image 20 is displayed, generates a video signal, and outputs the generated video signal to the control unit 201. In Fig. 1 , the left and right cameras 12 each output the generated video signal to the control unit 201. The control unit 201 detects the distribution of illuminance in front of the head mounted display 1 based on one frame of captured image included in the video signal from the camera 12.

[0047] The camera 12 may have the function of an illuminance detection unit as a function of a built-in processing circuit, and the illuminance detection unit may detect the distribution of illuminance in front of the head mounted display 1 using a video signal captured by the camera 12. In this case, the camera 12 sequentially outputs the illuminance detected by the illuminance detection unit to the control unit 201. The illuminance in front of the head mounted display 1 may be detected based on a plurality of frames of captured images included in the video signal, or may be detected based on one frame of captured images at intervals of a plurality of frames.

[0048] The control unit 201 includes an arithmetic processing unit such as a CPU or FPGA, and a memory. The control unit 201 controls the camera 12 and each unit of the projection unit 11. The control unit 201 transmits the orientation of the head mounted display 1 detected by the orientation sensor 6 to an external information terminal device via the communication unit 5, and receives information for displaying an image from the external information terminal device. The control unit 201 generates a video signal corresponding to one frame based on the information received via the communication unit 5, and outputs the generated video signal to the signal processing unit 300.

[0049] The first mirror drive circuit 211 drives the first mirror 141 of the first scanning unit 140 in response to a drive signal from the control unit 201. The second mirror drive circuit 212 drives the second mirror 161 of the second scanning unit 160 in response to a drive signal from the control unit 201.

[0050] The first mirror monitor sensor 213 is installed on the first mirror 141 and outputs a detection signal corresponding to the rotation of the first mirror 141 to the control unit 201. The second mirror monitor sensor 214 is installed on the second mirror 161 and outputs a detection signal corresponding to the rotation of the second mirror 161 to the control unit 201. Based on the detection signals from the first mirror monitor sensor 213 and the second mirror monitor sensor 214, the control unit 201 outputs drive signals to the first mirror drive circuit 211 and the second mirror drive circuit 212 so that the first mirror 141 and the second mirror 161 rotate with desired drive waveforms.

[0051] The signal processing unit 300 processes one frame of video signals from the control unit 201 and sequentially outputs one line of video signals to the line memory 221 .

[0052] The line memory 221 outputs the video signal for one line output from the signal processing unit 300 to the laser driving circuit 222. The laser driving circuit 222 drives the light emitting elements 101, 102, and 103 to emit light modulated by the video signal for one line output from the line memory 221. The first scanning unit 140 and the second scanning unit 160 generate a frame image 20 for one frame by raster scanning the beam emitted from the light source 170.

[0053] That is, a scanning line based on one line of video signal is formed by rotating the first mirror 141 from one side to the other in the X-axis direction (horizontal direction), and multiple scanning lines are formed in the Z-axis direction by rotating the second mirror 161 from one side to the other in the Z-axis direction (vertical direction), thereby generating the frame image 20.

[0054] Incidentally, both light from the scenery in front of the head-mounted display 1 and light from the image display device 3 are incident on the user's eyes E. This allows the user to see the frame image 20 superimposed on the scenery in a see-through manner. However, if the scenery includes a bright light source such as a light or the sun, the illuminance of the scenery increases. In this case, it becomes difficult for the user to see the frame image 20 displayed by the image display device 3.

[0055] Therefore, in this embodiment, the image display device 3 is controlled so that the frame image 20 displayed in a see-through manner can be easily viewed even when the illuminance of the scenery is high. This control will be described below.

[0056] 4A and 4B are diagrams schematically showing a captured image 30 captured by the camera 12. FIG.

[0057] 4A and 4B, for convenience, the X, Y, and Z axes when the head-mounted display 1 is placed as in FIG. 1 are added as the coordinates of the captured image 30.

[0058] The captured image 30 in FIG. 4( a) shows a state in which no high-intensity light source is captured in the landscape. In this case, there are no high-brightness areas in the entire captured image 30. On the other hand, the captured image 30 in FIG. 4( b) shows a high-intensity light source in part of the landscape, and the area in the Z-axis direction corresponding to the position where the high-intensity light source is captured is a high-brightness area R1. For convenience, the high-brightness area R1 is shown as a dotted line in FIG. 4( b). In this embodiment, when the captured image 30 includes the high-brightness area R1, the area of ​​the frame image 20 corresponding to the high-brightness area R1 is set to high brightness as shown below.

[0059] 5A and 5B are diagrams each showing a frame image 20 displayed in the display area 40 when the captured images 30 shown in FIGS. 4A and 4B are acquired.

[0060] 5(a) and 5(b), for convenience, the X, Y, and Z axes when the head-mounted display 1 is positioned as in Fig. 1 are indicated as coordinates of the display area 40 when viewed in the positive direction of the Y axis. The frame image 20 includes, for example, an image 21 showing the time, an image 22 showing the temperature, and an image 23 showing navigation information.

[0061] When the captured image 30 shown in Fig. 4(a) is acquired, a standard scanning range SR0 in which the beam is scanned in the standard scanning mode is set over the entire region of the frame image 20, as shown in Fig. 5(a). By scanning the standard scanning range SR0 with the beam in the standard scanning mode, the images included in the standard scanning range SR0 (images 21 to 23 in Fig. 5(a)) are displayed at standard brightness.

[0062] On the other hand, when the captured image 30 shown in Fig. 4(b) is acquired, as shown in Fig. 5(b), a high-luminance region R2 corresponding to the high-luminance region R1 is set in the frame image 20. A second scanning range SR2, which is scanned with the beam in the second scanning mode, is set for the high-luminance region R2, and a first scanning range SR1, which is scanned with the beam in the first scanning mode, is set for the region other than the high-luminance region R2.

[0063] By scanning the first scanning range SR1 with the beam in the first scanning mode, the images included in the first scanning range SR1 (images 21 and 22 in FIG. 5(b)) are displayed at standard brightness. By scanning the second scanning range SR2 with the beam in the second scanning mode, the image included in the second scanning range SR2 (image 23 in FIG. 5(b)) is displayed at high brightness. For convenience, in FIG. 5(b), the area of ​​frame image 20 that is displayed at high brightness in the second scanning mode is shown by dots.

[0064] Next, beam scanning in the standard scanning mode, the first scanning mode, and the second scanning mode will be described with reference to FIGS.

[0065] 6A and 6B are graphs that schematically show the rotation speed of the second mirror 161 when the frame images 20 of FIGS. 5A and 5B are displayed, respectively.

[0066] 6A and 6B, timings Ts and Te respectively indicate the timing at which the second mirror 161 starts to rotate and the timing at which the second mirror 161 stops to rotate in order to display the frame image 20. In this embodiment, the time interval from timing Ts to timing Te is set to a constant value corresponding to the frame rate (for example, 1 / 60 seconds) for displaying the frame image 20.

[0067] As shown in Figure 5(a), when the standard scanning range SR0 is set to cover the entire frame image 20, the second mirror 161 is rotated at a constant speed in the standard scanning range SR0 in the standard scanning mode, as shown in Figure 6(a).

[0068] Specifically, from timing Ts to timing Te, second mirror 161 is rotated at a constant speed in the Z-axis direction. Then, while second mirror 161 is rotated from one side to the other, first mirror 141 is repeatedly rotated along the X-axis direction at the same period. As a result, multiple scanning lines along the X-axis direction are formed in the Z-axis direction, and frame image 20 is displayed.

[0069] As shown in Figure 5(b), when a first scanning range SR1 and a second scanning range SR2 are set in the frame image 20, as shown in Figure 6(b), in the first scanning range SR1, the second mirror 161 is rotated at a constant speed in the first scanning mode, and in the second scanning range SR2, the second mirror 161 is rotated at a constant speed in the second scanning mode.

[0070] Specifically, in the second scanning range SR2, the second mirror 161 is rotated at a slower speed than in the case of FIG. 6A. The speed (slope of the graph) of the second mirror 161 in the second scanning range SR2 is smaller than the speed (slope of the graph) in FIG. 6A. The speed of the second mirror 161 in the second scanning range SR2 is a predetermined speed. This determines the length of the period T1 corresponding to the second scanning range SR2.

[0071] Furthermore, the speed (gradient of the graph) of the second mirror 161 in the first scanning range SR1 is determined based on the length of the second scanning range SR2 and the period T2. When multiple first scanning ranges SR1 exist as shown in FIG. 6B, the speeds (gradient of the graph) in the multiple first scanning ranges SR1 are set equal to each other. This also determines the position of the period T2. In this way, the speed (gradient of the graph) of the second mirror 161 in the first scanning range SR1 is greater than the speed (gradient of the graph) in FIG. 6A and the speed (gradient of the graph) in the second scanning range SR2.

[0072] 7A and 7B are diagrams each showing a schematic view of the scanning line L in the display area 40 when the second mirror 161 is rotated as shown in FIGS. 6A and 6B.

[0073] The scanning line L extends from one side to the other along the X-axis direction. The scanning line L corresponds to the trajectory of the beam irradiated onto the display area 40 to form the frame image 20. However, because the second mirror 161 rotates at a constant speed in the Z-axis direction, the scanning line L is actually slightly tilted with respect to the X-axis direction. However, for convenience, the scanning line L is illustrated in Figures 6(a) and 6(b) as being parallel to the X-axis direction.

[0074] When the beam scans from one side to the other along the scanning line L, the first mirror 141 is returned to one side along the X-axis direction. When the beam finishes scanning the last scanning line L, the first mirror 141 and the second mirror 161 are returned to their initial positions. In Figures 7(a) and 7(b), the scanning trajectories of the first mirror 141 and the second mirror 161 when they are returned are shown by dotted lines for convenience. In other words, when the scanning positions of the first mirror 141 and the second mirror 161 pass through the dotted lines, the light-emitting elements 101 to 103 are not lit, and the beam is not irradiated onto the display area 40.

[0075] When the second mirror 161 rotates in the Z-axis direction in the standard scanning mode as shown in FIG. 6(a), scanning lines L along the X-axis direction based on the first mirror 141 are formed in the standard scanning range SR0 at a standard pitch P0 in the Z-axis direction as shown in FIG. 7(a).

[0076] When the second mirror 161 rotates in the Z-axis direction in the first scanning mode and the second scanning mode as shown in Fig. 6B, the pitch of the scanning lines L changes depending on the rotation speed of the second mirror 161 as shown in Fig. 7B. That is, the scanning lines L in the first scanning range SR1 are formed side by side in the Z-axis direction at a first pitch P1, and the scanning lines L in the second scanning range SR2 are formed side by side in the Z-axis direction at a second pitch P2. The magnitude relationship among the three pitches P0, P1, and P2 is P2<P0<P1.

[0077] In this embodiment, the rotation speed of the first mirror 141 is constant, and the period from timing Ts to timing Te is constant as shown in Figures 6(a) and (b), so the number of scanning lines L formed in the display area 40 is constant.

[0078] Fig. 8 is a diagram schematically illustrating the overlap of beams on each scanning line L in Fig. 7(a) The left side of Fig. 8 shows the scanning lines L in the display area 40, and the right side of Fig. 8 shows the beam intensity and position in the Z-axis direction of each scanning line L.

[0079] In Fig. 8, two adjacent scanning lines L are arranged at a standard pitch P0. The beam of each scanning line L has a Gaussian distribution, with high intensity at the center and decreasing intensity toward the periphery. In this embodiment, the standard pitch P0 and the intensity and shape of the beams are set so that the resolution of the frame image 20 is maximized by arranging the scanning lines L at the standard pitch P0. For example, the beam intensity ST2 at the position where two adjacent beams overlap in the standard scanning range SR0 is 1 / e of the peak intensity ST1. ２ The standard pitch P0 is set to be larger than the peak intensity ST1 and smaller than half the peak intensity ST1.

[0080] FIG. 9 is a diagram schematically showing the overlap of beams on each scanning line L in FIG. 7(b).

[0081] 9, in the first scanning range SR1, two adjacent scanning lines L are arranged at a first pitch P1, and in the second scanning range SR2, two adjacent scanning lines L are arranged at a second pitch P2. In this embodiment, the standard pitch P0 and the beam intensity and shape are set as described with reference to FIG. 8, and the first pitch P1 is larger than the standard pitch P0. Therefore, the resolution of the frame image 20 in the first scanning range SR1 is slightly lower than the resolution of the frame image 20 in the case of FIG. 8. For example, if the beam intensity ST3 at the position where two adjacent beams overlap in the first scanning range SR1 is 1 / e of the peak intensity ST1, ２ The first pitch P1 is set so that:

[0082] Furthermore, in this embodiment, the second pitch P2 is smaller than the standard pitch P0. Therefore, the brightness of the frame image 20 in the second scanning range SR2 is higher than the brightness of the frame image 20 in the standard scanning range SR0 (see FIG. 8). For example, the second pitch P2 is set so that the beam intensity ST4 at the position where two adjacent beams overlap in the second scanning range SR2 is half the peak intensity ST1.

[0083] As mentioned above, the standard pitch P0 is set so that the resolution of the frame image 20 is the highest, and therefore the resolution of the frame image 20 formed by the second pitch P2 will be lower than the resolution of the frame image 20 formed by the standard pitch P0.

[0084] In this way, in the second scanning range SR2, the scanning lines L are arranged at the second pitch P2, which is smaller than the standard pitch P0, so that although the resolution is lower than that of the frame image 20 formed at the standard pitch P0, the brightness of the frame image 20 is increased. This makes it easier for the user to view the frame image 20 even in the high-brightness region R2 (see FIG. 5B ) that includes a high-illumination landscape.

[0085] As long as the pitch relationship in each scanning range is maintained as described above and the frame image 20 is displayed with high brightness in the second scanning range SR2, the setting of the degree of overlap of the beams in each scanning range is not limited to the setting described with reference to Figures 8 and 9.

[0086] Next, the processing performed by the image display device 3 will be described.

[0087] FIG. 10 is a flowchart showing the driving process of the image display device 3.

[0088] When the control unit 201 of the image display device 3 receives an instruction to start image display (S11: YES), it controls the laser drive circuit 222 to start emitting laser light at a predetermined power (S12), and then controls the first mirror drive circuit 211 and the second mirror drive circuit 212 to start driving the first mirror 141 and the second mirror 161 (S13).

[0089] As a result, the first mirror 141 is repeatedly rotated a predetermined number of times from one side to the other in the X-axis direction during the period in which the frame image 20 is displayed (timings Ts to Te in FIGS. 6(a) and 6(b)). Furthermore, the second mirror 161 is rotated from one side to the other in the Z-axis direction during the period in which the frame image 20 is displayed. The driving process for the second mirror 161 at this time will be described later with reference to FIGS. 11 and 12. Then, for each scanning line L, the laser driving circuit 222 is driven to turn on or off as appropriate, and the frame image 20 is displayed in the display area 40.

[0090] When the control unit 201 receives the instruction to end image display (S14: YES), it controls the laser drive circuit 222 to end the laser light emission process (S15), and then controls the first mirror drive circuit 211 and the second mirror drive circuit 212 to end the drive process of the first mirror 141 and the second mirror 161 (S16).

[0091] FIG. 11 is a flowchart showing the driving process of the second mirror 161 when a frame image 20 corresponding to one frame is displayed.

[0092] The control unit 201 controls the camera 12 to acquire the captured image 30, identifies a high-brightness region R1 (see Figure 4(b)) where the illuminance is above a predetermined threshold based on the captured image 30, and identifies a high-brightness region R2 (see Figure 5(b)) in the display area 40 where the illuminance is above a predetermined threshold based on the high-brightness region R1 (S101).

[0093] If a high-brightness region R2 is present (S102: YES), the control unit 201 sets the scanning range in the vertical direction (Z-axis direction) corresponding to the high-brightness region R2 as the second scanning range SR2, and sets the scanning range in the display area 40 other than the second scanning range SR2 as the first scanning range SR1 (S103). On the other hand, if a high-brightness region R2 is not present (S102: NO), the control unit 201 sets the entire scanning range in the vertical direction in the display area 40 as the standard scanning range SR0 (S104). Then, the control unit 201 executes mirror drive processing for the second mirror 161 (S105).

[0094] FIG. 12 is a flowchart showing the mirror driving process for the second mirror 161.

[0095] If the rotation position of the second mirror 161 is within the standard scanning range SR0 (S201: YES), the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 in the standard scanning mode (S202). If the rotation position of the second mirror 161 is within the first scanning range SR1 (S201: NO, S203: YES), the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 in the first scanning mode (S204). If the rotation position of the second mirror 161 is within the second scanning range SR2 (S201: NO, S203: NO), the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 in the second scanning mode (S205).

[0096] In parallel with the processing of steps S201 to S205, the control unit 201 controls the first mirror drive circuit 211 so that the first mirror 141 repeatedly rotates at the same cycle, and drives the laser drive circuit 222 based on the video signal for one line input to the line memory 221. As a result, one scanning line L is formed in the display area 40.

[0097] Until the display of the frame image 20 corresponding to one frame is completed (S206: NO), the control unit 201 returns the process to S201 and repeatedly executes the processes of S201 to S205, thereby forming a plurality of scanning lines L and displaying the frame image 20 corresponding to one frame.

[0098] On the other hand, when the display of the frame image 20 corresponding to one frame has ended (S206: YES), the control unit 201 ends the mirror driving process for the second mirror 161 in Fig. 12. Thereafter, the control unit 201 repeatedly executes the process in Fig. 11 until it is determined in step S14 in Fig. 10 that the image display has ended.

[0099] <Effects of First Embodiment> According to the first embodiment, the following effects are achieved.

[0100] The image display device 3 is a device that displays a frame image 20 (image) in a display area 40 that allows a viewer to see behind it. The image display device 3 includes a light source 170 that emits a beam modulated by a video signal, a first scanning unit 140 and a second scanning unit 160 (scanning units) that raster scan the beam to generate the frame image 20 (one frame's worth of image), a camera 12 that captures an image of the scenery in the area corresponding to the display area 40, and a control unit 201 that controls the light source 170 and the first scanning unit 140 and the second scanning unit 160 (scanning units). The control unit 201 identifies a high-brightness area R2 in the display area 40 from the captured image 30 of the camera 12, where the illuminance is equal to or greater than a predetermined threshold, and performs vertical raster scanning in a second scanning range SR2 (scanning range) in the vertical direction corresponding to the high-brightness area R2 so that the beam is closer in the vertical direction to the second scanning range SR1 (other scanning ranges) and has a higher luminance.

[0101] According to this configuration, the frame image 20 in the high-brightness region R2 is made brighter, so that the frame image 20 displayed in a see-through manner can be viewed smoothly even when the illumination of the scenery is high.

[0102] In order to make the frame image 20 superimposed on a high-illuminance background easier to see, it is also possible to increase the brightness of the frame image 20 by emitting high-output light from the light-emitting elements 101 to 103. In contrast, according to the first embodiment, high brightness is achieved by bringing the beams closer together in the vertical direction in the second scanning range SR2, so it is not necessarily necessary to increase the output of the light-emitting elements 101 to 103. Therefore, according to the first embodiment, the power consumption of the image display device 3 can be reduced.

[0103] As shown in FIG. 6B, the control unit 201 reduces the vertical scanning speed of the second scanning range SR2 (scanning range) corresponding to the high-brightness region R2 compared to the first scanning range SR1 (other scanning ranges).

[0104] According to this configuration, by adjusting the scanning speed in the vertical direction, it is possible to switch the pitch of the scanning lines L of the beam between the first scanning range SR1 and the second scanning range SR2.

[0105] The image display device 3 includes a first scanning unit 140 and a second scanning unit 160 (scanning units) that display a frame image 20 (image) by raster scanning a beam modulated by a video signal, and a control unit 201 that controls the first scanning unit 140 and the second scanning unit 160 (scanning units). The control unit 201 includes a standard scanning mode (first mode) that displays the entire frame image 20 (image) at a uniform resolution, and a second scanning mode (second mode) that displays the frame image 20 (image) in a second scanning range SR2 (predetermined scanning range) perpendicular to the scanning line L with higher brightness and lower resolution than in the standard scanning mode (first mode), and in the second scanning mode (second mode), a second pitch P2 (pitch) between the scanning lines L in the second scanning range SR2 (predetermined scanning range) is reduced from the standard pitch P0 (pitch) between the scanning lines L in the standard scanning mode (first mode).

[0106] According to this configuration, when a display range occurs in the frame image 20 where it is desired to increase the brightness even at the expense of resolution, the brightness of the frame image 20 in the second scanning range SR2 corresponding to this display range can be effectively increased by executing the second scanning mode.

[0107] As shown in FIG. 1 , the head-mounted display 1 includes an image display device 3 and a frame 2 that holds the image display device 3 .

[0108] According to this configuration, by wearing the head-mounted display 1 on the head, the user can view the frame image 20 displayed by the image display device 3 together with the scenery ahead, without having to hold the image display device 3.

[0109] In a second embodiment, the head mounted display 1 of the first embodiment is further provided with a viewpoint detection unit 13 that detects the viewpoint of a user in the display area 40. Then, the scanning mode is changed based on the viewpoint of the user in the display area 40 and the high-brightness area R2.

[0110] FIG. 13 is a perspective view schematically illustrating the configuration of a head-mounted display 1 according to the second embodiment.

[0111] 1, the image display device 3 of the second embodiment includes a pair of viewpoint detection units 13. The pair of viewpoint detection units 13 are installed on the inner surface of the front surface 2a between the pair of mirrors 4. The viewpoint detection units 13 are used to detect the viewpoint of the user.

[0112] FIG. 14 is a block diagram showing the configuration of the projection unit 11 and the viewpoint detection unit 13 according to the second embodiment.

[0113] The viewpoint detection unit 13 includes a light source 13a and an image sensor 13b, and is connected to the control unit 201 of the projection unit 11. The light source 13a is, for example, an LED that emits light of an infrared wavelength. The image sensor 13b is, for example, a CMOS image sensor or a CCD image sensor. The light source 13a irradiates light onto the user's eye E in response to an instruction from the control unit 201. The image sensor 13b captures an image of the user's eye E in response to an instruction from the control unit 201, and outputs the captured image to the control unit 201.

[0114] The control unit 201 detects the user's viewpoint by, for example, the dark pupil method, the bright pupil method, the corneal reflex method, or the like, based on the captured image from the viewpoint detection unit 13. Based on the detected user's viewpoint, the control unit 201 acquires the viewpoint position in the frame image 20 formed on the user's retina.

[0115] 15A and 15B are diagrams schematically showing a captured image 30 captured by the camera 12. FIG.

[0116] In the captured image 30 of Figures 15(a) and 15(b), as in the first embodiment, the region in the Z-axis direction corresponding to the position where the high-illuminance light source is captured is the high-brightness region R1. In the second embodiment, a predetermined region in the Z-axis direction corresponding to the viewpoint is set as a first region R31, and the region other than the first region R31 is set as a second region R32. For convenience, in Figures 15(a) and 15(b), the user's viewpoint obtained from the captured image acquired by the viewpoint detection unit 13 is displayed superimposed on the captured image 30. In Figure 15(a), the high-brightness region R1 and the first region R31 including the viewpoint do not overlap, while in Figure 15(b), the high-brightness region R1 and the first region R31 including the viewpoint overlap.

[0117] 16A and 16B are diagrams each showing a frame image 20 displayed in the display area 40 when the captured images 30 of FIGS. 15A and 15B are acquired.

[0118] 16(a), when the captured image 30 shown in Fig. 15(a) is acquired, a first region R41 corresponding to the first region R31 and a second region R42 corresponding to the second region R32 are set in the display area 40. In this case, because the high-brightness region R2 and the first region R41 do not overlap, the frame image 20 is not brightened, and the second region R42 that does not include the viewpoint is displayed at a lower resolution than the first region R41 that includes the viewpoint. In this case, a third scanning range SR3 in which the beam is scanned in the third scanning mode is set for the first region R41, and a fourth scanning range SR4 in which the beam is scanned in the fourth scanning mode is set for the second region R42.

[0119] By scanning the beam in the third scanning range SR3 in the third scanning mode and the beam in the fourth scanning range SR4 in the fourth scanning mode, the image included in the fourth scanning range SR4 (images 21 to 23 in FIG. 6(a)) is displayed at a lower resolution than the image included in the third scanning range SR3 (no image in FIG. 16(a)). In FIG. 16(a), for convenience, the area of ​​frame image 20 that is displayed at a higher resolution in the third scanning mode is shown with light halftone dots.

[0120] In this way, when the first region R41 including the viewpoint and the high-brightness region R2 including the high-illuminance scenery do not overlap, the image in the first region R41 is visible without any problems, and therefore the brightness of the frame image 20 is not increased. On the other hand, if the entire frame image 20 has a high resolution, the user's eyes E become easily fatigued. In contrast, in the second region R42 outside the first region R41 including the viewpoint, the resolution of the frame image 20 is set lower than that of the first region R41. This reduces the user's eye fatigue.

[0121] Similarly, when the captured image 30 shown in FIG. 15B is acquired, a high-brightness region R2, a first region R41, and a second region R42 are set in the frame image 20, as shown in FIG. 16B. However, in the case of FIG. 16B, because the first region R41 and the high-brightness region R2 overlap, the entire frame image 20 within the first region R41 is made brighter. In this case, as in the first embodiment, a second scanning range SR2 is set for the first region R41, and the region other than the second scanning range SR2 is set as the first scanning range SR1. As a result, the image included in the first scanning range SR1 (images 21 and 22 in FIG. 16B) is displayed at standard brightness, and the image included in the second scanning range SR2 (image 23 in FIG. 16B) is displayed at high brightness.

[0122] 17A and 17B are graphs that schematically show the rotation speed of the second mirror 161 when the frame images 20 of FIGS. 16A and 16B are displayed, respectively.

[0123] As shown in Figure 16(a), when the third scanning range SR3 and the fourth scanning range SR4 are set in the frame image 20, as shown in Figure 17(a), in the third scanning range SR3, the second mirror 161 is rotated at a constant speed in the third scanning mode, and in the fourth scanning range SR4, the second mirror 161 is rotated at a constant speed in the fourth scanning mode.

[0124] Specifically, in the third scanning range SR3, the second mirror 161 is rotated at a speed slower than that in the standard scanning range SR0 in Fig. 6A and faster than that in the second scanning range SR2 in Fig. 6B. The speed of the second mirror 161 in the third scanning range SR3 is a predetermined speed. The length of the period T2 corresponding to the third scanning range SR3 is then determined, and the slope of the graph in the fourth scanning range SR4 is set to be equal to that of the period T2, thereby determining the position of the period T2.

[0125] 16B, when a first scanning range SR1 and a second scanning range SR2 are set in the frame image 20, the slopes of the graphs in the first scanning range SR1 and the second scanning range SR2 and the position of the period T1 are determined as shown in FIG. 17B, similarly to embodiment 1. Then, in the first scanning range SR1, the second mirror 161 is rotated at a constant speed in the first scanning mode, and in the second scanning range SR2, the second mirror 161 is rotated at a constant speed in the second scanning mode.

[0126] 18A and 18B are diagrams each showing a schematic view of the scanning line L in the display area 40 when the second mirror 161 is rotated as shown in FIGS. 17A and 17B.

[0127] 17A, when the second mirror 161 rotates in the Z-axis direction in the third scanning mode and the fourth scanning mode, as shown in Fig. 18A, the pitch of the scanning lines L changes depending on the rotation speed of the second mirror 161. That is, the scanning lines L in the third scanning range SR3 are formed side by side in the Z-axis direction at a third pitch P3, and the scanning lines L in the fourth scanning range SR4 are formed side by side in the Z-axis direction at a fourth pitch P4.

[0128] 17B, ​​when the second mirror 161 rotates in the Z-axis direction in the first scanning mode and the second scanning mode, as shown in FIG. 18B, the pitch of the scanning lines L becomes the first pitch P1 or the second pitch P2 depending on the rotation speed of the second mirror 161, as in the first embodiment. The magnitude relationship among the four pitches P1 to P4 is P2<P3<P4<P1.

[0129] FIG. 19 is a flowchart showing the driving process of the second mirror 161 when a frame image 20 corresponding to one frame is displayed, according to the second embodiment.

[0130] In the second embodiment, steps S111 to S114 are added instead of steps S102 to S104, as compared with the first embodiment shown in FIG.

[0131] The control unit 201 controls the viewpoint detection unit 13 to acquire a captured image, detects the user's viewpoint based on the captured image, identifies a first region R41 of a predetermined width in the Z-axis direction centered on the viewpoint, and identifies a second region R42 other than the first region R41 (S111).

[0132] The control unit 201 determines whether the first region R41 includes the high-brightness region R2 (S112). If the first region R41 includes the high-brightness region R2 (S112: YES), the control unit 201 sets the entire first region R41 as the second scanning range SR2 and the second region R42 as the first scanning range SR1 (S113). On the other hand, if the first region R41 does not include the high-brightness region R2 (S112: NO), the control unit 201 sets the first region R41 as the third scanning range SR3 and the second region R42 as the fourth scanning range SR4 (S114).

[0133] FIG. 20 is a flowchart showing a mirror driving process for the second mirror 161 according to the second embodiment.

[0134] In the second embodiment, steps S201 and S202 are deleted and steps S211 to S214 are added, compared to the first embodiment shown in FIG.

[0135] If the rotation position of the second mirror 161 is included in the first scanning range SR1 (S203: YES), the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 in the first scanning mode (S204).If the rotation position of the second mirror 161 is included in the second scanning range SR2 (S203: NO, S211: YES), the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 in the second scanning mode (S205).

[0136] If the rotation position of the second mirror 161 is included in the third scanning range SR3 (S203: NO, S211: NO, S212: YES), the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 in the third scanning mode (S213).If the rotation position of the second mirror 161 is included in the fourth scanning range SR4 (S203: NO, S211: NO, S212: NO), the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 in the fourth scanning mode (S214).

[0137] <Effects of Second Embodiment> According to the second embodiment, the following effects are achieved.

[0138] The image display device 3 is a device that displays a frame image 20 (image) in a display area 40 that allows a user to see through to the rear. The image display device 3 includes a light source 170 that emits a beam modulated by a video signal, a first scanning unit 140 and a second scanning unit 160 (scanning units) that generate the frame image 20 (one frame's worth of image) by raster scanning the beam, a camera 12 that captures the scenery in the area corresponding to the display area 40, a viewpoint detection unit 13 that detects the user's viewpoint in the display area 40, and a control unit 201 that controls the light source 170 and the first scanning unit 140 and second scanning unit 160 (scanning units). The control unit 201 identifies a high-brightness region R2 in the display area 40 from the image 30 captured by the camera 12, where the illuminance is above a predetermined threshold, and divides the display area 40 into a first region R41 including the viewpoint and a second region R42 in the vertical direction of the raster scan.The control unit 201 performs vertical scanning in the raster scan so that the second vertical scanning range SR2 (scanning range) of the first region R41 corresponding to at least the high-brightness region R2 is closer in the vertical direction than the second region R42 and is therefore brighter.

[0139] According to this configuration, when the frame image 20 in the area viewed by the user overlaps with a highly illuminant landscape, at least the second scanning range SR2 that overlaps with the highly illuminant landscape in the frame image 20 is made brighter. This allows the frame image 20 displayed in a see-through manner to be viewed smoothly even when the landscape is highly illuminant.

[0140] In the second embodiment, since high brightness is achieved by bringing the beams closer together in the vertical direction in the second scanning range SR2, it is not necessary to increase the output of the light-emitting elements 101 to 103. Therefore, according to the second embodiment, the power consumption of the image display device 3 can be reduced.

[0141] When the first region R41 includes the high-brightness region R2, the control unit 201 performs vertical scanning in the raster scan so that the beam approaches the entire first region R41 in the vertical direction more than the second region R42.

[0142] Generally, people view a range of a predetermined angle from their line of sight. According to the above configuration, the entire first region R41 including the user's viewpoint is uniformly brightened, allowing the user to view the entire frame image 20 near the viewpoint without feeling uncomfortable.

[0143] When the first region R41 does not include the high-brightness region R2, the control unit 201 performs vertical raster scanning on the second region R42 so that the beam is farther away in the vertical direction than the first region R41.

[0144] According to this configuration, the resolution of the second region R42 located around the first region R41 including the viewpoint is lowered, thereby reducing eye fatigue of the user.

[0145] The resolution of the frame image 20 (image) in the scanning range corresponding to the high-brightness region R2 of the first region R41 is lower than the resolution of the frame image 20 (image) in the second region R42.

[0146] With this configuration, the resolution of the frame image 20 in the high-brightness region R2 of the first region R41 is lower than the resolution of the frame image 20 in the second region R42 due to the overlap of the beams in the vertical direction, but the overlap of the beams in the vertical direction can significantly increase the brightness. Therefore, even if the background scenery is highly illuminant, the frame image 20 can be displayed with good visibility.

[0147] The image display device 3 includes a first scanning unit 140 and a second scanning unit 160 (scanning units) that display a frame image 20 (image) by raster scanning a beam modulated by a video signal, and a control unit 201 that controls the first scanning unit 140 and the second scanning unit 160 (scanning units). The control unit 201 includes a standard scanning mode (first mode) that displays the entire frame image 20 (image) at a uniform resolution, and a third scanning mode (second mode) that displays the frame image 20 (image) in a third scanning range SR3 (predetermined scanning range) perpendicular to the scanning line L with higher brightness and lower resolution than in the standard scanning mode (first mode), and in the third scanning mode (second mode), a third pitch P3 (pitch) between the scanning lines L in the third scanning range SR3 (predetermined scanning range) is reduced from the standard pitch P0 (pitch) between the scanning lines L in the standard scanning mode (first mode).

[0148] According to this configuration, when a display range occurs in the frame image 20 where it is desired to increase the brightness even at the expense of resolution, the brightness of the frame image 20 in the third scanning range SR3 corresponding to this display range can be effectively increased by executing the third scanning mode.

[0149] <Modification of Embodiment 2> In Embodiment 2, the area of ​​the first region R41 other than the high-brightness region R2 is also set to the second scanning range SR2, but this is not limited to this and may also be set to the third scanning range SR3.

[0150] FIG. 21A is a diagram schematically showing a frame image 20 displayed in the display area 40 when the captured image 30 in FIG. 15B is acquired according to this modified example.

[0151] In the display area 40 of this modified example, the area of ​​the first region R41 other than the high-brightness region R2 is set as a third scanning range SR3, as compared to the second embodiment shown in Fig. 16(b). As a result, the image included in the first scanning range SR1 (images 21 and 22 in Fig. 21(a)) is displayed at standard brightness and low resolution, the image included in the second scanning range SR2 (image 23 in Fig. 21(a)) is displayed at high brightness, and the image included in the third scanning range SR3 (no image in Fig. 21(a)) is displayed at high resolution.

[0152] FIG. 21B is a graph schematically showing the rotation speed of the second mirror 161 when the frame image 20 of FIG. 21A is displayed according to this modified example.

[0153] When a first scanning range SR1, a second scanning range SR2, and a third scanning range SR3 are set in the frame image 20 as shown in Fig. 21(a), the rotation speeds (slope of the graph) in the first scanning range SR1, the second scanning range SR2, and the third scanning range SR3 and the positions of the periods T1 and T2 corresponding to the second scanning range SR2 and the third scanning range SR3, respectively, are determined as shown in Fig. 21(b). The rotation speeds (slope of the graph) in the second scanning range SR2 and the third scanning range SR3 in this case are the same as those in Figs. 17(a) and 17(b), and are preset values.

[0154] FIG. 22 is a flowchart showing the driving process of the second mirror 161 when a frame image 20 corresponding to one frame is displayed according to this modified example.

[0155] In this modification, step S121 is added instead of step S113, as compared with the second embodiment shown in FIG.

[0156] If the first region R41 includes the high-brightness region R2 (S112: YES), the control unit 201 sets the high-brightness region R2 in the first region R41 to the second scanning range SR2, sets the area other than the high-brightness region R2 in the first region R41 to the third scanning range SR3, and sets the area other than the second scanning range SR2 and the third scanning range SR3 to the first scanning range SR1 (S121).

[0157] <Advantages of Modification of Second Embodiment> According to this modification, the following advantages are achieved.

[0158] When the first region R41 includes the high-brightness region R2, the control unit 201 performs vertical scanning in the raster scan so that the beam is farther away vertically from the third vertical scanning range SR3 (scanning range) of the first region R41 that does not correspond to the high-brightness region R2 than from the second scanning range SR2 (scanning range) that corresponds to the high-brightness region R2.

[0159] As the beams overlap in the vertical direction, the brightness of the frame image 20 increases, but the resolution of the frame image 20 decreases. According to the above configuration, beam overlap is suppressed in the third scanning range SR3 other than the high-brightness region R2 in the first region R41 that includes the user's viewpoint, so the user can clearly view the frame image 20 in areas where the scenery behind is not highly illuminant.

[0160] Third Embodiment In the second embodiment, the beam scanning of the frame image 20 is switched based on the high-brightness region R1 and the first region R31 of the captured image 30. In contrast to this, in the third embodiment, the beam scanning of the frame image 20 is switched further based on an ROI (Region of Interest) region R51 of the captured image 30. The ROI region is a region of interest that is likely to interest the user.

[0161] FIG. 23A is a diagram schematically showing a captured image 30 captured by the camera 12. As shown in FIG.

[0162] 23(a), when a predetermined scene such as a road sign is captured in captured image 30, the region in the Z-axis direction corresponding to the scene becomes ROI region R51. The scene targeted by ROI region R51 is stored in advance in the memory of control unit 201, and ROI region R51 is set when the target scene is extracted by control unit 201. In FIG. 23(a), ROI region R51 is included in upper high-brightness region R1, and a first region R31 corresponding to the viewpoint overlaps with lower high-brightness region R1.

[0163] In embodiment 3, even if the first region R31 including the viewpoint and the high-brightness region R1 do not overlap, the first region R41 of the display region 40 is not displayed at high resolution, but as in embodiment 2, the first region R41 of the display region 40 may be displayed at high resolution.

[0164] FIG. 23B is a diagram that schematically shows a frame image 20 displayed in the display area 40 when the captured image 30 in FIG. 23A is acquired.

[0165] When the captured image 30 shown in Fig. 23(a) is acquired, as shown in Fig. 23(b), an ROI region R61 corresponding to the ROI region R51 is set in the display area 40. In Fig. 23(b), the ROI region R61 is included in the high-brightness region R2, so the second scanning range SR2 is set to cover the entire ROI region R61.

[0166] Furthermore, an image 24 corresponding to a predetermined scene (a road sign in FIG. 23A) in the captured image 30 is displayed in the ROI region R61. The control unit 201 analyzes the scene captured in the ROI region R51 of the captured image 30, reads out a corresponding pre-stored scene image from memory, and displays the read scene image as an image 24 at a corresponding position in the ROI region R61. In this way, the pre-stored image 24 is displayed corresponding to the predetermined scene that is difficult to see because it overlaps with the high-illumination background, allowing the user to clearly see the predetermined scene (such as a road sign).

[0167] FIG. 24 is a flowchart showing the driving process of the second mirror 161 when a frame image 20 corresponding to one frame is displayed.

[0168] In the third embodiment, steps S131 to S136 are added instead of steps S113 and S114, as compared with the second embodiment shown in FIG.

[0169] If the first region R41 includes the high-brightness region R2 (S112: YES), the control unit 201 sets the entire first region R41 as the second scanning range SR2 (S131). Next, the control unit 201 extracts a predetermined scene (e.g., a road sign) based on the captured image 30 and identifies an ROI region R61 in the Z-axis direction corresponding to the scene in the display area 40 (S132). If the ROI region R61 exists and includes the high-brightness region R2 (S133: YES), the control unit 201 sets the ROI region R61 as the second scanning range SR2 (S134).

[0170] The control unit 201 sets the area other than the second scanning range SR2 as the first scanning range SR1 (S135). Then, the control unit 201 displays the image 24 corresponding to the predetermined scene extracted based on the captured image 30 in step S132 in the ROI region R61 of the frame image 20 (S136). Specifically, the control unit 201 generates a video signal corresponding to one frame so that the image 24 is superimposed, and outputs the generated video signal to the signal processing unit 300.

[0171] <Effects of Third Embodiment> According to the third embodiment, the following effects are achieved.

[0172] The control unit 201 identifies an ROI region R61 from the captured image 30 that is likely to interest the user, and based on the fact that the ROI region R61 includes a high-brightness region R2, displays an image 24 corresponding to the ROI region R61 in the ROI region R61 in the frame image 20 (image) by scanning in the same vertical direction as the high-brightness region R2 in the first region R41.

[0173] According to this configuration, even if the user has difficulty in directly viewing the object in the ROI region R61, the user can easily view the object in the ROI region R61 by referring to the high-brightness frame image 20 corresponding to the ROI region R61.

[0174] <Modification of Third Embodiment> In the third embodiment, the beam scanning of the frame image 20 is switched based on the ROI region R51 of the captured image 30, but instead, the beam scanning of the frame image 20 may be switched based on the motion region R52 of the captured image 30. The motion region is a region that includes a moving object in the captured images 30 obtained in time series.

[0175] FIG. 25A is a diagram schematically showing a captured image 30 captured by the camera 12 according to this modified example.

[0176] 25A, when a moving object is captured in the captured image 30, the region in the Z-axis direction corresponding to the scene is a motion region R52. In FIG. 25A, the motion region R52 is included in the lower high-brightness region R1, and the upper high-brightness region R1 overlaps with a first region R31 corresponding to the viewpoint.

[0177] FIG. 25B is a diagram that schematically shows the frame image 20 displayed in the display area 40 when the captured image 30 in FIG. 25A is acquired.

[0178] When the captured image 30 shown in Fig. 25(a) is acquired, as shown in Fig. 25(b), a movement region R62 corresponding to the movement region R52 is set in the display region 40. In Fig. 25(b), the movement region R62 is included in the high-brightness region R2, and therefore a second scanning range SR2 in which the beam is scanned in the second scanning mode is set over the entire movement region R62.

[0179] Additionally, in the motion region R62, a frame image 25 is displayed surrounding the position of the moving object (a ball-shaped object in FIG. 25( a)) detected based on the captured image 30. The control unit 201 detects the position of the moving object captured in the motion region R52 of the captured image 30, and displays the frame image 25 at the position in the motion region R62 that corresponds to the detected position. In this way, the frame image 25 is displayed corresponding to the moving object that is difficult to see because it overlaps with the bright background, so the user can accurately grasp the position of the moving object.

[0180] FIG. 26 is a flowchart showing the driving process of the second mirror 161 when a frame image 20 corresponding to one frame is displayed.

[0181] In this modification, steps S141 to S144 are added instead of steps S132 to S134 and S136, as compared with the third embodiment shown in FIG.

[0182] The control unit 201 extracts a moving object from the captured image 30 and identifies a movement region R62 in the Z-axis direction corresponding to the moving object in the display area 40 (S141). If the movement region R62 exists and includes the high-brightness region R2 (S142: YES), the control unit 201 sets the movement region R62 to the second scanning range SR2 (S143).

[0183] The control unit 201 displays the frame image 25 corresponding to the predetermined scene extracted based on the captured image 30 in step S141 in the moving area R62 of the frame image 20 (S144). Specifically, the control unit 201 generates a video signal corresponding to one frame so that the frame image 25 surrounds the moving object, and outputs the generated video signal to the signal processing unit 300.

[0184] <Effects of Modification of Third Embodiment> According to this modification, the following effects are achieved.

[0185] The control unit 201 identifies a motion area R62 where movement has occurred in the captured image 30, and based on the fact that the motion area R62 includes a high-brightness area R2, displays a frame image 25 (an image that calls attention) in the motion area R62 in the frame image 20 (image) by scanning in the same vertical direction as the high-brightness area R2 in the first area R41.

[0186] With this configuration, even if the user has difficulty directly viewing the object in the motion area R62, the user can easily view the object in the motion area R62 by referring to the high-brightness frame image 25 that corresponds to the motion area R62 and alerts the user.

[0187] Fourth Embodiment In the first to third embodiments, one line of video signal is output to the line memory 221 based on one type of video signal in the signal processing unit 300. In contrast, in the fourth embodiment, two buffers are provided in the signal processing unit 300, and one line of video signal is selectively output to the line memory 221 from two types of video signals stored in each buffer.

[0188] FIG. 27 is a block diagram showing the configuration of the signal processing unit 300.

[0189] The signal processing unit 300 includes an input processing unit 310 , a first buffer 321 , and a second buffer 322 .

[0190] Input processing unit 310 outputs one frame of video signal input from control unit 201 as a first video signal directly to first buffer 321. Input processing unit 310 also performs averaging processing on one frame of video signal input from control unit 201, and outputs the processed video signal as a second video signal to second buffer 322. The first video signal and the second video signal have the same number of lines. The averaging processing will be described later with reference to Figures 28(a) and 28(b).

[0191] First buffer 321 temporarily stores one frame of the first video signal that has not been averaged by input processing unit 310. Second buffer 322 temporarily stores one frame of the second video signal that has been averaged by input processing unit 310. The second video signal stored in second buffer 322 is a video signal used to display high-luminance region R2, and the first video signal stored in first buffer 321 is a video signal used to display regions other than high-luminance region R2.

[0192] Either the first video signal in the first buffer 321 or the second video signal in the second buffer 322 is output to the line memory 221 line by line by switching by the control unit 201 .

[0193] 28A and 28B are diagrams conceptually showing the first video signal in the first buffer 321 and the second video signal in the second buffer 322, respectively.

[0194] For example, let us assume that the first and second video signals are each composed of 15 line data. In this case, as shown in Fig. 28(a), the first video signal includes 15 independent line data. In Fig. 28(a), each line data is shown in a different color to indicate that the line data are different from each other.

[0195] On the other hand, as shown in Fig. 28(b), the second video signal is obtained by averaging the line data within each pixel block, each pixel block consisting of a predetermined number of pixels in both the vertical and horizontal directions. In the example shown in Fig. 28(b), the pixel block is shown to be composed of five pixels in both the vertical and horizontal directions, and for convenience, the line data within each pixel block is shown in a different color to indicate that the line data within each pixel block arranged vertically are different from each other. As a result of the averaging process, the line data within the same pixel block become the same data, and the second video signal has a lower resolution than the first video signal.

[0196] The size of one pixel block (number of pixels in both the vertical and horizontal directions) can be set in relation to the second pitch P2 in the second scanning range SR2 so that an image with a resolution that is easier to see is displayed compared to embodiments 1 to 3.

[0197] <Effects of Fourth Embodiment> According to the fourth embodiment, the following effects are achieved.

[0198] The image display device 3 includes a first buffer 321 that stores a first video signal and a second buffer 322 that stores a second video signal having a lower resolution than the first video signal. The control unit 201 acquires the second video signal from the second buffer 322 in a second scanning range SR2 (scanning range) corresponding to the high-luminance region R2 and causes the light source 170 to emit a beam, and acquires the first video signal from the first buffer 321 and causes the light source 170 to emit the beam in scanning ranges (e.g., second, third, and fourth scanning ranges) corresponding to regions other than the high-luminance region R2.

[0199] In the second scanning range SR2 corresponding to the high-brightness region R2, the amount of beam overlap in the vertical direction is large, resulting in a decrease in the resolution of the frame image 20. For this reason, in this scanning range, the frame image 20 is easier to see if an image based on a video signal with a lower resolution is displayed than in the other scanning ranges. Therefore, with the above configuration, it is possible to provide the user with an easy-to-see frame image 20 in the second scanning range SR2 corresponding to the high-brightness region R2.

[0200] The second video signal stored in the second buffer 322 is a video signal whose resolution is reduced by integrating the first video signal for each pixel block including a plurality of pixels.

[0201] According to this configuration, the second video signal can be smoothly generated from the first video signal.

[0202] Note that the integration of the first video signals is not limited to being performed by averaging processing. For example, the second video signal may be generated by unifying the first video signals of any of the pixel blocks.

[0203] 6A and 6B, in the first embodiment, the rotation speed of the second mirror 161 is changed to increase the brightness of the frame image 20 corresponding to the high-brightness region R2. In this case, each scanning range expands or contracts slightly in the vertical direction depending on the rotation speed of the second mirror 161. In contrast, in the fifth embodiment, the configuration of the video signal used for beam scanning is changed so that the rotation speed of the second mirror 161 in the vertical direction is constant regardless of the beam scanning range. This makes it possible to prevent each scanning range from expanding or contracting in the vertical direction.

[0204] In the fifth embodiment, an input processing section 310, a first buffer 321, and a second buffer 322 are provided in a signal processing section 300, as in the fourth embodiment shown in FIG.

[0205] 29A and 29B are diagrams conceptually showing the first video signal in the first buffer 321 and the second video signal in the second buffer 322, respectively, according to the fifth embodiment.

[0206] 29(a), the input processing unit 310 performs processing on the video signal input from the control unit 201 to insert a predetermined number of blank lines of data (for example, one line) that do not contain any data between adjacent line data, and outputs the result as a first video signal to the first buffer 321. The first video signal is used when beam scanning is performed on an area other than the high-brightness area R2, and is configured so that the frame image 20 becomes an image with a standard resolution when the second mirror 161 rotates at a predetermined speed.

[0207] 29(b), the input processing unit 310 outputs the video signal input from the control unit 201 as a second video signal directly to the second buffer 322. The second video signal is used when the beam is scanned over the high-brightness region R2, and is configured so that the frame image 20 becomes a high-brightness, low-resolution image when the second mirror 161 rotates at a predetermined speed.

[0208] 30A and 30B are diagrams schematically showing scanning lines L in a display area 40 when the first and second video signals are used according to the fifth embodiment.

[0209] As shown in Fig. 30(a), when only the standard scanning range SR0 is set, scanning lines L are formed based on the first video signal shown in Fig. 29(a). In this case, there are lines between adjacent scanning lines L where no beam is irradiated, and the pitch of the scanning lines L is the standard pitch P0.

[0210] On the other hand, when a first scanning range SR1 and a second scanning range SR2 are set as shown in FIG. 30(b), the scanning lines L of the first scanning range SR1 are formed based on the first video signal shown in FIG. 29(a), and the scanning lines L of the second scanning range SR2 are formed based on the second video signal shown in FIG. 29(b). In this case, in the first scanning range SR1, as in FIG. 30(a), there are lines between adjacent scanning lines L that are not irradiated with the beam, and the pitch of the first scanning range SR1 is the first pitch P1. In the second scanning range SR2, the beam is irradiated on each line, and the pitch of the second scanning range SR2 is the second pitch P2. In this embodiment, the standard pitch P0 and the first pitch P1 are equal to each other.

[0211] <Effects of Fifth Embodiment> According to the fifth embodiment, the following effects are achieved.

[0212] The control unit 201 sets the vertical scanning speed to be the same throughout the entire display area 40, and reduces the frequency of scanning lines L for image display in the standard scanning range SR0 and the first scanning range SR1 (other scanning ranges) compared to the second scanning range SR2 (scanning range) corresponding to the high-brightness area R2.

[0213] According to this configuration, it is possible to prevent the frame image 20 from expanding or contracting in the vertical direction, and the frame image 20 can be displayed without creating an awkward feeling.

[0214] Note that also in the third embodiment, the first region R41 and the ROI region R61 overlapping the high-luminance region R2 may be scanned with a beam based on the first video signal shown in Fig. 29(a) as in the fifth embodiment. Similarly, also in the modification of the third embodiment, the first region R41 and the motion region R62 overlapping the high-luminance region R2 may be scanned with a beam based on the first video signal shown in Fig. 29(a) as in the fifth embodiment.

[0215] Furthermore, in the fifth embodiment, the signal processing unit 300 may be provided with other buffers in addition to the first buffer 321 and the second buffer 322. By storing video signals in three or more buffers, each of which has a different number of lines between adjacent scanning lines L that are not irradiated with a beam, it is possible to display frame images 20 with different variations in brightness and resolution in multiple scanning ranges.

[0216] Also in this embodiment, the frame image 20 may be displayed using one type of video signal similar to those in embodiments 1 to 4. In this case, for example, line data as shown in Figures 29(a) and 29(b) may be generated from the original video signal at the scanning timing of each scanning line L, and the scanning line L may be scanned with a beam based on the generated line data.

[0217] Sixth Embodiment In a sixth embodiment, the present invention is applied to an in-vehicle head-up display.

[0218] 31(a) to 31(c) are diagrams schematically illustrating usage patterns of a head-up display 400 according to a sixth embodiment. FIG. 31(a) is a schematic diagram illustrating the interior of a passenger vehicle 411 seen from the side of the passenger vehicle 411. FIG. 31(b) is a diagram illustrating the front in the direction of travel from the inside of the passenger vehicle 411. FIG. 31(c) is a schematic diagram illustrating a state in which a frame image 20 is formed as a virtual image. In FIGS. 31(a) to 31(c), components similar to those in the first to fifth embodiments are denoted by the same reference numerals as those in the first to fifth embodiments for convenience.

[0219] 31(a), head-up display 400 is installed inside dashboard 412 of passenger vehicle 411. As shown in Figures 31(a) and 31(b), head-up display 400 projects display light modulated by a video signal onto display area 40 located near the driver's seat below windshield 413. The projected display light is reflected by display area 40 and irradiated onto a horizontally elongated area (eye box area) around the position of eye E of driver 420.

[0220] As shown in FIG. 3C , the head-up display 400 includes one image display device 3 and an optical system 401 .

[0221] The optical system 401 is configured, for example, by a single mirror having a curved reflective surface. The optical system 401 reflects light emitted from the image display device 3 toward the windshield 413, and generates a virtual image in front of the windshield 413 using this light. The light reflected by the optical system 401 and the windshield 413 is irradiated onto a horizontally elongated area (eye box area) around the position of the eyes E of the driver 420. The optical system of the image display device 3 and the optical system 401 are designed so that the frame image 20 is displayed as a virtual image at a predetermined size in front of the windshield 413.

[0222] The optical system 401 does not necessarily have to be composed of only one mirror, but may include, for example, a plurality of mirrors, a lens, or the like.

[0223] The communication unit 5 of this embodiment is a communication unit that can communicate with a car navigation system installed in the passenger vehicle 411. The control unit 201 of this embodiment receives the time, temperature, navigation information, and the like from the car navigation system via the communication unit 5.

[0224] With the above configuration and control similar to those of the first to fifth embodiments, the frame image 20 is displayed as a virtual image in the field of view ahead of the driver 420. The driver 420 can see the frame image 20, which is a virtual image, superimposed on the scenery ahead of the windshield 413.

[0225] <Effects of Sixth Embodiment> According to the sixth embodiment, the following effects are achieved.

[0226] The head-up display 400 projects light onto a display area 40 of a windshield 413 to display a virtual image of a frame image 20 (image) in front of the windshield 413. The head-up display 400 includes an image display device 3 similar to those in the first to fifth embodiments, and an optical system 401 that displays a virtual image of the frame image 20 (image) in front of the windshield 413 based on the light emitted from the image display device 3.

[0227] This configuration achieves the same effects as those of the first or second aspect. In addition, the driver 420 can see a virtual image of the frame image 20 in front of the windshield 413, and therefore can check the frame image 20 displayed by the image display device 3 while looking at the scenery in front of the windshield 413.

[0228] <Other Modifications> The configurations of the image display device 3, the head-mounted display 1, and the head-up display 400 can be modified in various ways in addition to the configurations shown in the above-described embodiment and modifications.

[0229] In the third embodiment, in step S133 of Fig. 24 , it is determined whether or not the ROI region R61 in the Z-axis direction includes the high-brightness region R2, but it may also be determined whether or not a high-illuminance range in the X-Y plane overlaps with a range in the X-Y plane in which a predetermined object, such as a road sign, appears in the captured image 30. Similarly, in the modification of the third embodiment, in step S142 of Fig. 26 , it is determined whether or not the movement region R62 in the Z-axis direction includes the high-brightness region R2, but it may also be determined whether or not a high-illuminance range in the X-Y plane overlaps with a range in the X-Y plane in which a moving object appears in the captured image 30.

[0230] In the above embodiment and modified example, the first region R41, ROI region R61, motion region R62, etc. are identified for each frame, but this is not limited to this, and they may be identified for each predetermined number of frames (two or more). However, with this method, the identification of the above regions is delayed by several frame periods, making it more difficult to quickly reflect the above regions in the frame image 20 compared to when the above regions are identified for each frame. Therefore, in order to quickly reflect the above regions in the frame image 20, it is preferable to identify the above regions for each frame, as in the above embodiment and modified example.

[0231] In the above embodiment and modified example, for example, as shown in Fig. 5(a), the frame image 20 includes images showing the time, temperature, navigation information (route information), etc., but the images included in the frame image 20 are not limited to these. For example, in the case of the head-mounted display 1, an image showing store information within the field of view may be displayed as navigation information. In the case of the head-up display 400, an image showing the speed of the passenger vehicle 411, an image showing road congestion information, etc. may be displayed.

[0232] In the above embodiments 1 to 5 and modified examples, two sets of image display devices 3 and mirrors 4 are provided on the head-mounted display 1 to correspond to the user's pair of eyes E, but only one set may be provided on the head-mounted display 1 to correspond to only one of the user's eyes E.

[0233] In the above-described first to fifth embodiments and modified examples, the light scanned by the first scanning unit 140 and the second scanning unit 160 is directed to the user's eye E via the mirror 4, but this is not limiting, and the light may be directed to the user's eye E via an optical system other than a mirror (for example, a lens, etc.). In this case, the optical system may be, for example, a combination of multiple mirrors, a combination of a mirror and a lens, or a combination of multiple lenses.

[0234] In the above embodiment and modified examples, the first mirror 141 and the second mirror 161 are provided separately, but instead of the first mirror 141 and the second mirror 161, a single mirror that rotates about two axes may be provided.

[0235] The embodiments of the present invention can be modified in various ways as appropriate within the scope of the technical idea defined in the claims.

[0236] (Additional Notes) The above description of the embodiments discloses the following techniques.

[0237] (Technology 1) An image display device that displays an image in a display area that allows a user to see through to the rear, comprising: a light source that emits a beam modulated by a video signal; a scanning unit that generates one frame of an image by raster scanning the beam; a camera that captures an image of a scene in an area corresponding to the display area; and a control unit that controls the light source and the scanning unit, wherein the control unit identifies, from the image captured by the camera, a high-brightness area in the display area where the illuminance is equal to or greater than a predetermined threshold, and performs vertical scanning in the raster scan so that the beam is closer in the vertical direction to the vertical scanning range corresponding to the high-brightness area and is therefore brighter than other scanning ranges.

[0238] This technology increases the brightness of images in high-brightness areas, allowing images displayed in see-through mode to be viewed smoothly even in high-illumination scenes.

[0239] (Technology 2) An image display device that displays an image in a display area that allows a user to see through to the rear, comprising: a light source that emits a beam modulated by a video signal; a scanning unit that generates one frame of an image by scanning the beam using raster scanning; a camera that captures an image of a scene in an area corresponding to the display area; a viewpoint detection unit that detects a user's viewpoint in the display area; and a control unit that controls the light source and the scanning unit, wherein the control unit identifies, from the image captured by the camera, a high-brightness area in the display area where the illuminance is equal to or greater than a predetermined threshold, divides the display area into a first area that includes the viewpoint and a second area in the vertical direction of the raster scanning, and performs vertical scanning in the raster scanning so that the beam is closer in the vertical direction to and higher in the vertical scanning range of at least the high-brightness area in the first area than in the second area.

[0240] With this technology, when the image of the area viewed by the user overlaps with a highly illuminated scene, at least the scanning range of the image that overlaps with the highly illuminated scene is made brighter, allowing the image displayed in see-through mode to be viewed smoothly even when the scene is highly illuminated.

[0241] (Technology 3) In the image display device described in Technology 2, the control unit is configured to, when the first region includes the high-brightness region, perform vertical scanning in the raster scan so that the beam approaches the entire first region in the vertical direction more than the second region.

[0242] Generally, people view an area within a predetermined angle from their line of sight. According to the above technology, the entire first region including the user's viewpoint is uniformly brightened, allowing the user to view the entire image near the viewpoint without feeling uncomfortable.

[0243] (Technology 4) In the image display device described in Technology 2, the control unit, when the first region includes the high-brightness region, performs vertical scanning in the raster scan so that the beam is farther away in the vertical direction in a vertical scanning range of the first region that does not correspond to the high-brightness region than in the scanning range that corresponds to the high-brightness region.

[0244] The more the beams overlap in the vertical direction, the greater the brightness of the image, but the lower the image resolution. According to the above technology, beam overlap is suppressed in the scanning range other than the high-brightness area in the first area including the user's viewpoint, so the user can clearly see the image in the area where the scenery behind them is not high-brightness.

[0245] (Technology 5) In the image display device described in any one of Technologies 2 to 4, the control unit is configured to, when the first region does not include the high-brightness region, perform vertical scanning in the raster scan so that the beam is farther away in the vertical direction in the second region than in the first region.

[0246] According to this technique, the resolution of the second area located around the first area including the viewpoint is lowered, thereby reducing eye strain on the user.

[0247] (Technology 6) In the image display device according to Technology 5, a resolution of the image in the scanning range corresponding to the high-brightness area of ​​the first area is lower than a resolution of the image in the second area.

[0248] With this technology, the image resolution in the high-brightness area of ​​the first region is lower than that in the second region due to the vertical beam overlap, but the vertical beam overlap significantly increases the brightness, allowing the image to be displayed with good visibility even when the background scenery is highly illuminating.

[0249] (Technology 7) In the image display device described in any one of technologies 2 to 6, the control unit identifies an ROI region in the captured image that is likely to interest a user, and based on the ROI region including the high-brightness region, displays an image corresponding to the ROI region in the ROI region in the image by scanning in the vertical direction similar to the high-brightness region in the first region.

[0250] According to this technique, even if a user has difficulty directly viewing an object in an ROI region, the user can easily view the object in the ROI region by referring to a high-luminance image corresponding to the ROI region.

[0251] (Technology 8) In the image display device described in any one of technologies 2 to 7, the control unit identifies a motion area in the captured image where motion has occurred, and, based on the fact that the motion area includes the high-brightness area, displays an image that calls attention in the motion area in the image by scanning in the vertical direction in the same way as the high-brightness area in the first area.

[0252] According to this technology, even if a user has difficulty directly viewing an object in a moving area, the user can easily view the object in the moving area by referring to a high-brightness image corresponding to the moving area that calls attention to the object.

[0253] (Technology 9) In the image display device described in any one of technologies 1 to 8, the control unit reduces the vertical scanning speed of the scanning range corresponding to the high-brightness area compared to other scanning ranges.

[0254] According to this technique, by adjusting the scanning speed in the vertical direction, it is possible to switch the pitch of the scanning lines of the beam between the scanning range corresponding to the high-brightness area and the other scanning ranges.

[0255] (Technology 10) In the image display device described in any one of technologies 1 to 9, the control unit sets the vertical scanning speed to be the same across the entire range of the display area, and reduces the frequency of scanning lines for image display in other scanning ranges compared to the scanning range corresponding to the high-brightness area.

[0256] This technique can prevent the displayed image from expanding or contracting in the vertical direction, allowing the displayed image to be displayed without creating an awkward feeling.

[0257] (Technology 11) An image display device according to any one of technologies 1 to 10, comprising: a first buffer for storing a first video signal; and a second buffer for storing a second video signal having a lower resolution than the first video signal, wherein the control unit acquires the second video signal from the second buffer in a scanning range corresponding to the high-brightness area and causes the light source to emit the beam, and acquires the first video signal from the first buffer in a scanning range corresponding to an area other than the high-brightness area.

[0258] In the scanning range corresponding to the high-brightness area, the amount of beam overlap in the vertical direction is large, resulting in a decrease in image resolution. Therefore, in this scanning range, it is easier to see the image by displaying an image based on a video signal with a lower resolution than in other scanning ranges. Therefore, the above technology can provide the user with an easy-to-see image in the scanning range corresponding to the high-brightness area.

[0259] (Technology 12) In the image display device described in Technology 11, the second video signal stored in the second buffer is a video signal obtained by integrating the first video signal for each pixel block including a plurality of pixels to reduce the resolution.

[0260] According to this technique, the second video signal can be smoothly generated from the first video signal.

[0261] (Technology 13) An image display device comprising: a scanning unit that displays an image by raster scanning a beam modulated by a video signal; and a control unit that controls the scanning unit, wherein the control unit has a first mode that displays the entire image at a uniform resolution, and a second mode that displays the image at a higher brightness and lower resolution than in the first mode in a predetermined scanning range perpendicular to the scanning line, and in the second mode, reduces the pitch between scanning lines in the predetermined scanning range compared to the pitch between scanning lines in the first mode.

[0262] According to this technology, when a display range occurs in a display image where it is desired to increase the brightness even at the expense of resolution, the second mode can be executed to effectively increase the brightness of the image in a specified scanning range corresponding to this display range.

[0263] (Technology 14) A head-mounted display comprising: the image display device according to any one of technologies 1 to 12; and a frame that holds the image display device.

[0264] The head-mounted display according to this technology has the same effects as the above technology. In addition, by wearing the head-mounted display on the user's head, the user can view the image displayed by the image display device together with the scenery ahead, without holding the image display device.

[0265] (Technology 15) A head-up display that projects light onto a display area of ​​a windshield to display a virtual image of an image in front of the windshield, comprising: an image display device according to any one of technologies 1 to 12; and an optical system that displays the virtual image of the image in front of the windshield based on the light emitted from the image display device.

[0266] The head-up display according to this technology has the same effect as the above technology, and also allows the driver to see a virtual image of the image ahead of the windshield, so that the driver can check the image displayed by the image display device while looking at the scenery ahead of the windshield.

[0267] REFERENCE SIGNS LIST 1 Head-mounted display 2 Frame 3 Image display device 12 Camera 13 Viewpoint detection unit 20 Frame image (image) 24, 25 Images 30 Captured image 40 Display area 140 First scanning unit (scanning unit) 160 Second scanning unit (scanning unit) 170 Light source 201 Control unit 321 First buffer 322 Second buffer 400 Head-up display 401 Optical system 413 Windshield L Scanning line P0 Standard pitch (pitch) P2 Second pitch (pitch) P3 Third pitch (pitch) R2 High-brightness area R41 First area R42 Second area R61 ROI area R62 Movement area SR1 First scanning range (other scanning range) SR2 Second scanning range (scanning range, predetermined scanning range) SR3 Third scanning range (scanning range, other scanning range, predetermined scanning range) SR4 4th scanning range (other scanning range)

Claims

1. An image display device that displays an image in a display area that allows the user to see through to the rear, comprising: a light source that emits a beam modulated by a video signal; a scanning unit that generates one frame of an image by raster scanning the beam; a camera that captures an image of an area corresponding to the display area; and a control unit that controls the light source and the scanning unit, wherein the control unit identifies, from the image captured by the camera, a high-brightness area in the display area where the illuminance is equal to or greater than a predetermined threshold, and performs vertical scanning in the raster scan so that the beam is closer in the vertical direction to the vertical scanning range corresponding to the high-brightness area and is therefore brighter than other scanning ranges.

2. An image display device that displays an image in a display area that allows the user to see through to the rear, comprising: a light source that emits a beam modulated by a video signal; a scanning unit that generates one frame of an image by scanning the beam using a raster scan; a camera that captures an image of a scene in an area corresponding to the display area; a viewpoint detection unit that detects the user's viewpoint in the display area; and a control unit that controls the light source and the scanning unit, wherein the control unit identifies, from the image captured by the camera, a high-brightness area in the display area where the illuminance is equal to or greater than a predetermined threshold, divides the display area into a first area that includes the viewpoint and a second area in the vertical direction of the raster scan, and performs vertical scanning in the raster scan so that the beam is closer in the vertical direction to and higher in brightness in at least the vertical scanning range of the first area that corresponds to the high-brightness area than in the second area.

3. An image display device according to claim 2, characterized in that, when the first region includes the high-brightness region, the control unit performs vertical scanning in the raster scan so that the beam approaches the entire first region in the vertical direction more closely than the second region.

4. An image display device according to claim 2, characterized in that, when the first region includes the high-brightness region, the control unit performs vertical scanning in the raster scan so that the beam is farther away in the vertical direction in the vertical scanning range of the first region that does not correspond to the high-brightness region than in the scanning range that corresponds to the high-brightness region.

5. An image display device according to claim 2, characterized in that, when the first region does not include the high-brightness region, the control unit performs vertical scanning in the raster scan so that the beam is farther away in the vertical direction in the second region than in the first region.

6. An image display device according to claim 5, wherein the resolution of the image in the scanning range corresponding to the high brightness area of ​​the first area is lower than the resolution of the image in the second area.

7. An image display device according to claim 2, wherein the control unit identifies an ROI region in the captured image that is likely to interest the user, and, based on the fact that the ROI region includes the high-brightness region, displays an image corresponding to the ROI region in the ROI region in the image by scanning in the vertical direction in the same manner as the high-brightness region in the first region.

8. An image display device according to claim 2, wherein the control unit identifies a motion area in the captured image where motion has occurred, and, based on the fact that the motion area includes the high-brightness area, displays an image calling attention in the motion area in the image by scanning in the same vertical direction as the high-brightness area in the first area.

9. An image display device according to claim 1 or 2, characterized in that the control unit reduces the vertical scanning speed of the scanning range corresponding to the high-brightness area compared to other scanning ranges.

10. An image display device according to claim 1 or 2, characterized in that the control unit sets the same vertical scanning speed over the entire range of the display area, and reduces the frequency of scanning lines for image display in other scanning ranges compared to the scanning range corresponding to the high-brightness area.

11. An image display device according to claim 1 or 2, comprising: a first buffer for storing a first video signal; and a second buffer for storing a second video signal having a lower resolution than the first video signal, wherein the control unit acquires the second video signal from the second buffer in a scanning range corresponding to the high-brightness area and causes the light source to emit the beam, and acquires the first video signal from the first buffer in a scanning range corresponding to an area other than the high-brightness area and causes the light source to emit the beam.

12. An image display device according to claim 11, wherein the second video signal stored in the second buffer is a video signal obtained by integrating the first video signal for each pixel block containing a plurality of pixels to reduce the resolution.

13. An image display device comprising: a scanning unit that displays an image by raster scanning a beam modulated by a video signal; and a control unit that controls said scanning unit, wherein said control unit has a first mode in which the entire image is displayed at a uniform resolution, and a second mode in which the image is displayed at a higher brightness and lower resolution than in said first mode in a predetermined scanning range perpendicular to the scanning line, and in said second mode the pitch between scanning lines in said predetermined scanning range is reduced compared to the pitch between scanning lines in said first mode.

14. A head-mounted display comprising: an image display device according to claim 1 or 2; and a frame that holds the image display device.

15. A head-up display that projects light onto a display area of ​​a windshield to display a virtual image of an image in front of the windshield, comprising: an image display device according to claim 1 or 2; and an optical system that displays the virtual image of the image in front of the windshield based on the light emitted from the image display device.

Images