Image generation device and head-mounted display

Abstract

This image generation device comprises: an imaging processing unit (230) including a camera (13) for imaging a field-of-view range, the imaging processing unit (13) outputting a first captured video signal for forming a frame image having a first definition and a second captured video signal for forming a frame image having a second definition; a first frame buffer (341) in which the first captured video signal is stored; and a second frame buffer (342) in which the second captured video signal is stored. A specific region including at least part of a region toward which a gaze is likely to be directed is set in the frame images. An image is generated in a first image region and the specific region by applying the first captured video signal from the first frame buffer (341), and an image is generated in a second image region by applying the second captured video signal from the second frame buffer (342).

Description

Image generating device and head-mounted display

[0001] The present invention relates to an image generating device that generates an image by scanning light and a head-mounted display.

[0002] Conventionally, as an image generating device that generates an image by scanning light, for example, a head-mounted display such as goggles or glasses that realize AR (Augmented Reality) or VR (Virtual Reality) has been known. In these devices, for example, light based on a video signal is irradiated onto a translucent display, and the reflected light is irradiated onto a user's eyes. Alternatively, light based on the video signal is irradiated directly onto a user's eyes.

[0003] Japanese Patent Application Laid-Open No. 2006-124494 (Patent Document 1) describes an apparatus that realizes a first line density in a first portion of an image and a second line density lower than the first line density in a second portion of the image by controlling the rotation of the fast axis and the slow axis of a MEMS mirror, and determines the position of the first portion of the image based on the line of sight of the eye. As a result, the resolution of the image in the second portion that does not correspond to the line of sight is lower than the resolution of the image in the first portion that corresponds to the line of sight, thereby reducing eye fatigue for the user.

[0004] U.S. Patent No. 9,986,215

[0005] In the head-mounted display described above, a video signal obtained by capturing an image in front of the user can be used as a video signal for modulating light for generating an image. This allows the user to grasp the scenery in front of them from the captured image even if the goggles or glasses described above are not particularly see-through.

[0006] In this case, it is preferable to make the image definition of the portion corresponding to the user's line of sight different from the image definition of the other portion so that the user can view the image more comfortably.

[0007] However, because the user's line of sight can change dynamically, generating video signals of different resolutions in response to the line of sight would result in delays in display, as the video signals would not be generated in time. Such display delays would distort the image and cause discomfort to the user.

[0008] In view of the above problem, an object of the present invention is to provide an image generating device and a head-mounted display that can more smoothly display high-definition images in an area near the user's line of sight.

[0009] According to a first aspect of the present invention, an image generating device includes a camera that captures an image of a field of view and an imaging processing unit that outputs a first captured video signal for constructing a frame image with a first resolution and a second captured video signal for constructing a frame image with a second resolution lower than the first resolution, a first frame buffer that stores the first captured video signal, a second frame buffer that stores the second captured video signal, a light source that emits light for constructing the frame image, a scanning unit that scans the light emitted from the light source, a detection unit that detects a user's line of sight, and a control unit. The control unit sets a specific region in the frame image that includes at least a portion of an area toward which the user's line of sight is likely to be directed. The control unit controls the light source and the scanning unit to apply the first captured video signal from the first frame buffer to a first image region in the frame image that includes a viewpoint position corresponding to the line of sight and the specific region to generate an image, and controls the light source and the scanning unit to apply the second captured video signal from the second frame buffer to a second image region of the frame image other than the first image region and the specific region to generate an image.

[0010] According to the image generating device of this aspect, a first captured video signal stored in a first frame buffer and a second captured video signal stored in a second frame buffer are selectively used in accordance with the user's line of sight to generate one frame of image. This allows a higher-resolution image to be smoothly displayed in the first image area near the user's line of sight than in the second image area. Furthermore, the first captured video signal is also applied to a specific area including at least a portion of an area likely to be gazed at, thereby displaying a high-resolution image. Therefore, when the user subsequently gazes at this specific area, a high-resolution image can be displayed in that area without delay.

[0011] A head-mounted display according to a second aspect of the present invention comprises an image generating device according to the first aspect, a frame for holding the image generating device, and an optical system for directing light from the image generating device to the eyes of the user wearing the head-mounted display on their head.

[0012] The head-mounted display according to this aspect has the same effects as those of the first aspect. In addition, by wearing the head-mounted display on the head, the user can grasp the scenery captured by the camera, etc., from the frame image generated by the image generating device.

[0013] As described above, the present invention can provide an image generating device and a head-mounted display that can more smoothly display high-definition images in an area near the user's line of sight.

[0014] The effects and significance of the present invention will become more apparent from the following description of the embodiments, however, the embodiments shown below are merely examples of how the present invention can be implemented, and the present invention is not limited to the embodiments described below.

[0015] FIG. 1 is a perspective view schematically showing the configuration of AR glasses according to the first embodiment. FIG. 2 is a diagram schematically showing the configuration of a projection unit according to the first embodiment. FIG. 3 is a block diagram showing the configurations of a projection unit and a detection unit according to the first embodiment. FIG. 4 is a block diagram showing the configuration of a signal processing unit according to the first embodiment. FIG. 5 is a diagram schematically showing first and second captured video signals acquired by a camera according to the first embodiment. FIG. 6 is a schematic diagram for explaining thinning processing by an input processing unit according to the first embodiment. FIG. 7( a) is a diagram schematically showing generation of a frame image according to a comparative example. FIG. 7( b) is a diagram schematically showing generation of a frame image according to the first embodiment. FIG. 8 is a flowchart showing frame image generation processing performed by an image generating device according to the first embodiment. FIG. 9 is a flowchart showing details of storage processing according to the first embodiment. FIG. 10 is a block diagram showing the configuration of a signal processing unit according to a first modification of the first embodiment. FIG. 11 is a schematic diagram for explaining thinning processing by an input processing unit according to the first modification of the first embodiment. FIG. 12 is a block diagram showing the configuration of a signal processing unit according to a second modification of the first embodiment. FIG. 13 is a diagram schematically illustrating that the first image area is set to one of five areas based on the viewpoint position according to a third modification of the first embodiment. FIG. 14 is a diagram illustrating the scanning speed of the second mirror when five areas are each set as the first image area according to a third modification of the first embodiment. FIG. 15 is a block diagram illustrating the configuration of a signal processing unit according to a second embodiment. FIG. 16 is a diagram schematically illustrating generation of a frame image according to the second embodiment. FIG. 17 is a flowchart illustrating frame image generation processing performed by the image generating device according to the second embodiment. FIGS. 18(a) and 18(b) are diagrams schematically illustrating generation of a frame image according to a third embodiment. FIG. 19 is a block diagram illustrating the configurations of a projection unit and a detection unit according to another modification. FIG. 20 is a block diagram illustrating the configuration of a signal processing unit according to yet another modification.

[0016] However, the drawings are for illustrative purposes only and do not limit the scope of the present invention.

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, an example in which the present invention is applied to an image generation device for a head-mounted display is shown. Examples of head-mounted displays include AR glasses, AR goggles, VR glasses, and VR goggles. The head-mounted display in the following embodiments is AR glasses. However, the following embodiments are one embodiment of the present invention, and the present invention is not limited to the following embodiments. For example, the present invention is not limited to image generation devices for head-mounted displays, but can also be applied to image generation devices such as in-vehicle head-up displays.

[0018] First Embodiment FIG. 1 is a perspective view schematically illustrating the configuration of AR glasses 1. FIG.

[0019] 1 also shows mutually orthogonal X, Y, and Z axes in addition to the front, back, left, right, up, and down directions of the AR glasses 1. The positive X-axis direction, positive Y-axis direction, and positive Z-axis direction correspond to the rightward, backward, and upward directions of the AR glasses 1, respectively.

[0020] The AR glasses 1 include a frame 2, a pair of image generating devices 3, and a pair of mirrors 4. The AR glasses 1 are worn on the user's head, similar to ordinary eyeglasses.

[0021] The frame 2 holds a pair of image generating devices 3 and a pair of mirrors 4. The frame 2 is composed of a front portion 2a and a pair of support portions 2b. The pair of support portions 2b extend rearward from the right and left ends of the front portion 2a. When the frame 2 is worn by a user, the front portion 2a is positioned in front of a pair of eyes E of the user. The frame 2 is composed of an opaque material. The frame 2 may also be composed of a transparent material.

[0022] The pair of image generation devices 3 are symmetrical with respect to the YZ plane passing through the center of the AR glasses 1. The image generation devices 3 generate images at the eyes E of a user wearing the AR glasses 1 on their head.

[0023] The mirror 4 has a concave reflective surface and is installed on the inner surface of the front surface 2a of the frame 2. The mirror 4 almost totally reflects the light projected from the corresponding projection unit 11 and guides it to the user's eye E.

[0024] The image generating device 3 includes a projection unit 11 , a detection unit 12 , and a camera 13 .

[0025] The projection unit 11 is installed on the inner surface of the support unit 2b. The projection unit 11 projects light modulated by a video signal onto the corresponding mirror 4. The light from the projection unit 11 reflected by the mirror 4 is irradiated onto the fovea centralis, which is located at the center of the retina in the eye E. This allows the user to visually grasp the frame image 20 (see FIG. 2) generated by the image generation device 3.

[0026] The pair of detectors 12 are installed on the inner surface of the front surface 2a between the pair of mirrors 4. The detectors 12 are used to detect the line of sight of the user.

[0027] The pair of cameras 13 are installed on the outer surface of the front surface 2a in front of the pair of mirrors 4. The cameras 13 capture an image within the field of view of the cameras 13. In this embodiment, the field of view of the cameras 13 is in front of the AR glasses 1.

[0028] FIG. 2 is a diagram schematically illustrating the configuration of the projection unit 11. As shown in FIG.

[0029] The projection unit 11 includes light sources 101 , 102 , and 103 , collimator lenses 111 , 112 , and 113 , apertures 121 , 122 , and 123 , a mirror 131 , dichroic mirrors 132 and 133 , a first scanning unit 140 , a relay optical system 150 , and a second scanning unit 160 .

[0030] Light sources 101, 102, and 103 are, for example, semiconductor laser light sources. Light source 101 emits laser light with a red wavelength in the range of 635 nm to 645 nm, light source 102 emits laser light with a green wavelength in the range of 510 nm to 530 nm, and light source 103 emits laser light with a blue wavelength in the range of 440 nm to 460 nm.

[0031] In the first embodiment, a color image is generated as the frame image 20 described below, and therefore the projection unit 11 includes light sources 101, 102, and 103 capable of emitting red, green, and blue laser light. When a monochromatic image is displayed as the frame image 20, the projection unit 11 may include only one light source corresponding to the color of the image. Alternatively, the projection unit 11 may include two light sources with different emission wavelengths.

[0032] The light beams emitted from the light sources 101, 102, and 103 are converted into parallel light beams by collimator lenses 111, 112, and 113, respectively. The light beams transmitted through the collimator lenses 111, 112, and 113 are shaped into approximately circular beams by apertures 121, 122, and 123, respectively.

[0033] Mirror 131 almost totally reflects the red light that has passed through aperture 121. Dichroic mirror 132 reflects the green light that has passed through aperture 122 and transmits the red light reflected by mirror 131. Dichroic mirror 133 reflects the blue light that has passed through aperture 123 and transmits the red light and green light that have passed through dichroic mirror 132. Mirror 131 and the two dichroic mirrors 132 and 133 are arranged to align the optical axes of the light of each color emitted from light sources 101, 102, and 103.

[0034] The first scanning unit 140 reflects the light that has passed through the dichroic mirror 133. The first scanning unit 140 is, for example, a MEMS (Micro Electro Mechanical System) mirror. The first scanning unit 140 is configured to rotate a first mirror 141, onto which the light that has passed through the dichroic mirror 133 is incident, around an axis 141a parallel to the Z-axis direction in response to a drive signal. The rotation of the first mirror 141 changes the reflection direction of the light. As a result, the light reflected by the first mirror 141 is scanned along a scanning line extending in the X-axis direction on the retina of the eye E, as described below.

[0035] The relay optical system 150 directs light reflected by the first scanning unit 140 toward the center of the second mirror 161 of the second scanning unit 160. That is, light incident on the first scanning unit 140 is deflected by the first mirror 141 at a predetermined deflection angle. The relay optical system 150 directs light at each deflection angle toward the center of the second mirror 161. The relay optical system 150 also has multiple mirrors, and reflects the light reflected by the first scanning unit 140 by the multiple mirrors to direct the light toward the second scanning unit 160. This makes it possible to achieve a long optical path length inside the relay optical system 150 and reduce the deflection angle of the light when viewed from the second mirror 161.

[0036] The second scanning unit 160 reflects the light that has passed through the relay optical system 150. The second scanning unit 160 is, for example, a MEMS mirror. The second scanning unit 160 has a configuration that rotates a second mirror 161, onto which the light that has passed through the relay optical system 150 is incident, around an axis 161a parallel to the XY plane in response to a drive signal. The rotation of the second mirror 161 changes the reflection direction of the light. As a result, the scanning line on the retina of the eye E along which the light is scanned by the first scanning unit 140 is changed in the Z-axis direction, as described below.

[0037] The light reflected by the second scanning unit 160, that is, the light emitted from the projection unit 11, is reflected by the mirror 4 and forms a frame image 20 on the retina of the eye E.

[0038] FIG. 3 is a block diagram showing the configuration of the projection unit 11 and the detection unit 12. As shown in FIG.

[0039] The detection unit 12 includes a light source 12a and an image sensor 12b, and is connected to the control unit 201 of the projection unit 11. The light source 12a is, for example, an LED that emits light in the infrared wavelength range. The image sensor 12b is, for example, a CMOS image sensor or a CCD image sensor. The light source 12a irradiates light onto the user's eye E in response to an instruction from the control unit 201. The image sensor 12b captures an image of the user's eye E in response to an instruction from the control unit 201, and outputs the captured image to the control unit 201.

[0040] In response to instructions from the control unit 201, the cameras 13 capture images of their field of view, generate video signals, and output the generated video signals to the signal processing unit 300 of the corresponding projection unit 11. In Fig. 1 , the left camera 13 outputs the generated video signals to the signal processing unit 300 of the left projection unit 11, and the right camera 13 outputs the generated video signals to the signal processing unit 300 of the right projection unit 11. The cameras 13 of the first embodiment output a first captured video signal for high resolution and a second captured video signal for low resolution, as will be described later.

[0041] The camera 13 also has a function of a moving object detection unit 13a as a function of its built-in processing circuit. The moving object detection unit 13a detects a moving object region in a frame image 20 using the first captured video signal or the second captured video signal. The moving object detection unit 13a identifies a region of an object moving in a frame image (a moving object region) using a plurality of consecutive frame images 20. For example, the moving object detection unit 13a calculates the difference in the captured video signal for each pixel between a plurality of consecutive frame images 20, and identifies a region of an object moving in a frame image (a moving object region) from the calculated difference. The moving object detection unit 13a sequentially outputs information indicating the position of the moving object region in the frame image 20 to the control unit 201.

[0042] The control unit 201 may have the function of the moving object detection unit 13a. In this case, the first captured video signal or the second captured video signal is output from the camera 13 to the control unit 201. The control unit 201 detects a moving object area moving on the frame image 20 from the input captured video signal.

[0043] The projection unit 11 includes a control unit 201, a first mirror driving circuit 211, a second mirror driving circuit 212, a first mirror monitor sensor 213, a second mirror monitor sensor 214, a signal processing unit 300, a line memory 221, and a laser driving circuit 222.

[0044] The control unit 201 includes an arithmetic processing unit such as a CPU or FPGA, and a memory. The control unit 201 detects the user's line of sight based on the captured image from the detection unit 12, for example, by the dark pupil method, the bright pupil method, or the corneal reflex method. The control unit 201 acquires the viewpoint position in the frame image 20 formed on the user's retina based on the detected line of sight of the user. The control unit 201 also controls the signal processing unit 300 to process video signals from the camera 13 and external devices.

[0045] The first mirror drive circuit 211 drives the first mirror 141 of the first scanning unit 140 in response to a drive signal from the control unit 201. The second mirror drive circuit 212 drives the second mirror 161 of the second scanning unit 160 in response to a drive signal from the control unit 201.

[0046] The first mirror monitor sensor 213 is installed on the first mirror 141 and outputs a detection signal corresponding to the rotation of the first mirror 141 to the control unit 201. The second mirror monitor sensor 214 is installed on the second mirror 161 and outputs a detection signal corresponding to the rotation of the second mirror 161 to the control unit 201. Based on the detection signals from the first mirror monitor sensor 213 and the second mirror monitor sensor 214, the control unit 201 outputs drive signals to the first mirror drive circuit 211 and the second mirror drive circuit 212 so that the first mirror 141 and the second mirror 161 rotate with desired drive waveforms.

[0047] The signal processing unit 300 processes the video signals from the camera 13 and external devices to generate a video signal for one line. The configuration of the signal processing unit 300 will be described later with reference to FIG.

[0048] The line memory 221 outputs the video signal for one line output from the signal processing unit 300 to the laser driving circuit 222. The laser driving circuit 222 drives the light sources 101, 102, and 103 to emit light modulated by the video signal for one line output from the line memory 221.

[0049] FIG. 4 is a block diagram showing the configuration of the signal processing unit 300.

[0050] The signal processing unit 300 includes a first buffer 301, a second buffer 302, an input processing unit 310, a first buffer 321, a second buffer 322, a signal synthesis unit 330, a first frame buffer 341, and a second frame buffer 342.

[0051] In the first embodiment, the imaging processing unit 230 is configured by the camera 13. The camera 13 captures an image of the field of view and generates a first imaging video signal for high resolution and a second imaging video signal for low resolution. The first buffer 301 is a memory that temporarily stores the first imaging video signal output from the camera 13 (imaging processing unit 230). The second buffer 302 is a memory that temporarily stores the second imaging video signal output from the camera 13 (imaging processing unit 230).

[0052] FIG. 5 is a diagram schematically showing the first and second image pickup video signals acquired by the camera 13. As shown in FIG.

[0053] The camera 13 generates a first imaging video signal in a first imaging period set within one frame, and generates a second imaging video signal in a second imaging period different from the first imaging period within one frame. For example, the first imaging period is the first half period within one frame, and the second imaging period is the second half period within one frame, and the lengths of the first imaging period and the second imaging period are the same.

[0054] During the first imaging period, the camera 13 drives all of the light receiving units (pixels) in the camera 13 to generate a first imaging video signal for high resolution. Meanwhile, during the second imaging period, the camera 13 drives every other row of the light receiving units arranged horizontally in the camera 13 to generate a second imaging video signal for low resolution.

[0055] In Fig. 5, the video signals stored in the first buffer 301 and the second buffer 302 for each line are indicated by solid and dashed lines. Of the first captured video signals stored in the first buffer 301, the odd-numbered first captured video signals from the top are indicated by solid lines, and the even-numbered first captured video signals from the top are indicated by dashed lines. For convenience, the first buffer 301 in Fig. 5 shows 17 lines of first captured video signals, but the actual number of lines is several levels greater. In the second captured video signals stored in the second buffer 302, the dashed lines of the first buffer 301 are omitted and the second captured video signals are indicated by solid lines.

[0056] 5, the second imaged video signal in the second buffer 302 is a signal in which every other line of the first imaged video signal in the first buffer 301 is thinned out by driving each light receiving unit in the camera 13. However, the way in which the lines are thinned out is not limited to this, and for example, the second imaged video signal in the second buffer 302 may be a signal in which every three or more lines of the first imaged video signal in the first buffer 301 are thinned out by one line.

[0057] Returning to Fig. 4, the video signal from the external device is, for example, a video signal related to CG (Computer Graphics) that is superimposed on a predetermined region of the frame image 20. This video signal has the same resolution as the first captured video signal output from the camera 13. The input processing unit 310 performs thinning processing on the video signal input from the external device. The first buffer 321 is a memory that temporarily stores the video signal input from the external device, i.e., the first input video signal that has not been thinned out by the input processing unit 310. The second buffer 322 is a memory that temporarily stores the second input video signal that has been thinned out by the input processing unit 310.

[0058] FIG. 6 is a schematic diagram for explaining the thinning process performed by the input processing unit 310. In FIG.

[0059] The input processing unit 310 generates a first input video signal and a second input video signal having different resolutions from a video signal from an external device.

[0060] In Fig. 6, the video signals for each line stored in the first buffer 321 and the second buffer 322 are indicated by solid and dashed lines. Of the first input video signals stored in the first buffer 321, the odd-numbered first input video signals from the top are indicated by solid lines, and the even-numbered first input video signals from the top are indicated by dashed lines. For convenience, the first buffer 321 in Fig. 6 shows 17 lines of first input video signals, but the actual number of lines is several levels greater. The second input video signal stored in the second buffer 322 is a signal obtained by thinning out the dashed lines of the first buffer 321.

[0061] 6, the second input video signal of the second buffer 322 is a signal obtained by thinning out every other line of the first input video signal of the first buffer 321. However, the method of thinning out the lines is not limited to this. For example, the second input video signal may be a signal obtained by thinning out every three or more lines of the first input video signal of the first buffer 321. Furthermore, the second input video signal may be generated by mixing adjacent lines of the first input video signal stored in the first buffer 321. In the mixing process, for example, two adjacent lines are replaced with one line calculated as the average value of the signals of these two lines.

[0062] 4 , the signal combining unit 330 combines the first captured video signal for high resolution stored in the first buffer 301 with the first input video signal for high resolution stored in the first buffer 321 to generate a first combined video signal for high resolution for one frame. The signal combining unit 330 also combines the second captured video signal for low resolution stored in the second buffer 302 with the second input video signal for low resolution stored in the second buffer 322 to generate a second combined video signal for low resolution for one frame.

[0063] The first frame buffer 341 stores one frame of the first synthesized video signal for high resolution generated by the signal synthesis unit 330. The second frame buffer 342 stores one frame of the second synthesized video signal for low resolution generated by the signal synthesis unit 330.

[0064] In response to a control signal from the control unit 201 (see FIG. 3 ), the first frame buffer 341 sequentially outputs one line of the first composite video signal out of one frame of the stored first composite video signal to the line memory 221. In response to a control signal from the control unit 201, the second frame buffer 342 sequentially outputs one line of the second composite video signal out of one frame of the stored second composite video signal to the line memory 221. The line memory 221 receives either one line of the first composite video signal from the first frame buffer 341 or one line of the second composite video signal from the second frame buffer 342.

[0065] Next, a method for generating a frame image 20 according to a comparative example and a method for generating a frame image 20 according to the first embodiment will be described in order.

[0066] FIG. 7A is a diagram schematically showing the generation of a frame image 20 according to a comparative example.

[0067] The first scanning unit 140 scans light in the X-axis direction along the scanning line, and the second scanning unit 160 changes the scanning line in the Z-axis direction, thereby generating a frame image 20 on the retina of the user's eye E. When the scanning line is changed, the scanning positions of the first scanning unit 140 and the second scanning unit 160 are moved with the light sources 101, 102, and 103 turned off, as shown by the dotted lines in FIG. 7A. When changing the scanning line, the scanning position is moved to the beginning of the scanning line of the next row when scanning of each row of scanning lines is completed, except for the bottom row of scanning lines. When scanning of the bottom row of scanning lines is completed, the scanning position is moved to the beginning of the top row of scanning lines. In the comparative example, the frame image 20 is generated based only on a high-resolution video signal.

[0068] However, if the entire frame image 20 has a high resolution as shown in Fig. 7(a), the user's eyes will become tired easily. Therefore, in the first embodiment, the resolution (number of scanning lines) of the frame image 20 is set low in the area outside a predetermined range including the user's viewpoint position P10, as shown in Fig. 7(b). This reduces the user's eye fatigue.

[0069] However, if the resolution (number of scanning lines) is to be changed within the frame image 20, scanning must be performed using a video signal that corresponds to the changed resolution (number of scanning lines) as needed. However, because the user's viewpoint position P10 can change dynamically, if video signals for each resolution (number of scanning lines) are to be generated as needed in response to changes in the viewpoint position P10, the generation of the video signals may not be able to keep up, resulting in a delay in display. If such a display delay occurs, the frame image 20 will be distorted, causing discomfort to the user.

[0070] Therefore, in the first embodiment, as described above, a first composite video signal with a large number of scanning lines (high resolution) is stored in advance in the first frame buffer 341, and a second composite video signal with a small number of scanning lines (low resolution) is stored in the second frame buffer 342. Then, the control unit 201 switches between the first composite video signal from the first frame buffer 341 and the second composite video signal from the second frame buffer 342 based on the viewpoint position P10 and an area to which the user's gaze is likely to be directed, thereby generating the frame image 20. This makes it possible to suppress the above-mentioned display delay and realize image generation that tracks the user's viewpoint position P10.

[0071] FIG. 7B is a diagram schematically showing the generation of a frame image 20 according to the first embodiment.

[0072] In the first embodiment, the control unit 201 detects the line of sight of the user based on the captured image acquired by the detection unit 12, and acquires the viewpoint position P10 on the frame image 20 based on the detected line of sight. The control unit 201 controls the light sources 101, 102, 103, the first scanning unit 140, and the second scanning unit 160 so that an image is generated by applying a high-resolution first composite video signal from the first frame buffer 341 to a first image region R1 of a predetermined number of scanning lines that includes the viewpoint position P10 on the frame image 20.

[0073] The control unit 201 also sets a specific region R3 on the frame image 20, which includes at least a portion of a region toward which the gaze is likely to be directed. In the first embodiment, a moving object region R10 that moves on the frame image 20 is set as a region toward which the gaze is likely to be directed. As described above, the moving object region R10 is detected by the moving object detection unit 14a and output to the control unit 201. The control unit 201 controls the light sources 101, 102, 103, the first scanning unit 140, and the second scanning unit 160 so that an image is generated by applying a high-resolution first combined video signal from the first frame buffer 341 to the specific region R3 on the frame image 20, which has a predetermined number of scanning lines and includes the moving object region R10.

[0074] In addition, the control unit 201 controls the light sources 101, 102, 103, the first scanning unit 140 and the second scanning unit 160 so that an image is generated by applying a low-resolution second composite video signal from the second frame buffer 342 to the second image area R2 other than the first image area R1 and the specific area R3 on the frame image 20.

[0075] Here, the control unit 201 sets the specific region R3 to be smaller than the first image region R1. Specifically, the control unit 201 sets the width of the specific region R3 in the Z-axis direction to be smaller than the width of the first image region R1 in the Z-axis direction. More specifically, the control unit 201 sets the number of scanning lines included in the specific region R3 to be smaller than the number of scanning lines included in the first image region R1.

[0076] The width of the first image region R1 in the Z-axis direction is predetermined. The control unit 201 sets the first image region R1 so that the viewpoint position P10 is aligned with the center of this width. The width of the first image region R1 may be set taking into account the width of the range in the Z-axis direction around the viewpoint that can normally be clearly recognized by a person. For example, the width of the first image region R1 in the Z-axis direction may be set to approximately the width of the range in the Z-axis direction around the viewpoint that can normally be clearly recognized by a person.

[0077] The width of the specific region R3 in the Z-axis direction is predetermined. The control unit 201 sets the specific region R3 so that the center of gravity of the moving object region R10 is aligned with the center of this width. If the width of the moving object region R10 in the Z-axis direction is smaller than the specified width of the specific region R3, the control unit 201 may reduce the width of the specific region R3 from the specified width to the width of the moving object region R10.

[0078] It is not necessary to set a specified width for the width of the specific region R3 in the Z-axis direction. That is, the control unit 201 may dynamically change the width of the specific region R3 according to the size of the moving object region R10 so that the width of the specific region R3 approximately matches the width of the moving object region R10.

[0079] However, in this case, if the moving object region R10 becomes larger because the moving object is closer to the camera 13, the width of the specific region R3 in the Z-axis direction becomes larger than the width of the first image region R1, and the proportion of the region where the high-resolution image is displayed in the frame image 20 increases. This may reduce the effect of alleviating the user's eyestrain.

[0080] In contrast to this, by setting the specific region R3 smaller than the first image region R1 as described above, it is possible to primarily increase the resolution of the first image region R1, which is the region the user's gaze is directed at, while subsidiarily increasing the resolution of the specific region R3, which is the region the gaze is likely to be directed at next, thereby appropriately reducing the proportion of the region in which the high-resolution image is displayed in the frame image 20. This makes it possible to smoothly and quickly display a high-resolution image at the position of the user's gaze by following changes in the user's gaze while appropriately reducing eye fatigue.

[0081] When the width of the specific region R3 is dynamically changed according to the size of the moving object region R10 as described above, a predetermined upper limit may be set for the width of the specific region R3 in the Z-axis direction. For example, as with the width of the first image region R1, this upper limit may be set taking into account the width in the Z-axis direction of the range around the viewpoint that can normally be clearly recognized by a human. This prevents the specific region R3 from becoming significantly larger, thereby appropriately reducing eye fatigue in the user.

[0082] In FIG. 7(b), for convenience, about five scanning lines are shown in the first image region R1, a total of about four scanning lines are shown in the second image region R2, and about three scanning lines are shown in the specific region R3, but the actual number of scanning lines is several levels higher than this.

[0083] 7B, the first image region R1 has ranges with the same number of scanning lines above and below the viewpoint position P10, but the number of scanning lines corresponding to the upper range and the number of scanning lines corresponding to the lower range may be different from each other.

[0084] FIG. 8 is a flowchart showing the process of generating a frame image 20 performed by the image generating device 3.

[0085] The processing in steps S11 to S20 is related to the generation of a frame image 20 corresponding to one frame.

[0086] The control unit 201 performs a storage process (S11), whereby a first composite video signal for one frame of high resolution and a second composite video signal for one frame of low resolution are stored in the first frame buffer 341 and the second frame buffer 342, respectively, based on the video signal from the camera 13 and the video signal from the external device.

[0087] Fig. 9 is a flowchart showing details of the storage process in step S11 in Fig. 8. The control unit 201 controls the image capture processing unit 230 and the signal processing unit 300 to execute the process in Fig. 9.

[0088] 5, the imaging processing unit 230 (camera 13 in the first embodiment) generates a first imaging video signal for high resolution during a first imaging period, and generates a second imaging video signal for low resolution during a second imaging period. Then, the imaging processing unit 230 stores the generated first imaging video signal and second imaging video signal in the first buffer 301 and the second buffer 302, respectively (S101).

[0089] 6, the input processing unit 310 generates a first input video signal for high resolution and a second input video signal for low resolution based on a video signal input from an external device. Then, the input processing unit 310 stores the generated first input video signal and second input video signal in the first buffer 321 and the second buffer 322, respectively (S102). Note that the processes of steps S101 and S102 are performed in parallel.

[0090] The signal synthesis unit 330 synthesizes the first captured video signal stored in the first buffer 301 with the first input video signal of the same line as the first captured video signal among the first input video signals stored in the first buffer 321, to generate a first synthesized video signal for one frame of high resolution. The signal synthesis unit 330 then stores the generated first synthesized video signal in the first frame buffer 341 (S103).

[0091] The signal synthesis unit 330 synthesizes the second captured video signal stored in the second buffer 302 with the second input video signal of the same line as the second captured video signal, among the second input video signals stored in the second buffer 322, to generate a second synthesized video signal for one frame of low resolution. The signal synthesis unit 330 then stores the generated second synthesized video signal in the second frame buffer 342 (S104). Note that the processes of steps S103 and S104 are performed in parallel.

[0092] Returning to FIG. 8, the processes in steps S12 to S19 are related to the generation of one line of an image.

[0093] In parallel with the processing of steps S12 to S19, the control unit 201 controls the first mirror driving circuit 211 so that the first mirror 141 rotates repeatedly at the same period in the processing of each line, and drives the laser driving circuit 222 based on the video signal for one line input to the line memory 221.

[0094] The control unit 201 detects the user's viewpoint position P10 based on the captured image acquired by the detection unit 12 (S12). The control unit 201 sets a first image region R1 and a second image region R2 based on the viewpoint position P10 detected in step S12 (S13). Furthermore, the control unit 201 sets a specific region R3 as described above based on the detection result from the moving object detection unit 13a (S14).

[0095] The control unit 201 determines whether the current scanning line is within the first image region R1 or the specific region R3 (S15).

[0096] If the current scan line is within the first image region R1 or the specific region R3 (S15: YES), the control unit 201 causes the first frame buffer 341 to output a high-resolution first composite video signal to the line memory 221 (S16). As a result, one line of image is generated using the first composite video signal from the first frame buffer 341. In parallel with this, the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 at the first scanning speed (S17). As a result, the spacing between vertically adjacent scan lines becomes narrower, as shown in the first image region R1 or the specific region R3 in FIG. 7B.

[0097] On the other hand, if the current scan line is within the second image region R2 (S15: NO), the control unit 201 causes the second frame buffer 342 to output a low-resolution second composite video signal to the line memory 221 (S18). As a result, one line of image is generated using the second composite video signal from the second frame buffer 342. In parallel with this, the control unit 201 controls the second mirror drive circuit 212 to rotate the second mirror 161 at a second scanning speed that is faster than the first scanning speed (S19). As a result, the spacing between vertically adjacent scanning lines becomes wider, as shown in the second image region R2 in FIG. 7B.

[0098] In steps S16 and S18, the first composite video signal or the second composite video signal is output to the line memory 221 so that, at the start of scanning in steps S17 and S19, the first composite video signal or the second composite video signal corresponding to that scan line is held in the line memory 221. That is, when the thinning process shown in FIG. 6 is performed, the output frequency of the first composite video signal to the line memory 221 is about twice the output frequency of the second composite video signal to the line memory 221.

[0099] The control unit 201 determines whether or not image generation for one frame has been completed (S20). If image generation for one frame has not been completed (S20: NO), the process returns to step S12, and steps S12 to S19 are performed again. When image generation for one frame is completed in this manner (S20: YES), the process of Fig. 8 ends. By repeating the process of Fig. 8, frame images 20 are continuously generated.

[0100] If the specific region R3 set in step S14 overlaps the first image region R1 in part or in whole, the specific region R3 and the first image region R1 are merged into a single region. In this case, step S15 determines whether the current scanning line is within the merged region, and steps S16 and S17 are performed while the scanning line is within the merged region.

[0101] <Effects of First Embodiment> According to the first embodiment, the following effects are achieved.

[0102] The imaging processing unit 230 outputs a first imaging video signal for constructing a frame image 20 with a high resolution (first definition) and a second imaging video signal for constructing a frame image 20 with a low resolution (second definition). The first frame buffer 341 stores the first composite video signal (first imaging video signal), and the second frame buffer 342 stores the second composite video signal (second imaging video signal). The control unit 201 sets a specific region R3 in the frame image 20 that includes at least a portion of an area to which the gaze is likely to be directed (moving object region R10). The control unit 201 controls the light sources 101, 102, 103, the first scanning unit 140, and the second scanning unit 160 so that images are generated by applying the first composite video signal from the first frame buffer 341 to the first image region R1 and the specific region R3. In addition, the control unit 201 controls the light sources 101, 102, 103, the first scanning unit 140 and the second scanning unit 160 so that an image is generated in the second image region R2 by applying the second composite video signal from the second frame buffer 342.

[0103] With this configuration, one frame of image is generated by selectively using the first captured video signal stored in the first frame buffer 341 and the second captured video signal stored in the second frame buffer 342 according to the user's line of sight. This allows a higher-definition (higher resolution) image to be smoothly displayed in the first image region R1 near the user's line of sight than in the second image region R2. Furthermore, the first captured video signal is also applied to a specific region R3 including at least a portion of the moving object region R10, which is likely to be the target of the user's line of sight, to display a high-definition (high-resolution) image. Therefore, when the user subsequently directs their gaze toward this specific region R3, a high-definition image can be displayed in that region without delay.

[0104] As shown in Figure 3, the image generating device 3 is equipped with a moving object detection unit 13a that detects a moving object region R10 that moves on the frame image, and as shown in Figure 7 (b), the control unit 201 sets a specific region R3 in the detected moving object region R10 as a region that is likely to be directed at.

[0105] When a user sees movement in a position other than the user's line of sight, the user is likely to shift their gaze to that position. With this configuration, the specific region R3 is set to include at least a portion of the moving object region R10, so a high-resolution image can be displayed in advance in the moving object region R10, which is the region where the user's gaze is likely to be directed next. Therefore, when the user's gaze is actually directed to the moving object region R10 in response to movement, a high-resolution image can be displayed in that region without delay.

[0106] As shown in FIG. 3, the moving object detector 13 a is disposed in the camera 13 that acquires frame images for display, and detects a moving object region R10 from the image signal captured by the camera 13 .

[0107] According to this configuration, the camera 13 is used both to acquire frame images for display and to detect the moving object region R10, which allows the configuration of the image generating device 3 to be simplified.

[0108] As shown in FIG. 7B, the control unit 201 sets the specific region R3 to be smaller than the first image region R1.

[0109] This allows the first image region R1, which the user's gaze is directed at, to be primarily made high-resolution, while the specific region R3, which the gaze is likely to be directed at next, to be made high-resolution secondarily, thereby appropriately reducing the proportion of the region where the high-resolution image is displayed in the frame image 20. Therefore, it is possible to smoothly and quickly display a high-resolution image at the position of the user's gaze by following changes in the user's gaze while appropriately reducing eye fatigue.

[0110] The frame image 20 of the first image region R1 and the specific region R3 has a high resolution (first resolution), while the frame image 20 of the second image region R2 has a low resolution (second resolution). This resolution is determined by the scanning speed of the second scanning unit 160 in steps S17 and S19 of FIG. 8.

[0111] With this configuration, a high-resolution frame image 20 is displayed in the first image region R1 near the user's line of sight, and a low-resolution frame image 20 is displayed in the second image region R2 around the user's line of sight. This reduces eye fatigue for the user. Furthermore, even if the user has poor eyesight, the user can clearly grasp the subject or scenery by referring to the high-resolution frame image 20 in the first image region R1. Furthermore, because a high-resolution frame image 20 is displayed in the specific region R3, the user can immediately and clearly grasp the subject or scenery the next time he or she moves his or her line of sight to this specific region R3.

[0112] As shown in Figure 5, camera 13 outputs a first imaging video signal corresponding to high resolution during a first imaging period within one frame period, and outputs a second imaging video signal corresponding to low resolution during a second imaging period different from the first imaging period within one frame period.

[0113] According to this configuration, the first imaged video signal with high resolution and the second imaged video signal with low resolution can be generated smoothly.

[0114] Based on a video signal from an external device, the input processing unit 310 outputs a first input video signal for constructing a high-resolution (first definition) frame image 20 and a second input video signal for constructing a low-resolution (second definition) frame image 20. The signal combining unit 330 stores a first combined video signal, obtained by combining the first captured video signal and the first input video signal, in a first frame buffer 341, and stores a second combined video signal, obtained by combining the second captured video signal and the second input video signal, in a second frame buffer 342.

[0115] According to this configuration, the video signal from the imaging processing unit 230 and the video signal from the external device can be combined to generate the frame image 20. This allows a CG image or the like based on the video signal to be added to the image captured by the camera 13 and displayed, allowing the user to grasp the additional information based on the video signal.

[0116] As shown in Figure 1, the mirror 4 (optical system) guides the light scanned by the first scanning unit 140 and the second scanning unit 160 (scanning unit) to the eyes E of a user wearing AR glasses 1 (head-mounted display) on their head.

[0117] According to this configuration, by wearing the AR glasses 1 on the head, the user can grasp the scenery captured by the camera 13 through the frame image 20 generated by the image generation device 3.

[0118] <Modification 1 of Embodiment 1> In Embodiment 1, camera 13 outputs both the first captured video signal and the second captured video signal, but this is not limited to this. Camera 13 may output one type of video signal, and a processing unit located downstream of camera 13 may generate video signals with different resolutions.

[0119] FIG. 10 is a block diagram showing the configuration of a signal processing unit 300 according to this modification.

[0120] 4, the signal processing unit 300 in Fig. 10 includes an input processing unit 350 between the camera 13 and the first and second buffers 301 and 302. The imaging processing unit 230 in this modified example is made up of the camera 13 and the input processing unit 350.

[0121] The camera 13 of this modified example outputs only a high-resolution video signal similar to the first captured video signal of embodiment 1. The input processing unit 350 performs a thinning process on the video signal input from the camera 13 similar to the thinning process performed by the input processing unit 310. The first buffer 301 temporarily stores the first captured video signal output from the camera 13 that has not been thinned out by the input processing unit 350. The second buffer 302 temporarily stores the second captured video signal that has been thinned out by the input processing unit 350.

[0122] FIG. 11 is a schematic diagram for explaining the thinning process by the input processing unit 350. In FIG.

[0123] The input processing unit 350 generates, from the video signal from the camera 13, a first captured video signal and a second captured video signal having different resolutions.

[0124] In Fig. 11, the video signals for each line stored in the first buffer 301 and the second buffer 302 are indicated by solid and dashed lines. Of the first imaged video signals stored in the first buffer 301, the odd-numbered first imaged video signals from the top are indicated by solid lines, and the even-numbered first imaged video signals from the top are indicated by dashed lines. For convenience, the first buffer 301 in Fig. 11 shows 17 lines of first imaged video signals, but the actual number of lines is several levels greater. The second imaged video signal stored in the second buffer 302 is a signal obtained by thinning out the dashed lines from the first buffer 301.

[0125] 11 , the second imaged video signal in the second buffer 302 is a signal obtained by thinning out every other line of the first imaged video signal in the first buffer 301. However, the manner of thinning out the lines is not limited to this. For example, the second imaged video signal may be a signal obtained by thinning out every three or more lines of the first imaged video signal in the first buffer 301. The second imaged video signal may also be generated by mixing adjacent lines of the first imaged video signal stored in the first buffer 301. In the mixing process, for example, two adjacent lines are replaced with one line calculated as the average value of the signals of these two lines.

[0126] <Effects of Modification 1 of Embodiment 1> The camera 13 outputs a first imaging video signal corresponding to a high resolution, and the imaging processing unit 230 includes an input processing unit 350 that thins out or mixes the first imaging video signal of high resolution to generate a second imaging video signal corresponding to a low resolution.

[0127] According to this configuration, the camera 13 only needs to output one type of first imaging video signal, thereby simplifying the configuration and processing of the camera 13. Furthermore, in a configuration in which the first imaging video signal and the second imaging video signal are obtained in two different imaging periods as shown in the first embodiment, if the subject moves at high speed, the position of the subject will differ between the two types of video signals. However, according to this modified example, because the camera 13 only outputs one type of first imaging video signal, the position of the subject will be the same between the first imaging video signal and the second imaging video signal generated by the input processing unit 350, even if the subject moves at high speed. This avoids the user feeling uncomfortable due to the first imaging video signal and the second imaging video signal.

[0128] <Modification 2 of Embodiment 1> In the first embodiment, both the video signal from the camera 13 and the video signal from the external device are input to the signal processing unit 300, but as will be described below, only the video signal from the camera 13 may be input to the signal processing unit 300.

[0129] FIG. 12 is a block diagram showing the configuration of a signal processing unit 300 according to this modification.

[0130] 4, the signal processing unit 300 in Fig. 12 does not include the input processing unit 310, the first buffer 321, the second buffer 322, and the signal combining unit 330. The first captured video signal from the camera 13 is temporarily stored in the first buffer 301, and then one frame of the first captured video signal is output to the first frame buffer 341. The second captured video signal from the camera 13 is temporarily stored in the second buffer 302, and then one frame of the second captured video signal is output to the second frame buffer 342.

[0131] In this configuration, the frame image generation process is the same as that shown in FIG.

[0132] Effect of Modification 2 of First Embodiment The first frame buffer 341 stores the first captured video signal from the first buffer 301, and the second frame buffer 342 stores the second captured video signal from the second buffer 302. Images are generated for the first image region R1 and the specific region R3 by applying the first captured video signal from the first frame buffer 341, and an image is generated for the second image region R2 by applying the second captured video signal from the second frame buffer 342.

[0133] According to this configuration, one frame of image is generated by selectively using the first captured video signal stored in the first frame buffer 341 and the second captured video signal stored in the second frame buffer 342 in accordance with the user's line of sight. Therefore, as in the first embodiment, a higher-resolution image than the second image region R2 can be smoothly displayed in the first image region R1 near the user's line of sight. Furthermore, the first captured video signal is also applied to a specific region R3 including at least a portion of the moving object region R10, which is likely to be the target of the user's line of sight, to display a high-resolution image. Therefore, when the user subsequently directs their gaze toward this specific region R3, a high-resolution image can be displayed in that region without delay.

[0134] In this modified example, as in modified example 1 shown in Figure 10, one type of video signal may be output from camera 13, and input processing unit 350 may be arranged between camera 13 and first buffer 301 and second buffer 302.

[0135] <Modification 3 of Embodiment 1> In Embodiment 1, the first image area R1 is set based on the viewpoint position P10 based on the user's line of sight, but this is not limited to this, and the first image area R1 may be fixed in advance.

[0136] FIG. 13 is a diagram showing, in this modified example, a schematic diagram illustrating that the first image region R1 is set to one of five regions R11 to R15 based on the viewpoint position P10.

[0137] When viewpoint position P10 is included in viewpoint regions R01 to R05 in frame image 20, control unit 201 sets regions R11 to R15 as the first image region R1, respectively. Viewpoint regions R01 to R05 are obtained by dividing frame image 20 into five regions in the vertical direction (Z-axis direction). Regions R11 to R15 are regions set corresponding to viewpoint regions R01 to R05, and each region includes a predetermined number of scanning lines.

[0138] FIG. 14 is a diagram showing the scanning speed of the second mirror 161 when five regions R11 to R15 are each set in the first image region R1 according to this modified example.

[0139] When regions R11 to R15 are set as the first image region as shown in the upper part of Fig. 14, the scanning speed of second mirror 161 is set as shown in the graph in the lower part of Fig. 14. The graph in the lower part of Fig. 14 shows the scanning speed of second mirror 161 for scanning one line. In each of the five graphs in the lower part of Fig. 14, the speed of second mirror 161 is slower in regions R11 to R15. As a result, when five regions R11 to R15 are set as the first image region R1, the image resolution is increased in the ranges corresponding to each of regions R11 to R15.

[0140] As for the specific region R3, as described above, among the multiple selection candidates for the specific region R3 set in advance, the selection candidate that includes the center of gravity of the moving object region R10 may be set as the specific region R3 for displaying a high-resolution image.

[0141] In this modified example, similarly to the first embodiment, the area other than the first image area R1 and the specific area R3 is set as the second image area R2.

[0142] <Effects of Modification 3 of Embodiment 1> The control unit 201 sets the area including the viewpoint position P10 as the first image area R1 from among multiple areas R11 to R15 that are formed by dividing the frame image 20 in advance in a direction intersecting the scanning line (Z-axis direction).

[0143] With this configuration, multiple regions R11 to R15 are prepared in advance, making it possible to smoothly set the first image region R1 that includes the viewpoint position P10. When the same process is performed on the specific region R3, it is possible to smoothly set the specific region R3 that includes the center of gravity of the moving object region R10.

[0144] In the first modification of the first embodiment, the input processing units 350 and 310 output two types of video signals, one for high resolution and one for low resolution. In contrast, in the second embodiment, the input processing units 350 and 310 output video signals for high gradation and one for low gradation.

[0145] FIG. 15 is a block diagram showing the configuration of a signal processing unit 300 according to the second embodiment.

[0146] The signal processing unit 300 in FIG. 15 differs from the first modification of the first embodiment in FIG. 10 in that the two types of video signals output from the input processing units 350 and 310 are different.

[0147] The input processing unit 350 outputs the first imaging video signal for high gradation output from the camera 13 as is to the first buffer 301, and outputs a second imaging video signal generated by reducing the gradation of the first imaging video signal for high gradation output from the camera 13 to the second buffer 302. The first imaging video signal output from the camera 13 is, for example, a video signal in which shading is expressed in 256 gradations, and the second imaging video signal generated by the gradation reduction by the input processing unit 350 is, for example, a two-gradation video signal in which the gradation is reduced to two using a threshold value. Both the first imaging video signal and the second imaging video signal are video signals for high resolution.

[0148] Input processing unit 310 outputs a first input video signal for high gradation output from an external device directly to first buffer 321, and outputs a second input video signal generated by reducing the gradation of the first input video signal for high gradation output from the external device to second buffer 322. The first input video signal output from the external device is, for example, a video signal expressed in 256 gradations, and the second input video signal generated by the gradation reduction by input processing unit 310 is, for example, a two-level video signal reduced to two gradations using a threshold value. Both the first input video signal and the second input video signal are video signals for high resolution.

[0149] The signal combining unit 330 combines the first captured video signal for high gradation stored in the first buffer 301 with the first input video signal for high gradation stored in the first buffer 321 to generate a first combined video signal for high gradation for one frame. The signal combining unit 330 also combines the second captured video signal for low gradation stored in the second buffer 302 with the second input video signal for low gradation stored in the second buffer 322 to generate a second combined video signal for low gradation for one frame.

[0150] FIG. 16 is a diagram schematically illustrating generation of a frame image 20 according to the second embodiment.

[0151] The control unit 201 sets a first image region R31 of a predetermined size that includes the viewpoint position P10. As in the first embodiment, the size of the first image region R1 may be set taking into account the range around the viewpoint that can usually be clearly recognized by a person.

[0152] Furthermore, the control unit 201 sets the specific region R33 so as to include at least a portion of the moving object region R10 detected by the moving object detection unit 13a. For example, the specific region R33 is set so that the center of gravity of the moving object region R10 is aligned with the center. As in the first embodiment, the specific region R33 is set smaller than the first image region R31. In the example of Fig. 16, the size of the specific region R33 is fixed to a specified size. If the moving object region R10 is smaller than the specified size of the specific region R33, the specific region R33 may be reduced to a size that inscribes the moving object region R10.

[0153] The size of the specific region R33 may also change depending on the size of the moving object region R10. In this case, as described in the first embodiment, it is preferable to set an upper limit to the change in size of the specific region R33. In this case, the upper limit may be set, for example, taking into account the range around the viewpoint that can be clearly recognized by a person.

[0154] The control unit 201 applies the first composite video signal from the first frame buffer 341 to the first image region R31 including the viewpoint position P10 and the specific region R33. The control unit 201 also applies the second composite video signal from the second frame buffer 342 to the second image region R32 other than the first image region R31 and the specific region R33.

[0155] FIG. 17 is a flowchart showing the process of generating a frame image 20 performed by the image generating device 3 according to the second embodiment.

[0156] In the process of Figure 17, steps S15 to S19 are replaced with steps S21 to S23 compared to the process of embodiment 1 shown in Figure 8. In the process of Figure 17, the second mirror 161 is rotated in the negative direction of the Z axis at a constant scanning speed. This scanning speed is the same as the scanning speed when a high-resolution image is displayed in embodiment 1. Below, we will explain the process that differs from Figure 8.

[0157] The control unit 201 determines whether the current scanning line is included only in the second image region R32 (S21).

[0158] If the current scan line is included only in the second image region R32 (S21: YES), the control unit 201 outputs the second composite video signal from the second frame buffer 342 to the line memory 221 (S22). As a result, one line of image is generated by the second composite video signal for low gradation.

[0159] On the other hand, if the current scanning line is included in the first image area R31, the second image area R32, and the specific area R33 (S21: NO), the control unit 201 selectively outputs the first composite video signal of the first frame buffer 341 or the second composite video signal of the second frame buffer 342 to the line memory 221 depending on the positions of the first image area R31 and the specific area R33 (S23).

[0160] That is, for addresses on the line memory 221 corresponding to the first image region R31 and the specific region R33, the control unit 201 causes the first frame buffer 341 to output the first composite video signal corresponding to these addresses. Also, for addresses on the line memory 221 corresponding to the second image region R32, the control unit 201 causes the second frame buffer 342 to output the second composite video signal corresponding to this address.

[0161] As a result, the image of the first image region R31 and the specific region R33 of one line of the image is generated by the first composite video signal for high gradation, and the image of the second image region R32 of one line of the image is generated by the second composite video signal for low gradation.

[0162] Through this processing, a high-gradation image is displayed in the first image area R31 near the viewpoint position P10 and in the specific area R33 to which the gaze is likely to be directed, and a low-gradation image is displayed in the second image area R32 other than the first image area R31 and the specific area R33.

[0163] Effect of Second Embodiment The imaging processing unit 230 outputs a first imaging video signal for constructing a frame image 20 with high gradation (first definition) and a second imaging video signal for constructing a frame image 20 with low gradation (second definition). The first frame buffer 341 stores the first composite video signal (first imaging video signal), and the second frame buffer 342 stores the second composite video signal (second imaging video signal). The frame image 20 in the first image region R31 has high gradation (first definition, first gradation), and the frame image 20 in the second image region R32 has low gradation (second definition, second gradation). In this case, the first gradation and the second gradation define the luminance resolution of the first imaging video signal and the second imaging video signal, respectively.

[0164] With this configuration, a high-gradation frame image 20 is displayed in the first image region R31 near the user's line of sight, and a low-gradation frame image 20 is displayed in the second image region R32 around the user's line of sight. This reduces eye fatigue for the user. Even if the user has poor eyesight, the user can clearly grasp the subject by referring to the high-gradation frame image 20 in the first image region R31. Furthermore, the first captured video signal is also applied to a specific region R3 that includes at least a portion of the moving object region R10, which is likely to be the subject's line of sight, to display a high-definition (high-gradation) image. Therefore, when the user subsequently directs their gaze toward this specific region R3, a high-definition image can be displayed in that region without delay.

[0165] The camera 13 outputs a first imaging video signal corresponding to a high gradation (first gradation), and the imaging processing unit 230 has an input processing unit 350 that performs a gradation reduction process on the high-gradation first imaging video signal to generate a second imaging video signal corresponding to a low gradation (second gradation).

[0166] According to this configuration, high-gradation and low-gradation video signals can be generated smoothly.

[0167] The input processing unit 350 may be integral with the camera 13. The video signals from the external device may be a first input video signal for high gradation and a second input video signal for low gradation. In this case, the input processing unit 310 is omitted.

[0168] Also, similar to the second modification of the first embodiment, the configuration for processing video signals from an external device, i.e., the input processing unit 310, the first buffer 321, the second buffer 322, and the signal synthesis unit 330, may be omitted from the configuration of FIG. 15 .

[0169] Third Embodiment In the first and second embodiments, the moving object region R10 is exemplified as a region to which the gaze is likely to be directed, but the region to which the gaze is likely to be directed is not limited to this.

[0170] For example, a region of interest that the user is likely to pay attention to may be set in advance as a region to which the user's gaze is likely to be directed. In this case, the control unit 201 sets a specific region to include the region of interest. The region of interest is, for example, near the bottom edge of the frame image 20. When walking while wearing the AR glasses 1, the user is particularly concerned about whether there are any obstacles or the like under their feet. For this reason, the user occasionally shifts their gaze to their feet while looking at the image of the scenery ahead.

[0171] 18(a) and (b) are diagrams schematically showing the generation of a frame image 20 according to embodiment 3. Fig. 18(a) shows a scanning configuration in which a specific region R3 is set near the bottom edge of the frame image 20 in the configuration of embodiment 1, and Fig. 18(b) shows a scanning configuration in which a specific region R3 is set near the bottom edge of the frame image 20 in the configuration of embodiment 2. In this case, the region near the bottom edge of the frame image 20 is the region of interest, and this region of interest is set as is as the specific region R3.

[0172] In the example of Fig. 18(a), a specific region R3 is set near the bottom edge of the frame image 20, so the scanning lines are denser near this bottom edge, increasing the image resolution. Also, in the example of Fig. 18(b), a specific region R33 is set near the bottom edge of the frame image 20, so the gradation is increased near this bottom edge, increasing the image definition.

[0173] Advantages of Third Embodiment As described above, in the third embodiment, a region of interest (here, near the bottom edge of the frame image 20) that the user is likely to focus their attention on is set in advance in the control unit 201, and the control unit 201 sets specific regions R3 and R33 so as to include the region of interest. As a result, when the user shifts their gaze to the region of interest, they can immediately see a high-resolution image near the region of interest. In the examples of Figures 18(a) and 18(b), if the user occasionally shifts their gaze to the ground while looking at an image of the scenery ahead, they can immediately see a high-resolution image near the ground.

[0174] In the examples of FIGS. 18A and 18B, since there is no need to detect the moving object region R10, the moving object detector 13a is omitted from the configuration of FIG.

[0175] 18(a) and 18(b) , regions such as traffic lights and traffic signs may be set as regions of interest. In this case, the control unit 201 is provided with a function for extracting regions such as traffic lights and traffic signs from the video signal from the camera 13. Using this function, the control unit 201 identifies these regions of interest on the frame image 20 and sets the specific regions R3 and R33 so as to include at least a portion of these regions. For example, the control unit 201 sets the specific regions R3 and R33 so that the centers of gravity of the identified regions of interest are aligned with the centers of the specific regions R3 and R33.

[0176] In the third embodiment, the frame image generation process is similar to the process shown in Fig. 8 or 17. In the third embodiment, the specific regions R3 and R33 are preferably set smaller than the first image region R1.

[0177] <Other Modifications> In the above-described first and second embodiments, the camera 13 used to create the frame image for display has the function of the moving object detection unit 13a, but a camera for acquiring the moving object region R10 may be provided separately from the camera 13.

[0178] FIG. 19 is a block diagram showing the configuration of the projection unit and the detection unit in this case.

[0179] In this configuration example, a camera 14 is added to the configuration in Fig. 3. The camera 14 has the function of a moving object detector 14a. The function of the moving object detector 14a is the same as the function of the moving object detector 13a in Fig. 3.

[0180] Camera 14 is placed close to camera 13 and is facing the same direction as camera 13. Therefore, the field of view of camera 14 substantially overlaps with the field of view of camera 13. For this reason, the position of the moving object region detected by moving object detection unit 14a on the frame image is substantially the same as the position of the moving object region on the frame image on the camera 13 side, and the former position can be used as the latter position. If there is a discrepancy between these two positions, the former position may be converted into the latter position, and the latter position may be obtained from the former position.

[0181] The control unit 201 acquires a moving object region R10 on the frame image 20 on the camera 13 side from the moving object region detected by the moving object detection unit 14a. When these two moving object regions substantially overlap as described above, the control unit 201 treats the moving object region detected by the moving object detection unit 14a as the moving object region R10 on the frame image 20. Then, as in the first and second embodiments, the control unit 201 sets specific regions R3 and R33 so as to include at least a portion of the moving object region R10. The frame image generation process is similar to the process in FIG. 8 or FIG. 17 .

[0182] This modification also provides the same effects as those of the first and second embodiments.

[0183] Furthermore, according to this modification, the function of the moving object detection unit 14a is provided in the other camera 14, so the camera 13 only needs to perform processing to capture an image for display and output it to the signal processing unit 300. Therefore, the video signal for display can be smoothly output to the signal processing unit 300.

[0184] Furthermore, in the above embodiments 1 to 3, the first frame buffer 341 for high definition and the second frame buffer 342 for low definition are arranged in the signal processing unit 300, but the number of frame buffers that hold video signals of different definitions is not limited to this, and three or more frame buffers may be arranged in the signal processing unit 300.

[0185] For example, as shown in FIG. 20, three frame buffers 371 to 373 may be arranged in the signal processing unit 300, each of which holds three types of composite video signals: high definition, medium definition, and low definition.

[0186] The input processing unit 350 outputs three types of captured video signals: high definition, medium definition, and low definition.

[0187] When definition is defined by resolution as in the first embodiment, for example, high definition (high resolution) is the resolution with the number of lines on the left side of Fig. 11, medium definition (medium resolution) is the resolution with the number of lines on the right side of Fig. 11, and low definition (low resolution) is the resolution obtained by further thinning out every other line from the resolution on the right side of Fig. 11. The medium resolution here corresponds to the low resolution in the case of Fig. 10.

[0188] Furthermore, when the definition is defined by the gradation as in the second embodiment, for example, high definition (high gradation) is 256 gradations, which are the gradations of the video signal output from the camera 13, medium definition (medium gradation) is 64 gradations, which is ¼ of 256 gradations, and low definition (low gradation) is 2 gradations. The low gradation here corresponds to the low gradation in the second embodiment.

[0189] Three buffers 351 to 353 for holding the image pickup video signals of the three different resolutions are arranged in front of the signal synthesis unit 330 .

[0190] The input processing unit 310 outputs three types of input video signals: high definition, medium definition, and low definition. These three types of definition are the same as the three types of captured video signals output by the input processing unit 350. Three buffers 361 to 363 that hold the input video signals of these definitions, respectively, are arranged before the signal combining unit 330.

[0191] The signal synthesis unit 330 acquires the captured video signal and the input video signal of the same resolution from the corresponding buffers, synthesizes the acquired video signals, and outputs the generated synthesized video signal to the corresponding frame buffer of the resolution among the three subsequent frame buffers 371 to 373.

[0192] In this case, the resolution applied to the first image region R1 and the specific region R3 and the resolution applied to the second image region R2 are set in advance by the user. The user can set the two resolutions so that the resolution applied to the first image region R1 and the specific region R3 is higher than the resolution applied to the second image region R2.

[0193] In accordance with this setting, the control unit 201 selects two frame buffers to be used for image display from the three frame buffers 371 to 373. The control unit 201 then performs the processing of Fig. 8 or Fig. 17 using the two selected frame buffers as the first and second frame buffers to generate a frame image. If the definition is the resolution, the first and second scanning speeds in steps S17 and S19 of Fig. 8 are adjusted to correspond to the definition (resolution) selected by the user.

[0194] For example, if the user selects high definition as the first resolution to be applied to the first image region R1 and the specific region R3, and selects low definition as the second resolution to be applied to the second image region R2, the control unit 201 performs the processing of Figure 8 or Figure 17 using the composite video signals stored in the frame buffers 371 and 373 as the first composite video signal and the second composite video signal, respectively.

[0195] In this case, frame buffers 371 and 373 correspond to the first and second frame buffers, respectively, and the captured video signals output from input processing unit 350 to buffers 351 and 353 correspond to the first and second captured video signals, respectively. Also, the input video signals output from input processing unit 310 to buffers 361 and 363 correspond to the first and second input video signals, respectively.

[0196] 20, as in the configuration of FIG. 12, the processing units for displaying an image based on an input video signal, that is, input processing unit 310, buffers 361 to 363, and signal synthesis unit 330 may be omitted.

[0197] Furthermore, in the above-described first and second embodiments, the region of a moving object moving in the vertical and horizontal directions (directions parallel to the X-Y plane) on the frame image 20 is detected as the moving object region R10. However, the region of a moving object moving in the forward-backward direction (Z-axis direction) may also be detected as the moving object region R10. In this case, the image generating device 3 includes a means for detecting the forward-backward distance for each position (e.g., pixel by pixel) within the field of view, such as a time-of-flight (TOF) camera. The TOF camera may be of a so-called flash type or a scanning type. In this case, as in the modified example of FIG. 19 , the TOF camera should be positioned close to the camera 13 so that its field of view substantially overlaps with the field of view of the camera 13.

[0198] In addition, in the above-described embodiment 2, the gradation of the composite video signal is changed between the first image region R1 and the specific region R3 and the second image region R2, but the resolution may be changed between the first image region R1 and the specific region R3 and the second image region R2 as in embodiment 1. Furthermore, the configuration of embodiment 3 may be added to the configurations of embodiments 1 and 2 to increase the definition of both the specific region including the moving object region and the specific region including the region of interest.

[0199] In the first modification of the first embodiment, the input processing unit 350 performs thinning and mixing processes on the input first captured video signal for high resolution to generate the second captured video signal for low resolution, as shown in Fig. 11. However, when the second captured video signal for low resolution is input from the camera 13, the input processing unit 350 may perform interpolation processes on the input second captured video signal for low resolution to generate the first captured video signal for high resolution. However, because the interpolation process requires a higher processing load than the thinning and mixing processes, it is preferable that the video signal input to the input processing unit 350 be the first captured video signal for high resolution.

[0200] In the second embodiment, the first image region R31 and the second image region R32 are set based on the viewpoint position P10, but this is not limiting, and the first image region R1 and the second image region R2 may be set based on the viewpoint position P10, as in the first embodiment shown in Fig. 7B. Furthermore, in the second embodiment, a plurality of regions R11 to R15 prepared in advance may be set based on the viewpoint position P10, as in the third modification of the first embodiment.

[0201] In the second embodiment, the input processing units 350 and 310 perform processing to reduce the number of gradations of the video signal to two gradations, but this is not limited to this, and the input processing units 350 and 310 may also perform processing to reduce the number of gradations of the video signal to a number other than two gradations (for example, 16 gradations).

[0202] In the above embodiment and modified example, the field of view of the camera 13 is in front of the AR glasses 1, but this is not limited thereto and may be above, below, or behind the AR glasses 1.

[0203] In the above embodiment and modified example, two sets of image generating devices 3 and mirrors 4 are provided in the AR glasses 1 to correspond to the user's pair of eyes E, but only one set may be provided in the AR glasses 1 to correspond to only one of the user's eyes E.

[0204] In the above embodiment and modified example, the light scanned by the first scanning unit 140 and the second scanning unit 160 is directed to the user's eye E via the mirror 4, but this is not limiting and the light may be directed to the user's eye E via an optical system other than a mirror (for example, a lens, etc.). In this case, the optical system may be, for example, a combination of multiple mirrors, a combination of a mirror and a lens, or a combination of multiple lenses.

[0205] In the above embodiment and modified examples, the first mirror 141 and the second mirror 161 are provided separately, but instead of the first mirror 141 and the second mirror 161, a single mirror that rotates about two axes may be provided.

[0206] Furthermore, the configurations of the image generating device 3 and the AR glasses 1 (head-mounted display) can be modified in various ways in addition to the configurations shown in the above-described embodiment and modified examples. Various modifications are possible as appropriate within the scope of the technical ideas shown in the scope of claims.

[0207] (Additional Notes) The above description of the embodiments discloses the following techniques.

[0208] (Technology 1) An image generating device comprising: an imaging processing unit that includes a camera that captures an image of a field of view, and that outputs a first captured video signal for constructing a frame image of a first resolution and a second captured video signal for constructing a frame image of a second resolution lower than the first resolution; a first frame buffer that stores the first captured video signal; a second frame buffer that stores the second captured video signal; a light source that emits light for constructing the frame image; a scanning unit that scans the light emitted from the light source; a detection unit that detects a user's line of sight; and a control unit, wherein the control unit sets a specific area on the frame image that includes at least a part of an area toward which the line of sight is likely to be directed, controls the light source and the scanning unit so that an image is generated by applying the first captured video signal from the first frame buffer to a first image area and the specific area that includes a viewpoint position on the frame image that corresponds to the line of sight, and controls the light source and the scanning unit so that an image is generated by applying the second captured video signal from the second frame buffer to a second image area of ​​the frame image other than the first image area and the specific area.

[0209] According to this technology, a first captured video signal stored in a first frame buffer and a second captured video signal stored in a second frame buffer are selectively used in accordance with the user's line of sight to generate one frame of image. This allows a higher-resolution image to be smoothly displayed in a first image area near the user's line of sight than in a second image area. Furthermore, the first captured video signal is also applied to a specific area including at least a portion of an area likely to be gazed at, thereby displaying a high-resolution image. Therefore, when the user subsequently gazes at this specific area, a high-resolution image can be displayed in that area without delay.

[0210] (Technology 2) The image generating device according to Technology 1 further includes a moving object detection unit that detects a moving object area on the frame image, and the control unit sets the specific area as an area to which the gaze is likely to be directed, based on the detected moving object area.

[0211] When a user sees movement in a position other than the user's line of sight, the user is likely to shift their gaze to that position. According to Technology 2, a specific region is set to include at least a portion of the moving object region, so a high-resolution image can be displayed in advance in the moving object region where the user's gaze is likely to be directed next. Therefore, when the user actually directs their gaze to a moving object region in response to the movement, a high-resolution image can be displayed in that region without delay.

[0212] (Technology 3) In the image generating device according to Technology 2, the moving object detection unit detects the moving object region from an image pickup video signal of the camera.

[0213] According to this technology, a camera is used both to acquire frame images for display and to detect moving object regions, which simplifies the configuration of the image generation device.

[0214] (Technology 4) The image generating device according to Technology 2, further comprising another camera separate from the camera, wherein the moving object detection unit detects the moving object region from an image signal captured by the other camera.

[0215] According to this technology, the function of the moving object detection unit is allocated to another camera, so the camera only needs to capture and output images for display, allowing for smooth output of video signals for display.

[0216] (Technology 5) In the image generating device described in any one of technologies 1 to 4, an area of ​​interest that a user is likely to focus their attention on is set in advance, and the control unit sets the specific area to include the area of ​​interest.

[0217] According to this technology, a specific area is set to include a region of interest, so that when a user moves their line of sight to the region of interest, they can immediately see a high-definition image in the vicinity of the region of interest.

[0218] (Technology 6) In the image generating device according to any one of techniques 1 to 5, the control unit sets the specific area to be smaller than the first image area.

[0219] This technology primarily increases the resolution of a first image area where the user's gaze is directed, while subsidiarily increasing the resolution of a specific area where the gaze is likely to be directed next, thereby appropriately reducing the proportion of the area where the high-resolution image is displayed relative to the frame image. This makes it possible to smoothly and quickly display a high-resolution image at the position of the user's gaze by following changes in the user's gaze while appropriately reducing eye fatigue.

[0220] (Technology 7) In the image generating device described in any one of Technologies 1 to 6, the first resolution and the second resolution are, respectively, a first resolution and a second resolution of the frame image determined by the scanning speed of the scanning unit, and the first resolution is higher than the second resolution.

[0221] According to this technology, a high-resolution frame image is displayed in a first image area near the user's line of sight and in a specific area where the user's line of sight is easily directed, and a low-resolution frame image is displayed in a second image area around the user's line of sight. This reduces eye fatigue for the user. Furthermore, even if the user has poor eyesight, the user can clearly grasp the subject or scenery by referring to the first resolution of the first image area and the frame image in the specific area.

[0222] (Technology 8) In the image generating device described in Technology 7, the camera outputs the first imaging video signal corresponding to the first resolution during a first imaging period in one frame period, and outputs the second imaging video signal corresponding to the second resolution during a second imaging period different from the first imaging period in one frame period.

[0223] According to this technique, the first video signal having the first resolution and the second resolution can be generated smoothly.

[0224] (Technology 9) In the image generating device described in Technology 7 or 8, the camera outputs the first captured video signal corresponding to the first resolution, and the image processing unit includes an input processing unit that performs thinning or mixing on the first captured video signal of the first resolution to generate the second captured video signal corresponding to the second resolution.

[0225] This technology simplifies the camera configuration and processing because the camera only needs to output one type of first captured video signal. Furthermore, in a configuration in which two types of video signals are obtained in two different imaging periods, if the subject moves at high speed, the position of the subject will differ between the two types of video signals. However, with the above configuration, because the camera only outputs one type of video signal, the position of the subject will be the same between the two types of video signals generated by the input processing unit, even if the subject moves at high speed. This avoids the user's discomfort caused by the first captured video signal and the second captured video signal.

[0226] (Technology 10) In the image generating device described in any one of technologies 1 to 6, the first resolution and the second resolution are first and second gradations that define the luminance resolution of the first and second imaged video signals, respectively, and the first gradation is higher than the second gradation.

[0227] According to this technology, a high-gradation frame image is displayed in a first image area near the user's line of sight and in a specific area where the user's line of sight is easily directed, and a low-gradation frame image is displayed in a second image area around the user's line of sight. This reduces eye fatigue for the user. Furthermore, even if the user has poor eyesight, the user can clearly grasp the subject by referring to the first-gradation frame image in the first image area and the specific area.

[0228] (Technology 11) In the image generating device described in Technology 10, the camera outputs the first imaging video signal corresponding to the first gradation, and the imaging processing unit includes an input processing unit that performs gradation reduction processing on the first imaging video signal of the first gradation to generate the second imaging video signal for the second gradation.

[0229] According to this technique, the video signals of the first and second gradations can be generated smoothly.

[0230] (Technology 12) The image generating device according to any one of technologies 1 to 11, further comprising: an input processing unit that outputs, based on a video signal from an external device, a first input video signal for constructing a frame image of the first resolution and a second input video signal for constructing a frame image of the second resolution; and a signal combining unit that stores, in the first frame buffer, a first combined video signal obtained by combining the first captured video signal and the first input video signal, and that stores, in the second frame buffer, a second combined video signal obtained by combining the second captured video signal and the second input video signal, wherein the control unit controls the light source and the scanning unit so that an image is generated by applying the first combined video signal from the first frame buffer to the first image region and the specific region, and controls the light source and the scanning unit so that an image is generated by applying the second combined video signal from the second frame buffer to the second image region.

[0231] This technology allows a frame image to be generated by combining a video signal from an imaging processor with a video signal from an external device. Also in this case, a first combined video signal stored in a first frame buffer and a second combined video signal stored in a second frame buffer are selectively used according to the user's line of sight to generate one frame of image. This allows smooth switching of image resolution between a first image region near the user's line of sight and a specific region likely to be targeted by the user's line of sight, and a second image region other than the first image region.

[0232] (Technology 13) A head-mounted display comprising: an image generating device according to any one of technologies 1 to 8; a frame for holding the image generating device; and an optical system for directing light from the image generating device to the eyes of the user wearing the head-mounted display on their head.

[0233] According to this technology, a user can wear a head-mounted display on his or her head and grasp a scene captured by a camera through a frame image generated by an image generating device.

[0234] 1 AR glasses (head mounted display) 2 Frame 3 Image generating device 4 Mirror (optical system) 12 Detection unit 13, 14 Camera 13a, 14a Moving object detection unit 20 Frame image 101, 102, 103 Light source 140 First scanning unit (scanning unit) 160 Second scanning unit (scanning unit) 201 Control unit 230 Image capturing processing unit 310 Input processing unit 330 Signal synthesis unit 341 First frame buffer 342 Second frame buffer 350 Input processing unit P10 Viewpoint position R10 Moving object region R1, R31 First image region R11 to R15 Region (first image region) R2, R32 Second image region R3, R33 Specific region

Claims

1. An image generating device comprising: an imaging processing unit including a camera that captures an image of a field of view, and that outputs a first imaging video signal for constructing a frame image of a first resolution and a second imaging video signal for constructing a frame image of a second resolution lower than the first resolution; a first frame buffer that stores the first imaging video signal; a second frame buffer that stores the second imaging video signal; a light source that emits light for constructing the frame image; a scanning unit that scans the light emitted from the light source; a detection unit that detects the user's line of sight; and a control unit, wherein the control unit sets a specific area on the frame image including at least a part of an area toward which the line of sight is likely to be directed, controls the light source and the scanning unit so that an image is generated by applying the first imaging video signal from the first frame buffer to a first image area and the specific area including a viewpoint position on the frame image corresponding to the line of sight, and controls the light source and the scanning unit so that an image is generated by applying the second imaging video signal from the second frame buffer to a second image area of ​​the frame image other than the first image area and the specific area.

2. An image generating device as described in claim 1, further comprising a motion detection unit that detects a moving object area moving on the frame image, and the control unit sets the specific area as an area to which the gaze is likely to be directed, the detected moving object area.

3. An image generating device according to claim 2, wherein the moving object detection section detects the moving object area from an image pickup video signal of the camera.

4. An image generating device according to claim 2, further comprising another camera separate from the camera, and wherein the motion detection unit detects the moving object area from an image signal captured by the other camera.

5. An image generating device according to claim 1, characterized in that an area of ​​interest that a user is likely to focus on is set in advance, and the control unit sets the specific area so as to include the area of ​​interest.

6. An image generating device according to claim 1, characterized in that the control unit sets the specific area to be smaller than the first image area.

7. An image generating device as described in claim 1, characterized in that the first resolution and the second resolution are, respectively, a first resolution and a second resolution of the frame image defined by the scanning speed of the scanning unit, and the first resolution is higher than the second resolution.

8. An image generating device as claimed in claim 7, characterized in that the camera outputs the first imaging video signal corresponding to the first resolution during a first imaging period in one frame period, and outputs the second imaging video signal corresponding to the second resolution during a second imaging period different from the first imaging period in one frame period.

9. An image generating device according to claim 7, characterized in that: the camera outputs the first imaging video signal corresponding to the first resolution; and the imaging processing unit includes an input processing unit that performs thinning or mixing on the first imaging video signal of the first resolution to generate the second imaging video signal corresponding to the second resolution.

10. An image generating device according to claim 1, wherein the first resolution and the second resolution are first and second gradations that define the luminance resolution of the first and second image-capturing video signals, respectively, and the first gradation is higher than the second gradation.

11. An image generating device according to claim 10, characterized in that the camera outputs the first imaging video signal corresponding to the first gradation, and the imaging processing unit includes an input processing unit that performs a gradation reduction process on the first imaging video signal of the first gradation to generate the second imaging video signal corresponding to the second gradation.

12. An image generating device as described in claim 1, comprising: an input processing unit which outputs a first input video signal for constructing a frame image of the first resolution and a second input video signal for constructing a frame image of the second resolution based on a video signal from an external device; and a signal synthesis unit which stores in the first frame buffer a first synthetic video signal obtained by synthesizing the first captured video signal and the first input video signal, and stores in the second frame buffer a second synthetic video signal obtained by synthesizing the second captured video signal and the second input video signal, wherein the control unit controls the light source and the scanning unit so that an image is generated by applying the first synthetic video signal from the first frame buffer to the first image region and the specific region, and controls the light source and the scanning unit so that an image is generated by applying the second synthetic video signal from the second frame buffer to the second image region.

13. A head-mounted display comprising: an image generating device according to any one of claims 1 to 12; a frame for holding the image generating device; and an optical system for directing light scanned by the scanning unit to the eyes of a user wearing the head-mounted display on his or her head.

Images