Display device, display control method, and display control program

Abstract

To provide the user with appropriate images.SOLUTION: A display 1 provides a stereoscopic image to a user and includes a display unit 10 by illuminating light, a lens 30 provided on the user side rather than the display unit 10 and capable of changing the focus, a gaze information acquisition unit that acquires the detection result of the user's gaze, and a focus position setting unit that sets the focus of the lens 30 based on the detection result of the user's gaze and the depth information indicating the depth direction position of the image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a display device, a display control method, and a display control program. [Background technology]

[0002] There is known a display device that displays a stereoscopic image by making the right and left eyes of a user view images with different parallax and utilizing the difference in convergence. One such display device is a so-called head-mounted display (HMD) that is mounted on the user's head. For example, Patent Document 1 describes a head-mounted display in which a microlens array is disposed between a display and an optical system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 044501 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, a display device that displays an image is required to provide the image appropriately to a user.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a display device, a display control method, and a display control program capable of providing an appropriate image to a user. [Means for solving the problem]

[0006] A display device according to one aspect of the present invention includes a display unit that provides an image to a user by irradiating light, a lens that is provided on the user side of the display unit and has a variable focus, a gaze information acquisition unit that acquires a detection result of the user's gaze, and a focus position setting unit that sets the focus of the lens based on the detection result of the user's gaze and depth information indicating a position in the depth direction of the image.The gaze information acquisition unit detects the gaze position in the image at which the user is gazing based on the gaze detection result, and the focus position setting unit sets the lens focus position at which the difference between the opening angle of the light beam of the light and the opening angle of the light beam when the light is irradiated toward the user from a depth position at the gaze position is within a predetermined range based on depth information of the gaze position, and further includes a drive control unit that moves the lens focus position at a predetermined period, and a timing setting unit that sets the irradiation timing of the light for each pixel based on the lens focus position.

[0007] A display control method according to one aspect of the present invention includes a display step of providing an image to a user by irradiating light from a display unit, a gaze information acquisition step of acquiring a detection result of the user's gaze, and a focus position setting step of setting a focus of a lens that is provided on the user side of the display unit and has a variable focus, based on the detection result of the user's gaze and depth information indicating a position in a depth direction of the image. fixed Steps and The method further includes the steps of: in the gaze information acquisition step, detecting a gaze position in the image at which the user is gazing based on a gaze detection result; and in the focus position setting step, setting, based on depth information of the gaze position, a lens focus position at which a difference between an opening angle of the light beam and an opening angle of the light beam when the light is irradiated toward the user from a depth position at the gaze position is within a predetermined range, and moving the lens focus position at a predetermined period; and setting the irradiation timing of the light for each pixel based on the lens focus position.

[0008] A display control program according to one aspect of the present invention includes a display step of providing an image to a user by irradiating light from a display unit, a gaze information acquisition step of acquiring a detection result of the user's gaze, and a focus position setting step of setting a focus of a lens that is provided closer to the user than the display unit and has a variable focus, based on the detection result of the user's gaze and depth information indicating a position in a depth direction of the image. fixed The steps are executed by a computer. In the gaze information acquisition step, the gaze position at which the user is gazing in the image is detected based on the gaze detection result, and in the focus position setting step, the lens focus position at which the difference between the opening angle of the light beam and the opening angle of the light beam when the light is irradiated toward the user from a depth position at the gaze position is within a predetermined range based on depth information of the gaze position is set as the lens focus position, and the lens focus position is moved at a predetermined period, and the light irradiation timing is set for each pixel based on the lens focus position. Effect of the Invention

[0009] According to the present invention, an image can be appropriately provided to a user. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram for explaining the convergence accommodation conflict. [Diagram 2] FIG. 2 is a schematic diagram of the display device according to the first embodiment. [Diagram 3] FIG. 3 is a schematic diagram of each component of the display device according to the first embodiment. [Figure 4] FIG. 4 is a schematic block diagram of the control device according to the first embodiment. [Diagram 5] FIG. 5 is a schematic diagram for explaining the setting of the irradiation timing. [Figure 6] FIG. 6 is a schematic diagram for explaining the setting of the irradiation timing. [Figure 7] FIG. 7 is a schematic diagram for explaining the setting of the irradiation timing. [Figure 8] FIG. 8 is a flowchart illustrating a process flow of the control device according to the present embodiment. [Figure 9] FIG. 9 is a schematic block diagram of a control device according to the second embodiment. [Figure 10] FIG. 10 is a flowchart illustrating a process flow of the control device according to the present embodiment. [Figure 11] FIG. 11 is a schematic diagram of each component of the display device according to the third embodiment. [Figure 12] FIG. 12 is a schematic block diagram of a control device according to the third embodiment. [Figure 13] FIG. 13 is a flowchart illustrating a process flow of the control device according to the present embodiment. [Figure 14] FIG. 14 is a schematic diagram of a display device according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment described below.

[0012] (Convergence regulation conflict) FIG. 1 is a schematic diagram for explaining the convergence accommodation conflict. A display device for displaying a stereoscopic image displays a stereoscopic image by making the right eye and the left eye of a user view images with different parallax and utilizing the difference in convergence. When displaying a stereoscopic image, the display surface on which the image is actually displayed becomes the focal position of the user's eyes, and the position where the lines of sight of the left and right eyes intersect becomes the convergence position. However, as shown in the example of FIG. 1, in a stereoscopic image, the positions of the focal position PO1 and the convergence position PO2 in the Z direction, which is the depth direction of the stereoscopic image, may shift. When the positions of the focal position PO1 and the convergence position PO2 shift, a so-called convergence accommodation conflict occurs, which causes eye strain and so-called 3D sickness. For this reason, it is required to suppress the convergence accommodation conflict. Note that FIG. 1 (A) shows an example in which the focal position PO1 is closer to the user's eye EY than the convergence position PO2, and FIG. 1 (B) shows an example in which the convergence position PO2 is closer to the user's eye EY than the focal position PO1.

[0013] (First embodiment) (Overall configuration of the display device) FIG. 2 is a schematic diagram of a display device according to the first embodiment. The display device 1 according to the first embodiment is a display device that displays a stereoscopic image. As shown in FIG. 2, the display device 1 is a so-called HMD (Head Mount Display) that is worn on the head of a user U. For example, the display device 1 has a display unit 10 worn at a position facing the eye EY of the user U. The display device 1 displays an image on the display unit 10 to provide content to the user U. Note that the configuration of the display device 1 shown in FIG. 2 is an example. For example, the display device 1 may include an audio output unit (speaker) that is worn on the ear of the user U.

[0014] Since the display device 1 is worn by the user U in this manner, the position of the display device 1 with respect to the eyes EY of the user U is fixed. The display device 1 is not limited to being an HMD worn by the user U, and may be a display device fixed to equipment, etc. Even in such a case, it is preferable that the position of the display device 1 with respect to the eyes EY of the user U is fixed, for example, by fixing the position of the display device 1 with respect to the seat of the user U.

[0015] Fig. 3 is a schematic diagram of each component of the display device according to the first embodiment. As shown in Fig. 3, the display device 1 has a display unit 10, an eyepiece lens 20, a lens 30, a gaze detection unit 40, and a control device 50.

[0016] (Display) The display unit 10 is a device that displays a stereoscopic image. The display unit 10 includes a display panel 12 and a light source unit 14. The display panel 12 is a display having a plurality of pixels arranged in a matrix. In the example of this embodiment, the display panel 12 is a liquid crystal display panel having a plurality of pixel electrodes arranged in a matrix and a liquid crystal layer filled with liquid crystal elements. The surface on which an image is displayed on the display panel 12 is defined as a display surface 10A. Hereinafter, the direction from the display surface 10A toward the eye EY of the user U is defined as a direction Z1, and the opposite direction to the direction Z1, that is, the direction from the eye EY of the user U toward the display surface 10A is defined as a direction Z2. When the directions Z1 and Z2 are not distinguished from each other, they are referred to as a direction Z. In FIG. 3, the surface of the display panel 12 on the eye EY side of the user U is defined as the display surface 10A, but the display surface 10A is not limited to being the surface on the eye EY side of the user U, and may be located inside the surface on the eye EY side of the user U. The display unit 10 receives a drive signal for driving the display panel 12 and a control signal for controlling the timing of light irradiation by the light source unit 14 from a control device 50, which will be described later.

[0017] The light source unit 14 is a light source that irradiates light to the display panel 12. In this embodiment, the light source unit 14 is a so-called backlight that is provided on the surface of the display panel 12 opposite to the display surface 10A. The position at which the light source unit 14 is provided is not limited to this and is arbitrary, and may be, for example, a side light next to the display panel 12. In this embodiment, the light source unit 14 irradiates light uniformly to all pixels of the display panel 12, in other words, does not control the irradiation of light individually for each pixel. However, the light source unit 14 is not limited to one that irradiates light uniformly to all pixels of the display panel 12, and may be, for example, a so-called local dimming type that can adjust the irradiation of light for each of a plurality of pixels, for example, by dividing the entire screen into several sections and adjusting the light intensity for each section.

[0018] In this manner, the display unit 10 has a display panel 12 and a light source unit 14. The display unit 10 provides a stereoscopic image to the user U by making the image light L, which is light irradiated from the light source unit 14 to the display panel 12, reach the eye EY of the user U. In the example of the present embodiment, the light irradiated from the light source unit 14 passes through each pixel of the display panel 12 and enters the eye EY of the user U as the image light L. More specifically, the display panel 12 is irradiated with light from the light source unit 14 toward the display panel 12 while controlling, for example, the voltage of the pixel electrode so that an image for the left eye and an image for the right eye are provided. Of the light from the light source unit 14, the image light L that has passed through the pixel corresponding to the image for the left eye enters the left eye of the user U, and the image light L that has passed through the pixel corresponding to the image for the right eye enters the left eye of the user U, thereby providing a stereoscopic image to the user U.

[0019] The configuration of the display unit 10 is not limited to the above, and may be, for example, a self-luminous display capable of controlling the lighting of each pixel, such as an OLED (Organic Light Emitting Diode) or a so-called micro LED. The display unit 10 may also be a reflective liquid crystal display device.

[0020] (Eyepiece) The eyepiece 20 is provided on the Z1 direction side of the display unit 10. The eyepiece 20 is an optical element that transmits light (image light). More specifically, the eyepiece 20 is the optical element (lens) that is closest to the eye EY of the user U, excluding the lens 30, in the display device 1. The image light L emitted from the display panel 12 passes through the eyepiece 20 and the lens 30 and enters the eye EY of the user U. Note that in this embodiment, the optical axis direction in the optical path of the image light L from the lens 30 to the eye EY of the user U can also be referred to as the Z direction.

[0021] In the example of FIG. 3, only the eyepiece lens 20 and the lens 30 are shown as optical elements on the Z1 direction side of the display unit 10, but the present invention is not limited to this and other optical elements may be provided.

[0022] (lens) The lens 30 is provided on the user U side of the display unit 10 in the optical axis direction of the image light L. More specifically, the lens 30 is provided on the user U side of the eyepiece lens 20. However, the present invention is not limited to this, and the lens 30 may be provided on the display unit 10 side of the eyepiece lens 20. The lens 30 is a so-called variable focus lens that can change the position of the focal point (the distance from the lens 30 to the focal point) in the Z direction (the optical axis direction of the image light L). The lens 30 may have any structure, but may be capable of changing the position of the focal point by changing the thickness of the lens 30 by applying a voltage, for example. More specifically, for example, the lens 30 may have a lens core made of an optical liquid sealed in an elastic polymer film and an electromagnetic actuator attached to the lens core, and the actuator is driven by applying a voltage to the actuator, and the lens core is deformed by the pressure applied by the actuator, thereby changing the thickness of the lens 30 and changing the position of the focal point. The actuator of the lens 30 obtains a signal for controlling the position of the focal point from a control device 50 described later.

[0023] (Gaze detection section) The gaze detection unit 40 detects the direction in which the eyes EY of the user U are facing, that is, the gaze of the user U. The gaze detection unit 40 may be any sensor capable of detecting the gaze of the user U, and may be, for example, a camera that captures an image of the eyes EY of the user U. The gaze detection unit 40 sends the gaze detection result to the control device 50.

[0024] (Control device) The control device 50 is a device that controls each part of the display device 1. FIG. 4 is a schematic block diagram of the control device according to the first embodiment. In this embodiment, the control device 50 is a computer, and has a storage unit 52 and a control unit 54. The storage unit 52 is a memory that stores various information such as the calculation contents and programs of the control unit 54, and includes at least one of a main storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), and an external storage device such as a HDD (Hard Disk Drive). The program for the control unit 54 stored in the storage unit 52 may be stored in a recording medium that can be read by the control device 50.

[0025] The control unit 54 is a calculation device and includes a calculation circuit such as a CPU (Central Processing Unit). The control unit 54 includes an image information acquisition unit 60, a line of sight information acquisition unit 62, an irradiation control unit 64, a focal position setting unit 66, and a drive control unit 68. The control unit 54 realizes the image information acquisition unit 60, the line of sight information acquisition unit 62, the irradiation control unit 64, the focal position setting unit 66, and the drive control unit 68 by reading and executing a program (software) from the storage unit 52, and executes the processes. The control unit 54 may execute these processes by one CPU, or may include multiple CPUs and execute the processes by the multiple CPUs. At least one of the image information acquisition unit 60, the line of sight information acquisition unit 62, the irradiation control unit 64, the focal position setting unit 66, and the drive control unit 68 may be realized by a hardware circuit.

[0026] (Image information acquisition section) The image information acquisition unit 60 acquires image data of a stereoscopic image displayed by the display unit 10. That is, the image information acquisition unit 60 acquires image data of an image for the left eye and image data of an image for the right eye. The image information acquisition unit 60 also acquires depth information indicating a position in the depth direction of the stereoscopic image. The position in the depth direction of the stereoscopic image refers to a position in the depth direction of a virtual image visually recognized by the user U when an image is displayed on the display surface 10A. The depth direction can also be said to be a direction perpendicular to the display surface 10A of the display unit 10, and is the Z direction in this embodiment. The depth information is associated with the image data. Furthermore, a position in the depth direction is set for each image included in one frame of the stereoscopic image, in other words, a position in the depth direction is set for each position on the display surface 10A. Therefore, it can be said that the image information acquisition unit 60 acquires depth information for each position on the display surface 10A for the stereoscopic image. In addition, in a stereoscopic image, a position in the depth direction is set for each pixel, but the positions in the depth direction may be set to be the same for multiple pixels constituting one image. The image information acquisition unit 60 may acquire image data and depth information by any method, and may read image data and depth information stored in advance in the storage unit 52, or may receive image data and depth information via a communication unit (not shown). The image information acquisition unit 60 may also acquire depth information by calculating a position in the depth direction based on the image data.

[0027] (Gaze information acquisition unit) The line-of-sight information acquisition unit 62 controls the line-of-sight detection unit 40 to cause the line-of-sight detection unit 40 to detect the line of sight of the user U, and acquires the line-of-sight detection result by the line-of-sight detection unit 40. In this embodiment, the line-of-sight information acquisition unit 62 detects the gaze position at which the user U is gazing based on the line-of-sight detection result. The gaze position refers to the position on the display surface 10A of the image at which the user U is gazing among the stereoscopic images, in other words, the position at which the user U is gazing among the entire area of ​​the display surface 10A. In this embodiment, the line-of-sight information acquisition unit 62 detects the convergence position of the user U from the line-of-sight detection result of the user U, and detects the position on the display surface 10A of the image corresponding to the virtual image at the position overlapping with the convergence position as the gaze position. That is, for example, when the convergence position of the user U overlaps with the virtual image of a car in the stereoscopic image, the position on the display surface 10A of the image of the car is set as the gaze position. However, the method of detecting the gaze position is not limited thereto, and may be detected by any method from the detection result of the line of sight of the user U.

[0028] (Irradiation control unit) The irradiation control unit 64 causes the display unit 10 to emit image light L toward the user U based on the image data. In this embodiment, the irradiation control unit 64 drives each pixel of the display panel 12 based on the image data while emitting light from the light source unit 14, thereby emitting image light L from each pixel toward the user U, and providing a stereoscopic image to the user U. More specifically, the irradiation control unit 64 causes the light source unit 14 to emit light while controlling the applied voltage to each pixel electrode of the display panel 12 based on the image data to orient the liquid crystal element. As a result, the light from the light source unit 14 passes through the liquid crystal layer and is emitted toward the eye EY of the user Y as image light L for a stereoscopic image. Note that, for example, when the display unit 10 is a self-luminous type, the irradiation control unit 64 causes each pixel to emit light based on the image data, and causes each pixel to emit image light L.

[0029] (Focus position setting section) Based on the detection result of the line of sight of the user U, the focal position setting unit 66 sets a set focal position that is the focal position of the lens 30 in the optical axis direction (direction Z) of the image light L. More specifically, the focal position setting unit 66 sets the set focal position based on the detection result of the line of sight of the user U and depth information of the stereoscopic image. Setting of the set focal position will be described in more detail below.

[0030] 5 to 7 are schematic diagrams for explaining the setting of the irradiation timing. For example, as shown in FIG. 5, the image light L emitted from the display unit 10 is incident on the eye EY of the user U as a light beam having a predetermined opening angle. In this case, the user U unconsciously changes the thickness of the crystalline lens of the eyeball to adjust the focus on the retina to match the opening angle of the light beam. The convergence accommodation conflict refers to a mismatch between the degree of convergence of the left and right eyes (degree of deviation of the left and right eyes) and the state in which the focus is adjusted to the opening angle of the light beam. In contrast, the display device 1 of the present embodiment emits the image light L so that the difference between the virtual image opening angle (angle θ1A in the example of FIG. 5), which is the opening angle when it is assumed that light is emitted from the virtual image (convergence position) and enters the eye EY, and the opening angle of the image light L when the image light L is actually emitted from the display unit 10 and enters the eye EY, becomes small. Here, since the luminous flux opening angle of the image light L changes (i.e., is refracted) when passing through the lens 30, the degree of change in the luminous flux opening angle is determined according to the focal position of the lens 30. Therefore, the focal position setting unit 66 sets the focal position of the lens 30 where the difference between the virtual image opening angle and the opening angle of the image light L becomes small as the set focal position.

[0031] More specifically, the focal position setting unit 66 acquires depth information at the gaze position detected by the line-of-sight information acquisition unit 62. That is, the focal position setting unit 66 acquires information on the position in the depth direction (Z direction) of the gaze position. In other words, the focal position setting unit 66 acquires information on the position in the Z direction of the virtual image on which the user U is gazing. Then, the focal position setting unit 66 sets the set focal position based on the depth information at the gaze position. The focal position setting unit 66 sets the position of the focus of the lens 30 in the Z direction when the opening angle of the image light L matches the opening angle of the light beam (the virtual image opening angle from the gaze position) when light is irradiated to the eye EY from the position in the depth direction at the gaze position (the position of the virtual image on which the user U is gazing) as the set focal position. However, the set focal position is not limited to the position where the virtual image opening angle and the opening angle of the actual image light L strictly match. For example, the focal position of the lens 30 when the difference between the virtual image opening angle and the opening angle of the actual image light L falls within a predetermined range may be set as the set focal position. The predetermined range may be set arbitrarily, but it is preferable to set it to a value that reduces the difference between the virtual image opening angle and the opening angle of the image light L in order to reduce the convergence accommodation contradiction. In addition, the focal position setting unit 66 may use a value quantized in the depth direction as the position in the depth direction of the gaze position used to set the set focal position. That is, for example, the depth direction is divided into a plurality of numerical ranges, and a predetermined value within the numerical range is set as a reference position for each numerical range. Then, the focal position setting unit 66 extracts a numerical range that includes the position in the depth direction of the acquired gaze position, and treats the reference position for the numerical range as the position in the depth direction of the gaze position used to set the set focal position.

[0032] The drive control unit 68 controls the lens 30 to move the focal position of the lens 30 in the Z direction. The drive control unit 68 moves the focal position of the lens 30 in the Z direction, for example, by controlling application of a voltage to an actuator of the lens 30. The drive control unit 68 moves the Z direction position of the focal position of the lens 30 to a set focal position. Therefore, the opening angle of the light flux of the image light L incident on the eye EY becomes close to the virtual image opening angle from the gaze position (the virtual image gazed at by the user U), and the convergence accommodation conflict can be reduced.

[0033] An example of setting the set focal position described above will be described with reference to Fig. 5 to Fig. 7. In the following, an example will be taken in which an image of a house, an image of a car, and an image of a helicopter are displayed as a stereoscopic image, and the positions in the depth direction (Z direction) of the image of the car, the image of the house, and the image of the helicopter are farther away from the eye EY of the user U in the order of the image of the car, the image of the house, and the image of the helicopter. That is, the virtual image P2 of the image of the car is located on the Z1 side of the virtual image P1 of the image of the house, and the virtual image P3 of the image of the helicopter is located on the Z2 side of the virtual image P1 of the image of the house.

[0034] FIG. 5 is an example in which the user U is gazing at a virtual image P1 of a house image. In the example of FIG. 5, the opening angle of the light flux (virtual image opening angle from the gaze position) when light is irradiated from the virtual image P1 (position in the depth direction of the gaze position) to the eye EY is set to angle θ1A. Then, when the position of the focus of the lens 30 in the Z direction is the first position, the opening angle of the light flux of the image light L from the pixel constituting the image of the house is set to angle θ1A. In this case, the first position becomes the set focus position, and the focus position setting unit 66 moves the focus of the lens 30 to the first position. As a result, the virtual image opening angle from the virtual image P1 of the house image and the opening angle of the image light L that actually enters the eye EY of the user U match, and the convergence accommodation contradiction can be reduced. In this embodiment, since light is irradiated to the entire display panel 12, the virtual image P2 of the car image and the virtual image P3 of the helicopter image are also visually recognized by the user U. In this case, the virtual image opening angle for the virtual images P2 and P3 does not match the opening angle of the image light L that actually enters the eye EY of the user U. However, since the virtual images P2 and P3 are outside the range of the area gazed upon by the user U, the impact on the user U is small, and since the virtual image opening angle of the virtual image P1, which is within the range of the area gazed upon by the user U, matches, the burden on the user U can be reduced. In the above description, the first position where the virtual image opening angle and the opening angle of the image light L match is set as the set focal position, but this is not limited thereto, and a position where the difference between the virtual image opening angle and the opening angle of the image light L is within a predetermined range may be set as the set focal position.

[0035] FIG. 6 shows an example in which the user U is gazing at a virtual image P2 of a car image. In the example of FIG. 6, the opening angle of the light flux (virtual image opening angle) when light is irradiated from the virtual image P2 (the depth direction position of the pixel P constituting the image of the car) to the eye EY is set to angle θ2A. When the position of the focal point of the lens 30 in the Z direction is the second position, the opening angle of the light flux of the image light L from the pixel P constituting the image of the car is set to angle θ2A. Note that since the virtual image P2 is viewed in the Z1 direction more than the virtual image P1, the angle θ2A is larger than the angle θ1A in FIG. 5, and the degree of enlargement (or the degree of reduction) of the opening angle of the light flux by the lens 30 is larger when the second position is the focal point than when the first position is the focal point. In this case, the second position becomes the set focal position, and the focal position setting unit 66 moves the focal point of the lens 30 to the second position. This allows the virtual image opening angle from the virtual image P2 of the image of the car to match the opening angle of the image light L that actually enters the eye EY of the user U, thereby reducing the convergence accommodation contradiction. In the above description, the second position where the virtual image opening angle and the opening angle of the image light L match is set as the set focal position, but this is not limited thereto, and a position where the difference between the virtual image opening angle and the opening angle of the image light L is within a predetermined range may be set as the set focal position.

[0036] FIG. 7 shows an example in which the user U is gazing at a virtual image P3 of a helicopter image. In the example of FIG. 7, the opening angle of the light flux (virtual image opening angle) when light is irradiated from the virtual image P3 (the depth direction position of the pixel P constituting the helicopter image) to the eye EY is set to angle θ3A. When the position of the focal point of the lens 30 in the Z direction is the third position, the opening angle of the light flux of the image light L from the pixel P constituting the helicopter image is set to angle θ3A. Note that since the virtual image P3 is viewed in the Z2 direction more than the virtual image P1, the angle θ3A is smaller than the angle θ1A in FIG. 5, and the degree of enlargement (or the degree of reduction) of the opening angle of the light flux by the lens 30 is smaller when the third position is the focal point than when the first position is the focal point. In this case, the third position becomes the set focal position, and the focal position setting unit 66 moves the focal point of the lens 30 to the third position. This makes the virtual image opening angle from the virtual image P2 of the image of the car closer to the opening angle of the image light L actually incident on the eye EY of the user U, thereby reducing the convergence accommodation conflict. In this case, the third position becomes the set focal position, and the focal position setting unit 66 moves the focal point of the lens 30 to the third position.

[0037] (Processing flow) Next, the process flow of the control device 50 described above will be described. FIG. 8 is a flowchart for explaining the process flow of the control device according to this embodiment. The control device 50 drives the pixels of the display panel 12 based on image data using the irradiation control unit 64, while emitting light from the light source unit 14. The control device 50 acquires the detection result of the line of sight of the user U using the line of sight information acquisition unit 62 (step S10), and detects the gaze position based on the line of sight detection result (step S12). Then, the control device 50 sets the set focal position from the depth information of the gaze position using the focus position setting unit 66 (step S14). The control device 50 sets the focal position of the lens 30 to the set focal position using the drive control unit 68 (step S16). If the process is not to be ended (step S18; No), the process returns to step S10 and continues, and if the process is to be ended (step S18; Yes), the process ends.

[0038] (effect) As described above, the display device 1 according to this embodiment provides a stereoscopic image to the user U, and includes the display unit 10, the lens 30, the line-of-sight information acquisition unit 62, the focal position setting unit 66, and the drive control unit 68. The display unit 10 includes a plurality of pixels, and provides a stereoscopic image to the user U by making the image light L reach the user U. The line-of-sight information acquisition unit 62 acquires a detection result of the line of sight of the user U. The focal position setting unit 66 sets a set focal position that is the focal position of the lens 30 in the optical axis direction of the image light L, based on the detection result of the line of sight of the user U and depth information indicating the position in the depth direction of the stereoscopic image. The drive control unit 68 moves the focal position of the lens 30 to the set focal position.

[0039] Here, when displaying a stereoscopic image, it is required to provide the user with an appropriate stereoscopic image. In contrast, in this embodiment, the focal position of the lens 30 is moved to a set focal position set based on the viewpoint of the user U and the position in the depth direction of the stereoscopic image. Therefore, according to this embodiment, it is possible to make the image light L reach the user U at an appropriate light beam opening angle based on the line of sight of the user U and the position in the depth direction of the stereoscopic image, and the stereoscopic image can be appropriately provided to the user U. Furthermore, as described above, when displaying a stereoscopic image, a convergence accommodation conflict may occur. In contrast, in this embodiment, the convergence accommodation conflict can be reduced by appropriately adjusting the opening angle of the light beam of the image light L.

[0040] Further, the line-of-sight information acquisition unit 62 detects the gaze position of the user U in the stereoscopic image based on the line-of-sight detection result. Then, the focal position setting unit 66 sets the focal position of the lens 30 in the optical axis direction of the image light L as the set focal position such that the difference between the opening angle of the light beam of the image light L and the opening angle of the light beam when the image light L is irradiated from the depth direction position of the gaze position toward the user U (the virtual image opening angle from the gaze position) falls within a predetermined range based on the depth information of the gaze position. According to this embodiment, the opening angle of the light beam of the image light L can be brought close to the virtual image opening angle from the virtual image that the user U is gazing at, so that the convergence accommodation conflict can be appropriately reduced.

[0041] Moreover, the display device 1 is a head-mounted display, and therefore, a stereoscopic image can be appropriately provided by the head-mounted display.

[0042] Second embodiment Next, a second embodiment will be described. The second embodiment differs from the first embodiment in that the focal position of the lens 30 is changed at a predetermined cycle. Descriptions of the configurations of the second embodiment and the first embodiment will be omitted.

[0043] FIG. 9 is a schematic block diagram of a control device according to the second embodiment. As shown in FIG. 9, the control unit 54a of the control device 50a according to the second embodiment includes an image information acquisition unit 60, a line of sight information acquisition unit 62, an irradiation control unit 64a, a timing setting unit 66a, and a drive control unit 68a. The control unit 54a reads out a program (software) from the storage unit 52 and executes it to realize the image information acquisition unit 60, the line of sight information acquisition unit 62, the irradiation control unit 64a, the timing setting unit 66a, and the drive control unit 68a, and executes the processes. The control unit 54a may execute these processes by one CPU, or may include multiple CPUs and execute the processes by the multiple CPUs. At least one of the image information acquisition unit 60, the line of sight information acquisition unit 62, the irradiation control unit 64a, the timing setting unit 66a, and the drive control unit 68a may be realized by a hardware circuit.

[0044] (Drive control unit) In the second embodiment, the drive control unit 68a moves the focal position of the lens 30 in a predetermined cycle along the Z direction. The drive control unit 68a moves the focal position of the lens 30 so that the focal position of the lens 30 repeats a reciprocating movement (oscillation) in the Z direction in which the focal position of the lens 30 moves a predetermined distance in the Z1 direction and then moves a predetermined distance in the Z2 direction. In other words, the drive control unit 68a causes the focal position of the lens 30 to move back and forth in the Z direction in a predetermined cycle. In this embodiment, the cycle of the reciprocating movement in the Z direction (the time it takes for the focal position to return to its original position in the Z direction) is constant, but is not limited to being constant and may be changed.

[0045] (Timing setting unit and irradiation control unit) The timing setting unit 66a sets the timing of emitting light from the light source unit 14. The illumination control unit 64a causes the light source unit 14 to emit light at the illumination timing set by the timing setting unit 66a. The timing setting unit 66a sets the illumination timing based on the detection result of the line of sight of the user U and the position of the focal point of the lens 30 in the Z direction (the optical axis direction of the image light L). More specifically, the timing setting unit 66a sets the illumination timing based on the detection result of the line of sight of the user U, depth information of the stereoscopic image, and the position of the focal point of the lens 30 in the Z direction. The settings of the timing setting unit 66a will be described in more detail below.

[0046] The timing setting unit 66a sets, as the irradiation timing, the timing when the focus of the lens 30 reaches a position where the difference between the virtual image opening angle from the gaze position and the opening angle of the image light L becomes small. More specifically, the timing setting unit 66a acquires depth information at the gaze position detected by the line-of-sight information acquisition unit 62. That is, the timing setting unit 66a acquires information on the position in the depth direction (Z direction) of the gaze position. Then, the timing setting unit 66a sets, based on the depth information of the gaze position, the irradiation position which is the position of the focus of the lens 30 when the light source unit 14 starts emitting light. The timing setting unit 66a sets, as the irradiation position, the position of the focus of the lens 30 in the Z direction when the opening angle of the image light L that actually enters the eye EY from the pixel P through the lens 30 matches the opening angle (virtual image opening angle) of the light beam when light is irradiated to the eye EY from the position in the depth direction at the gaze position (the position of the virtual image that the user U is gazing at). The timing setting unit 66a sets the timing when the focal point of the lens 30 reaches a position whose distance from the irradiation position is within a predetermined distance range as the irradiation timing. The predetermined distance here may be set arbitrarily, but it is preferable to set it to a distance that reduces the difference between the virtual image opening angle from the gaze position and the opening angle of the image light L in order to reduce the convergence accommodation contradiction.

[0047] In this embodiment, the timing setting unit 66a sequentially acquires information on the position of the focal point of the lens 30 in the Z direction, and when the position of the focal point of the lens 30 in the Z direction approaches the irradiation position by a predetermined distance, it determines that the irradiation timing has arrived. The timing setting unit 66a may acquire information on the position of the focal point of the lens 30 in the Z direction by any method. For example, when the focal point of the lens 30 is reciprocated in the Z direction at a predetermined period, the position (predicted position) of the focal point of the lens 30 in the Z direction at each time can be grasped. Therefore, the timing setting unit 66a may acquire information on the position of the focal point of the lens 30 in the Z direction from the time information. In this case, the timing setting unit 66a may set the time at which the focal point of the lens 30 reaches a predetermined distance from the irradiation position as the irradiation timing based on the information on the predicted position of the focal point of the lens 30 at each time and the information on the irradiation position, and when the current time reaches the irradiation timing, it may determine that the focal point of the lens 30 has reached the irradiation position, and cause the light source unit 14 to irradiate the image light L.

[0048] The timing setting unit 66a sets the irradiation timing in this manner. Furthermore, the timing setting unit 66a sets a timing after the irradiation timing as the irradiation stop timing. When the timing setting unit 66a determines that the irradiation timing has been reached, the irradiation control unit 64a causes the light source unit 14 to start emitting light. The irradiation control unit 64a causes the light source unit 14 to emit light from the irradiation timing to the irradiation stop timing, and when the irradiation stop timing is reached, causes the light source unit 14 to stop emitting light. The irradiation stop timing may be set arbitrarily. For example, a predetermined time after the irradiation timing may be set as the irradiation stop timing, or the timing at which the distance between the focus of the lens 30 and the irradiation position falls outside a predetermined distance range immediately after the irradiation timing may be set as the irradiation stop timing.

[0049] In the second embodiment, the light source unit 14 is caused to emit light when the irradiation timing is reached, and the irradiation of light is stopped when the irradiation stop timing is reached. The image light L is incident on the eye EY of the user U between the irradiation timing and the irradiation stop timing. Therefore, the opening angle of the light flux of the image light L incident on the eye EY becomes close to the virtual image opening angle from the gaze position (the virtual image that the user U is gazing at), and the convergence accommodation contradiction can be reduced. In addition, since the focal point of the lens 30 moves back and forth in the Z direction, the distance from the irradiation position repeatedly becomes within a predetermined distance range and out of the predetermined distance range. The control device 50a causes the light source unit 14 to emit light every time the distance between the focal point of the lens 30 and the irradiation position becomes within the predetermined distance range, that is, every time the irradiation timing is reached. Therefore, the user U visually recognizes a stereoscopic image as a moving image. In addition, since the focal point of the lens 30 moves back and forth in the Z direction, there are two times during one period of the back and forth movement when the distance from the irradiation position becomes a predetermined distance. Therefore, it is desirable to set the frequency of the reciprocating movement of the focal point of the lens 30 to at least half the frame rate of the stereoscopic image. However, the frequency (period) of the reciprocating movement of the focal point of the lens 30 may be set arbitrarily.

[0050] (Processing flow) Next, the process flow of the control device 50a described above will be described. FIG. 10 is a flowchart for explaining the process flow of the control device according to this embodiment. The control device 50a drives the pixels of the display panel 12 based on image data using the irradiation control unit 64a, while the drive control unit 68a moves the focal point of the lens 30 back and forth in the Z direction at a predetermined period. The control device 50a acquires the detection result of the line of sight of the user U using the line of sight information acquisition unit 62 while driving the pixels and moving the focal point of the lens 30 back and forth (step S20), and detects the gaze position based on the line of sight detection result (step S22). Then, the control device 50a sets the irradiation position from the depth information of the gaze position using the timing setting unit 66a (step S24). The control device 50a sequentially acquires the position of the focal point of the lens 30 in the Z direction, and determines whether the focal point of the lens 30 has reached within a predetermined distance from the irradiation position (step S26). When the focal point of the lens 30 reaches within a predetermined distance from the irradiation position (step S26; Yes), the timing setting unit 66a determines that the irradiation timing has been reached, and the irradiation control unit 64a causes the light source unit 14 to irradiate light (step S28). After that, if the irradiation is stopped at the irradiation stop timing and the process is not to be ended (step S30; No), the process returns to step S20 and continues. On the other hand, if the focal point of the lens 30 has not reached within a predetermined distance from the irradiation position (step S26; No), the process returns to step S26 and does not cause the light source unit 14 to irradiate light until the focal point of the lens 30 reaches within a predetermined distance from the irradiation position. Note that if the focal point of the lens 30 has not reached within a predetermined distance from the irradiation position, that is, if the answer is No in step S26, the process may also return to step S20. If the process is to be ended in step S30 (step S30; Yes), the process ends.

[0051] (effect) As described above, the display device 1a according to the second embodiment provides a stereoscopic image to the user U, and includes the display unit 10, the lens 30, the line-of-sight information acquisition unit 62, the drive control unit 68a, the timing setting unit 66a, and the irradiation control unit 64a. The display unit 10 has a display panel 12 including a plurality of pixels, and a light source unit 14 that irradiates the display panel 12 with light, and provides a stereoscopic image to the user U by causing the image light L, which is light irradiated from the light source unit 14 to the display panel 12, to reach the user U. The lens 30 is provided on the user U side of the display unit 10 in the optical axis direction of the image light L, and is capable of changing its focus. The line-of-sight information acquisition unit 62 acquires a detection result of the line of sight of the user U. The drive control unit 68a moves the focus of the lens 30 along the optical axis direction of the image light L (Z direction in this embodiment) at a predetermined period. The timing setting unit 66a sets the light emission timing of the light source unit 14 based on the detection result of the line of sight of the user U and the position in the optical axis direction (Z direction in this embodiment) of the focal point of the lens 30. The emission control unit 64a causes the light source unit 14 to emit light at the emission timing.

[0052] Here, when displaying a stereoscopic image, it is required to provide the user with a stereoscopic image appropriately. In contrast, in this embodiment, the focus of the lens 30 is moved in the optical axis direction, and the timing of light irradiation is set based on the viewpoint of the user U and the position of the focus of the lens 30 in the optical axis direction. Therefore, according to this embodiment, it is possible to make the image light L reach the user U at an appropriate timing based on the viewpoint of the user U and the position of the focus of the lens 30 in the optical axis direction, and a stereoscopic image can be appropriately provided to the user U. Furthermore, in this embodiment, the focus of the lens 30 is moved in the optical axis direction, and the timing of light irradiation is set based on the viewpoint of the user U and the position of the focus of the lens 30, thereby appropriately adjusting the opening angle of the light flux of the image light L and reducing the convergence accommodation contradiction. Furthermore, in this embodiment, the focus of the lens 30 is not moved each time to a position where the opening angle of the light flux of the image light L is appropriate, but the focus of the lens 30 is moved at a predetermined period, and the light irradiation of the light source unit 14 is controlled at a timing where the opening angle of the light flux of the image light L is appropriate. By controlling the light irradiation in this manner, control delays are suppressed, and the opening angle of the light flux of the image light L can be appropriately adjusted.

[0053] The timing setting unit 66a also sets the irradiation timing based on depth information indicating the position in the depth direction (Z direction in this embodiment) of the stereoscopic image. According to this embodiment, since the irradiation timing is set using the depth information as well, it is possible to appropriately adjust the opening angle of the light flux of the image light L according to the stereoscopic image to be displayed, and it is possible to reduce the convergence accommodation contradiction.

[0054] Further, the line-of-sight information acquisition unit 62 detects the gaze position in the stereoscopic image at which the user is gazing, based on the line-of-sight detection result. The timing setting unit 66a acquires information on the irradiation position, which is a position in the optical axis direction of the focus of the lens 30 such that the opening angle of the light beam of the image light L corresponds to the opening angle of the light beam (virtual image opening angle) when the image light L is irradiated from the depth direction position toward the user U, based on the depth information of the gaze position. The timing setting unit 66a sets the timing at which the focus of the lens 30 is within a predetermined distance range from the irradiation position as the irradiation timing. According to this embodiment, the opening angle of the light beam of the image light L can be brought close to the virtual image opening angle from the virtual image at which the user U is gazing, so that the convergence accommodation conflict can be appropriately reduced.

[0055] Third embodiment Next, a third embodiment will be described. The third embodiment differs from the second embodiment in that the display unit is a self-luminous type. Descriptions of the configurations of the third embodiment and the second embodiment will be omitted.

[0056] FIG. 11 is a schematic diagram of each component of the display device according to the third embodiment. As shown in FIG. 11, the display device 1b according to the third embodiment has a display unit 10b. The display device 1b does not have a gaze detection unit 40. The display unit 10b is a display having a plurality of pixels P (display elements) arranged in a matrix and self-emitting. Since each pixel P of the display unit 10b self-emitting, the display unit 10b can control the light emission (light irradiation) of each pixel P individually. Each pixel P of the display unit 10b may be, for example, an organic light-emitting diode or an inorganic light-emitting diode. In the third embodiment, the display unit 10b receives a control signal from the control device 50b for controlling the timing of light irradiation of the pixel P. In addition, the actuator of the lens 30 obtains a signal for controlling the position of the focus from the control device 50b.

[0057] Fig. 12 is a schematic block diagram of a control device according to the third embodiment. As shown in Fig. 12, a control unit 54b of a control device 50b according to the third embodiment includes an image information acquisition unit 60, an irradiation control unit 64b, a timing setting unit 66b, and a drive control unit 68b. In the third embodiment, a line of sight information acquisition unit 62 is not included.

[0058] (Drive control unit) As in the second embodiment, the drive control unit 68b moves the focal position of the lens 30 in the Z direction at a predetermined cycle.

[0059] (Timing setting unit and irradiation control unit) The timing setting unit 66b sets the irradiation timing of the image light L for each pixel P. The irradiation control unit 64b controls the pixel P based on the image data to irradiate the pixel P with the image light L. The irradiation control unit 64b irradiates the pixel P with the image light L at the irradiation timing for each pixel P set by the timing setting unit 66b. That is, the irradiation control unit 64b irradiates the pixel P with the image light L at the irradiation timing set for the pixel P. The timing setting unit 66b sets the irradiation timing based on the position of the focal point of the lens 30 in the Z direction (the optical axis direction of the image light L). More specifically, the timing setting unit 66b sets the irradiation timing based on the depth information of the stereoscopic image and the position of the focal point of the lens 30 in the Z direction. This will be described in more detail below.

[0060] The timing setting unit 66b acquires depth information for each pixel P. That is, the timing setting unit 66b acquires information on the position in the depth direction (Z direction) for each pixel P. Then, based on the depth information of the pixel P, the timing setting unit 66b sets an irradiation position, which is the position of the focus of the lens 30 when starting irradiation of the image light L to the pixel P. The timing setting unit 66b sets, as the irradiation position for the pixel P, the position of the focus of the lens 30 in the Z direction when the opening angle of the image light L that actually enters the eye EY from the pixel P through the lens 30 matches the opening angle of the light flux (virtual image opening angle) when light is irradiated to the eye EY from the position in the depth direction of the portion of the stereoscopic image displayed by the pixel P (the position of the virtual image formed by the pixel P). Then, the timing setting unit 66b sets, as the irradiation timing for the pixel P, the timing at which the focus of the lens 30 reaches a position whose distance from the irradiation position is within a predetermined distance range. The timing setting unit 66b sets the irradiation position for each pixel P and sets the irradiation timing for each pixel P. In this way, the timing setting unit 66b sets the irradiation position and irradiation timing for each pixel P, but the irradiation position and irradiation timing are not limited to being different for each pixel P, and for example, the irradiation position and irradiation timing of a group of pixels P that make up one image (such as a group of pixels P displaying an image of a house in Figure 5) may be set to be the same.

[0061] In the third embodiment, since the illumination timing is set for each pixel P, there are cases where only some of the pixels P are lit in one frame. For example, only the image of the house is displayed at the timing of FIG. 5, only the image of the car is displayed at the timing of FIG. 6, and only the image of the helicopter is displayed at the timing of FIG. 7. However, due to the afterimage effect caused by consecutively displaying a plurality of frames, the user U recognizes that he or she is in one image of the house, the car, and the helicopter. Also, the entire image (here, all of the house, the car, and the helicopter) may be displayed within the display time of one frame. In this case, the drive control unit 68b may move the focal position of the lens 30 by at least half the period of the reciprocating movement within the display time of one frame. This makes it possible to cover all the focal positions in the reciprocating movement within the display time of one frame, and to display the entire image.

[0062] (Processing flow) Next, the process flow of the control device 50b described above will be described. FIG. 13 is a flowchart for explaining the process flow of the control device according to this embodiment. The control device 50b causes the drive control unit 68b to move the focal position of the lens 30 back and forth in the Z direction at a predetermined cycle. The control device 50b sets the irradiation position for each pixel P from the depth information for each pixel P using the timing setting unit 66b (step S40). The control device 50b sequentially acquires the position of the focal position of the lens 30 in the Z direction, and judges for each pixel P whether the focal position of the lens 30 has reached within a predetermined distance from the irradiation position (step S42). When the focal position of the lens 30 has reached within a predetermined distance from the irradiation position (step S42; Yes), that is, when there is a pixel P for which it is judged that the focal position of the lens 30 has reached within a predetermined distance from the irradiation position, the timing setting unit 66b judges that the pixel P has reached the irradiation timing, and the irradiation control unit 64b causes the pixel P to be irradiated with image light L based on the image data (step S44). Thereafter, the irradiation of the image light L is stopped at the irradiation stop timing, and if the process is not to be ended (step S46; No), the process returns to step S40 and continues. On the other hand, if the focus of the lens 30 has not reached within a predetermined distance from the irradiation position (step S42; No), that is, if there is no pixel P for which it is determined that the focus of the lens 30 has reached within a predetermined distance from the irradiation position, the process returns to step S42 and light is not irradiated onto the pixel P until the focus of the lens 30 reaches within a predetermined distance from the irradiation position. If the process is to be ended in step S46 (step S46; Yes), the process ends.

[0063] (effect) As described above, the display device 1b according to the third embodiment provides a stereoscopic image to the user U, and includes the display unit 10, the lens 30, the drive control unit 68b, the timing setting unit 66b, and the irradiation control unit 64b. The display unit 10 includes a plurality of self-emitting pixels P, and provides a stereoscopic image to the user U by making the image light L from the pixels P reach the user U. The lens 30 is provided closer to the user U than the display unit 10 in the optical axis direction of the image light L, and is capable of changing its focus. The drive control unit 68b moves the position of the focus of the lens 30 along the optical axis direction of the image light L (Z direction in this embodiment) at a predetermined period. The timing setting unit 66b sets the irradiation timing of the image light L for each pixel P based on the position of the focus of the lens 30 in the optical axis direction (Z direction in this embodiment). The irradiation control unit 64b causes the pixel P to be irradiated with the image light L at the irradiation timing.

[0064] Here, when displaying a stereoscopic image, it is required to provide the user with a stereoscopic image appropriately. In contrast, in this embodiment, the focal point of the lens 30 is moved in the optical axis direction, and the timing of irradiating the image light L is set based on the position of the focal point. Therefore, according to this embodiment, it is possible to make the image light L reach the user U at an appropriate timing based on the position of the focal point in the optical axis direction, and a stereoscopic image can be appropriately provided to the user U. Furthermore, as described above, when displaying a stereoscopic image, a convergence accommodation conflict may occur. In contrast, in this embodiment, the focal point of the lens 30 is moved in the optical axis direction, and the timing of irradiating the image light L is set based on the position of the focal point in the optical axis direction, thereby appropriately adjusting the opening angle of the light flux of the image light L and reducing the convergence accommodation conflict.

[0065] The timing setting unit 66b also sets the irradiation timing based on depth information indicating the position in the depth direction (Z direction in this embodiment) of the stereoscopic image. According to this embodiment, since the irradiation timing is set using the depth information as well, it is possible to appropriately adjust the opening angle of the light flux of the image light L according to the stereoscopic image to be displayed, and the convergence accommodation conflict can be reduced.

[0066] Further, the timing setting unit 66b acquires information on the irradiation position, which is a position in the optical axis direction of the focus of the lens 30 such that the opening angle of the light beam of the image light L from the pixel P corresponds to the opening angle of the light beam (virtual image opening angle) when the image light L is irradiated toward the user U from a position in the depth direction in the portion of the stereoscopic image displayed by the pixel P. The timing setting unit 66b sets the timing when the focus of the lens 30 is within a predetermined distance range from the irradiation position as the irradiation timing for the pixel P. According to this embodiment, it is possible to cause the pixel P whose opening angle of the light beam and the virtual image opening angle are close to each other to emit light, and not cause the pixel P whose opening angle of the light beam and the virtual image opening angle are far from each other to emit light, so that the convergence accommodation contradiction can be appropriately reduced.

[0067] (Modification) Next, a modified example will be described. The display device 1c according to the modified example differs from the first embodiment in that the display is enlarged by a concave mirror 20C instead of the eyepiece lens 20. The description of the configuration of the modified example and the first embodiment will be omitted.

[0068] FIG. 14 is a schematic diagram of a display device according to a modified example. As shown in FIG. 14, the display device 1c according to the modified example does not have an eyepiece lens 20, and a half mirror 20B and a concave mirror 20C are provided closer to the eye EY of the user U than the display unit 10 in the optical axis direction of the image light L. The half mirror 20B and the concave mirror 20C can also be considered as optical elements. In the modified example, the image light L emitted from the display unit 10 is reflected by the half mirror 20B and enters the concave mirror 20C. The image light L that enters the concave mirror 20C becomes almost parallel light with a slight spread angle at the concave mirror 20C, and passes through the half mirror 20B to enter the eye EY of the user U.

[0069] In the modified example, the focal position of the lens 30 is controlled in the same manner as in the first embodiment. Therefore, even with a configuration like the modified example, a stereoscopic image can be appropriately provided to the user U, and the opening angle of the light flux of the image light L can be appropriately adjusted to reduce the convergence accommodation conflict, like in the first embodiment.

[0070] The modified examples are also applicable to the second and third embodiments. The display device may have a configuration other than those of the embodiments and the modified examples shown in FIG.

[0071] Although the embodiments of the present invention have been described above, the embodiments are not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined, and the configurations of each embodiment can also be combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the above-described embodiments. [Explanation of symbols]

[0072] 1 Display device 10 Display 20 Eyepiece 30 Lenses 50 Control device 60 Image information acquisition unit 62 Gaze information acquisition unit 64 Irradiation control unit 66 Focus position setting section 68 Drive control unit L Image light U User

Claims

1. A display unit that provides an image to a user by emitting light; a lens that is provided closer to the user than the display unit and has a variable focus; a gaze information acquisition unit for acquiring a detection result of the user's gaze; a focal position setting unit that sets a focal point of the lens based on a detection result of the user's line of sight and depth information indicating a position in a depth direction of the image; Including, The line-of-sight information acquisition unit detects a gaze position at which the user is gazing in the image based on a result of the line-of-sight detection, the focal position setting unit sets, based on depth information of the gaze position, a focal position of the lens such that a difference between an opening angle of a luminous flux of the light and an opening angle of a luminous flux when the light is irradiated from a position in a depth direction at the gaze position toward the user falls within a predetermined range; and a drive control unit that moves a focal position of the lens at a predetermined cycle; and a timing setting unit that sets the irradiation timing of the light for each pixel based on the focal position of the lens; Further comprising: Display device.

2. a display step of providing an image to a user by irradiating light from a display unit; A gaze information acquisition step of acquiring a detection result of the user's gaze; a focus position setting step of setting a focus of a lens that is provided on the user side of the display unit and has a variable focus based on a result of the detection of the user's line of sight and depth information indicating a position in a depth direction of the image; Including, In the line-of-sight information acquisition step, a gaze position at which the user is gazing in the image is detected based on a result of the line-of-sight detection, In the focal position setting step, a difference between an opening angle of a luminous flux of the light and an opening angle of a luminous flux when the light is irradiated from a depth direction position at the gaze position toward the user is within a predetermined range based on depth information of the gaze position, and moving the focal position of the lens at a predetermined period; setting the light irradiation timing for each pixel based on a focal position of the lens; Further comprising: Display control method.

3. a display step of providing an image to a user by irradiating light from a display unit; A gaze information acquisition step of acquiring a detection result of the user's gaze; a focus position setting step of setting a focus of a lens that is provided on the user side of the display unit and has a variable focus based on a result of the detection of the user's line of sight and depth information indicating a position in a depth direction of the image; The computer executes the following: In the line-of-sight information acquisition step, a gaze position at which the user is gazing in the image is detected based on a result of the line-of-sight detection, In the focal position setting step, a difference between an opening angle of a luminous flux of the light and an opening angle of a luminous flux when the light is irradiated from a depth direction position at the gaze position toward the user is within a predetermined range based on depth information of the gaze position, and moving the focal position of the lens at a predetermined period; setting the light irradiation timing for each pixel based on a focal position of the lens; Further comprising: Display control program.

Images