Display device, display method, and program

Abstract

To allow for optimally displaying images of virtual objects superimposed on an actual landscape.SOLUTION: A display device is provided, comprising an external light measurement unit for measuring external light, an image generation unit for generating object images, a first display unit for displaying the object images, a second display unit located on the outer side of the first display unit, and a control unit capable of adjusting transmittance of a region of the second display unit, where the control unit adjusts transmittance of the region of the second display unit according to brightness of the external light.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display device, a display method, and a program.

Background Art

[0002] In recent years, as a technology for realizing virtual reality (VR: Virtual Reality) and augmented reality (AR: Augmented Reality), for example, a head-mounted display device (head-mounted display: HMD: Head Mounted Display) that a user wears on the head is known. Such a display device can display an image of a virtual object superimposed on an actual image acquired from the outside. Examples of such a display device include those described in Patent Document 1 below.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, while the actual image acquired from the outside is a clear display image, the image of the virtual object is a semi-transparent display image. The user views the image of the virtual object superimposed on the clear display image. However, since the image of the virtual object is semi-transparent, the user can see the actual image through the virtual object, and there is a problem that the reality of the image of the virtual object becomes insufficient.

[0005] The present invention has been made in view of the above, and an object thereof is to optimally display an image of a virtual object superimposed on an actual landscape.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, a display device according to the present invention includes an external light measurement unit that measures external light, a video generation unit that generates an object image, a first display unit that displays an image of the object, a second display unit that is disposed outside the first display unit, and a control unit that can adjust the transmittance of a region of the second display unit. The control unit adjusts the transmittance of the region of the second display unit according to the brightness of the external light.

[0007] A display method according to the present invention includes a step of measuring external light, a step of generating an object image, a step of displaying an image of the object on a first display unit, and a step of adjusting the transmittance of a region of a second display unit disposed outside the first display unit according to the brightness of the external light.

[0008] A program according to the present invention causes a computer operating as a display device to execute a step of measuring external light, a step of generating an object image, a step of displaying an image of the object on a first display unit, and a step of adjusting the transmittance of a region of a second display unit disposed outside the first display unit according to the brightness of the external light.

Advantages of the Invention

[0009] According to the present invention, there is an effect that an image of a virtual object can be optimally displayed by overlapping it on an actual landscape.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4-1

Figure 4-2

Figure 4-3

Figure 4-4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11-1

Figure 11-2

Figure 11-3

Figure 11-4

Figure 11-5

Figure 11-6

Figure 12-1

Figure 12-2

Figure 12-3

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of a display device, a display method, and a program according to the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited by the following embodiments.

[0012] <First Embodiment> [Display Device] The display device of the first embodiment will be described by applying it to a head-mounted display device that realizes augmented reality.

[0013] FIG. 1 is a block configuration diagram showing the display device according to the first embodiment.

[0014] In the first embodiment, as shown in FIG. 1, the display device 10 includes a first display unit 11, a second display unit 12, a video generation unit 13, a light control unit (control unit) 14, an illuminance sensor (external light measurement unit) 15, and a storage unit 16.

[0015] The first display unit 11 is disposed inside the display device 10. The first display unit 11 acquires a video signal related to an image of a virtual object from the video generation unit 13 and displays the image of the virtual object.

[0016] The second display unit 12 is disposed outside the image of the virtual object. The second display unit 12 acquires a control signal from the light control unit 14 and adjusts the transmittance of the second display unit 12.

[0017] The video generation unit 13 generates a virtual object based on the video signal and causes the first display unit 11 to display an image of the virtual object. Further, the video generation unit 13 can adjust the brightness and color of the image of the virtual object according to the brightness of the real space. Note that the video generation unit 13 transmits the video signal related to the image of the virtual object acquired from the storage unit 16 to the first display unit 11 and the second display unit 12. Also, the video generation unit 13 transmits the information regarding the brightness of the real space acquired from the illuminance sensor 15 to the light control unit 14.

[0018] The light control unit 14 can adjust the transmittance of the second display unit 12 according to the brightness of the real space input from the video generation unit 13. The light control unit 14 can adjust the transmittance of the area facing the image of the virtual object. The light control unit 14 determines whether the brightness of the image of the object existing in the real space is brighter than the brightness of the image of the virtual object from the video signal related to the image of the virtual object acquired from the video generation unit 13 and the information regarding the brightness of the real space. The light control unit 14 transmits a control signal to the second display unit 12.

[0019] The illuminance sensor 15 measures external light. That is, the illuminance sensor 15 measures the brightness (illuminance) of the real space by acquiring information on the image in the vicinity of the display device 10, particularly the front image, that is, the image of the real space. For example, the illuminance sensor 15 is a CMOS image sensor, but is not limited to this configuration. The illuminance sensor 15 transmits the information regarding the brightness of the real space to the video generation unit 13.

[0020] The storage unit 16 stores a video signal and the like related to the image of the virtual object. The storage unit 16 transmits the video signal related to the image of the virtual object to the video generation unit 13. Note that the video signal related to the image of the virtual object stored in the storage unit 16 may be stored in advance, or may acquire and store the video signal related to the image of the virtual object via a communication unit (not shown) connectable to the outside. The storage unit 16 is, for example, a memory or an HDD (Hard Disk Drive).

[0021] Note that the video generation unit 13 and the light control unit 14 are configured by at least one of, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a RAM (Random Access Memory), and a ROM (Read Only Memory).

[0022] FIG. 2 is a schematic diagram showing a specific configuration of the display device.

[0023] As shown in FIG. 2, the first display unit 11 is supported by the exterior 20. The first display unit 11 includes a first display panel 21, a half mirror 22, and a combiner mirror 23. The first display panel 21 is horizontally arranged at the upper part of the exterior 20. The first display panel 21 has a planar shape, and various display panels such as a liquid crystal panel, an organic EL panel, and a plasma panel can be applied, for example. The first display panel 21 has a display surface 21a capable of displaying an image of the virtual object A on the lower surface. The display surface 21a can irradiate display light La downward, that is, toward the inside of the exterior 20.

[0024] The half mirror 22 is arranged inside the exterior 20 below the first display panel 21. The half mirror 22 is arranged at a predetermined angle with respect to the first display panel 21. The half mirror 22 is provided with a reflective coating 22a on the upper surface side and an anti-reflective coating 22b on the lower surface side. The half mirror 22 reflects light from above and transmits light from the front. That is, the half mirror 22 reflects the display light La irradiated from the first display panel 21 toward the combiner mirror 23. Also, the half mirror 22 transmits the reflected light Lb reflected by the combiner mirror 23 rearward.

[0025] The combiner mirror 23 is disposed in front of the half mirror 22 inside the exterior 20. The combiner mirror 23 is disposed vertically at the front part of the exterior 20. The combiner mirror 23 has a concave shape. The combiner mirror 23 is provided with a reflective coating 23a on the inner surface side. The combiner mirror 23 reflects the display light La irradiated from the first display panel 21 and reflected by the half mirror 22, and irradiates the half mirror 22 as reflected light Lb.

[0026] The first display unit 11 reflects the display light La irradiated from the first display panel 21 forward by the half mirror 22, reflects the display light La backward by the combiner mirror 23, and transmits the reflected light Lb through the half mirror 22 to guide it to the user's eyeball. Therefore, the user visually recognizes the image of the virtual object A displayed by the first display unit 11 as being located in front of the display device 10.

[0027] In addition, the combiner mirror 23 takes in the real image light Lc that constitutes the image of the object B existing in the real space and transmits it from the outside to the half mirror 22 side. The first display unit 11 allows the real image light Lc that constitutes the image of the object B to reach the user's left and right eyeballs through the combiner mirror 23 and the half mirror 22. Therefore, the user directly visually recognizes the image of the object B existing in the real space.

[0028] At this time, the reflected light Lb (display light La) that generates the image of the virtual object A and the real image light Lc that generates the image of the object B existing in the real space reach the user's eyeball. Therefore, the user visually recognizes a composite image in which the image of the virtual object A overlaps the image of the object B existing in the real space.

[0029] The second display unit 12 has a second display panel 31. The second display panel 31 is supported along the vertical direction at the front part of the exterior 20. The second display panel 31 is arranged at a predetermined interval outside the combiner mirror 23. The second display panel 31 has a planar shape, and various display panels such as a liquid crystal panel, an organic EL panel, and a plasma panel can be applied. The second display panel 31 is provided with pixels in a matrix, and each pixel can be adjusted and controlled to be transparent to opaque. Therefore, the second display panel 31 can adjust the transmittance between 0% and 100%.

[0030] As shown in FIGS. 1 and 2, the illuminance sensor 15 measures the brightness of the real space. The light control unit 14 can adjust the transmittance of the region of the second display panel 31 facing the image of the virtual object A (here, a triangular image corresponding to the shape of the virtual object A) according to the brightness of the image of the light incident from the real space (the image of the object B).

[0031] Specifically, since the light rays of the virtual image (image) of the virtual object A displayed on the first display panel 21 cross the second display panel 31, the transmittance of the region of this crossing part is adjusted. Here, the center part of the field of view is close to the second display panel 31 and the combiner mirror 23, but the distance is farther at the peripheral part of the field of view. Therefore, a larger panel is required to cover the entire field of view. The second display panel 31 shown in FIG. 2 is a planar panel, but it may be a curved panel instead of the planar panel. The curved surface shape of this curved panel is preferably a shape along the curved surface shape of the combiner mirror 23. From the center to the peripheral part of the field of view, adjustment and control can be performed in a shape corresponding to the shape of the image of the virtual object A. The device can be miniaturized and adjusted and controlled neatly up to the peripheral part of the field of view.

[0032] Note that the video generation unit 13 and the light control unit 14 may be installed at any position.

[0033] [Display method] FIG. 3 is a flowchart showing the display method according to the first embodiment.

[0034] As shown in FIGS. 1 to 3, in step S11, the video generation unit 13 acquires and displays the image information of the virtual object A based on the video signal. In step S12, the illuminance sensor 15 acquires the image information of the object B existing in the real space and acquires the brightness of the image of the object B existing in the real space. In step S13, the light control unit 14 determines whether the brightness of the image of the object B existing in the real space is brighter than the brightness of the image of the virtual object A. Note that the brightness determination process of the image in step S13 is determined based on a preset determination value. The determination value is set in advance by experiments, simulations, etc. based on the sharpness of the image of the virtual object A.

[0035] Here, when the light control unit 14 determines (Yes) that the brightness of the image of the object B existing in the real space is brighter than the brightness of the image of the virtual object A, the process proceeds to step S14. In step S14, the light control unit 14 reduces the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes less permeable to light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes darker. Therefore, the user can clearly visually recognize the image of the virtual object A without being affected by the brightness of the image of the object B.

[0036] On the other hand, in step S13, when the light control unit 14 determines (No) that the brightness of the image of the object B existing in the real space is not brighter than the brightness of the image of the virtual object A, the process proceeds to step S15. In step S15, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes more permeable to light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can visually recognize the image of the object B through the image of the virtual object A, and can clearly visually recognize both the image of the virtual object A and the image of the object B.

[0037] In step S15, not only the transmittance of a predetermined area of the second display unit 12 is simply increased, but the transmittance of the predetermined area of the second display unit 12 may be adjusted according to the brightness of the image of the object B and the brightness of the image of the virtual object A.

[0038] FIG. 4-1 is a schematic diagram showing an image of the first display unit, FIG. 4-2 is a schematic diagram showing an image of the second display unit, FIG. 4-3 is a schematic diagram showing an image in which the image of the first display unit is superimposed on the image of the second display unit, and FIG. 4-4 is a schematic diagram showing an image in which the image of the first display unit is superimposed on the image of the second display unit and subjected to an enhancement process.

[0039] As shown in FIG. 4-1, an image of the virtual object A is displayed on the first display unit 11. As shown in FIG. 4-2, the transmittance of a predetermined area B1 facing the image of the virtual object A in the second display unit 12 is lowered to make it darker. Then, as shown in FIG. 4-3, the image of the virtual object A of the first display unit 11 is displayed on the darkened predetermined area B1 of the second display unit 12. At this time, it is preferable that the size of the predetermined area B1 to be darkened is the same as or larger than the size of the image of the virtual object A. Therefore, the user can clearly visually recognize the image of the virtual object A. Further, as shown in FIG. 4-4, the virtual object A image can be clearly displayed by the enhancement process in which the video generation unit 13 forms a border C around the image of the virtual object A. The border C is preferably applied with white, black, or the complementary color of the color of the image of the virtual object A.

[0040] [Modification Example] FIG. 5 is a flowchart showing a modification example of the display method according to the first embodiment.

[0041] As shown in FIGS. 1, 2, and 5, in step S21, the video generation unit 13 acquires and displays the image information of the virtual object A based on the video signal. In step S22, the illuminance sensor 15 acquires the image information of the object B existing in the real space and acquires the brightness of the image of the object B existing in the real space. In step S23, the light control unit 14 determines whether the chromaticity of the image of the object B existing in the real space is close to the chromaticity of the image of the virtual object A. The chromaticity is the property of a color ignoring brightness, the color of light excluding lightness, and a quantitative representation of hue and chroma. Note that the chromaticity determination process of the image in step S13 is determined based on a preset determination value. The determination value is set in advance based on experiments, simulations, etc. for the image of the virtual object A based on sharpness.

[0042] Here, when the light control unit 14 determines (Yes) that the chromaticity of the image of the object B existing in the real space is close to the chromaticity of the image of the virtual object A, the process proceeds to step S24. In step S24, the light control unit 14 decreases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes less permeable to light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes darker. Therefore, the user can clearly view the image of the virtual object A without being affected by the chromaticity of the image of the object B. At this time, the video generation unit 13 may adjust the color of the image of the virtual object A so that the image of the virtual object A becomes clearer.

[0043] On the other hand, if in step S13, the light control unit 14 determines (No) that the chromaticity of the image of the object B existing in the real space is not close to the chromaticity of the image of the virtual object A, the process proceeds to step S25. In step S25, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes more permeable to the light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can visually recognize the image of the object B through the image of the virtual object A, and can clearly visually recognize both the image of the virtual object A and the image of the object B.

[0044] <Second Embodiment> FIG. 6 is a block configuration diagram showing a display device according to the second embodiment.

[0045] In the second embodiment, as shown in FIG. 6, the display device 10A includes a first display unit 11, a second display unit 12, a video generation unit 13, a light control unit 14, an illuminance sensor 15, a storage unit 16, and a posture sensor (walking detection unit) 17. Here, the first display unit 11, the second display unit 12, the video generation unit 13, the light control unit 14, the illuminance sensor 15, and the storage unit 16 are the same as those in the first embodiment, and the description thereof is omitted.

[0046] The posture sensor 17 detects the walking of the user. The posture sensor 17 transmits the detection result of the walking to the video generation unit 13. The video generation unit 13 can adjust the brightness and color of the image of the virtual object according to the detection result of the posture sensor 17. Note that the video generation unit 13 transmits the detection result of the walking acquired from the posture sensor 17 to the light control unit 14. The light control unit 14 can adjust the transmittance of the second display unit 12 according to the detection result of the posture sensor 17 input from the video generation unit 13. Specifically, when the posture sensor 17 detects the walking of the user, the light control unit 14 adjusts the transmittance of a predetermined area of the second display unit 12 to be increased.

[0047] FIG. 7 is a flowchart showing a display method according to the second embodiment.

[0048] As shown in FIGS. 6 and 7, in step S31, the video generation unit 13 acquires and displays the image information of the virtual object A based on the video signal. In step S32, the illuminance sensor 15 acquires the image information of the object B existing in the real space and acquires the brightness of the image of the object B existing in the real space. In step S33, the walking state of the user detected by the attitude sensor 17 is acquired.

[0049] In step S34, the light control unit 14 determines whether the user is walking based on the walking state of the user detected by the attitude sensor 17. Here, if the light control unit 14 determines (No) that the user is not walking, the process proceeds to step S35. In step S35, the light control unit 14 determines whether the brightness of the image of the object B existing in the real space is brighter than the brightness of the image of the virtual object A. Here, if the light control unit 14 determines (Yes) that the brightness of the image of the object B existing in the real space is brighter than the brightness of the image of the virtual object A, the process proceeds to step S36. In step S36, the light control unit 14 reduces the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes less permeable to light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes darker. Therefore, the user can clearly view the image of the virtual object A without being affected by the brightness of the image of the object B.

[0050] On the other hand, in step S35, if the light control unit 14 determines (No) that the brightness of the image of the object B existing in the real space is not brighter than the brightness of the image of the virtual object A, the process proceeds to step S37. In step S37, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes more permeable to light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can view the image of the object B through the image of the virtual object A and can clearly view both the image of the virtual object A and the image of the object B.

[0051] Also, in step S34, when the light control unit 14 determines (Yes) that the user is walking, it proceeds to step S37. In step S37, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes more permeable to light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can visually recognize the image of the object B through the image of the virtual object A. That is, although the user visually recognizes the image of the virtual object A while walking, the user can visually recognize the image of the object B, that is, the surrounding background image, through the image of the virtual object A and can walk safely.

[0052] Note that when it is determined that the user is walking, it is preferable that the light control unit 14 increases the transmittance of the predetermined area of the second display unit 12 more than when it is determined that the brightness of the image of the object B existing in the real space is not brighter than the brightness of the image of the virtual object A. That is, when it is determined that the user is walking, it is preferable to increase the transmittance of the predetermined area of the second display unit 12 to near 100%. Also, at this time, an emphasis process by bordering the image of the virtual object A may be performed.

[0053] [First Modification Example] FIG. 8 is a flowchart showing a first modification example of the display method according to the second embodiment.

[0054] As shown in FIGS. 6 and 8, in step S41, the video generation unit 13 acquires and displays the image information of the virtual object A based on the video signal. In step S42, the illuminance sensor 15 acquires the image information of the object B existing in the real space and acquires the brightness of the image of the object B existing in the real space. In step S43, the walking state of the user detected by the attitude sensor 17 is acquired.

[0055] In step S44, the light control unit 14 determines whether the user is walking based on the walking state of the user detected by the attitude sensor 17. Here, if the light control unit 14 determines (No) that the user is not walking, it proceeds to step S45. In step S45, the light control unit 14 determines whether the chromaticity of the image of the object B existing in the real space is close to the chromaticity of the image of the virtual object A. Here, if the light control unit 14 determines (Yes) that the chromaticity of the image of the object B existing in the real space is close to the chromaticity of the image of the virtual object A, it proceeds to step S46. In step S46, the light control unit 14 reduces the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes less permeable to the light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes darker. Therefore, the user can clearly view the image of the virtual object A without being affected by the brightness of the image of the object B.

[0056] On the other hand, in step S45, if the light control unit 14 determines (No) that the chromaticity of the image of the object B existing in the real space is not close to the chromaticity of the image of the virtual object A, it proceeds to step S47. In step S47, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes more permeable to the light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can view the image of the object B through the image of the virtual object A and can clearly view both the image of the virtual object A and the image of the object B.

[0057] Also, in step S44, when the light control unit 14 determines (Yes) that the user is walking, it proceeds to step S47. In step S47, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, in the second display unit 12, light from the real space becomes more likely to pass through the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can visually recognize the image of the object B through the image of the virtual object A. That is, although the user visually recognizes the image of the virtual object A while walking, the user can visually recognize the image of the object B, that is, the surrounding background image, through the image of the virtual object A and can walk safely.

[0058] Note that when it is determined that the user is walking, it is preferable that the light control unit 14 increases the transmittance of the predetermined area of the second display unit 12 more than when it is determined that the brightness of the image of the object B existing in the real space is not brighter than the brightness of the image of the virtual object A. That is, when it is determined that the user is walking, it is preferable to increase the transmittance of the predetermined area of the second display unit 12 to near 100%. Also, at this time, an emphasis process by outlining the image of the virtual object A may be performed.

[0059] [Second Modification Example] FIG. 9 is a flowchart showing a second modification example of the display method according to the second embodiment.

[0060] As shown in FIGS. 6 and 9, in step S51, the video generation unit 13 acquires and displays the image information of the virtual object A based on the video signal. In step S52, the illuminance sensor 15 acquires the image information of the object B existing in the real space through the combiner mirror 23 and acquires the brightness of the image of the object B existing in the real space. In step S53, the walking state of the user detected by the attitude sensor 17 is acquired.

[0061] In step S54, the light control unit 14 determines whether the user is walking based on the walking state of the user detected by the attitude sensor 17. Here, if the light control unit 14 determines (No) that the user is not walking, it proceeds to step S55. In step S55, the light control unit 14 determines whether the brightness of the image of the object B existing in the real space is brighter than the brightness of the image of the virtual object A. Here, if the light control unit 14 determines (Yes) that the brightness of the image of the object B existing in the real space is brighter than the brightness of the image of the virtual object A, it proceeds to step S56. In step S56, the light control unit 14 decreases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes less permeable to the light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes darker. Therefore, the user can clearly visually recognize the image of the virtual object A without being affected by the brightness of the image of the object B.

[0062] On the other hand, in step S55, if the light control unit 14 determines (No) that the brightness of the image of the object B existing in the real space is not brighter than the brightness of the image of the virtual object A, it proceeds to step S57. In step S57, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, the second display unit 12 becomes more permeable to the light from the real space in the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can visually recognize the image of the object B through the image of the virtual object A and can clearly visually recognize both the image of the virtual object A and the image of the object B.

[0063] Also, in step S54, when the light control unit 14 determines (Yes) that the user is walking, the light control unit 14 transmits the determination result to the video generation unit 13 and proceeds to step S58. In step S58, the video generation unit 13 determines whether the image of the virtual object A is at the center of the user's field of view. In this case, for example, it is determined whether a part of the image of the virtual object A overlaps the center of the user's field of view. Also, it may be determined whether a part of the image of the virtual object A is in a predetermined area centered on the center of the user's field of view. Here, when the video generation unit 13 determines (Yes) that the image of the virtual object A is at the center of the user's field of view, the video generation unit 13 transmits this determination result to the light control unit 14. In step S59, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. At this time, the video generation unit 13 may adjust to lower the brightness of the image of the virtual object A.

[0064] Then, in step S60, the video generation unit 13 moves the image of the virtual object A to the peripheral part of the user's field of view, that is, to the edge of the field of view. Then, the second display unit 12 becomes more permeable to light from the real space in a predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can visually recognize the image of the object B through the image of the virtual object A. Also, the image of the virtual object A will be displayed at the edge rather than the center of the user's field of view. Therefore, the user can visually recognize the image of the object B without the image of the virtual object A getting in the way. That is, although the user visually recognizes the image of the virtual object A while walking, by clearly visually recognizing the surrounding background image, the user can walk safely.

[0065] On the other hand, in step S58, when the video generation unit 13 determines (No) that the image of the virtual object A is not at the center of the user's field of view, the video generation unit 13 transmits this determination result to the light control unit 14 and proceeds to step S57. In step S57, the light control unit 14 increases the transmittance of a predetermined area of the second display unit 12 facing the image of the virtual object A. Then, in the second display unit 12, light from the real space is more likely to pass through the predetermined area, and the background side of the image of the virtual object A becomes brighter. Therefore, the user can visually recognize the image of the object B through the image of the virtual object A. That is, although the user visually recognizes the image of the virtual object A while walking, the user can visually recognize the image of the object B, that is, the surrounding background image, through the image of the virtual object A and can walk safely.

[0066] <Third Embodiment> FIG. 10 is a schematic diagram showing a specific configuration of the display device according to the third embodiment.

[0067] In the third embodiment, as shown in FIG. 1, the display device 10B includes a first display unit 11, a video generation unit 13, an illuminance sensor 15, and a storage unit 16.

[0068] Here, since the first display unit 11, the video generation unit 13, the illuminance sensor 15, and the storage unit 16 have the same configurations and functions as those in the first embodiment, the description thereof is omitted.

[0069] The display device 10B causes the reflected light Lb (display light La) that generates the image of the virtual object A by the first display unit 11 and the real image light Lc that generates the image of the object B existing in the real space to reach the user's eyeball. Therefore, the user visually recognizes a composite image in which the image of the virtual object A overlaps the image of the object B existing in the real space.

[0070] At this time, when the light control unit 14 determines that the brightness of the image of the object B existing in the real space is brighter than the brightness of the image of the virtual object A, and when it determines that the brightness of the image of the virtual object A is approximated to the brightness of the image of the object B existing in the real space, the light control unit 14 transmits the determination result to the video generation unit 13. The video generation unit 13 performs an enhancement process on the image of the virtual object A based on the transmitted determination result.

[0071] FIG. 11-1 is a schematic diagram showing an image of a virtual object, FIG. 11-2 is a schematic diagram showing an example of an image of the real space, FIG. 11-3 is a schematic diagram showing an example of an image in which the image of the virtual object is superimposed on the image of the real space, FIG. 11-4 is a schematic diagram showing an example of an image of the real space, FIG. 11-5 is a schematic diagram showing an example of an image in which the image of the virtual object is superimposed on the image of the real space, and FIG. 11-6 is a schematic diagram showing an image obtained by superimposing the image of the virtual object on the image of the real space and performing an enhancement process.

[0072] As shown in FIG. 11-1, an image of the virtual object A is displayed on the first display unit 11. As shown in FIG. 11-2, an image of the object B with a low brightness existing in the real space is acquired. As shown in FIG. 11-3, a composite image in which the image of the virtual object A is superimposed on the image of the object B with a low brightness existing in the real space is displayed. Here, since the brightness of the image of the object B existing in the real space is darker than the brightness of the image of the virtual object A, the user can clearly visually recognize the image of the virtual object A.

[0073] On the other hand, as shown in FIG. 11-4, an image of a bright object B existing in the real space is acquired. As shown in FIG. 11-5, a composite image is displayed in which the image of the virtual object A is superimposed on the image of the bright object B existing in the real space. Here, since the brightness of the image of the object B existing in the real space and the brightness of the image of the virtual object A are similar, it is difficult for the user to clearly view the image of the virtual object A. Therefore, as shown in FIG. 11-6, the image generator 13 performs an emphasis process to form a border C around the image of the virtual object A, thereby clearly displaying the image of the virtual object A. It is preferable that the border C is white, black, or a color opposite to the color of the image of the virtual object A.

[0074] Figure 12-1 is a schematic diagram showing an example of an image in which an image of a virtual object is superimposed on an image in real space, Figure 12-2 is a schematic diagram showing an example of an image in which an image of a virtual object is superimposed on an image in real space, and Figure 12-3 is a schematic diagram showing an image in which an image of a virtual object is superimposed on an image in real space and subjected to enhancement processing.

[0075] As shown in Fig. 12-1, a composite image is displayed in which a three-dimensional image of a virtual object D is superimposed on an image of a dark object B existing in real space. Since the brightness of the image of object B existing in real space is darker than the brightness of the three-dimensional image of virtual object D, the user can clearly view the three-dimensional image of virtual object D.

[0076] On the other hand, as shown in FIG. 12-2, a composite image is displayed in which a three-dimensional image of a virtual object D is superimposed on an image of a bright object B existing in real space. Here, since the brightness of the image of the object B existing in real space and the brightness of the three-dimensional image of the virtual object D are similar, it is difficult for the user to clearly view the three-dimensional image of the virtual object D. Therefore, as shown in FIG. 12-3, the image generator 13 performs an emphasis process to form a border E around the three-dimensional image of the virtual object D, thereby clearly displaying the three-dimensional image of the virtual object D.

[0077] [Operation and Effect of Embodiment] In this embodiment, an illuminance sensor (outside light measurement unit) 15 for measuring outside light, a video generation unit 13 for generating an object image, a first display unit 11 for displaying an object image, a second display unit 12 disposed outside the first display unit 11, and a light control unit (control unit) 14 capable of adjusting the transmittance of the region of the second display unit 12 are provided. The light control unit adjusts the transmittance of the region of the second display unit 12 according to the brightness of the outside light.

[0078] Therefore, by adjusting the transmittance of a predetermined region of the second display unit 12 according to the brightness of the outside light from the real space, the user can clearly visually recognize the object image. As a result, the image of the virtual object can be optimally displayed by overlapping it on the actual scenery. For example, when the image of the outside light from the real space is brighter than the image of the virtual object, the transmittance of a predetermined region of the second display unit 12 is lowered to make it darker. Then, the user can clearly visually recognize the image of the virtual object.

[0079] In this embodiment, a posture sensor (walking detection unit) 17 for detecting the walking state of the user is provided, and the light control unit 14 adjusts the transmittance of the region of the second display unit 12 based on the walking state of the user detected by the posture sensor 17. Therefore, when the user is walking, the transmittance of a predetermined region of the second display unit 12 facing the image of the virtual object is adjusted, and the user can visually recognize the image of the light incident from the real space through the image of the virtual object. As a result, the user can walk safely while clearly visually recognizing the surrounding background image.

[0080] In this embodiment, the video generation unit 13 adjusts the color of the object image according to the brightness of the outside light. Therefore, the image of the virtual object can be clearly displayed.

[0081] Although the display device 10 according to the present invention has been described so far, it may be implemented in various different forms other than the above-described embodiments.

[0082] Each component of the illustrated display device 10 is conceptually functional and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to that shown in the figure, and all or part of it may be functionally or physically distributed or integrated in arbitrary units according to the processing load of each device, usage conditions, etc.

[0083] The configuration of the display device 10 is realized, for example, by a program loaded into a memory as software. In the above embodiment, it has been described as a functional block realized by the cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof.

[0084] The above-described components include those that can be easily assumed by those skilled in the art and substantially identical ones. Furthermore, the above-described configurations can be combined as appropriate. Also, various omissions, substitutions, or changes in the configuration are possible without departing from the gist of the present invention.

[0085] Also, in the above embodiment, the first display unit 11 is configured from the first display panel 21, the half mirror 22, and the combiner mirror 23, but is not limited to this configuration and arrangement.

Explanation of Reference Numerals

[0086] 10, 10A, 10B Display device 11 First display unit 12 Second display unit 13 Video generation unit 14 Light control unit (control unit) 15 Illuminance sensor (external light measurement unit) 16 Storage unit 17 Attitude sensor (walking detection unit) 21 First display panel 22 Half mirror 23 Combiner mirror 31 Second display panel La Display light Lb Reflected light Lc real image light

Claims

1. An external light measurement unit that measures the brightness of a predetermined object existing in the real space, a video generation unit that generates an image of the object, a first display unit that displays the image of the object, a second display unit that is disposed outside the first display unit as viewed by the user and transmits light from the real space facing the image of the object, a control unit that can adjust the transmittance of the region of the second display unit, comprising: when the control unit determines that the brightness of the predetermined object overlapping the image of the object is brighter than the brightness of the image of the object, it reduces the transmittance of a predetermined region overlapping the image of the object and consisting of the size of the image of the object in the second display unit; when it determines that the brightness of the predetermined object overlapping the image of the object is not brighter than the brightness of the image of the object, it increases the transmittance of the predetermined region in the second display unit, a display device.

2. further comprising a walking detection unit that detects the walking state of the user, wherein the control unit adjusts the transmittance of the region of the second display unit based on the walking state of the user detected by the walking detection unit, the display device according to Claim 1.

3. the video generation unit adjusts the color of the image of the object according to the brightness of the external light, the display device according to Claim 1 or Claim 2.

4. A step of measuring the brightness of a predetermined object existing in the real space, a step of generating an image of the object, a step of displaying the image of the object on a first display unit, a step of adjusting the transmittance of the region of a second display unit that is disposed outside the first display unit as viewed by the user and transmits light from the real space facing the image of the object according to the brightness of the external light, when it is determined that the brightness of the predetermined object overlapping the image of the object is brighter than the brightness of the image of the object, a step of reducing the transmittance of a predetermined region overlapping the image of the object and consisting of the size of the image of the object in the second display unit; when it is determined that the brightness of the predetermined object overlapping the image of the object is not brighter than the brightness of the image of the object, a step of increasing the transmittance of the predetermined region in the second display unit, a display method including.

5. A step of measuring the brightness of a predetermined object existing in the real space, a step of generating an image of the object, The step of displaying an image of the object on a first display unit; The step of adjusting the transmittance of a region of a second display unit, which is disposed outside the first display unit as viewed by a user and transmits light from the real space facing the image of the object, according to the brightness of external light; When it is determined that the brightness of the predetermined object overlapping the image of the object is brighter than the brightness of the image of the object, reducing the transmittance of a predetermined region that overlaps the image of the object and has the size of the image of the object in the second display unit, and when it is determined that the brightness of the predetermined object overlapping the image of the object is not brighter than the brightness of the image of the object, increasing the transmittance of the predetermined region in the second display unit; A program that causes a computer operating as a display device to execute.

Images