Virtual image display device

Abstract

The purpose of the present invention is to provide a virtual image display device having a function of projecting a predetermined virtual image into the field of view of a user, the virtual image display device satisfactorily improving the problems of physiological burden on the user and narrow viewpoint range when the virtual image is viewed. In order to achieve the above purpose, a virtual image display device provided with a function of projecting a predetermined virtual image into a field of view of a user is configured so as to comprise: a visual distance detection unit for detecting a visual distance of at least some of a visual object watched by the user or a plurality of objects in the field of view; detecting an actual visual distance from the user to the object; a virtual image light generation unit having a function of generating a plurality of virtual image lights that are independent from each other; a virtual image light transmission and projection unit having a function of transmitting and projecting the virtual image light to the left and right eyes of the user via substantially the same optical path so that a predetermined virtual image is visually recognized at a predetermined position within the field of view of the user; and a virtual image viewing distance control unit having a function of variably controlling the viewing distance of the virtual image viewed by the user using the plurality of projected virtual image lights.

Description

[0001] This application is a divisional application of an application with an application date of October 30, 2018, an application number of "201880070301.8", and an invention title of "Virtual image display device and head-mounted display using the same". Technical Field

[0002] The present invention relates to a virtual image display device having a function of allowing an arbitrary image to be visually recognized as a virtual image by a user through a predetermined optical system, and particularly to a virtual image display device having a function of overlapping and displaying a virtual image within an actual space visual field visually recognized by the user. Background Art

[0003] As an example of a virtual image display device having a function of overlapping and displaying a predetermined virtual image within an actual space visual field visually recognized by a user, a head-mounted display (hereinafter referred to as an HMD), a head-up display (hereinafter referred to as an HUD), etc. are known. In these virtual image display devices, the presence or absence of a function of matching the visual recognition distance of the virtual image overlapped and displayed within the actual space visual field of the user to the actual space visual recognition distance of the actual object within the visual field (hereinafter, referred to as the virtual image visual recognition distance matching function) becomes an important performance requirement that contributes to improving the visual recognition of the user by reducing the physiological burden on the user.

[0004] Conventionally, as a method for realizing the virtual image visual recognition distance matching function in an HMD particularly worn on the head of a user, for example, in Patent Document 1, a so-called "binocular parallax method" virtual image visual recognition distance matching method is disclosed, in which a visual recognition distance (actual visual recognition distance) from an actual space object being gazed at by the user is detected by a predetermined detection unit, and the relative deviation amount of the left-eye visual recognition image and the right-eye visual recognition image of the user generated when gazing at the object at the actual visual recognition distance, that is, the "binocular parallax", is given to the left-eye display virtual image visually recognized only by the left eye of the user and the right-eye display virtual image visually recognized only by the right eye of the user, so that the user has an optical illusion as if the visual recognition distance of the virtual image matches the actual space visual recognition distance.

[0005] Prior Art Documents

[0006] Patent Documents

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-186561 Summary of the Invention

[0008] However, in the virtual image visual recognition distance matching method based on the "binocular parallax method" disclosed in Patent Document 1, due to the binocular parallax, there is a deviation between the perceived virtual image distance perceived by the user and the actual virtual image distance at which the user's eyes are actually focused on the virtual image, that is, the actual virtual image distance, so there is a problem that it imposes a physiological burden on the user and the user's fatigue is great when using it for a long time. In addition, the binocular parallax itself depends not only on the visual recognition distance, but also on the direction of the user's gaze, so there is also a problem that the user's viewpoint range that can accurately visually recognize the virtual image at a predetermined visual recognition distance, the so-called eye box, is extremely narrow. Furthermore, in this "binocular parallax method", different virtual images with different positions of the display image items must be displayed when used for the left eye and when used for the right eye, so it is necessary to prepare two independent virtual image display optical systems in the HMD when used for the left eye and when used for the right eye, which has a significant limitation in the miniaturization, lightness, and cost reduction of the HMD.

[0009] Based on the above background, an object of the present invention is to satisfactorily improve the above-mentioned problems of physiological burden on the user and narrow visual range.

[0010] In view of the above-mentioned background technology and problems, the present invention, if taking one example, is configured to include: a visual recognition distance detection unit for detecting the actual visual recognition distance from the user to the visual recognition object that the user is gazing at or at least a part of a plurality of objects in the field of view; a virtual image light generating unit for generating a plurality of virtual image lights that are independent of each other; a virtual image light transmission and projection unit for transmitting and projecting the virtual image lights to the left and right eyes of the user via substantially the same optical path so that the predetermined virtual image is visually recognized at a predetermined position in the field of view of the user; and a virtual image visual recognition distance control unit for variably controlling the visual recognition distance of the virtual image visually recognized by the user using the plurality of projected virtual image lights.

[0011] According to the present invention, it is possible to provide a virtual image display device capable of realizing a virtual image visual recognition distance matching function in a good visual recognition environment that reduces the physiological burden on a user, and a head mounted display using the virtual image display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic perspective view of the HMD in Example 1.

[0013] Figure 2 This is a schematic perspective view showing a usage scenario of the HMD in the first embodiment.

[0014] Figure 3 is a schematic top view showing an example of a visual recognition image within the user's visual field in a usage scenario of Figure 2 .

[0015] Figure 4 is a schematic perspective view showing another usage scenario of the HMD in Embodiment 1.

[0016] Figure 5 is a schematic top view showing an example of a visual recognition image within the user's visual field in a usage scenario of Figure 4 .

[0017] Figure 6 is a schematic top view showing the visual recognition state within the user's visual field in yet another usage scenario of the HMD in Embodiment 1.

[0018] Figure 7 is a block diagram showing the schematic structure of the HMD in Embodiment 1.

[0019] Figure 8 is a flowchart showing the operation process of the HMD in Embodiment 1.

[0020] Figure 9 is a schematic diagram for explaining the operation principle of the actual visual recognition distance detection unit of the visual recognition object of the HMD in Embodiment 1.

[0021] Figure 10 is a flowchart showing the operation process of the actual visual recognition distance detection unit of the visual recognition object of the HMD in Embodiment 1.

[0022] Figure 11 is a top view showing the schematic structure of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 1.

[0023] Figure 12 is a schematic diagram for explaining the general operation of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 1.

[0024] Figure 13 is a schematic diagram for explaining other general operations of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 1.

[0025] Figure 14 is a schematic diagram for explaining the outline of the variable control of the virtual image visual recognition distance of the HMD in Embodiment 1.

[0026] Figure 15 is a schematic diagram for explaining the operation principle of the actual visual recognition distance detection unit of the visual recognition object of the HMD in Embodiment 2.

[0027] Figure 16 This is a flowchart showing the operation process of the actual visual recognition distance detection unit for the visual recognition object of the HMD in Embodiment 2.

[0028] Figure 17 This is a top view showing the schematic structure of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 3.

[0029] Figure 18 This is a schematic diagram for explaining the outline of the operation of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 3.

[0030] Figure 19 This is a schematic diagram for explaining another outline of the operation of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 3.

[0031] Figure 20 This is a top view showing the schematic structure of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 4.

[0032] Figure 21 This is a top view showing the schematic structure of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 5.

[0033] Figure 22 This is a top view showing the schematic structure of the virtual image light generation unit and the virtual image light transmission and projection unit of the HMD in Embodiment 6.

[0034] Figure 23 This is a schematic perspective view showing the usage scenario of the portable information terminal in Embodiment 7.

[0035] Figure 24 This is a schematic front view showing the usage scenario of the vehicle head-up display in Embodiment 8.

[0036] (Symbol Explanation)

[0037] 1: HMD; 4, 5: Light guide plate; 6, 7: Incident hologram; 8a, 8b, 9a, 9b: Emission hologram; 10: Virtual image light transmission and projection unit; 20: Small electronic camera for outdoor scene imaging; 30: Drive control circuit; 34: Actual visual recognition distance detection unit for visual recognition object; 35: Virtual image visual recognition distance control unit; 40, 41: Virtual image light generation unit; 42, 43: Display for virtual image display; 46, 47: Projection lens system for virtual image light generation; 49: Trapezoidal PBS (Polarizing Beam Splitter); 50: Portable information terminal; 100: User. Detailed Embodiments

[0038] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0039] Example 1

[0040] Figure 1 is a schematic perspective view of a glasses-type HMD as the virtual image display device in this example. As Figure 1 shown, the glasses-type HMD 1 is worn on the head of the user 100. The HMD 1 stores an electronic circuit and an optical engine to be described later at positions 2 and 3 corresponding to the temple parts of glasses for resting on the left and right ears. In addition, at positions corresponding to the glasses lenses, transparent thin plates 4 and 5 made of optical glass or plastic parts for optical members are arranged. The transparent thin plates 4 and 5 are optical devices called "light guide plates". In this example, they are structured such that the light guide plates 4 and 5 are laminated, and the parts corresponding to the left and right glasses lenses are covered by the same light guide plate. Further, on the light guide plates 4 and 5, at parts in front of the left and right eyes, predetermined emission holograms Figure 8 a, 8b, 9a, 9b are arranged. In addition, the details of the functions of the light guide plates 4 and 5 and the emission holograms Figure 8 a, 8b, 9a, 9b will be described later.

[0041] In addition, in this example, the temple parts 2 and 3 are equipped with small electronic cameras 20 for photographing the external scene images in front of the user's eyes. In particular, in this example, the small electronic cameras 20 are two-eye cameras installed on both the left and right temple parts 2 and 3, and are configured to also have the function of measuring the distance to the photographed object through the so-called stereo camera function. However, the external scene photographing camera mounted on the HMD as described in this example is not limited to the above two-eye stereo camera. Of course, it can also be a single-eye camera, and its installation position is not limited to the position as Figure 1 shown.

[0042] In addition, in this example, a predetermined sensor box 21 is arranged on the upper part of the glasses frame. In the sensor box 21, for example, a so-called eye movement tracking camera for monitoring the positions of the left and right pupils of the user 100, or TOF( T ime O f Fsensors such as an eye movement tracking camera, a time-of-flight (TOF) sensor, etc., which measure the distance to any object in front of the user's eyes. In addition, regarding the sensors provided in the sensor box 21, of course, they are not limited to the eye movement tracking camera and the TOF sensor. As long as it is a sensor that detects an information signal, etc., required to detect the distance information by using a function of measuring the distance to any object in the external scene in front of the user's eyes or a predetermined method, it can be any sensor. In addition, the installation position of the sensor box 21 itself is not limited to the position shown in the figure. Furthermore, it can also be a structure in which the sensor box itself is removed and each sensor is assembled to a predetermined position inside the HMD1, for example.

[0043] However, in the present embodiment, the HMD1 has a function of wirelessly communicating with a portable information terminal 50 carried by the user, such as a smartphone, and is configured to obtain data related to a predetermined virtual image to be displayed on the HMD1 from the portable information terminal 50. However, this configuration is not limited to the above configuration. For example, it can also be a configuration in which the HMD itself has a part or all of the functions of the above portable information terminal 50 and stores data related to the virtual image to be displayed inside the HMD1. Hereinafter, the structure, etc., of the HMD1 in the present embodiment will be specifically described.

[0044] In addition, before specifically describing the present embodiment, in order to make the functions to be achieved by the virtual display device of the present embodiment clearer, several scenario examples of the user using the HMD of the virtual display device of the present embodiment in daily life will be introduced.

[0045] For example, Figure 2 is a schematic perspective view showing a scene where the user 100 wearing the HMD1 of the present invention is looking at the document 60 held in the hand. Figure 3An example of the actual visual recognition image of the document 60 visually recognized at this time within the field of view 101 (the area surrounded by the dashed line) of the user 100 and the virtual image 70 that is superimposed and displayed within the same field of view 101 by the HMD1 is shown. At this time, of course, the two eyes of the user 100 are focused on the hand-held document 60, and the visual recognition distance a is up to about several tens of centimeters. In such a usage scenario, for example, when associated information or explanatory information related to certain text paragraphs recorded on the document 60 is superimposed and displayed as a virtual image 70 based on the HMD1 at a predetermined position within the field of view 101, the visual recognition distance of the virtual image 70 needs to be at least approximately the same as the visual recognition distance a at which the two eyes of the user 100 are focused at that time. The reason is that if the visual recognition distances of the two are not the same, there will be image items with different visual recognition distances (i.e., the distances at which the eyes are focused) simultaneously present within the field of view of the user 100, and it will be difficult for the HMD user to visually recognize them simultaneously without discomfort or a sense of pressure.

[0046] Similarly, for example Figure 4 is a schematic perspective view showing a scene where the user 100 wearing the above-mentioned HMD1 is looking at a predetermined object 61 (a painting in the example of Figure 4 ) from a position several meters away. Figure 5 An example of the actual visual recognition image of the object 61 visually recognized at this time within the field of view 101 (the area surrounded by the dashed line) of the user 100 and the virtual image 71 that is superimposed and displayed within the same field of view 101 by the HMD1 is shown. In this scene as well, similar to the document viewing scene described above, the two eyes of the user 100 are focused on visually recognizing the object 61, but the visual recognition distance b extends to several meters, which is different from the document viewing case. In such a usage scenario as well, for example, when associated information or explanatory information related to the object 61 is superimposed and displayed as a virtual image 71 based on the HMD1 at a predetermined position within the field of view 101, when the visual recognition distance of the virtual image 71 is not approximately the same as the visual recognition distance b at which the two eyes of the user 100 are focused, it will be difficult for the user 100 to visually recognize the virtual image 71 and the object 61 simultaneously without discomfort or a sense of pressure.

[0047] In this way, in order for the HMD user to continuously visually recognize the real image seen within their field of view and the virtual image superimposed and displayed by the HMD without discomfort or a sense of pressure, it is necessary to make the visual recognition distance of the virtual image superimposed and displayed by the HMD approximately the same as the visual recognition distance of the object actually visually recognized by the user each time.

[0048] Furthermore, in order to use the HMD to achieve so-called augmented reality ( A ugmented RIn an Augmented Reality (AR) visual recognition environment, at least for any object seen within the field of view of an HMD user, a predetermined virtual image needs to be overlapped and displayed as a virtual image in association with the object. Therefore, it is necessary to make the visual recognition distance of the virtual image approximately the same as the actual visual recognition distance of the object, or to display it at a predetermined visual recognition distance associated with the actual visual recognition distance.

[0049] For example, Figure 6 An example of an actual visual recognition object seen within the field of view when an HMD user observes the interior of a certain room is shown. However, each furniture item 62 to 67 in the room, such as the table 62, the sofa 65, and the curtain 67, of course has a different visual recognition distance corresponding to the standing position of the user at that time point according to its installation position. Moreover, this visual recognition distance changes one by one according to the change in the standing position of the user. Therefore, for example, when implementing an AR environment by using an HMD to overlap and display certain virtual images 72 to 77 as virtual images for these respective furniture items, it is necessary to detect the actual visual recognition distance of each furniture item one by one, and make the visual recognition distance of the corresponding virtual image approximately the same as the detected actual visual recognition distance, or to display it at a predetermined visual recognition distance associated with the actual visual recognition distance.

[0050] To meet the requirements described above, the present embodiment provides a virtual image display device such as an HMD having the following functions: detecting the actual visual recognition distance of the object being looked at by the user or the object seen within the field of view by the user one by one, and almost in real time displaying a predetermined virtual image at a visual recognition distance approximately the same as the actual visual recognition distance or at a predetermined visual recognition distance associated with the actual visual recognition distance.

[0051] In addition, in Figure 3 , Figure 5 , Figure 6 , for the purpose of making the description easier to understand, as an example of the virtual image item overlapped and displayed within the field of view of the user by the HMD, it is disclosed by limiting it to an information display item in the so-called "prompt window" method. However, of course, the virtual image item to be displayed is not limited to such a "bubble" method, and it can also be an image item in any method.

[0052] Next, the specific structure of the HMD shown in Figure 1 of the present embodiment will be described. Figure 7 is a block diagram showing the schematic structure of the HMD1 in the present embodiment. In Figure 7In this case, the information signal receiving unit 31 for displaying a virtual image receives, through mutual communication with the portable information terminal 50 carried by the user 100, the required data related to the virtual image to be superimposed and displayed using the HMD1. At the same time, in the image information acquisition unit 32, the external scene image in front of the user captured by a small electronic camera 20 for external scene shooting or the like and the images of the binocular pupils of the user 100 captured by an eye movement tracking camera or the like in the sensor box 21 are acquired as predetermined image data. Further, based on the virtual image data received by the information signal receiving unit 31 for displaying a virtual image and the predetermined image data acquired by the image information acquisition unit 32, in the display virtual image image data generation unit 33, display image data for generating a virtual image 78 or 79 to be superimposed and displayed in the user's visual field is generated.

[0053] On the other hand, in the actual visual recognition distance detection unit 34 for the visual recognition object, the actual visual recognition distance of the user 100 is detected one by one using a predetermined method. In addition, in the virtual image visual recognition distance control unit 35, predetermined control processing is performed for variably controlling the visual recognition distance of the displayed virtual image according to the actual visual recognition distance detected by the actual visual recognition distance detection unit 34 for the visual recognition object. In addition, the detection of the actual visual recognition distance in the actual visual recognition distance detection unit 34 for the visual recognition object and the variable control of the visual recognition distance of the displayed virtual image in the virtual image visual recognition distance control unit 35 will be described later.

[0054] Next, the memory 36 has a function of temporarily storing at least a part of various data signals generated by the above-mentioned respective blocks, but it is not necessarily configured inside the HMD1. For example, a memory having the same function may be configured on the side of the portable information terminal 50. In addition, the above-mentioned blocks 31 to 36 are appropriately controlled in their operations by the control unit 37.

[0055] In addition, each of the blocks of the information signal receiving unit 31 to the control unit 37 for displaying a virtual image (the blocks within the area surrounded by a one-dot chain line in the figure) is a block composed only of electronic circuits, and in the following description, they are collectively referred to as the HMD drive control circuit 30 for simplicity.

[0056] On the other hand, the virtual image light generation units 40 and 41 are each composed of a predetermined display for displaying a virtual image and a drive circuit (display driver) of the display and a predetermined optical system. The subsequent virtual image light transmission and projection unit 10 is composed only of predetermined optical members, and has a function of transmitting virtual image light to the left and right eyes of the user via substantially the same optical path and projecting it so that a predetermined virtual image is visually recognized at a predetermined position in the user's visual field. In addition, specific structural examples of the virtual image light generation units 40 and 41 and the virtual image light transmission and projection unit 10 will also be described later.

[0057] Figure 8 is a flowchart showing Figure 1 and Figure 7 the operation process of the HMD in the present embodiment shown. In Figure 8 , after the process starts, first in step 201 (hereinafter, steps will be referred to as S in the figure and described as S201), each image data of N display virtual image items (S1, S2,...: S N ) sent from the portable information terminal 50 is received. At the same time, in step 202, each image data of the actual object within the user's field of view is obtained from the outdoor scene image in front of the user taken by the small electronic camera 20 for outdoor shooting, etc. Further, in step 203, the image data of the pupils of the user's both eyes is obtained by an eye movement tracking camera, etc.

[0058] Next, in step 204, a predetermined number variable n is set to 1, and in the next step 205, an object that becomes the link display object of the nth received display virtual image item Sn is selected from each image data of the actual object within the field of view obtained in step 202. Then, in the next subroutine 300 (S300), the actual visual recognition distance of this object is detected in the manner described later, and the visual recognition distance of the display virtual image item Sn is determined. Further, in the next subroutine 400 (S400), as data required for a predetermined control method used in the present embodiment for variably controlling the visual recognition distance of the display virtual image item Sn, the brightness ratio distribution data of the long-distance reference virtual image and the short-distance reference virtual image is estimated. The details of the virtual image visual recognition distance variable control based on the far and near reference virtual images and their brightness ratio distribution will be described later.

[0059] Next, in step 206, image display data such as the display position and display size of the display virtual image item Sn on the display for virtual image display is determined. Then, in step 207, it is determined whether n = N. If not, n = n + 1 and it is fed back to step 205.

[0060] By repeating the above processing flow, for all of the N received display virtual image items (S1, S2,...: S N ), the above image display data and the above far and near reference virtual image brightness ratio distribution data are determined.

[0061] After that, in step 208, using the above-determined N display virtual image items (S1, S2,...: S Nimage display data related to each of the virtual image display item images and the far - near reference virtual image brightness ratio allocation data, generate an image data signal input to the virtual image display monitor, and in the next step 209, actually drive the virtual image display monitor to perform a predetermined image display. Then, through a further predetermined optical system, generate a predetermined far - near reference virtual image light according to the image light emitted from the above - mentioned monitor, and transmit the virtual image light to the user's binoculars for projection.

[0062] Next, regarding Figure 8 the actual visual recognition distance detection of the user's visual recognition object performed in the sub - routine 300 (S300) in the flowchart of

[0063] Figure 9 and Figure 10 are a schematic diagram and a flowchart for explaining the detection process and principle of the actual visual recognition distance. First, use an eye - movement tracking camera or the like to photograph the binocular pupils of the user 100, and detect from the image data the offsets of the left and right eye pupil centers 102 and 103 shown in the figure from the reference positions (pupil center positions when looking at infinity) 104 and 105, that is, the convergence amounts P L and P R and the left - right eye pupil center interval W LR .

[0064] Now, when the distances from the rotation centers 106 and 107 of the left and right eyeballs to the left and right eye pupil centers 102 and 103, that is, the eyeball rotation radii, are set to R, the line - of - sight inclination angles α L α R of the left and right eyes of the user 100 towards the fixation point 120 on an arbitrary visual recognition object 68 are obtained by the following relational expressions (1) and (2).

[0065] tan(α L ) = P L / R---(1)

[0066] tan(α R ) = P R / R---(2)

[0067] Furthermore, the actual visual recognition distance S, which is the distance from the center position of the left - right pupil interval of the user 100 to the above - mentioned fixation point 120, is also obtained by the following relational expression (3).

[0068] S = W LR / {tan(α L ) + tan(α R )} = R×W LR / {P L+P R}---(3)

[0069] By performing such a processing procedure one by one, the actual visual recognition distance from the object visually recognized by the user 100 can be detected almost in real time.

[0070] In addition, regarding Figure 10 the processing procedures of each step in the processing steps S301 to S306 in the flowchart of, since they are repetitive with the above description, the description thereof is omitted.

[0071] In addition, regarding the above-mentioned eyeball rotation radius R, it only needs to be pre-detected before use by a predetermined calibration process (estimating the above formulas (1) to (3) by performing the same detection process using a calibration object with a known actual visual recognition distance for inverse operation).

[0072] Next, a specific example of the variable control of the visual recognition distance of the virtual image generated by the virtual image light generation units 40 and 41 and the virtual image light transmission and projection unit 10 in the block diagram of Figure 7 and the virtual image image that is overlapped and displayed by the virtual image light generation units 40 and 41 and the virtual image light transmission and projection unit 10 in the user's visual field will be described.

[0073] Figure 11 is a top view showing Figure 1 and Figure 7 the schematic structure of the virtual image light generation units 40 and 41 and the virtual image light transmission and projection unit 10 of the HMD in the present embodiment shown. Inside the part corresponding to the temple of the glasses of the HMD1 shown in Figure 1 the above-mentioned drive control circuit 30 and the independent virtual image light generation units 40 and 41 (the parts surrounded by the single-dot chain line in the figure) are arranged. In addition, inside the virtual image light generation units 40 and 41, virtual image image display monitors 42 and 43 composed of image display monitors such as liquid crystal displays, monitor drive circuits 44 and 45 for driving the monitors, and virtual image light generation projection lens systems 46 and 47 are respectively arranged, for example. In addition, the virtual image image display monitors 42 and 43 are not limited to the above-mentioned liquid crystal displays. For example, as long as they are small image display monitors such as organic electroluminescent displays (abbreviation: OLED), they can be monitors of any method and structure.

[0074] On the other hand, in the virtual image light transmission and projection unit 10 (the part surrounded by the dotted line in the figure), as Figure 1As also shown in the figure, two laminated transparent light guide plates 4 and 5 are arranged. The light guide plates 4 and 5 have the function of enclosing and transmitting the light incident on the light guide plate at a predetermined incident angle through repeated total internal reflections on the upper and lower surfaces of the light guide plate into the interior of the light guide plate. In addition, respectively arranged in the light guide plates 4 and 5 are: an incident holographic Figure 6 and 7, which have the function of incident light waves into the light guide plate; and an emission holographic Figure 8 a, 8, and 9a, 9b, which have the function of emitting the light waves traveling in the light guide plate to the outside. Regarding such a light guide plate equipped with an incident hologram and an emission hologram, it is already well-known as an optical device for virtual image light transmission and projection for HMD, so the above detailed description thereof is omitted.

[0075] In this embodiment, a predetermined command signal or control signal from the HMD drive control circuit 30 is received, and a predetermined image light is generated by projecting an image equivalent to the display virtual image through the virtual image display monitors 42 and 43.

[0076] Then, the first image light generated by the monitor 42 in the above image light first enters the virtual image light generating projection lens system 46 as Figure 12 shown, and is transformed into the first virtual image light having a predetermined visual recognition distance here. Then, next, it enters the interior of the light guide plate 4 through the incident holographic Figure 6 arranged in the light guide plate 4. After traveling toward the right in the figure in the light guide plate 4, it exits from the emission holographic Figure 8 a and 8b and enters the two eyes 112a and 112b of the user 100 respectively. At this time, the user 100 visually recognizes the virtual image 78 through the first virtual image light at a position with a predetermined visual recognition distance D L where the transparent light guide plates 4 and 5 are passed through. In this specification, the virtual image 78 will be hereinafter referred to as a long-distance reference virtual image.

[0077] On the other hand, the second image light generated by the monitor 43 enters the virtual image light generating projection lens system 47 as Figure 13 shown, and is transformed into the second virtual image light having a predetermined visual recognition distance here. Then, next, it enters the interior of the light guide plate 5 through the incident holographic Figure 7 arranged in the light guide plate 5. After traveling toward the left in the figure in the light guide plate 5, it exits from the emission holographic Figure 9 a and 9b and enters the two eyes 112a and 112b of the user 100 respectively in the same manner as the virtual image light exiting from the above emission holographic Figure 8 a and 8b. At this time, the user 100 visually recognizes through the second virtual image light at a predetermined visual recognition distance D S (DS <D L ) at the position of the visually recognizable virtual image 79. In this specification, this virtual image 79 will be hereinafter referred to as the near-distance reference virtual image.

[0078] That is, in such a manner that the far-distance reference virtual image 78 is accurately visually recognized at the position of a predetermined visual recognition distance D when observed from the user 100 through the transparent light guide plates 4 and 5, and the near-distance reference virtual image 79 is accurately visually recognized at the position of the visual recognition distance D L The optical constants and lens positions of the projection lens systems 46 and 47 for virtual image light generation are appropriately set. S (D S <D L ) at the position of the near-distance reference virtual image 79 is accurately visually recognized, the optical constants and lens positions of the projection lens systems 46 and 47 for virtual image light generation are appropriately set.

[0079] Moreover, in such a manner that the far-distance reference virtual image 78 and the near-distance reference virtual image 79 are visually recognized in an overlapping manner according to a complete projective relationship as Figure 14 shown, the image display positions in the displays 42 and 43 for virtual image display and the visual recognition magnification of the virtual images in the projection lens systems 46 and 47 for virtual image light generation are appropriately set. Specifically, for the visual recognition magnification M of the far-distance reference virtual image 78, the visual recognition magnification M' of the near-distance reference virtual image 79 is set to M' = (D S / D L ) × M.

[0080] Furthermore, in this state, based on the following predetermined relational expressions (4) and (5), when the brightnesses V L and V S of the far-distance reference virtual image 78 and the near-distance reference virtual image 79 are proportionally allocated, the user 100 has an illusion as if visually recognizing the virtual image at an arbitrary distance Do at the midpoint between the distances D L and D S . That is,

[0081] V L = V O × (D O - D S ) / (D L - D S )---(4)

[0082] V S = V O × (D L - D O ) / (D L - D S )---(5)

[0083] In addition, V Ois the brightness of the virtual image that is misperceived at an arbitrary distance Do.

[0084] Such an illusion phenomenon is generally referred to as a stereoscopic illusion phenomenon ( D epth- f used 3 D : abbreviated as DFD). Regarding DFD as a physiological phenomenon, since it is already a well-known phenomenon, a detailed description of its physiological occurrence mechanism and the like is omitted. However, compared with the binocular parallax method, which is the most widely used stereoscopic observation method based on illusion in the past, the stereoscopic vision using this DFD has the characteristics of less fatigue for the observer and a wider viewing point area (EyeBox) that can accurately perform stereoscopic observation. In addition, in the stereoscopic vision using this DFD, it is possible to match the visual recognition distance of the illusory virtual image to an arbitrary distance between the far and near reference virtual images by a relatively simple method of controlling the brightness ratio distribution of the far and near reference virtual images. Therefore, it is a method suitable for a situation such as almost real-time detection of the actual visual recognition distance of the user and variable control of the visual recognition distance of the display virtual image suitable for the actual visual recognition distance as in this embodiment. Therefore, in this embodiment, variable control of the visual recognition position of the display virtual image using this DFD is implemented.

[0085] That is, in this embodiment, as a virtual image display device, there are provided: a visual recognition distance detection unit that detects the actual visual recognition distance from a predetermined object in the user's visual field; a virtual image light generation unit that has a function of generating at least two types of reference virtual image lights with different visual recognition distances; a virtual image light transmission projection unit that has a function of transmitting and projecting the reference virtual image light to the user's binoculars through a transparent light guide plate provided with a hologram for light wave incidence and emission; and a virtual image visual recognition distance control unit that can variably control the visual recognition distance to a predetermined distance by controlling the image brightness ratio distribution of the two types of virtual image lights according to the detected actual visual recognition distance.

[0086] Thus, according to this embodiment, it is possible to provide a virtual image display device and an HMD using the device in a good visual recognition environment that reduces the physiological burden on the user by making the visual recognition distance of the virtual image visually recognized by the user match the actual visual recognition distance detected by the visual recognition distance detection unit through variable control of the visual recognition position of the display virtual image using DFD. In addition, it is possible to further provide a virtual image display device suitable for miniaturization, light weight, and low cost of the device.

[0087] In addition, the variable control of the visual recognition position of the display virtual image using this DFD is, as described above, by controlling the brightness ratio distribution of the reference virtual images for near and far, so that the visual recognition distance of the false visual image matches any distance between the reference virtual images for near and far. Therefore, its variable control range, that is, the dynamic range, is limited to the visual recognition distance range between the two reference virtual images for near and far. Thus, for example, by mechanically adjusting the lens intervals of the respective projection lenses that make up the projection lens systems 46 and 47 for generating virtual image light, etc., the visual recognition distances of the two reference virtual images for near and far themselves are made variable, and thus the dynamic range of the visual recognition distance of the false visual image can also be arbitrarily enlarged or reduced. However, in such a case, in order to always visually recognize the two reference virtual images for near and far as being overlapped in a complete projective relationship, it is also necessary to appropriately control the image display positions in each display and the virtual image projection magnifications of the projection lens systems 46 and 47 for generating virtual image light.

[0088] In addition, the reference virtual images that determine the dynamic range of the variable control of the visual recognition position of the display virtual image based on the DFD are not limited to the two, namely, the long-distance reference virtual image 78 and the short-distance reference virtual image 79, as described above. By adding the virtual image light generation unit and the virtual image light transmission and projection unit as described above, one or more intermediate-distance reference virtual images are set at arbitrary visual recognition distances between the farthest-distance reference virtual image and the nearest-distance reference virtual image. By arbitrarily selecting two reference virtual images from these multiple reference virtual images and reusing them as the long-distance reference virtual image 78 and the short-distance reference virtual image 79 and performing the brightness ratio distribution control as described above, the dynamic range of the variable control of the visual recognition position of the display virtual image can be arbitrarily varied.

[0089] Furthermore, the present embodiment is not limited to the variable control of the visual recognition position of the display virtual image based on the DFD. As long as it is a variable control applicable to the present embodiment, it can of course be in any manner. For example, in the fifth embodiment described later, an example of applying the variable control of the visual recognition position of the display virtual image based on the binocular parallax method, which has been widely used in the past, is also disclosed.

[0090] Embodiment 2

[0091] In Embodiment 1, for detecting the actual visual recognition distance of the object visually recognized by the user, a unit that detects only based on the convergence amount of the user's binocular pupils, as described in Figure 9 and Figure 10 is used, but of course, the detection of the actual visual recognition distance is not limited to this.

[0092] Hereinafter, as this embodiment, a new detection of the actual visual recognition distance different from the above detection unit will be described. Figure 15 is a schematic diagram for explaining its detection principle, Figure 16 is a flowchart summarizing its detection processing procedure. First, useFigure 16 This will be described in terms of the detection process.

[0093] In Figure 16 , first, in step 351, for the outdoor scene image in front of the user captured by the small electronic camera 20 for outdoor scene image shooting or the like, each image data of the object in the field of view is obtained. Next, in step 352, from the obtained image data, for each object in the user's field of view, the relative position of its user base point is detected. Then, further, in step 353, based on the detected relative position, the user base point azimuth angle (estimated angle when the user is looking at the object) β of each object in the field of view is calculated.

[0094] On the other hand, simultaneously, in step 354, the captured image data of the user's binocular pupils captured by the eye movement tracking camera or the like is obtained. In the next step 355, based on the above image data, the pupil center positions of the left and right eyes are detected. Then, further, in the next step 356, the line-of-sight tilt angles α L 、α R (see Figure 15 ) of the left and right eyes are calculated. In addition, the processes from step 354 to step 356 are the same as the processes of steps 301 to 305 in the first embodiment related to the detection of the actual visual recognition distance of the user described in Figure 9 and Figure 10 .

[0095] In this embodiment, in the next step 357, based on the line-of-sight tilt angles α L 、α R of the left and right eyes calculated in the above process, the following relational expression (6) is used to calculate the gaze line-of-sight tilt angle γ of the user 100 towards the gaze point 120 on the visual recognition object 68 currently gazed at by the user (see Figure 15 ).

[0096] tan(γ) = sin(α L -α R ) / {cos(α L +α R ) + cos(α L -α R )} --- (6)

[0097] Next, in step 358, the user base azimuth angle β of the object in the field of view calculated in the above process is compared with the inclination angle γ of the user's gaze line of sight, and the object in the field of view where β and γ are consistent, that is, the object currently being looked at by the user, is determined. In addition, a ranging sensor equipped on the HMD is used to measure the effective distance S from the determined object being looked at, that is, the actual visual recognition distance of the user at the current time point. In addition, as this ranging sensor, any sensor can be used as long as it is an existing ranging sensor such as a so-called stereo camera, TOF sensor, or ultrasonic sensor introduced in Figure 1 The so-called stereo camera, TOF sensor, or ultrasonic sensor introduced in

[0098] However, the embodiments 1 and 2 described above have always aimed to detect the actual visual recognition distance of the HMD user and make the displayed virtual image match this actual visual recognition distance. However, of course, this embodiment is not limited to such an aim.

[0099] For example, it can also be a display unit as follows: regardless of whether the user is looking, for all the objects that the user sees in the field of view, the effective distance from the user is detected for each, and the visual recognition distances of the display virtual images that are overlapped and displayed in association with each object are made to match the effective distances of the respective associated actual objects, thereby achieving so-called AR (augmented reality) response.

[0100] Furthermore, the virtual image can be displayed at an arbitrary visual recognition distance completely unrelated to the effective distance of each object in the field of view or the actual visual recognition distance of the user, or it can be a display method such as dynamically changing the visual recognition position of the display virtual image between arbitrary visual recognition distances by making real-time use of the variable control of the visual recognition distance of the display virtual image.

[0101] Embodiment 3

[0102] Figure 17 It is a top view showing the schematic structure of the virtual image light generation unit and the virtual image light transmission and projection unit in this embodiment. In addition, in Figure 17 For Figure 11 The same reference numerals are assigned to the same components as those shown.

[0103] In Figure 11In this case, in order to separately generate long-distance reference virtual image light and short-distance reference virtual image light, two independent virtual image light generation units 40 and 41 are arranged in the HMD. In addition, the virtual image light transmission and projection unit 10 is configured to be composed of two transparent light guide plates 4 and 5 arranged in a stacked manner, and the virtual image light generated by the virtual image light generation units 40 and 41 respectively is transmitted in one of the two light guide plates 4 and 5. However, it is also possible to have a structure in which the long-distance reference virtual image light and the short-distance reference virtual image light are transmitted together in a single light guide plate instead of being transmitted separately in individual light guide plates. In addition, by studying the optical system, the two independent virtual image light generation units as described above can also be integrated into one unit. Figure 17 The present embodiment shown is an embodiment for achieving such an object.

[0104] First, as Figure 17 shown, the HMD1 in the present embodiment is different from the embodiment 1 shown in Figure 11 that it only has a single virtual image light generation unit 40 (the part surrounded by the one-dot chain line in the figure). The virtual image light generation unit 40 includes a display 42 for virtual image display, a display drive circuit 44 for driving it, a first virtual image light generation projection lens system 46a and a second virtual image light generation projection lens system 46b that are divided into two lens groups, a trapezoidal PBS (polarizing beam splitter) 49, and a polarization conversion element 48 having a function of appropriately converting the polarization direction of the incident light according to a predetermined electrical signal into mutually perpendicular P-polarized light and S-polarized light, etc. In addition, as the above polarization conversion element, an existing device such as a liquid crystal type polarization conversion element used in a liquid crystal display can also be used.

[0105] In addition, in the present embodiment, only a single light guide plate 4 is arranged in the virtual image light transmission and projection unit 10 (the part surrounded by the dotted line in the figure). Further, two incident holograms Figure 6 and 7 and two exit holograms Figure 8 a and 8b are provided in the light guide plate 4.

[0106] In the present embodiment, first, in the above display 42 for virtual image display, for each predetermined frame frequency, the image for the long-distance reference virtual image light and the image for the short-distance reference virtual image light are alternately switched and displayed. In addition, the generated image light is incident on the first virtual image light generation projection lens system 46a, and here it is converted into virtual image light having a predetermined visual recognition distance, for example, the long-distance reference virtual image light having a visual recognition distance D L and then is incident on the polarization conversion element 48. The polarization conversion element 48 is set to switch to P-polarized light or S-polarized light of the incident light synchronously with the timing of the display switching of the image for the long-distance reference virtual image light and the image for the short-distance reference virtual image light displayed on the display 42.

[0107] Now, the virtual image light incident on the polarization conversion element 48 through the above-mentioned polarization conversion element 48 is, for example, set to be converted into P-polarized light at the timing of displaying the image of the long-distance reference virtual image light on the display 42, and conversely, to be converted into S-polarized light at the timing of displaying the image of the short-distance reference virtual image light. In addition, hereinafter, the virtual image light to be converted into the above-mentioned P-polarized light will be referred to as the first virtual image light, and the virtual image light to be converted into S-polarized light will be referred to as the second virtual image light.

[0108] Thus, the first virtual image light converted into P-polarized light (the polarization direction indicated by the double-headed arrow line in the figure) by the polarization conversion element 48 is as Figure 18 shown, transmits through the polarization separation film 49a of the trapezoidal PBS 49, and then enters the inside of the light guide plate 4 through the first incident hologram Figure 6 arranged on the light guide plate 4. Then, after traveling in the right direction in the figure in the light guide plate 4, it exits from the emission holograms Figure 8 a and 8b and enters the right eyes 112a and 112b of the user 100 respectively.

[0109] At this time, since the first virtual image light is generated as light equivalent to the above-mentioned long-distance reference virtual image light by the first virtual image light generation projection lens system 46a, the user 100 visually recognizes the long-distance reference virtual image 78 at a predetermined visual recognition magnification at a position at a predetermined visual recognition distance D L through the transparent light guide plate 4.

[0110] On the other hand, the second virtual image light converted into S-polarized light, which is a polarization direction perpendicular to the P-polarized light, by the above-mentioned polarization conversion element 48 is as Figure 19 shown, is reflected on the polarization separation film 49a of the trapezoidal PBS 49, and then is further reflected on the total reflection surface 49b, and enters the second virtual image light generation projection lens system 46b through an optical path different from that of the first virtual image light. The second virtual image light generation projection lens system 46b has a function of reconverting the second virtual image light, which is temporarily generated as light equivalent to the long-distance reference virtual image light by the first virtual image light generation projection lens system 46a, into light equivalent to the short-distance reference virtual image light having a predetermined visual recognition distance D S (D S < D L ).

[0111] Then, the second virtual image light reconverted into light equivalent to the short-distance reference virtual image light then enters the inside of the light guide plate 4 through the second incident hologram Figure 7 arranged on the light guide plate 4. Then, after traveling in the right direction in the figure in the light guide plate 4 in the same manner as the above-mentioned first virtual image light, it exits from the emission hologram Figure 8It is emitted from a and 8b and respectively enters the two eyes 112a and 112b of the user 100. At this time, since the second virtual image light passes through both the virtual image light generation projection lens systems 46a and 46b as described above and is transformed into virtual image light equivalent to the above-mentioned short-distance reference virtual image light, the user 100 visually recognizes the short-distance reference virtual image 79 at a predetermined visual recognition magnification at a position at a predetermined visual recognition distance D through which the transparent light guide plate 4 passes. S That is, the user 100 alternately visually recognizes the long-distance reference virtual image 78 and the short-distance reference virtual image 79 that are switched at a predetermined frame rate. However, since the user 100 cannot recognize each of the rapidly switched virtual images, it is perceived as if the long-distance reference virtual image 78 and the short-distance reference virtual image 79 are visually recognized simultaneously. Therefore, similar to the above-described first and second embodiments, it is possible to perform display virtual image visual recognition distance variable control using the stereoscopic illusion phenomenon (DFD).

[0112] That is, the user 100 alternately visually recognizes the long-distance reference virtual image 78 and the short-distance reference virtual image 79 that are switched at a predetermined frame rate. However, since the user 100 cannot recognize each of the rapidly switched virtual images, it is perceived as if the long-distance reference virtual image 78 and the short-distance reference virtual image 79 are visually recognized simultaneously. Therefore, similar to the above-described first and second embodiments, it is possible to perform display virtual image visual recognition distance variable control using the stereoscopic illusion phenomenon (DFD).

[0113] As described above, by adopting the structure shown in this embodiment, it is possible to integrate and reduce the light guide plates in the virtual image light generation unit and the virtual image light transmission and projection unit into one, and significant effects can be obtained in terms of miniaturization, weight reduction, and cost reduction of the HMD compared to the above-described first embodiment.

[0114] In addition, of course, the relationship between the far / near reference virtual image light and the P / S polarized light described in this embodiment can be completely reversed. In this case, the first virtual image light generation projection lens system 46a has the function of generating light equivalent to the short-distance reference virtual image light, and the second virtual image light generation projection lens system 46b has the function of further transforming this light into light equivalent to the long-distance reference virtual image light.

[0115] Embodiment 4

[0116] Figure 20 It is a top view showing a schematic structure of a virtual image light generation unit and a virtual image light transmission and projection unit in this embodiment. In addition, in Figure 20 , the same reference numerals are assigned to the components that are the same as the components shown in Figure 11 and Figure 17 .

[0117] In Embodiment 3, a single display for virtual image display is used to time-divisionally switch and display the long-distance reference virtual image light image and the short-distance reference virtual image light image at a predetermined frame rate, and the polarization direction of the virtual image light is also time-divisionally switched in synchronization with the frame rate, so that the user 100 perceives as if visually recognizing the long-distance reference virtual image and the short-distance reference virtual image simultaneously. In contrast, this embodiment is configured to project the two reference virtual image lights of different distances onto the two eyes 112a and 112b of the user 100 simultaneously instead of the time-division display as described above.

[0118] In addition, in this embodiment, similar to Figure 17 In the virtual image light transmission and projection unit 10 (the part surrounded by the dashed line in the figure), only a single light guide plate 4 is disposed. In addition, one incident hologram Figure 6 and 2 one emission hologram Figure 8 a and 8b.

[0119] In this embodiment, in the virtual image display monitor 42 in the virtual image light generation unit 40, the image display surface is divided into two display areas 42a and 42b, and independent images can be simultaneously displayed in each display area, or images that are the same but the brightness ratio distribution can be independently controlled for each display item. Further, immediately behind each display area, a phase plate or polarization filters 11 and 12 are respectively disposed. The phase plate or polarization filters 11 and 12 have the function of aligning the polarization directions of the image lights emitted from the respective divided display areas 42a and 42b of the monitor 42 to predetermined polarization directions. That is, the phase plate or polarization filter 11 aligns the first image light emitted from the divided display area 42a to P-polarized light (the polarization direction indicated by the double-headed arrow line in the figure), and the polarization filter 12 aligns the second image light emitted from the divided display area 42b to S-polarized light perpendicular to the P-polarized light.

[0120] Therefore, for example, an image corresponding to the long-distance reference virtual image light is simultaneously displayed in the divided display area 42a, and an image corresponding to the short-distance reference virtual image light is displayed in the divided display area 42b.

[0121] First, the first image light emitted from the divided display area 42a is incident on the trapezoidal PBS 49 via the phase plate or polarization filter 11. However, since the polarization direction of the first image light is aligned to P-polarized light by passing through the phase plate or polarization filter 11 as described above, it transmits through the polarization beam splitting film 49a of the trapezoidal PBS 49 and is incident on the first virtual image light generation projection lens system 46a. Then, after generating the long-distance reference virtual image light with the visual recognition distance D L the incident hologram disposed on the light guide plate 4 is passed through,Figure 6 The light is incident on the inside of the light guide plate 4. After traveling to the right in the figure, the light is emitted from the holographic Figure 8 The light rays 8a and 8b are emitted and enter the eyes 112a and 112b of the user 100 respectively.

[0122] On the other hand, the second image light emitted from the divided display area 42b is incident on the trapezoidal PBS 49 via the phase plate or polarization filter 12 and the second virtual image light generation projection lens system 46b, and reaches the polarization separation film 49a after being reflected on its total reflection surface 49b. At this time, the second image light is aligned to the S polarized light by the phase plate or polarization filter 12, so it is reflected on the polarization separation film 49a and synthesized into the optical path of the first image light, and is also incident on the first virtual image light generation projection lens system 46a. That is, the second image light passes through both the second virtual image light generation projection lens system 46b and the first virtual image light generation projection lens system 46a through the trapezoidal prism 49, and is generated by the combination of the lens systems to have a visual recognition distance D. S In addition, the short-distance reference virtual image light is generated similarly to the long-distance reference virtual image light generated from the first image light, and is transmitted through the incident holographic image light arranged on the light guide plate 4. Figure 6 The light is incident on the inside of the light guide plate 4, travels to the right in the light guide plate 4, and then exits from the holographic Figure 8 The light rays 8a and 8b are emitted and enter the eyes 112a and 112b of the user 100 respectively.

[0123] That is, through the structure of this embodiment as described above, the user 100 can visually recognize the long-distance reference virtual image light and the short-distance reference virtual image light at the same time. Therefore, the display virtual image visual recognition distance variable control using the stereoscopic illusion phenomenon (DFD) can be achieved in the same way as in the first, second, and third embodiments.

[0124] In the structure shown in this embodiment, high-speed switching control of image light and relatively expensive optical devices such as polarization conversion elements as shown in the third embodiment are not required, so a more significant effect can be obtained in reducing the cost of HMD.

[0125] In addition, of course, the relationship between the long / short distance reference virtual image light and the P / S polarized light described in this embodiment can also be completely opposite. In this case, the first virtual image light generating projection lens system 46a has the function of generating light equivalent to the short distance reference virtual image light, and the projection lens system formed by combining the first virtual image light generating projection lens system 46a and the second virtual image light generating projection lens system 46b has the function of generating light equivalent to the long distance reference virtual image light.

[0126] Example 5

[0127] In Embodiment 4, an example of using the stereoscopic illusion phenomenon (DFD) for variable control of the visual recognition distance of a displayed virtual image is shown. However, an HMD using the most general binocular parallax method in the past can also be implemented for variable control of the visual recognition distance of a displayed virtual image with substantially the same optical structure. In the present embodiment, an example of applying the invention of the present application to an HMD using the binocular parallax method will be described.

[0128] Figure 21 is a top view showing a schematic structure of a virtual image light generation unit and a virtual image light transmission and projection unit in the HMD of the present embodiment. In addition, in Figure 21 , the same reference numerals are assigned to the same components as those Figure 20 shown.

[0129] In Figure 21 , it has the same virtual image light generation unit and virtual image light transmission and projection unit as Figure 20 , but the only difference is that a polarization filter 13 is additionally arranged in the optical path between the emission hologram Figure 8 a and the right eye 112a of the user 100, and a polarization filter 14 is additionally arranged in the optical path between the emission hologram Figure 8 b and the left eye 112b of the user 100. Among them, the polarization filter 13 has, for example, a function of transmitting P-polarized light (the polarization direction indicated by the double-headed arrow line in the figure) and reflecting or absorbing S-polarized light. Conversely, the polarization filter 14 has a function of transmitting S-polarized light and reflecting or absorbing P-polarized light. However, of course, the combination of P / S polarized light can be exactly the opposite.

[0130] With such a structure, it is possible to visually recognize only the virtual image light generated from the display image of the divided display area 42a of the display 42 and incident on the right eye 112a of the user 100, and only the virtual image light generated from the display image of the divided display area 42b and incident on the left eye 112b of the user 100. Therefore, by displaying the right-eye image and the left-eye image with the display position of the image item corresponding to the binocular parallax corresponding to the desired visual recognition distance shifted in the divided display areas 42a and 42b, respectively, variable control of the visual recognition distance of the displayed virtual image based on the binocular parallax method can be achieved. In addition, since the specific process, principle, etc. of the binocular parallax method are already well-known, detailed description thereof is omitted.

[0131] Among them, in Figure 21In the optical structure of the present embodiment shown, the optical path lengths of the first image light emitted from the divided display area 42a of the display 42 and the second image light emitted from the divided display area 42b are different up to the first virtual image light generation projection lens system 46a. (For the second image light emitted from the divided display area 42b, the optical path length from the total reflection surface 49b in the trapezoidal PBS 49 to the polarization beam splitter film 49a is longer.) Therefore, regarding the virtual image generated by the first virtual image light generation projection lens system 46a, there are also differences in the visual recognition distance and the visual recognition image magnification between the first virtual image light generated based on the first image light and the second virtual image light generated based on the second image light.

[0132] However, in the case of performing variable control of the visual recognition distance of the virtual image in the binocular parallax display mode, the visual recognition distances and the image sizes of the virtual image recognized by the right eye and the virtual image recognized by the left eye must be exactly the same. Therefore, in the present embodiment, the second virtual image light generation projection lens system 46b is provided with a function of correcting the differences in the visual recognition distance and the visual recognition image magnification of the second virtual image light relative to the first virtual image light caused by the difference in the optical path length. By adopting such an optical structure, in the present embodiment, it is also possible to make the visual recognition distances and the image sizes of the virtual image recognized by the right eye and the virtual image recognized by the left eye the same, so that variable control of the visual recognition distance of the virtual image based on the binocular parallax display mode can be achieved.

[0133] The binocular parallax type HMD using the optical structure shown in the present embodiment does not need to be equipped with two independent virtual image light generation units, i.e., image display optical engines, separated for the left and right eyes as in the conventional binocular parallax type HMD. Therefore, significant effects can be obtained in the miniaturization, weight reduction, and cost reduction of the binocular parallax type HMD.

[0134] However, in the present embodiment, an example is shown in which the optical structure of Embodiment 4 shown Figure 20 is applied to the binocular parallax type HMD. Of course, for the optical structure of Embodiment 3 shown Figure 17 a binocular parallax type HMD with the same effect can also be achieved by setting a filter exactly the same as the polarization filter.

[0135] Embodiment 6

[0136] Figure 22 is a top view showing the schematic structure of the virtual image light generation unit and the virtual image light transmission and projection unit in the HMD of the present embodiment. In addition, in Figure 22 the same reference numerals are assigned to the components that are the same as those in Embodiment 4 shown Figure 20 and Embodiment 5 shown Figure 21 having the same components.

[0137] In this embodiment, the HMD1 also has the same virtual image light generation unit and virtual image light transmission and projection unit as those in Figure 20 Embodiment 4 shown, but the only difference is that an optical device capable of controlling light shielding property by a predetermined electric signal, i.e., a light shielding property controllable device 15, is additionally arranged in front of the light guide plate 4. As such an optical device with controllable light shielding property, for example, there are an active polarization filter using liquid crystal and a dimming glass. By arranging such an optical device in front of the light guide plate 4, it is possible to arbitrarily block the external scene light flying into the left and right eyes of the user 100 through the light guide plate 4, and only clearly visually recognize the displayed virtual image in front of the user's eyes. By having such a function, the user can arbitrarily switch between and distinguish the use of an HMD for so-called augmented reality (AR) that can perform fusion visual recognition of an actual visual recognition image and a displayed virtual image and an HMD for so-called virtual reality (VR) that blocks the external scenery and only visually recognizes the displayed virtual image with one HMD, which has a significant effect in expanding the versatility of the HMD.

[0138] In addition, in Figure 22 , the light shielding property controllable device 15 is arranged to cover the entire surface of the light guide plate 4, but it can also be arranged to cover only the openings of the holograms 8a and 8b for outgoing light and the vicinity thereof. Figure 8 a and 8b and the vicinity thereof.

[0139] Embodiment 7

[0140] The previously described embodiments are mainly embodiments related to the HMD, but this embodiment is an example of applying the present invention to a portable information terminal or a portable personal computer (PC) 16 equipped with a transparent display.

[0141] Figure 23 is a schematic perspective view showing the usage scenario of the portable information terminal in this embodiment. As Figure 23 shown, inside the main body of the portable terminal or the portable PC 16, at least the drive control circuit and the virtual image light generation unit disclosed in any one of Embodiments 1 to 6 are assembled, and as a see-through type image display, there is equipped with a virtual image light transmission and projection unit 10 composed of a light guide plate 4 with holograms for virtual image light incident and outgoing as shown in Figure 11 , Figure 17 , Figures 20 to 22 shown. By applying the present invention to the portable terminal or the portable PC in this way, different from the head-mounted type HMD, it is possible to realize augmented reality (AR) or virtual reality (VR) based on a portable device.

[0142] Embodiment 8

[0143] Figure 24An embodiment in which the present invention is applied to a head-up display (HUD) for vehicle use is shown. In this example, at least the drive control circuit and the virtual image light generation unit disclosed in any one of Embodiments 1 to 6 are assembled to the HUD main body 17 assembled in or mounted on the instrument panel of the vehicle, and as a so-called HUD synthesizer or a perspective display for HUD, a virtual image light transmission projection unit 10 composed of the light guide plate 4 with the hologram for virtual image light incident and emitted is provided.

[0144] In addition, with regard to the virtual image light transmission projection unit 10, by providing it on a part or the whole of the windshield provided on the entire surface of the vehicle driver's seat, or forming a part of the windshield itself with the light guide plate 4 with the hologram for virtual image light incident and emitted, the windshield itself can also have the function of a perspective display for HUD.

Claims

1. A virtual image display device, comprising: A generation unit that generates first image light and second image light with different visual recognition distances; A projection unit that projects the first image light and the second image light onto the left and right eyes of a user, enabling the user to visually recognize a long-distance reference virtual image and a short-distance reference virtual image respectively; and A control unit that controls the positions and sizes of the first image light and the second image light in such a way that the long-distance reference virtual image and the short-distance reference virtual image visually recognized by the user overlap, and proportionally distributes the brightness of the long-distance reference virtual image and the short-distance reference virtual image based on the visual recognition distance of the visually recognized virtual image misperceived from the long-distance reference virtual image and the short-distance reference virtual image, the visual recognition distance of the long-distance reference virtual image, and the visual recognition distance of the short-distance reference virtual image, Set the visual recognition distance of the visually recognized virtual image to be misjudged as D O Set the brightness of the visually recognized virtual image to be misjudged as V O Set the visual recognition distance of the long-distance reference virtual image to D L Set the brightness of the long-distance reference virtual image to V L Set the visual recognition distance of the short-distance reference virtual image to D S Set the brightness of the short-distance reference virtual image to V S When The control unit performs proportional distribution of the brightness of the long-distance reference virtual image and the short-distance reference virtual image in a manner that satisfies V L = V O × (D O - D S ) / (D L - D S ) V S = V O × (D L - D O ) / (D L - D S ) .

2. The virtual image display device according to claim 1, characterized in that, It further comprises: A visual recognition distance detection unit that detects the convergence amount of the pupil centers of the left and right eyes of the user to obtain the line-of-sight inclination angle of the left and right eyes, and estimates the actual visual recognition distance from the user to the object being gazed at by the user based on this line-of-sight inclination angle, The control unit makes the visual recognition distance of the visually recognized virtual image misperceived by the user match the actual visual recognition distance.

3. The virtual image display device according to claim 1, wherein The first image light and the second image light have substantially perpendicular polarization directions, and the optical paths of the first image light and the second image light are separated or combined by a polarization beam splitter disposed within the generation unit.

4. The virtual image display device according to claim 1, wherein The projection unit is composed of one or multiple stacked light guide plates, and each of the multiple light guide plates has an incident hologram with a function of allowing light waves to enter the light guide plate and an exit hologram with a function of allowing the light waves traveling within the light guide plate to exit to the outside.

5. The virtual image display device according to claim 1, wherein The virtual image display device is a head-mounted display device.

Images