Virtual image display device and head mounted display using the same

Abstract

The purpose is to provide a virtual image display device having a function of projecting a predetermined virtual image into a user's field of view, which effectively improves the physiological burden of the user and the narrow viewpoint range when visually recognizing the virtual image. In order to achieve the above-mentioned purpose, in a virtual image display device having a function of projecting a predetermined virtual image into a user's field of view, it is configured to have: a visual recognition distance detection unit, which detects the actual visual recognition distance from the user to the object with respect to the visual recognition object that the user is looking at or at least a part of the multiple objects in the field of view; a virtual image light generating unit, which has a function of generating multiple virtual image lights that are independent of each other; a virtual image light transmission and projection unit, which has a function of transmitting and projecting the virtual image light to the left and right eyes of the user via a substantially same optical path so that the predetermined virtual image is visually recognized at a predetermined position in the user's field of view; and a virtual image visual recognition distance control unit, which has a function of variably controlling the visual recognition distance of the virtual image visually recognized by the user using the multiple projected virtual image lights.

Description

Technical Field

[0001] The present invention relates to a virtual image display device having a function of allowing a user to visually recognize an arbitrary image as a virtual image through a predetermined optical system, and more particularly to a virtual image display device having a function of displaying a virtual image superimposed on an actual spatial field of view recognized by the user. Background Art

[0002] As examples of virtual image display devices having a function of displaying a predetermined virtual image in a superimposed manner within a user's visual field of real space, known are head mounted displays (hereinafter referred to as HMDs), heads-up displays (hereinafter referred to as HUDs), etc. In these virtual image display devices, the presence or absence of a function (hereinafter referred to as a virtual image visual recognition distance matching function) of matching the visual recognition distance of a virtual image displayed in a superimposed manner within a user's real space visual field with the real space visual recognition distance of an actual object within the visual field becomes an important performance requirement that reduces the physiological burden on the user and contributes to improving the user's visual recognition.

[0003] In the past, as a method for implementing a virtual image visual recognition distance matching function in an HMD that is particularly worn on the user's head, for example, Patent Document 1 discloses a virtual image visual recognition distance matching method based on a so-called "binocular parallax method", wherein a visual recognition distance (actual visual recognition distance) from an actual space object that the user is gazing at is detected by a predetermined detection unit, and the relative deviation between the user's left eye visual recognition image and the right eye visual recognition image generated when the user is gazing at the object at the actual visual recognition distance, i.e., "binocular parallax", is assigned to a virtual image displayed for the left eye that is visually recognized only by the user's left eye and a virtual image displayed for the right eye that is visually recognized only by the user's right eye, thereby giving the user an optical illusion (optical illusion) as if the visual recognition distance of the virtual image matches the actual space visual recognition distance.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2016-186561 Summary of the invention

[0007] However, in the virtual image visual recognition distance matching method based on the "binocular parallax method" disclosed in Patent Document 1, due to the binocular parallax, there is a deviation between the perceived virtual image distance perceived by the user and the actual virtual image distance at which the user's eyes are actually focused on the virtual image, that is, the actual virtual image distance, so there is a problem that it imposes a physiological burden on the user and the user's fatigue is great when using it for a long time. In addition, the binocular parallax itself depends not only on the visual recognition distance, but also on the direction of the user's gaze, so there is also a problem that the user's viewpoint range that can accurately visually recognize the virtual image at a predetermined visual recognition distance, the so-called eye box, is extremely narrow. Furthermore, in this "binocular parallax method", different virtual images with different positions of the display image items must be displayed when used for the left eye and when used for the right eye, so it is necessary to prepare two independent virtual image display optical systems in the HMD when used for the left eye and when used for the right eye, which has a significant limitation in the miniaturization, lightness, and cost reduction of the HMD.

[0008] Based on the above background, an object of the present invention is to satisfactorily improve the above-mentioned problems of physiological burden on the user and narrow visual range.

[0009] In view of the above-mentioned background technology and problems, the present invention, if taking one example, is configured to include: a visual recognition distance detection unit for detecting the actual visual recognition distance from the user to the visual recognition object that the user is gazing at or at least a part of a plurality of objects in the field of view; a virtual image light generating unit for generating a plurality of virtual image lights that are independent of each other; a virtual image light transmission and projection unit for transmitting and projecting the virtual image lights to the left and right eyes of the user via substantially the same optical path so that the predetermined virtual image is visually recognized at a predetermined position in the field of view of the user; and a virtual image visual recognition distance control unit for variably controlling the visual recognition distance of the virtual image visually recognized by the user using the plurality of projected virtual image lights.

[0010] According to the present invention, it is possible to provide a virtual image display device capable of realizing a virtual image visual recognition distance matching function in a good visual recognition environment that reduces the physiological burden on a user, and a head mounted display using the virtual image display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a schematic perspective view of the HMD in Example 1.

[0012] Figure 2 This is a schematic perspective view showing a usage scenario of the HMD in the first embodiment.

[0013] Figure 3 It is shown Figure 2 A schematic top view of an example of a visually recognizable image within the user's field of view in a usage scenario.

[0014] Figure 4 This is a schematic perspective view showing another usage scenario of the HMD in the first embodiment.

[0015] Figure 5 It is shown Figure 4 A schematic top view of an example of a visually recognizable image within the user's field of view in a usage scenario.

[0016] Figure 6 This is a schematic plan view showing a visual recognition state in a user's field of view in still another usage scenario of the HMD in the first embodiment.

[0017] Figure 7 This is a block diagram showing a schematic configuration of an HMD in the first embodiment.

[0018] Figure 8 1 is a flowchart showing the operation procedure of the HMD in the first embodiment.

[0019] Fig. 9 This is a schematic diagram for explaining the operating principle of the visual object actual visual distance detection unit of the HMD in the first embodiment.

[0020] Fig.10 This is a flowchart showing the operation procedure of the visual object actual visual distance detection unit of the HMD in the first embodiment.

[0021] Fig.11 1 is a plan view showing a schematic configuration of a virtual image light generating unit and a virtual image light transmission and projection unit of the HMD in the first embodiment.

[0022] Fig.12 This is a schematic diagram for explaining the outline of the operation of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in the first embodiment.

[0023] Fig.13 This is a schematic diagram for explaining the outline of other operations of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in the first embodiment.

[0024] Fig.14 This is a schematic diagram for explaining the outline of the variable control of the virtual image visual recognition distance of the HMD in the first embodiment.

[0025] Fig.15 This is a schematic diagram for explaining the operating principle of the visual object actual visual distance detection unit of the HMD in the second embodiment.

[0026] Fig.16 This is a flowchart showing the operation procedure of the visual object actual visual distance detection unit of the HMD in the second embodiment.

[0027] Fig.17 1 is a plan view showing a schematic configuration of a virtual image light generating unit and a virtual image light transmission and projection unit of an HMD in Example 3.

[0028] Fig.18 This is a schematic diagram for explaining the outline of the operation of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in the third embodiment.

[0029] Fig.19 This is a schematic diagram for explaining the outline of other operations of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in the third embodiment.

[0030] Fig. 20 1 is a plan view showing a schematic configuration of a virtual image light generating unit and a virtual image light transmission and projection unit of an HMD in Example 4.

[0031] Fig.21 1 is a plan view showing a schematic configuration of a virtual image light generating unit and a virtual image light transmission and projection unit of the HMD in Example 5.

[0032] Fig. 22 1 is a plan view showing a schematic configuration of a virtual image light generating unit and a virtual image light transmission and projection unit of an HMD in Example 6.

[0033] Fig.23 This is a schematic perspective view showing a usage scenario of the portable information terminal in the seventh embodiment.

[0034] Fig.24 This is a schematic front view showing a usage scenario of the vehicle head-up display in the eighth embodiment.

[0035] (Explanation of symbols)

[0036] 1: HMD; 4, 5: light guide plate; 6, 7: incident hologram; 8a, 8b, 9a, 9b: emitting hologram; 10: virtual image light transmission projection unit; 20: small electronic camera for shooting outdoor images; 30: drive control circuit; 34: actual visual recognition distance detection unit for visual recognition object; 35: virtual image visual recognition distance control unit; 40, 41: virtual image light generation unit; 42, 43: display for displaying virtual images; 46, 47: projection lens system for virtual image light generation; 49: trapezoidal PBS (polarization beam splitter); 50: portable information terminal; 100: user. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0038] Example 1

[0039] Figure 1 : is a schematic three-dimensional diagram of a glasses-type HMD as a virtual image display device in this embodiment. Figure 1 As shown, the glasses-type HMD 1 is worn on the head of the user 100. The HMD 1 stores the electronic circuit and the optical engine described later in the parts 2 and 3 corresponding to the temples of the glasses for mounting on the left and right ears. In addition, transparent thin plates 4 and 5 made of optical glass or plastic parts for optical components are arranged in the parts corresponding to the lenses of the glasses. The transparent thin plates 4 and 5 are optical devices called "light guide plates", and in the present embodiment, the structure is a stack of light guide plates 4 and 5, and the parts corresponding to the left and right glasses lenses are covered by the same light guide plate. Furthermore, on the light guide plates 4 and 5, in the parts located in front of the left and right eyes, predetermined holographic light for emission is arranged. Figure 8 a, 8b, 9a, 9b. The light guide plates 4, 5 and the holographic projection plates for emission will be described later. Figure 8 Detailed contents of the functions of a, 8b, 9a, 9b, etc.

[0040] Furthermore, in this embodiment, the temples 2 and 3 are equipped with a small electronic camera 20 for photographing the scenery in front of the user's eyes. In particular, in this embodiment, the small electronic camera 20 is a two-eye camera installed on both of the left and right temples 2 and 3, and is configured to also have a function of measuring the distance to the photographed object through a so-called stereo camera function. However, the camera for photographing scenery mounted on the HMD as described in this embodiment is not limited to the above-mentioned two-eye stereo camera, and of course it can also be a single-eye camera, and its installation position is not limited to the above-mentioned stereo camera. Figure 1 Position shown.

[0041] In addition, in this embodiment, a predetermined sensor box 21 is arranged on the upper part of the eyeglass frame. In the sensor box 21, for example, a so-called eye tracking camera or a TOF (time of flight) sensor for monitoring the positions of the pupils of the left and right eyes of the user 100 is mounted. T ime O f FThe sensor box 21 may be a sensor that measures the distance to any object in front of the user's eyes, such as an eye tracking camera or a TOF sensor. The sensor in the sensor box 21 is not limited to an eye tracking camera or a TOF sensor. Any sensor may be used as long as it has a function of measuring the distance to any object in the external scene in front of the user's eyes or detects information signals required for detecting the distance information in a predetermined manner. The location of the sensor box 21 itself is not limited to the location shown in the figure. Specifically, the sensor box itself may be removed, and each sensor may be assembled at a predetermined location inside the HMD 1.

[0042] However, in this embodiment, the HMD 1 has a function of wirelessly communicating with a portable information terminal 50 carried by a user, such as a smartphone, and is a structure for acquiring data related to a predetermined virtual image to be displayed by the HMD 1 from the portable information terminal 50. However, this structure is not limited to the above structure, and for example, the HMD itself may have a part or all of the functions of the portable information terminal 50, and the data related to the virtual image to be displayed may be retained inside the HMD 1. The structure of the HMD 1 in this embodiment is specifically described below.

[0043] Furthermore, before describing the present embodiment in detail, in order to make the functions to be implemented by the virtual image display device of the present embodiment clearer, several examples of scenarios in which users use an HMD utilizing the virtual image display device of the present embodiment in daily life are introduced.

[0044] For example, Figure 2 is a schematic perspective view showing a scene in which a user 100 wearing the HMD 1 of the present invention is looking at a document 60 held by the user. Figure 3An example of the actual visual image of the document 60 visually recognized in the visual field 101 (area surrounded by dotted lines) of the user 100 at this time and the virtual image 70 displayed in an overlapping manner in the same visual field 101 by the HMD 1 is shown. At this time, the eyes of the user 100 are of course focused on the handheld document 60, and the visual recognition distance a is as high as several tens of centimeters. In such a usage scenario, for example, when the associated information or explanation information related to certain text paragraphs recorded on the document 60 is displayed as a virtual image 70 based on the HMD 1 in an overlapping manner at a predetermined position in the visual field 101, the visual recognition distance of the virtual image 70 needs to be at least roughly consistent with the visual recognition distance a at which the eyes of the user 100 are focused at that time point. The reason is that, if the visual recognition distances of the two are not consistent, there are image items with different visual recognition distances, i.e., the distances at which the eyes are focused, in the visual field of the user 100, and it is difficult for the HMD user to visually recognize them at the same time without discomfort or pressure.

[0045] Likewise, for example Figure 4 The figure shows that the user 100 wearing the HMD 1 is looking at a predetermined object 61 (at a distance of several meters). Figure 4 In the example of a painting, a rough three-dimensional image of the scene is Figure 5 An example is shown of an actual visually recognizable image of an object 61 visually recognized in the visual field 101 (area surrounded by a dotted line) of the user 100 at this time and a virtual image 71 displayed in an overlapping manner in the same visual field 101 by the HMD 1. In this scenario, as in the above-mentioned document viewing scenario, both eyes of the user 100 are focused on the visually recognizable object 61, but the visual recognition distance b thereof is different from the case of document viewing and extends to several meters. In such a usage scenario, for example, when related information or explanation information related to the object 61 is displayed as a virtual image 71 based on the HMD 1 in an overlapping manner at a predetermined position in the visual field 101, if the visual recognition distance of the virtual image 71 is not roughly consistent with the visual recognition distance b at which both eyes of the user 100 are focused, it is difficult for the user 100 to visually recognize the virtual image 71 and the object 61 at the same time without discomfort or pressure.

[0046] In this way, in order for the HMD user to simultaneously and continuously visually recognize the real image seen in his or her field of view and the virtual image superimposed and displayed by the HMD without feeling discomfort or pressure, the visual recognition distance of the virtual image superimposed and displayed by the HMD must be roughly consistent with the visual recognition distance of the object actually visually recognized by the user.

[0047] Furthermore, in order to realize the so-called augmented reality ( A ugmented RIn the present invention, a visual recognition environment of augmented reality (AR) needs to at least associate any object seen in the field of view of the HMD user with the object and display a predetermined virtual image as a virtual image in an overlapping manner. Therefore, the visual recognition distance of the virtual image needs to be roughly consistent with the actual visual recognition distance of the object, or it needs to be displayed at a predetermined visual recognition distance associated with the actual visual recognition distance.

[0048] For example, Figure 6 An example of an actual visually recognizable object seen in the field of view of an HMD user when viewing the interior of a room is shown, but each furniture item 62 to 67 in the room, such as a table 62, a sofa 65, and a curtain 67, naturally has a visually recognizable distance that is different from the user's standing position at that point in time, depending on its installation position. Moreover, the visually recognizable distance changes one by one according to the change in the user's standing position. Therefore, for example, when an AR environment is realized by using an HMD to display certain virtual images 72 to 77 as virtual images in an overlapping manner for each of these furniture items, it is necessary to detect the actual visually recognizable distance of each furniture item one by one, so that the visually recognizable distance of the corresponding virtual image is roughly consistent with the detected actual visually recognizable distance, or to display it at a predetermined visually recognizable distance associated with the actual visually recognizable distance.

[0049] In order to meet the needs described above, the present embodiment provides a virtual image display device such as an HMD having the following functions: detecting the actual visual recognition distance of the objects that the user is looking at or the objects that the user sees in the field of view one by one, and displaying a predetermined virtual image at a predetermined visual recognition distance that is roughly consistent with the actual visual recognition distance or associated with the actual visual recognition distance in almost real time.

[0050] In addition, Figure 3 , Figure 5 , Figure 6 In the present invention, for the purpose of making the description easy to understand, the examples of virtual image items that are displayed in an overlapping manner within the user's field of view through the HMD are limited to information display items in a so-called "prompt window" manner, but of course the display of virtual image items is not limited to such a "bubble" manner, and can also be image items in any manner.

[0051] Next, the present embodiment is described. Figure 1 The specific structure of the HMD shown. Figure 7 is a block diagram showing a schematic structure of the HMD 1 in this embodiment. Figure 7In the display of virtual images, the information signal receiving unit 31 receives necessary data related to the virtual image to be displayed in a superimposed manner using the HMD 1 by mutual communication with the portable information terminal 50 carried by the user 100. In addition, at the same time, the image information acquiring unit 32 acquires the image of the scenery in front of the user's eyes captured by the small electronic camera 20 for scenery photography and the image of the pupils of both eyes of the user 100 captured by the eye tracking camera in the sensor box 21 as predetermined image data. In addition, based on the virtual image data received by the information signal receiving unit 31 for display of virtual images and the predetermined image data acquired by the image information acquiring unit 32, the image data generating unit 33 for display of virtual images 78 or 79 to be displayed in a superimposed manner in the user's field of view is generated.

[0052] On the other hand, in the visual recognition object actual visual recognition distance detection unit 34, the actual visual recognition distance of the user 100 is detected one by one using a predetermined method. In addition, in the virtual image visual recognition distance control unit 35, a predetermined control process is performed for variably controlling the visual recognition distance of the displayed virtual image according to the actual visual recognition distance detected by the visual recognition object actual visual recognition distance detection unit 34. In addition, the detection of the actual visual recognition distance in the visual recognition object actual visual recognition distance detection unit 34 and the variable control of the visual recognition distance of the displayed virtual image in the virtual image visual recognition distance control unit 35 will be described later.

[0053] Next, the memory 36 has a function of temporarily storing at least a part of various data signals generated by the above-mentioned blocks, but it is not necessarily arranged in the HMD 1, and for example, a memory having the same function may be arranged on the side of the portable information terminal 50. In addition, the operations of the above-mentioned blocks 31 to 36 are appropriately controlled by the control unit 37.

[0054] In addition, each block from the virtual image display information signal receiving unit 31 to the control unit 37 (blocks in the area surrounded by a single-dot chain line in the figure) is a block composed only of electronic circuits, and in the following description, it is collectively described as the HMD drive control circuit 30 for simplicity.

[0055] On the other hand, the virtual image light generating units 40 and 41 are respectively composed of a predetermined virtual image display display, a drive circuit (display driver) of the display, and a predetermined optical system, and the subsequent virtual image light transmission projection unit 10 is composed only of predetermined optical components, and has a function of transmitting virtual image light to the left and right eyes of the user through a substantially same optical path and projecting it so that the user can visually recognize a predetermined virtual image at a predetermined position in the user's field of view. In addition, a specific structural example of the virtual image light generating units 40 and 41 and the virtual image light transmission projection unit 10 will also be described later.

[0056] Figure 8 It is shown Figure 1 as well as Figure 7 The flowchart of the operation process of the HMD in this embodiment is shown in FIG. Figure 8 After the process starts, first in step 201 (hereinafter referred to as step S in the figure, as recorded as S201), N display virtual image items (S1, S2,: S201) sent from the portable information terminal 50 are received. N ). At the same time, in step 202, image data of actual objects in the user's field of view are acquired from the scene image in front of the user's eyes captured by the small electronic camera 20 for scene photography. Furthermore, in step 203, image data of the user's pupils are acquired by an eye tracking camera.

[0057] Next, in step 204, the predetermined number variable n is set to 1, and in the next step 205, from each image data of the actual object in the field of view obtained in step 202, an object that becomes the link display object of the received nth display virtual image item Sn is selected. Then, in the next subroutine 300 (S300), the actual visual recognition distance of the object is detected by the method described later, and the visual recognition distance of the display virtual image item Sn is determined. Furthermore, in the next subroutine 400 (S400), as data required for the predetermined control method used in this embodiment to variably control the visual recognition distance of the display virtual image item Sn, the brightness ratio distribution data of the long-distance reference virtual image and the short-distance reference virtual image are estimated. The details of the variable control of the virtual image visual recognition distance based on the long-short reference virtual image and its brightness ratio distribution will be described later.

[0058] Next, in step 206, the display position and size of the virtual image item Sn on the virtual image display monitor are determined. Then, in step 207, it is determined whether n=N. If not, n=n+1 is set and feedback is given to step 205.

[0059] By repeatedly performing the above processing flow, the received N virtual image items (S1, S2, S3) are displayed. N ) to determine the above-mentioned image display data and the above-mentioned near and far reference virtual image brightness ratio distribution data.

[0060] Then, in step 208, the N virtual image items (S1, S2, S3) determined above are used to display the virtual image. N) and the near and far reference virtual image brightness ratio distribution data related to each virtual image item displayed by the display device, generate an image data signal input to the virtual image display display, and actually drive the virtual image display display to perform a predetermined image display in the next step 209. Then, by further passing through a predetermined optical system, a predetermined near and far reference virtual image light is generated based on the image light emitted from the above-mentioned display, and the virtual image light is transmitted to the user's eyes for projection.

[0061] Next, about Figure 8 The actual visual recognition distance detection of the object visually recognized by the user executed in the subroutine 300 (S300) in the flowchart of FIG. 1 is described in detail.

[0062] Fig. 9 as well as Fig.10 The following is a schematic diagram and a flow chart for explaining the actual visual recognition distance detection process and principle. First, the eye tracking camera or the like is used to capture the pupils of the user 100, and the offset of the left and right pupil centers 102 and 103 shown in the figure from the reference position (pupil center position when looking at infinity) 104 and 105, the so-called convergence amount P, is detected from the image data. L and P R The distance between the pupil centers of the left and right eyes is W LR .

[0063] Now, when the distance from the rotation centers 106 and 107 of the left and right eyeballs to the pupil centers 102 and 103 of the left and right eyeballs, i.e., the eyeball rotation radius is defined as R, the line of sight inclination angle α of the left and right eyes of the user 100 toward the gaze point 120 on the arbitrary visual recognition object 68 is L α R It is obtained using the following equations (1) and (2).

[0064] tan(α L )=P L / R---(1)

[0065] tan(α R )=P R / R---(2)

[0066] Furthermore, the actual visual recognition distance S, which is the distance from the center position of the left and right pupils of the user 100 to the gaze point 120, is also calculated using the following relational expression (3).

[0067] S=W LR / {tan(α L )+tan(α R )}=R×W LR / {P L+P R}---(3)

[0068] By executing such processing steps one by one, the actual visual recognition distance from the object visually recognized by the user 100 can be detected in almost real time.

[0069] In addition, regarding Fig.10 Since the processing procedures of each step in the processing steps S301 to S306 in the flowchart are repeated in the above description, the description is omitted.

[0070] The eyeball rotation radius R may be detected in advance before use by a predetermined calibration process (estimated by performing the same detection process using a calibration object whose actual visual recognition distance is known and inversely calculating the above equations (1) to (3)).

[0071] Next, explain Figure 7 The specific structure of the virtual image light generating units 40, 41 and the virtual image light transmission projection unit 10 in the block diagram and the specific embodiment of the variable control of the visual recognition distance of the virtual image displayed overlappingly in the user's field of view by the virtual image light generating units 40, 41 and the virtual image light transmission projection unit 10.

[0072] Fig.11 It is shown Figure 1 as well as Figure 7 FIG. 1 is a top view showing the schematic structure of the virtual image light generating units 40 and 41 and the virtual image light transmission projection unit 10 of the HMD in this embodiment. Figure 1 The driving control circuit 30 and independent virtual image light generating units 40 and 41 (parts surrounded by single-dot chain lines in the figure) are arranged in the parts corresponding to the temples of the glasses of the HMD 1 shown. In addition, in the virtual image light generating units 40 and 41, for example, virtual image display monitors 42 and 43 composed of image monitors such as liquid crystal displays, display driving circuits 44 and 45 for driving the monitors, and projection lens systems 46 and 47 for virtual image light generation are arranged respectively. In addition, the virtual image display monitors 42 and 43 are not limited to the above-mentioned liquid crystal displays, and can be any display in any form and structure as long as they are small image monitors such as organic electroluminescent displays (abbreviated as: OLED).

[0073] On the other hand, in the virtual image light transmission projection unit 10 (the portion surrounded by the dotted line in the figure), as shown in FIG. Figure 1As also shown in FIG. 1 , two stacked transparent light guide plates 4 and 5 are arranged. The light guide plates 4 and 5 have the function of sealing and transmitting light incident on the light guide plate at a predetermined incident angle to the inside of the light guide plate through repeated total reflection on the upper and lower surfaces of the light guide plate. In addition, the light guide plates 4 and 5 are respectively arranged with: a holographic light source for incident light; Figure 6 and 7, having the function of directing incident light waves into the light guide plate; and emitting holographic Figure 8 a, 8 and 9a, 9b have the function of emitting the light waves traveling in the light guide plate to the outside. As for the light guide plate having such an incident hologram and an emitting hologram, it is already known as an optical device for virtual image light transmission projection for HMD, so the above detailed description is omitted.

[0074] In the present embodiment, in response to a predetermined instruction signal or control signal from the HMD drive control circuit 30 , an image corresponding to a display virtual image is projected on the virtual image display monitors 42 , 43 to generate predetermined image light.

[0075] Then, the first image light generated by the display 42 among the image lights is firstly Fig.12 The light is incident on the projection lens system 46 for generating virtual image light, where it is converted into the first virtual image light having a predetermined visual recognition distance. Figure 6 The incident light enters the interior of the light guide plate 4, and then travels to the right in the figure. Figure 8 a and 8b are emitted and incident on the eyes 112a and 112b of the user 100 respectively. At this time, the user 100 is guided by the first virtual image light at a predetermined visual recognition distance D passing through the transparent light guide plates 4 and 5. L The virtual image 78 is visually recognized at the position of. In this specification, the virtual image 78 is hereinafter described as a long-distance reference virtual image.

[0076] On the other hand, the second image light generated by the display 43 is as follows: Fig.13 The light is incident on the projection lens system 47 for generating virtual image light, where it is converted into a second virtual image light having a predetermined visual recognition distance. Figure 7 The incident light enters the light guide plate 5, travels to the left in the light guide plate 5, and then exits the holographic Fig. 9 a and 9b are emitted and emitted from the above-mentioned holographic Figure 8 The virtual image lights emitted from the second virtual image light 8a and 8b are incident on the eyes 112a and 112b of the user 100 respectively. At this time, the user 100 is guided by the second virtual image light at a predetermined visual recognition distance D passing through the transparent light guide plates 4 and 5. S (DS <D L ) position to visually recognize the virtual image 79. In this specification, the virtual image 79 will be described as the short-distance reference virtual image.

[0077] That is, when the user 100 observes, the predetermined visual recognition distance D passing through the transparent light guide plates 4 and 5 is L The long-distance reference virtual image 78 is accurately visually recognized at the position of the visual recognition distance D S (D S <D L ) so that the close-range reference virtual image 79 can be accurately visually recognized at the position of the virtual image 79, and the optical constants and lens positions of the projection lens systems 46 and 47 for generating the virtual image light are appropriately set.

[0078] Furthermore, the long-distance reference virtual image 78 and the short-distance reference virtual image 79 are as shown in FIG. Fig.14 As shown in the figure, the image display positions in the virtual image display monitors 42 and 43 and the visual recognition magnifications of the virtual images of the virtual image light generation projection lens systems 46 and 47 are appropriately set so that the complete projective relationship is overlapped and visually recognized. Specifically, with respect to the visual recognition magnification M of the long-distance reference virtual image 78, the visual recognition magnification M' of the short-distance reference virtual image 79 is set to M'=(D S / D L )×M.

[0079] Furthermore, in this state, the brightness V of the long-distance reference virtual image 78 and the short-distance reference virtual image 79 are calculated based on the following predetermined relational expressions (4) and (5). L and V S When performing proportional distribution, the user 100 is at a distance D L With D S The illusion is that a virtual image is visually recognized at any distance Do in the middle of

[0080] V L =V O ×(D O -D S ) / (D L -D S )---(4)

[0081] V S =V O ×(D L -D O ) / (D L -D S )---(5)

[0082] In addition, V OIt is the brightness of the virtual image perceived at any distance Do.

[0083] This kind of optical illusion is generally called stereoscopic optical illusion. D epth- f used 3 D : DFD for short). As DFD is a physiological phenomenon, since it is a well-known phenomenon, detailed description of its physiological mechanism of occurrence is omitted. However, compared with the binocular parallax method that is the most widely used stereoscopic observation based on optical illusion, the stereoscopic vision using the DFD has the characteristics of less fatigue of the observer and a wide viewpoint area (EyeBox) that can accurately observe stereoscopically. In addition, in the stereoscopic vision using the DFD, the visual recognition distance of the optical illusion virtual image can be matched to an arbitrary distance between the near and far reference virtual images by a relatively simple method of controlling the brightness ratio distribution of the near and far reference virtual images. Therefore, it is a method that matches the situation such as performing variable control of the user's actual visual recognition distance detection and the display virtual image visual recognition distance suitable for the actual visual recognition distance in almost real time as in the present embodiment. Therefore, in the present embodiment, variable control of the visual recognition position of the displayed virtual image using the DFD is implemented.

[0084] That is, in the present embodiment, as a virtual image display device, there is provided: a visual recognition distance detection unit, which detects the actual visual recognition distance from a predetermined object in the user's field of view; a virtual image light generating unit, which has the function of generating at least two reference virtual image lights with different visual recognition distances; a virtual image light transmission projection unit, which has the function of transmitting and projecting the reference virtual image light to the user's eyes via a transparent light guide plate provided with a hologram for light wave incidence and emission; and a virtual image visual recognition distance control unit, which can variably control the visual recognition distance to a predetermined distance by controlling the image brightness ratio distribution of the two virtual image lights according to the detection of the actual visual recognition distance.

[0085] Thus, according to this embodiment, a virtual image display device in a good visual recognition environment can be provided, which reduces the physiological burden of the user by making the visual recognition distance of the user visually recognize the virtual image match the actual visual recognition distance detected by the visual recognition distance detection unit by performing variable control of the display virtual image visual recognition position using the DFD, and an HMD using the device. In addition, a small, lightweight, and low-cost virtual image display device suitable for the device can be further provided.

[0086] In addition, the variable control of the visual recognition position of the displayed virtual image using the DFD is a method of matching the visual recognition distance of the illusion virtual image with an arbitrary distance between the far and near reference virtual images by controlling the brightness ratio distribution of the far and near reference virtual images as described above, so its variable control range, that is, the dynamic range, is limited to the visual recognition distance range between the far and near reference virtual images. Therefore, for example, by mechanically adjusting the lens interval of each projection lens constituting the projection lens system 46 and 47 for generating virtual image light, the visual recognition distance of the far and near reference virtual images themselves can be made variable, thereby arbitrarily enlarging or reducing the dynamic range of the visual recognition distance of the illusion virtual image. However, in such a case, in order to make the far and near reference virtual images always overlap and visually recognize in accordance with a complete projection relationship, it is also necessary to appropriately control the image display position in each display and the virtual image projection magnification of the projection lens system 46 and 47 for generating virtual image light.

[0087] In addition, the reference virtual images that determine the dynamic range of the variable control of the visual recognition position of the displayed virtual image based on the DFD are not limited to the two of the long-distance reference virtual image 78 and the short-distance reference virtual image 79. By adding the virtual image light generating unit and the virtual image light transmission and projection unit as described above, one or more intermediate distance reference virtual images are set at any visual recognition distance between the farthest distance reference virtual image and the shortest distance reference virtual image, and two reference virtual images are arbitrarily selected from these multiple reference virtual images, and they are re-used as the long-distance reference virtual image 78 and the short-distance reference virtual image 79 to perform the brightness ratio distribution control as described above, so that the dynamic range of the variable control of the visual recognition position of the displayed virtual image can be arbitrarily changed.

[0088] Furthermore, the present embodiment is not limited to the variable control of the visual recognition position of the displayed virtual image based on DFD. As long as it is a variable control applicable to the present embodiment, it can be of course any method. For example, in the fifth embodiment described later, an example of the variable control of the visual recognition position of the displayed virtual image based on the binocular parallax method widely used in the past is also disclosed.

[0089] Example 2

[0090] In the first embodiment, the actual visual recognition distance of the user visually recognizes the object is detected by using the method described in Fig. 9 as well as Fig.10 The unit described in the above description is based only on the detection of the convergence amount of the user's pupils, but of course the actual visual recognition distance detection is not limited to this.

[0091] Hereinafter, as this embodiment, a new actual visual recognition distance detection different from the above-mentioned detection means will be described. Fig.15 It is a schematic diagram used to illustrate its detection principle. Fig.16 is a flowchart summarizing its detection process. First, use Fig.16 Let’s explain it from the perspective of the detection process.

[0092] exist Fig.16 In the present invention, first, in step 351, each image data of the object in the field of view is obtained from the external scene image in front of the user captured by the small electronic camera 20 for capturing external scene images. Next, in step 352, the relative position of the user base point of each object in the user's field of view is detected from the acquired image data. Then, in step 353, the user base point azimuth (estimated angle when the user is looking at the object) β of each object in the field of view is calculated based on the detected relative position.

[0093] On the other hand, at the same time, in step 354, the image data of the user's pupils taken by the eye tracking camera is obtained, and in the next step 355, the pupil center positions of the left and right eyes are detected based on the above image data. Then, in the next step 356, the line of sight tilt angle α of the left and right eyes is calculated. L , α R (Refer to Fig.15 ). In addition, the process from step 354 to step 356 is the same as in Fig. 9 as well as Fig.10 The process from step 301 to step 305 in the first embodiment related to the detection of the user's actual visual recognition distance described in is the same as the process.

[0094] In this embodiment, in the next step 357, according to the line of sight inclination angle α of the left and right eyes calculated in the above process, L , α R , the following relational expression (6) is used to calculate the gaze line inclination angle γ of the user 100 toward the gaze point 120 on the visual recognition object 68 currently being gazed by the user (refer to Fig.15 ).

[0095] tan(γ)=sin(α L -α R ) / {cos(α L +α R )+cos(α L -α R )}---(6)

[0096] Next, in step 358, the user base point azimuth angle β of the object in the field of view calculated in the above process and the user's gaze line inclination angle γ are compared to determine the object in the field of view that matches β and γ, that is, the object currently being gazed at by the user. In addition, the effective distance S from the determined gaze object, that is, the actual visual recognition distance of the user at the current time point, is measured using a distance measurement sensor equipped in the HMD. In addition, as long as the distance measurement sensor is Figure 1 Any sensor can be used instead of existing distance measuring sensors such as the so-called stereo camera, TOF sensor or ultrasonic sensor introduced in the text.

[0097] However, the first and second embodiments described above have been intended to detect the actual visual recognition distance of the HMD user and to display a virtual image in accordance with the actual visual recognition distance, but the present embodiment is of course not limited to such an objective.

[0098] For example, it may be entirely possible to have a display unit that detects the effective distances of all objects seen by the user in the field of view regardless of whether the user is looking at them or not, and makes the visual recognition distances of the virtual images displayed in overlap with the objects match the effective distances of the associated actual objects, thereby achieving so-called AR (augmented reality) response.

[0099] Furthermore, a virtual image can be displayed at an arbitrary visual recognition distance completely unrelated to the effective distance of each object in the field of view or the actual visual recognition distance of the user, or a display method can be used in which the visual recognition position of the displayed virtual image is dynamically changed between arbitrary visual recognition distances by utilizing variable control of the visual recognition distance of the displayed virtual image in real time.

[0100] Example 3

[0101] Fig.17 1 is a top view showing the schematic structure of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in this embodiment. Fig.17 In, with Fig.11 The same components as shown are denoted by the same reference numerals.

[0102] exist Fig.11In order to generate long-distance reference virtual image light and short-distance reference virtual image light separately, two independent virtual image light generating units 40 and 41 are arranged in the HMD. In addition, the virtual image light transmission projection unit 10 is composed of two transparent light guide plates 4 and 5 stacked in a stack, and the virtual image light generated by the virtual image light generating units 40 and 41 is transmitted in one of the two light guide plates 4 and 5. However, it is also possible to have a structure in which the long-distance reference virtual image light and the short-distance reference virtual image light are transmitted together in a single light guide plate instead of being transmitted in separate light guide plates. In addition, by studying the optical system, the two independent virtual image light generating units as described above can also be integrated into one. Fig.17 The present embodiment shown is an embodiment for achieving such an object.

[0103] First, if Fig.17 As shown, the HMD1 in this embodiment is Fig.11 Unlike the embodiment 1 shown, only a single virtual image light generating unit 40 (the portion surrounded by a single dot-dash line in the figure) is provided. The virtual image light generating unit 40 includes a display 42 for displaying virtual images, a display driving circuit 44 for driving the display, a first virtual image light generating projection lens system 46a and a second virtual image light generating projection lens system 46b divided into two lens groups, a trapezoidal PBS (polarization beam splitter) 49, and a polarization conversion element 48 having a function of appropriately converting the polarization direction of incident light into mutually perpendicular P polarized light and S polarized light according to a predetermined electrical signal. In addition, as the above-mentioned polarization conversion element, an existing device such as a liquid crystal type polarization conversion element used in a liquid crystal display can also be used.

[0104] In this embodiment, only a single light guide plate 4 is disposed in the virtual image light transmission projection unit 10 (the portion surrounded by the dotted line in the figure). Furthermore, two holographic lenses for incident light are provided in the light guide plate 4. Figure 6 And 7 and 2 injection holographic Figure 8 a and 8b.

[0105] In this embodiment, first, the image for the long-distance reference virtual image light and the image for the short-distance reference virtual image light are alternately switched and displayed at each predetermined frame rate on the virtual image display monitor 42. The generated image light is incident on the first virtual image light generating projection lens system 46a, where it is converted into virtual image light having a predetermined visual recognition distance, for example, having a visual recognition distance D L After the long-distance reference virtual image light is incident on the polarization conversion element 48, the polarization conversion element 48 is set to switch the incident light to P polarization light or S polarization light in synchronization with the timing of display switching of the long-distance reference virtual image light image and the short-distance reference virtual image light image displayed on the display 42.

[0106] Now, the virtual image light incident on the polarization conversion element 48 through the polarization conversion element 48 is set to be converted into P polarized light at the timing of displaying the image for the long-distance reference virtual image light on the display 42, and conversely converted into S polarized light at the timing of displaying the image for the short-distance reference virtual image light. In addition, the virtual image light converted into the P polarized light is hereinafter described as the first virtual image light, and the virtual image light converted into the S polarized light is hereinafter described as the second virtual image light.

[0107] Then, the first virtual image light converted into P polarized light (polarization direction indicated by the arrow lines at both ends in the figure) by the polarization conversion element 48 is shown as follows: Fig.18 As shown in FIG. 1 , the polarized light is transmitted through the polarized light separation film 49a of the trapezoidal PBS 49, and then passes through the first incident holographic Figure 6 The light is incident on the inside of the light guide plate 4. Then, after traveling to the right in the figure, it is emitted from the holographic Figure 8 The light rays 8a and 8b are emitted and enter the eyes 112a and 112b of the user 100 respectively.

[0108] At this time, the first virtual image light is generated by the first virtual image light generating projection lens system 46a as light corresponding to the long-distance reference virtual image light, so that the user 100 can see the predetermined visual recognition distance D passing through the transparent light guide plate 4. L The long-distance reference virtual image 78 is visually recognized at a predetermined visual recognition magnification at the position.

[0109] On the other hand, the second virtual image light is converted by the polarization conversion element 48 into S polarized light having a polarization direction perpendicular to the P polarized light. Fig.19 As shown in FIG. 1 , after being reflected on the polarization separation film 49a of the trapezoidal PBS 49, the light is further reflected on the total reflection surface 49b, and enters the second virtual image light generation projection lens system 46b through an optical path different from that of the first virtual image light. The second virtual image light generation projection lens system 46b has a function of converting the second virtual image light, which is temporarily generated as light corresponding to the long-distance reference virtual image light by the first virtual image light generation projection lens system 46a, into light corresponding to the predetermined visual recognition distance D. S (D S <D L )'s close-range reference virtual image light is equivalent to the function of light.

[0110] Then, the second virtual image light converted into the short-distance reference virtual image light corresponding light is transmitted through the second incident holographic image light arranged on the light guide plate 4. Figure 7 The light is incident on the inside of the light guide plate 4. Then, it travels in the right direction of the figure in the same way as the first virtual image light in the light guide plate 4, and then exits from the holographic Figure 8a and 8b, and enter the eyes 112a and 112b of the user 100 respectively. At this time, the second virtual image light is transformed into virtual image light corresponding to the above-mentioned short-distance reference virtual image light by passing through both the projection lens systems 46a and 46b for virtual image light generation as described above, so that the user 100 can see the predetermined visual recognition distance D of the user 100 after passing through the transparent light guide plate 4. S The close-range reference virtual image 79 is visually recognized at a predetermined visual recognition magnification at the position.

[0111] That is, the user 100 visually recognizes the long-distance reference virtual image 78 and the short-distance reference virtual image 79 alternately switched at a predetermined frame rate, but the user 100 cannot distinguish the virtual images switched at a high speed, and thus perceives as if visually recognizing the long-distance reference virtual image 78 and the short-distance reference virtual image 79 at the same time. Therefore, similarly to the first and second embodiments, the display virtual image visual recognition distance can be controlled variably using the stereoscopic illusion phenomenon (DFD).

[0112] As described above, by setting the structure shown in this embodiment, the light guide plates in the virtual image light generating unit and the virtual image light transmission projection unit can be integrated and reduced to one, which can achieve significant effects in miniaturization, weight reduction and cost reduction of the HMD compared to the above-mentioned first embodiment.

[0113] In addition, of course, the relationship between the long / short distance reference virtual image light and the P / S polarized light described in this embodiment can also be completely opposite. In this case, the first virtual image light generating projection lens system 46a has the function of generating light equivalent to the short distance reference virtual image light, and the second virtual image light generating projection lens system 46b has the function of further converting the light into light equivalent to the long distance reference virtual image light.

[0114] Example 4

[0115] Fig. 20 1 is a top view showing the schematic structure of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in this embodiment. Fig. 20 In, with Fig.11 as well as Fig.17 The same components as shown are denoted by the same reference numerals.

[0116] In the third embodiment, a single virtual image display is used to display the long-distance reference virtual image light image and the short-distance reference virtual image light image in a time-division manner at a predetermined frame rate, and the polarization direction of the virtual image light generated synchronously with the frame rate is also time-division switched, so that the user 100 perceives as if he or she is visually recognizing the long-distance reference virtual image and the short-distance reference virtual image at the same time. In contrast, the present embodiment is configured to project the long-distance and short-distance reference virtual image lights simultaneously to the eyes 112a and 112b of the user 100 instead of the time-division display as described above.

[0117] In addition, in this embodiment, Fig.17 Similarly, only a single light guide plate 4 is arranged in the virtual image light transmission projection unit 10 (the portion surrounded by the dotted line in the figure). In addition, one incident holographic Figure 6 and 2 Holographic Figure 8 a and 8b.

[0118] In this embodiment, in the virtual image display monitor 42 in the virtual image light generating unit 40, its image display surface is divided into two display areas 42a and 42b, and it is possible to simultaneously display independent images in each display area, or to independently control the brightness ratio distribution of each display item in spite of being the same image. Furthermore, phase plates or polarization filters 11 and 12 are respectively arranged immediately after each display area. The phase plates or polarization filters 11 and 12 have the function of aligning the polarization directions of the image light emitted from each divided display area 42a and 42b of the above-mentioned monitor 42 to predetermined polarization directions. That is, the phase plate or polarization filter 11 aligns the first image light emitted from the divided display area 42a to P polarized light (polarization direction indicated by the arrow lines at both ends in the figure), and the polarization filter 12 aligns the second image light emitted from the divided display area 42b to S polarized light perpendicular to the P polarized light.

[0119] Therefore, for example, an image corresponding to the long-distance reference virtual image light is displayed in the divided display area 42 a , and an image corresponding to the short-distance reference virtual image light is displayed in the divided display area 42 b at the same time.

[0120] First, the first image light emitted from the divided display area 42a enters the trapezoidal PBS 49 via the phase plate or the polarization filter 11. However, since the polarization direction of the first image light is aligned to P polarized light by the phase plate or the polarization filter 11 as described above, the first image light passes through the polarization separation film 49a of the trapezoidal PBS 49 and enters the projection lens system 46a for generating the first virtual image light. Then, a first virtual image light having a visual recognition distance D is generated here. L After the long-distance reference virtual image light is transmitted through the incident holographic Figure 6 The light is incident on the inside of the light guide plate 4. After traveling to the right in the figure, the light is emitted from the holographic Figure 8 The light rays 8a and 8b are emitted and enter the eyes 112a and 112b of the user 100 respectively.

[0121] On the other hand, the second image light emitted from the divided display area 42b is incident on the trapezoidal PBS 49 via the phase plate or polarization filter 12 and the second virtual image light generation projection lens system 46b, and reaches the polarization separation film 49a after being reflected on its total reflection surface 49b. At this time, the second image light is aligned to the S polarized light by the phase plate or polarization filter 12, so it is reflected on the polarization separation film 49a and synthesized into the optical path of the first image light, and is also incident on the first virtual image light generation projection lens system 46a. That is, the second image light passes through both the second virtual image light generation projection lens system 46b and the first virtual image light generation projection lens system 46a through the trapezoidal prism 49, and is generated by the combination of the lens systems to have a visual recognition distance D. S In addition, the short-distance reference virtual image light is generated similarly to the long-distance reference virtual image light generated from the first image light, and is transmitted through the incident holographic image light disposed on the light guide plate 4. Figure 6 The light is incident on the inside of the light guide plate 4, travels to the right in the light guide plate 4, and then exits from the holographic Figure 8 The light rays 8a and 8b are emitted and enter the eyes 112a and 112b of the user 100 respectively.

[0122] That is, through the structure of this embodiment as described above, the user 100 can visually recognize the long-distance reference virtual image light and the short-distance reference virtual image light at the same time. Therefore, the display virtual image visual recognition distance variable control using the stereoscopic illusion phenomenon (DFD) can be achieved in the same way as in the first, second, and third embodiments.

[0123] In the structure shown in this embodiment, high-speed switching control of image light and relatively expensive optical devices such as polarization conversion elements as shown in the third embodiment are not required, so a more significant effect can be obtained in reducing the cost of HMD.

[0124] In addition, of course, the relationship between the long / short distance reference virtual image light and the P / S polarized light described in this embodiment can also be completely opposite. In this case, the first virtual image light generating projection lens system 46a has the function of generating light equivalent to the short distance reference virtual image light, and the projection lens system formed by combining the first virtual image light generating projection lens system 46a and the second virtual image light generating projection lens system 46b has the function of generating light equivalent to the long distance reference virtual image light.

[0125] Example 5

[0126] In Example 4, an example of using a stereoscopic illusion phenomenon (DFD) as a variable control of the visual recognition distance of a virtual image is shown, but an HMD using the most common binocular parallax method as a variable control of the visual recognition distance of a virtual image can also be realized by using a substantially similar optical structure. In this embodiment, an example of applying the present invention to an HMD of the binocular parallax method is described.

[0127] Fig.21 1 is a top view showing the schematic structure of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in this embodiment. Fig.21 In, with Fig. 20 The same components as shown are denoted by the same reference numerals.

[0128] exist Fig.21 In, with Fig. 20 The same virtual image light generating unit and virtual image light transmission projection unit, but the only difference is that the holographic Figure 8 A polarizing filter 13 is additionally arranged in the optical path between the right eye 112a of the user 100, and the holographic Figure 8 A polarizing filter 14 is additionally arranged in the optical path between the left eye 112b and the user 100. The polarizing filter 13, for example, has the function of transmitting P polarized light (polarization direction indicated by the arrow lines at both ends in the figure) and reflecting or absorbing S polarized light, and conversely, the polarizing filter 14 has the function of transmitting S polarized light and reflecting or absorbing P polarized light. However, of course, the combination of P / S polarized light can be completely opposite.

[0129] By setting such a structure, only the virtual image light generated by the display image of the divided display area 42a of the display 42 can be incident on the right eye 112a of the user 100 for visual recognition, and only the virtual image light generated by the display image of the divided display area 42b can be incident on the left eye 112b for visual recognition. Therefore, by displaying the right eye image and the left eye image with the image item display position offset corresponding to the binocular parallax corresponding to the desired visual recognition distance in the divided display areas 42a and 42b, respectively, it is possible to realize the variable control of the display virtual image visual recognition distance based on the binocular parallax method. In addition, since the specific process and principle of the binocular parallax method are already well-known, the detailed description is omitted.

[0130] Among them, Fig.21In the optical structure of the present embodiment shown, the optical path lengths of the first image light emitted from the divided display area 42a of the display 42 and the second image light emitted from the divided display area 42b until reaching the projection lens system 46a for generating the first virtual image light are different. (The second image light emitted from the divided display area 42b is longer by the optical path length from the total reflection surface 49b in the trapezoidal PBS 49 to the polarization separation film 49a.) Therefore, the virtual image generated by the projection lens system 46a for generating the first virtual image light also has different visual recognition distances and visual recognition image magnifications between the first virtual image light generated from the first image light and the second virtual image light generated from the second image light.

[0131] However, when the display virtual image visual recognition distance is controlled variably by binocular parallax, the visual recognition distance and image size of the virtual image visually recognized by the right eye and the virtual image visually recognized by the left eye must be completely consistent. Therefore, in this embodiment, the projection lens system 46b for generating the second virtual image light has the function of correcting the difference in the visual recognition distance and visual recognition image magnification of the second virtual image light relative to the first virtual image light caused by the difference in optical path length. By setting such an optical structure, in this embodiment, the visual recognition distance and image size of the virtual image visually recognized by the right eye and the virtual image visually recognized by the left eye can be made consistent, so the display virtual image visual recognition distance variable control based on binocular parallax can be realized.

[0132] The binocular parallax HMD using the optical structure shown in this embodiment does not need to be equipped with two separate virtual image light generating units for the left and right eyes, that is, an optical engine for image display, as in the conventional binocular parallax HMD, so it is possible to achieve significant effects in miniaturization, weight reduction, and cost reduction of the binocular parallax HMD.

[0133] However, in this embodiment, Fig. 20 The optical structure of the fourth embodiment shown is applied to an example of a binocular parallax HMD, but of course it is also applicable to Fig.17 The optical structure of the third embodiment shown can also realize a binocular parallax type HMD having the same effect by using the same filter arrangement as the polarizing filter.

[0134] Example 6

[0135] Fig. 22 1 is a top view showing the schematic structure of the virtual image light generating unit and the virtual image light transmission and projection unit of the HMD in this embodiment. Fig. 22 In, with Fig. 20 The embodiment 4 shown and Fig.21 The same components as those in the fifth embodiment are given the same reference numerals.

[0136] In this embodiment, HMD 1 also has Fig. 20 The embodiment 4 shown in the figure has the same virtual image light generating unit and virtual image light transmission and projection unit, but the only difference is that an optical device capable of controlling the light shielding property by a predetermined electrical signal, that is, a light shielding property controllable device 15 is additionally arranged in front of the light guide plate 4. As such a light shielding property controllable optical device, there are, for example, an active polarization filter using liquid crystal and dimming glass. By arranging such an optical device in front of the light guide plate 4, it is possible to arbitrarily shield the external scene light that passes through the light guide plate 4 and flies into the left and right eyes of the user 100, so that only the displayed virtual image can be clearly visually recognized in front of the user's eyes. By having such a function, the user can arbitrarily switch between an HMD that can visually recognize the actual visual recognition image and the displayed virtual image by fusion and distinguish between an HMD that can visually recognize the so-called augmented reality (AR) and an HMD that can visually recognize the so-called virtual reality (VR) by shielding the external scenery and only visually recognizing the displayed virtual image, and use them separately, which has a significant effect in expanding the versatility of the HMD.

[0137] In addition, Fig. 22 In the embodiment, the light shielding controllable device 15 is arranged to cover the entire surface of the light guide plate 4, but it can also be arranged to cover only the holographic light for emission. Figure 8 a and the openings of 8b and their vicinity.

[0138] Example 7

[0139] The embodiments described above are mainly embodiments related to HMDs, but this embodiment is an example in which the present invention is applied to a portable information terminal or a portable personal computer (PC) 16 having a transparent display.

[0140] Fig.23 2 is a schematic three-dimensional diagram showing a usage scenario of the portable information terminal in this embodiment. Fig.23 As shown, in the main body of the portable terminal or portable PC16, at least the drive control circuit and the virtual image light generating unit disclosed in any of the embodiments 1 to 6 are assembled, and as a perspective image display, it is equipped with Fig.11 , Fig.17 , Figure 20 to Figure 22 The virtual image light transmission projection unit 10 is composed of a light guide plate 4 with a hologram for virtual image light incidence and emission. By applying the present invention to a portable terminal or a portable PC, unlike a head-mounted HMD, augmented reality (AR) or virtual reality (VR) based on a portable device can be realized.

[0141] Example 8

[0142] Fig.24An embodiment in which the present invention is applied to a head-up display (HUD) for vehicle use is shown. In this example, a HUD main body 17 assembled into or mounted on a vehicle dashboard is assembled with at least the drive control circuit and virtual image light generating unit disclosed in any of the first to sixth embodiments, and is equipped with a virtual image light transmission and projection unit 10 composed of the above-mentioned light guide plate 4 with a hologram for virtual image light incidence and emission, etc. as a so-called HUD synthesizer or a see-through display for HUD.

[0143] In addition, regarding the virtual image light transmission projection unit 10, by equipping it on a part or the entire surface of the windshield set on the entire surface of the vehicle driver's seat, or by forming a part of the windshield itself with the above-mentioned light guide plate 4 with a hologram for the incidence and emission of virtual image light, the windshield itself can also have the function of a see-through display for HUD.

Claims

1. A virtual image display device having the function of projecting a predetermined virtual image into a user's field of view, characterized in that: have: a visual recognition distance detection unit for detecting an actual visual recognition distance from the user to a visual recognition object that the user is gazing at or at least a part of a plurality of objects within the field of view; A virtual image light generating unit having the function of generating a plurality of virtual image lights independent of each other and transforming the polarization direction of the virtual image lights; a virtual image light transmission and projection unit having a function of transmitting the virtual image light via substantially the same optical path, projecting the virtual image light to the left and right eyes of the user, and allowing the user to visually recognize a predetermined virtual image at a predetermined position within the user's field of view; as well as The virtual image visual recognition distance control unit has a function of variably controlling the visual recognition distance of the virtual image visually recognized by the user using the plurality of virtual image lights. The virtual image light generating unit switches different virtual image lights generated based on the visual recognition distance at a frame rate, and changes polarization directions of the different virtual image lights obtained by switching the frame rates.

2. The virtual image display device according to claim 1, characterized in that: The visual recognition distance detection unit has the following functions: detecting the line of sight inclination angle of the left and right eyes of the user based on the convergence amount of the pupil centers of the left and right eyes, and estimating the actual visual recognition distance according to the line of sight inclination angle using a predetermined relationship.

3. The virtual image display device according to claim 1, characterized in that: The visual recognition distance detection unit has the following functions: determining the object that the user is looking at from multiple objects in the user's field of view based on the inclination angle of the lines of sight of the left and right eyes detected based on the convergence amount of the centers of the pupils of the user's left and right eyes, and detecting the actual distance from the user to the object that the user is looking at using a predetermined distance measurement method.

4. The virtual image display device according to claim 1, characterized in that: The virtual image visual distance control unit has a function of matching a visual distance at which the user visually recognizes a virtual image with the actual visual distance detected by the visual distance detection unit.

5. The virtual image display device according to claim 1, characterized in that: The virtual image light generating unit and the virtual image light transmission projecting unit have the following functions: projecting the first virtual image light and the second virtual image light having different visual recognition distances so that they are accurately projected and overlapped and visually recognized when viewed from the user, The virtual image visual recognition distance control unit has the following function: using a predetermined relationship, for each displayed image item, variably controlling the brightness ratio distribution of the two virtual image lights, thereby variably controlling the visual recognition distance of the user's visual recognition of the virtual image to be any distance between the visual recognition distances of the first virtual image light and the second virtual image light.

6. The virtual image display device according to claim 1, characterized in that: The virtual image light generating unit generates a first virtual image light and a second virtual image light which are independent of each other. The virtual image light transmission projection unit has a function of projecting the first virtual image light only onto the right eye of the user and projecting the second virtual image light only onto the left eye of the user. The virtual image visual recognition distance control unit has the following function: relative in-plane shifting of the position of each display image item in the virtual image visually recognized by the user through the first virtual image light and the second virtual image light by an amount corresponding to a distance corresponding to binocular parallax corresponding to a predetermined visual recognition distance, thereby variably controlling the visual recognition distance of the virtual image.

7. The virtual image display device according to claim 5, characterized in that: The first virtual image light and the second virtual image light have at least polarization directions substantially perpendicular to each other, and the optical paths of the first virtual image light and the second virtual image light are separated or combined by a polarization beam splitter disposed in the virtual image light generating unit.

8. The virtual image display device according to claim 6, characterized in that: The first virtual image light and the second virtual image light have at least polarization directions substantially perpendicular to each other, and the optical paths of the first virtual image light and the second virtual image light are separated or combined by a polarization beam splitter disposed in the virtual image light generating unit.

9. The virtual image display device according to claim 1, characterized in that: The virtual image light transmission projection unit is composed of one or multiple stacked light guide plates, each of which has an incident hologram with a function of allowing light waves to enter the light guide plate and an emitting hologram with a function of emitting light waves traveling in the light guide plate to the outside.

10. A head mounted display device, comprising a virtual image display device having a function of projecting a predetermined virtual image into a user's field of view, and being worn on the user's head, characterized in that: The virtual image display device comprises: a visual recognition distance detection unit for detecting an actual visual recognition distance from the user to a visual recognition object that the user is gazing at or at least a part of a plurality of objects within the field of view; A virtual image light generating unit having the function of generating a plurality of virtual image lights independent of each other and transforming the polarization direction of the virtual image lights; a virtual image light transmission and projection unit having a function of transmitting the virtual image light via substantially the same optical path, projecting the virtual image light to the left and right eyes of the user, and allowing the user to visually recognize a predetermined virtual image at a predetermined position within the user's field of view; as well as The virtual image visual recognition distance control unit has a function of variably controlling the visual recognition distance of the virtual image visually recognized by the user using the plurality of virtual image lights. The virtual image light generating unit switches different virtual image lights generated based on the visual recognition distance at a frame rate, and changes polarization directions of the different virtual image lights obtained by switching the frame rates.

Images