Head-mounted type display device

Abstract

To reduce the load on a control unit required for distortion correction of a video.SOLUTION: A head-mounted type display device 100 comprises: a control unit 88 that corrects a video; a display element 11 that displays the video corrected by the control unit 88; a first optical member 22 on which video light corresponding to the video is incident; and a reflecting member 23 that reflects the video light from the first optical member 22 to project a virtual image corresponding to the video. The first optical member 22 and the reflecting member 23 correct distortion related to the virtual image in a first direction, and the control unit 88 corrects the video according to distortion related to the virtual image in a second direction intersecting the first direction.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a head-mounted display device that enables observation of virtual images.

Background Art

[0002] In a display device for virtual images, a drive circuit that operates an image display element performs distortion correction that gives a distortion in the direction opposite to the distortion caused by an optical system for observing an image displayed on the image display element, in accordance with the vertical and horizontal distortions of the input image displayed on the image display element as a processing means, and causes the image display element to display the image on which the distortion processing has been performed (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the device of Patent Document 1 described above, since the processing means performs inverse correction according to the vertical and horizontal distortions of the image, that is, the vertical and horizontal distortions, the burden on the processing means increases.

Means for Solving the Problems

[0005] A head-mounted display device according to one aspect of the present invention includes a control unit that corrects an image, a display element that displays the image corrected by the control unit, a first optical member that receives image light corresponding to the image, and a reflection member that reflects the image light from the first optical member and projects a virtual image corresponding to the image. The first optical member and the reflection member correct the distortion in the first direction related to the virtual image, and the control unit corrects the image according to the distortion in the second direction that intersects the first direction related to the virtual image.

Brief Description of the Drawings

[0006] [Figure 1] It is an external perspective view for explaining the wearing state of the head-mounted display device of the first embodiment. [Figure 2] It is a side cross-sectional view for explaining the internal structure of the head-mounted display device. [Figure 3] It is a conceptual diagram for explaining an example of the arrangement of the optical system of the head-mounted display device. [Figure 4] It is a diagram for explaining the virtual image formed on the eye side and the original image displayed on the display element. [Figure 5] It is a diagram for explaining the distortion state of the virtual image formed by correcting the vertical distortion in the optical system of the head-mounted display device. [Figure 6] It is a conceptual diagram for explaining a specific example of the distortion state of the corrected virtual image formed by the optical system of the head-mounted display device. [Figure 7] It is a diagram for explaining the relationship between the range of the incident and exit angles of the image light of the reflection member and the horizontal or vertical angular field. [Figure 8] It is a block diagram for explaining the circuit configuration of the head-mounted display device. [Figure 9] It is a diagram for explaining the coordinate system before distortion conversion and the coordinate system after distortion conversion in the control unit. [Figure 10] It is a block diagram for partially explaining the display control device and the display element of the head-mounted display device. [Figure 11] It is a diagram showing the distortion state when each parameter at the time of coordinate conversion of an image is adjusted. [Figure 12] It is a diagram for explaining the correction of chromatic aberration in the optical system of the head-mounted display device. [Figure 13] It is a diagram for explaining linear interpolation for tone adjustment of an image. [Figure 14] It is a flowchart for explaining image processing in the head-mounted display device. [Figure 15] It is a side cross-sectional view for explaining the internal structure of the head-mounted display device of the second embodiment. [Figure 16]This figure illustrates the relationship between the range of the input and output angles of image light from the reflective member of the head-mounted display device of the second embodiment and the vertical field of view. [Figure 17] This is a conceptual diagram illustrating a specific example of the distortion state of the virtual image formed by the optical system of the head-mounted display device of the second embodiment. [Modes for carrying out the invention]

[0007] [First Embodiment] The head-mounted display device of the first embodiment according to the present invention will be described below with reference to Figures 1 to 3, etc.

[0008] Figure 1 illustrates the wearing state of the head-mounted display (hereinafter also referred to as HMD) 200. The HMD 200 causes the observer or wearer US to perceive an image as a virtual image. In Figure 1, etc., X, Y, and Z are Cartesian coordinate systems. The +X direction corresponds to the lateral direction in which the eyes (EY) of the observer or wearer US wearing the HMD 200 or head-mounted display device 100 are aligned. The +Y direction corresponds to the upward direction perpendicular to the lateral direction in which the eyes (EY) of the wearer US are aligned. The +Z direction corresponds to the forward or frontal direction in which the wearer US is aligned. The ±Y directions are parallel to the vertical axis or the vertical direction.

[0009] The HMD200 comprises a first display device 100A for the right eye, a second display device 100B for the left eye, a pair of temple-shaped support devices 100C that support the display devices 100A and 100B, and a user terminal 90 which is an information terminal. The first display device 100A consists of a display drive unit 102 located at the top and an exterior member 103 that is spectacle-lens-shaped and covers the area in front of the eyes. Similarly, the second display device 100B consists of a display drive unit 102 located at the top and an exterior member 103 that is spectacle-lens-shaped and covers the area in front of the eyes. The support devices 100C are mounting members that are attached to the head of the wearer US and support the upper end of the exterior member 103 via the display drive unit 102. The first display device 100A and the second display device 100B are optically reversed left and right, and a detailed explanation of the second display device 100B is omitted.

[0010] Figure 2 is a side cross-sectional view illustrating the optical structure of the first display device 100A. The first display device 100A comprises a display element 11, an imaging optical system 20, and a display control device 88. The imaging optical system 20 comprises a projection lens 21, a prism mirror 22, and a see-through mirror 23. Of the imaging optical system 20, the projection lens 21 and the prism mirror 22 correspond to a second optical member and a first optical member, respectively, to which image light or image light ML corresponding to an image or video is incident. The see-through mirror 23 corresponds to a reflective member that reflects the image light ML emitted from the optical members toward the eye EY or pupil position PP. Furthermore, of the imaging optical system 20, the projection lens 21 and the prism mirror 22 correspond to the display drive unit 102 shown in Figure 1, and the see-through mirror 23 corresponds to the exterior member 103 shown in Figure 1. The combination of the display element 11, projection lens 21, and prism mirror 22 is called the projection optical system 12, and these are fixed inside the case 51 in a state of mutual alignment. The case 51 is a housing or support member, formed of a light-shielding material, and supports the display control device 88 that operates the display element 11. The case 51 has an aperture 51a. The aperture 51a allows the projection optical system 12 to emit image light ML outwards.

[0011] The first display device 100A corrects distortion in a first direction related to the virtual image using a prism mirror 22, which is a first optical element, and a see-through mirror 23, which is a reflective element. In this embodiment, for example, the prism mirror 22 and the see-through mirror 23 correct distortion in a first direction related to the virtual image, specifically in the vertical direction. Furthermore, the first display device 100A corrects the image or video according to distortion in a second direction that intersects with the first direction related to the virtual image, using a display control device 88. In this embodiment, for example, the display control device 88 corrects distortion in the left-right direction inherent in the virtual image. Here, the second direction or left-right direction corresponds to the scanning direction of the display element 11. Distortion in the first direction related to the virtual image is eliminated in the first display device 100A by correction by the prism mirror 22 and the see-through mirror 23. Furthermore, any remaining distortion inherent in the virtual image is caused by the prism mirror 22 and the see-through mirror 23, and is canceled out by the distortion correction in the display control device 88, making it invisible to the eye EY. The projection lens 21 also affects the distortion of the virtual image, and the above correction is performed on the entire imaging optical system 20 of the first display device 100A.

[0012] The display element 11 is a self-emissive display device. The display element 11 is, for example, an organic electroluminescent (OLED) display, and forms a color still image or moving image on a two-dimensional display surface 11a. The display element 11 is arranged along an xy plane that is slightly rotated and tilted around the X axis relative to the XY plane. The display element 11 is driven by a display control device 88, which is the control unit, to perform display operations. In the example shown in Figure 2, the display element 11 is arranged in an inverted direction, with the upper +y direction of the display element 11 facing downwards, i.e., in the -Y direction, relative to the coordinates of the imaging optical system 20. In this case, due to the inverted characteristics of the imaging optical system 20, the image displayed on the display element 11 has the same orientation of display content as the original image before correction by the display control device 88.

[0013] The display element 11 is not limited to an organic EL display, but can be replaced with a display device using inorganic EL, organic LED, LED array, laser array, quantum dot light-emitting element, etc. The display element 11 is not limited to a self-emissive image light generation device, but may be composed of an LCD or other light modulation element, and an image may be formed by illuminating the light modulation element with a light source such as a backlight. Instead of an LCD, an LCOS (Liquid crystal on silicon, LCoS is a registered trademark) or a digital micromirror device may be used as the display element 11.

[0014] The imaging optical system 20 includes a projection lens 21 which comprises a first lens 21o, a second lens 21p, and a third lens 21q. The first lens 21o has an incident surface 21a and an exit surface 21b, the second lens 21p has an incident surface 21c and an exit surface 21d, and the third lens 21q has an incident surface 21e and an exit surface 21f. The projection lens 21 receives the image light ML emitted from the display element 11 and directs it onto the prism mirror 22. The projection lens 21 focuses the image light ML emitted from the display element 11 into a state close to a parallel beam. The prism mirror 22 has an incident surface 22a corresponding to the incident part, an internal reflective surface 22b corresponding to the reflective part, and an exit surface 22c corresponding to the exit part. The prism mirror 22 emits the image light ML incident from the front in a direction inclined with respect to the direction in which the incident direction is reversed (the direction of the light source as seen from the prism mirror 22). The see-through mirror 23 has a reflective surface 23a and an outer surface 23o. The see-through mirror 23 magnifies the intermediate image formed on the light emission side of the prism mirror 22.

[0015] The imaging optical system 20 is an off-axis optical system OS due to factors such as the see-through mirror 23 being a concave mirror. In this embodiment, the projection lens 21, prism mirror 22, and see-through mirror 23 are arranged non-axisymmetrically and have non-axisymmetric optical surfaces. The fact that the imaging optical system 20 is an off-axis optical system OS means that in the optical elements 21, 22, and 23 constituting the imaging optical system 20, the optical path as a whole is bent before and after the incidence of light rays onto multiple reflective or refractive surfaces. In this imaging optical system 20, or off-axis optical system OS, the optical axis AX is bent so that the optical axis AX extends along an off-axis plane (a plane parallel to the YZ plane) corresponding to the plane of paper. In this imaging optical system 20, the optical elements 21, 22, and 23 are arranged along this off-axis plane by bending the optical axis AX within the off-axis plane parallel to the YZ plane. The optical axis AX of the imaging optical system 20 is arranged along the off-axis plane (a plane parallel to the YZ plane), which is a reference plane extending in the vertical direction, and includes optical axis portions AX1, AX2, and AX3 that are inclined toward each other before and after the reflective surface. The optical axis AX as a whole extends along the optical path of the principal ray emitted from the center of the display element 11 and passes through the eye ring ER, which corresponds to the eye point, or the center of the pupil. When viewed in a cross-section parallel to the YZ plane, the optical axis AX is arranged in a Z shape by the multiple optical axis portions AX1, AX2, and AX3. In other words, on the off-axis plane parallel to the YZ plane, the optical path P1 from the projection lens 21 to the internal reflective surface 22b, the optical path P2 from the internal reflective surface 22b to the see-through mirror 23, and the optical path P3 from the see-through mirror 23 to the pupil position PP are arranged in a Z shape, folded back in two stages. The imaging optical system 20 is arranged vertically. Correspondingly, the off-axis plane (a plane parallel to the YZ plane), which is the reference plane, extends parallel to the vertical Y direction. In this case, the optical elements 21, 22, and 23 constituting the first display device 100A are arranged with their height positions changed in the vertical direction, thereby preventing an increase in the width of the first display device 100A.

[0016] In the imaging optical system 20, the optical path P1 from the projection lens 21 to the internal reflective surface 22b extends slightly diagonally upward or nearly parallel to the Z direction towards the rear with respect to the viewpoint. That is, in the optical path P1, the optical axis portion AX1 extends in a direction close to the Z axis between the -Z direction and the +Y direction. Note that the optical axis portion AX1 may also extend in a direction close to the Z axis between the -Z direction and the -Y direction. The optical path P2 from the internal reflective surface 22b to the see-through mirror 23 extends diagonally downward towards the front. That is, in the optical path P2, the optical axis portion AX2 extends in a direction close to the midpoint between the +Z direction and the -Y direction. With respect to the horizontal plane (XZ plane), the inclination of the optical path P2 is greater than the inclination of the optical path P1. The optical path P3 from the see-through mirror 23 to the pupil position PP extends slightly diagonally upward or nearly parallel to the Z direction towards the rear. In the illustrated example, the optical axis portion AX3 is at approximately -10° in the +Z direction, with downward being negative. In other words, the output optical axis EX, which is an extension of the optical axis portion AX3, extends at an angle of approximately 10° downward with respect to the central axis HX, which is parallel to the forward +Z direction. This is because the human gaze stabilizes in a slightly downward-looking position, tilted approximately 10° below the horizontal. The central axis HX, which extends horizontally with respect to the pupil position PP, is based on the assumption that the wearer US, wearing the first display device 100A, is in an upright, relaxed position, looking straight ahead and fixating on the horizontal direction or the horizontal line.

[0017] The incident surface 21a and the exit surface 21b of the first lens 21o constituting the projection lens 21 are asymmetrical with respect to the optical axis AX in the vertical direction parallel to the YZ plane and intersecting the optical axis AX, and are symmetrical with respect to the optical axis AX in the horizontal or X direction. The incident surface 21c and the exit surface 21d of the second lens 21p constituting the projection lens 21 are asymmetrical with respect to the optical axis AX in the vertical direction parallel to the YZ plane and intersecting the optical axis AX, and are symmetrical with respect to the optical axis AX in the horizontal or X direction. The incident surface 21e and the exit surface 21f of the third lens 21q constituting the projection lens 21 are asymmetrical with respect to the optical axis AX in the vertical direction parallel to the YZ plane and intersecting the optical axis AX, and are symmetrical with respect to the optical axis AX in the horizontal or X direction. The first lens 21o, the second lens 21p, and the third lens 21q are formed of, for example, resin, but can also be made of glass. Preferably, the first and third lenses 21o and 21q are formed from resins with a moderate refractive index and a high Abbe number, such as cyclic olefin copolymers (COC) or ring-opening polymerized hydrogenated resins (COP), and the second lens 21p is formed from a resin with a high refractive index and a low Abbe number. By providing a concave lens with a high refractive index and a low Abbe number as the second lens 21p, chromatic aberration of the imaging optical system 20 can be suppressed as an achromatic lens, and optical performance can be greatly improved. The incident surface 21a and the exit surface 21b of the first lens 21o are, for example, free-form surfaces. The incident surface 21a and the exit surface 21b are not limited to free-form surfaces, but can also be aspherical surfaces. The incident surface 21c and the exit surface 21d of the second lens 21p are, for example, free-form surfaces. The incident surface 21c and the exit surface 21d are not limited to free-form surfaces, but can also be aspherical surfaces. The incident surface 21e and the exit surface 21f of the third lens 21q are, for example, free-form surfaces. The incident surface 21e and the exit surface 21f are not limited to freeform surfaces, but can also be aspherical surfaces. By making the incident surfaces 21a, 21c, 21e or the exit surfaces 21b, 21d, 21f freeform surfaces or aspherical surfaces, aberrations can be reduced. In particular, when freeform surfaces are used, it becomes easier to improve the optical performance of the eccentric system, thus making it easier to reduce aberrations in the imaging optical system 20, which is a non-coaxial off-axis optical system OS. Detailed illustrations are omitted, but anti-reflective coatings are formed on the incident surfaces 21a, 21c, 21e and the exit surfaces 21b, 21d, 21f.

[0018] The prism mirror 22 is an optical component with refractive and reflective functions that combine the functions of a mirror and a lens, and it refracts and reflects the image light ML from the projection lens 21. Specifically, the prism mirror 22 allows the image light ML to enter the interior through the incident surface 22a, totally reflects the incident image light ML in a non-front direction by the internal reflection surface 22b, and emits the incident image light ML to the outside through the exit surface 22c. The incident surface 22a and the exit surface 22c are curved optical surfaces, which contribute to improved resolution compared to the case where only a reflection surface is used or where they are flat. The incident surface 22a, the internal reflection surface 22b, and the exit surface 22c, which constitute the prism mirror 22, have asymmetry with respect to the vertical direction parallel to the YZ plane and intersecting the optical axis AX, and have symmetry with respect to the optical axis AX in the horizontal or X direction. The prism mirror 22 is formed of, for example, resin, but can also be made of glass. Furthermore, it is preferable that the prism mirror 22 be formed from a resin with a moderate refractive index and a high Abbe number, such as a cyclic olefin copolymer (COC) or a ring-opening polymerized hydrogenated resin (COP). The refractive index of the body of the prism mirror 22 is set to a value that achieves total internal reflection on the inner surface, taking into consideration the reflection angle of the image light ML. The optical surfaces of the prism mirror 22, that is, the incident surface 22a, the internal reflection surface 22b, and the exit surface 22c, are, for example, free-form surfaces. The incident surface 22a, the internal reflection surface 22b, and the exit surface 22c are not limited to free-form surfaces, but can also be aspherical surfaces. In the prism mirror 22, aberrations can be reduced by making the optical surfaces 22a, 22b, and 22c free-form surfaces or aspherical surfaces, and in particular, when free-form surfaces are used, it becomes easier to improve the optical performance of the eccentric system. The internal reflection surface 22b is not limited to one that reflects the image light ML by total internal reflection, but can also be a reflective surface made of a metal film or a dielectric multilayer film. In this case, a reflective film consisting of a single layer or multilayer made of a metal such as Al or Ag is deposited on the inner reflective surface 22b by vapor deposition or other means, or a sheet-like reflective film made of metal is attached. Although detailed illustrations are omitted, an anti-reflective film is formed on the incident surface 22a and the exit surface 22c.

[0019] The see-through mirror 23 is a curved, plate-shaped reflective optical element that functions as a concave surface mirror and reflects image light ML from the prism mirror 22. In other words, the see-through mirror 23 reflects image light ML from the prism mirror 22, which is positioned in the emission region of the projection optical system 12, toward the pupil position PP. The see-through mirror 23 covers the pupil position PP where the eye EY or pupil is located and has a concave shape toward the pupil position PP and a convex shape toward the outside world. The see-through mirror 23 is a concave transparent mirror that covers the entire effective area of ​​the screen within the field of view. The see-through mirror 23 is a collimator with a focusing function and focuses the principal rays of image light ML emitted from each point of the display surface 11a, which have spread out due to imaging near the emission side of the prism mirror 22 of the projection optical system 12, toward the pupil position PP. The see-through mirror 23 is a mirror plate having a structure in which a transparent mirror film 23c is formed on the surface or back surface of a plate-like body 23b. The reflective surface 23a of the see-through mirror 23 is asymmetrical with respect to the optical axis AX in the vertical direction, which is parallel to the YZ plane and intersects the optical axis AX, and is symmetrical with respect to the optical axis AX in the horizontal or X direction. The reflective surface 23a of the see-through mirror 23 is, for example, a free-form surface. The reflective surface 23a is not limited to a free-form surface, but can also be an aspherical surface. By making the see-through mirror 23 a free-form surface or an aspherical surface, aberrations can be reduced, and in particular, when a free-form surface is used, it becomes easier to reduce aberrations in the imaging optical system 20, which is an off-axis optical system OS or a non-coaxial optical system.

[0020] The see-through mirror 23 is a transmissive reflective element that transmits some light during reflection, and the reflective surface 23a or mirror film 23c of the see-through mirror 23 is formed of a semi-transparent reflective layer. As a result, ambient light OL passes through the see-through mirror 23, enabling see-through viewing of the outside world and allowing a virtual image to be superimposed on the ambient image. In this case, if the plate-like body 23b supporting the mirror film 23c is thin to a few millimeters or less, the change in magnification of the ambient image can be kept small. The reflectivity of the mirror film 23c to the image light ML and ambient light OL is set to 10% to 50% within the expected incident angle range of the image light ML, from the viewpoint of ensuring the brightness of the image light ML and facilitating observation of the ambient image through see-through. The plate-like body 23b, which is the base material of the see-through mirror 23, is formed of resin, for example, but can also be made of glass. The plate-like body 23b is formed of the same material as the support plate 61 that supports it from the surroundings and has the same thickness as the support plate 61. The mirror film 23c is formed, for example, from a dielectric multilayer film consisting of multiple dielectric layers with adjusted film thicknesses. The mirror film 23c may also be a single-layer or multilayer film of a metal such as Al or Ag with adjusted film thickness. The mirror film 23c can be formed by lamination, but it can also be formed by attaching a sheet-like reflective film. An anti-reflective film is formed on the outer surface 23o of the plate-like body 23b.

[0021] To explain the optical path, the image light ML from the display element 11 enters the projection lens 21 and is emitted from the projection lens 21 in a nearly collimated state. The image light ML that has passed through the projection lens 21 enters the prism mirror 22, passes through the incident surface 22a while being refracted, is reflected by the internal reflective surface 22b with a high reflectivity of nearly 100%, and is refracted again by the exit surface 22c. The image light ML from the prism mirror 22 enters the see-through mirror 23 and is reflected by the reflective surface 23a with a reflectivity of about 50% or less. The image light ML reflected by the see-through mirror 23 enters the pupil position PP where the wearer's eye EY or pupil is located. External light OL that has passed through the see-through mirror 23 and the surrounding support plate 61 also enters the pupil position PP. In other words, the wearer US wearing the first display device 100A can observe a virtual image created by the image light ML superimposed on the external image.

[0022] Figure 3 is a conceptual diagram illustrating an example of the arrangement of the imaging optical system 20 of the first display device 100A. In Figure 3, area AR1 shows an example of arrangement focusing on the reflection angle of the prism mirror 22 and see-through mirror 23 along the optical axis AX, while area AR2 shows an example of arrangement focusing on the inclination of the prism mirror 22 and see-through mirror 23 with respect to the horizontal axis or the Z axis.

[0023] In the region AR1 of Figure 3, angle θ1 is the angle between the image light ML incident from the prism mirror 22 to the see-through mirror 23 and the image light ML reflected by the see-through mirror 23 to the pupil position PP. Angle θ2 is the angle between the image light ML incident on the inner reflective surface 22b, which is the reflective part of the prism mirror 22, and the image light ML reflected by the inner reflective surface 22b. Angle θ3 is the angle between the horizontal direction or the Z direction and the image light ML reflected by the see-through mirror 23 to the pupil position PP. Angle θ4 is the angle between the horizontal direction or the Z direction and the image light ML incident on the inner reflective surface 22b of the prism mirror 22. Angle θ4 is approximately the inclination of the image light ML emitted to the display element 11 with respect to the horizontal direction. The symbol L1 is the optical path length of the image light ML incident from the inner reflective surface 22b of the prism mirror 22 to the see-through mirror 23. The symbol L2 is the optical path length of the image light ML reflected by the see-through mirror 23 to the pupil position PP. The symbol L3 is the optical path length of the image light ML incident on the inner reflective surface 22b of the prism mirror 22 from the display element 11. Specifically, the optical path length L3 is the distance from the display surface 11a of the display element 11 to the inner reflective surface 22b of the prism mirror 22. Approximately, distances can be used for the optical path lengths L1 to L3.

[0024] In the region AR1 shown in the figure, the angle θ4 is expressed by the following equation. θ4 = -θ3 - θ1 + θ2 Here, angle θ4 is defined as negative on the lower side and positive on the upper side with respect to the horizontal axis or horizontal direction. Angle θ3 is defined as positive on the lower side with respect to the horizontal axis. Angle θ1 and θ2 are expressed as absolute values.

[0025] The distance Ln from the pupil position PP to the prism mirror 22 in the horizontal direction satisfies the following equation. Ln = L2cosθ3 - L1cos(θ3 + θ1) Here, distance Ln is the horizontal distance from pupil position PP to prism mirror 22, and is also called eye relief length. Angle θ3 determines the position of the image center or video center with respect to the horizontal line of sight (horizontal direction of the line of sight), and when viewing a video, it is preferable that it be below the horizontal line of sight. Specifically, when viewing a video, angle θ3 should be 5° to 20° below the horizontal axis. The value θ3 + θ1 in the formula represents the angle of L1 with respect to the horizontal axis.

[0026] Each variable satisfies the following ranges. 10°≦θ1≦70° 30°≦θ²≦90° -10°≦θ4≦+10° 15mm ≤ Ln ≤ 30mm 0.5≦L1cos(θ3+θ1) / L3cosθ4≦1.5

[0027] By positioning the see-through mirror 23 such that the angle θ1 falls within the above range, the arrangement angle of the see-through mirror 23 becomes relatively large, which suppresses eccentric aberration and improves optical performance. Furthermore, since the distance between the see-through mirror 23 and the prism mirror 22 can be made relatively small, the size of the imaging optical system 20 can be reduced in the thickness direction, i.e., in the lateral cross-sectional direction. Specifically, by keeping the angle θ1 above the lower limit, it is possible to prevent the size of the imaging optical system 20 from increasing on the upper or brow side. Also, by keeping the angle θ1 below the upper limit, the layout of the imaging optical system 20 can be made appropriate.

[0028] By positioning the prism mirror 22 such that the angle θ2 falls within the above range, the arrangement angle of the prism mirror 22 becomes relatively large, allowing for a reduction in the size of the imaging optical system 20 in the height direction, i.e., the vertical or longitudinal direction. Specifically, by setting the angle θ2 to be above the lower limit, the layout of the imaging optical system 20 can be made appropriate, allowing for miniaturization and suppression of eccentricity. Furthermore, by setting the angle θ2 to be below the upper limit, the layout of the imaging optical system 20 can be made appropriate, allowing for miniaturization.

[0029] By positioning the display element 11 such that the angle θ4 falls within the above range, eccentricity can be suppressed, thereby improving optical performance. Specifically, when the angle θ4 is above the lower limit, eccentricity can be suppressed while appropriately arranging the imaging optical system 20. When the angle θ4 is below the upper limit, the imaging optical system 20 can be miniaturized while suppressing eccentricity. In particular, tilting the angle θ4 downwards can further suppress eccentricity. Furthermore, when the angle θ4 is close to zero, the imaging optical system 20 can be made even smaller, and this also simplifies the overall design and contributes to a sleeker appearance when worn by the wearer (US).

[0030] By ensuring that the distance Ln from the pupil position PP to the prism mirror 22 is above the lower limit, it is possible to prevent the virtual image seen by the wearer US from being obscured even if they are wearing separate glasses. Furthermore, by ensuring that the distance Ln is below the upper limit, the imaging optical system 20 can be miniaturized.

[0031] As long as the value L1cos(θ3+θ1) / L3cosθ4 is greater than or equal to the lower limit, the layout of the imaging optical system 20 can be made appropriate even if the projection lens 21 has multiple lenses. The upper limit of the value L1cos(θ3+θ1) / L3cosθ4 is the maximum size in the thickness direction of the imaging optical system 20 when the projection lens 21 has approximately one lens.

[0032] In Figure 3, in region AR2, the arrangement angle of the see-through mirror 23 is θ1'. The arrangement angle of the reflective part of the prism mirror 22 is θ2'. The other angles and optical path lengths are the same as in region AR1.

[0033] In the region AR2 shown in the figure, the angle θ4 is expressed by the following equation. θ4 = -θ3 - 2θ1' + 2θ2' Here, angle θ4 is defined as negative on the lower side and positive on the upper side with respect to the horizontal axis or horizontal direction. Angles θ3 are defined as positive on the lower side with respect to the horizontal axis. Angles θ1' and θ2' are defined as positive on the upper side with respect to the horizontal axis.

[0034] The distance Ln from the pupil position PP to the prism mirror 22 in the horizontal direction is expressed by the following equation. Ln=L2cosθ3-L1cos(180-θ3-2θ1') The value 180-θ3-2θ1' in the formula represents the angle of L1 with respect to the horizontal axis.

[0035] Each variable satisfies the following ranges. 45°≦θ1'≦75° -10°≦θ4≦+10° 15mm ≤ Ln ≤ 30mm 0.5≦L1cos(180-θ3-2θ1') / L3cosθ4≦1.5

[0036] (Example 1) The following shows specific numerical examples for each variable. θ1 = 28° (θ1' = 63°) θ2 = 52° (θ2' = 67°) θ3 = 12° θ4 = -4° L1 = 35 mm L2 = 52 mm L3 = 26mm Ln=26mm L1cos(θ3+θ1) / L3cosθ4=1.01

[0037] In the above, the first display device 100A controls the position of the intermediate image by making the reflection angle θ1 or arrangement angle θ1' of the see-through mirror 23 relatively large, thereby increasing the tilt angle of the see-through mirror 23 compared to conventional devices. In other words, the shape of the intermediate image can be controlled by controlling the principal ray angle of the vertical angle-of-view rays (Y-direction angle-of-view rays) directed to the see-through mirror 23. Furthermore, by setting the variables of the imaging optical system 20 of the first display device 100A within the above range, distortion occurring in the vertical direction or up-down direction corresponding to the first direction is suppressed. In conventional devices, for example, the arrangement angle θ1' of the see-through mirror 23 was tilted at about 30° to 40° with respect to the horizontal axis. In this case, the image light ML is reflected more upwards. In conventional devices, in order to eliminate the eccentric aberration generated in the see-through mirror 23, the magnification of the intermediate image is adjusted by increasing the distance between the see-through mirror 23 and the reflective surface of the prism mirror 22 (increasing the distance between the intermediate image and the reflective surface), thereby suppressing the curvature of the reflective surface and reducing aberrations generated on the reflective surface. Therefore, there is a problem in that the size of the imaging optical system 20 increases towards the eyebrow side of the wearer's eye sockets. In the first display device 100A of this embodiment, the distance from the center of the pupil to the maximum ray height of the imaging optical system 20, which is the size on the eyebrow side, is about 20 mm, which is about 10 mm smaller than conventional devices. In addition, in conventional devices, the angle θ4, which is the angle of incidence to the prism mirror 22, was about 30°, so the housing structure of the optical system was complex. However, in the first display device 100A of this embodiment, the angle θ4 is within ±10°, preferably about 5°, so the housing case 51 is not very complex, and the ease of assembly and robustness can be improved.

[0038] As described above, in this embodiment, by raising the see-through mirror 23 at an angle θ1 or arrangement angle θ1' within a predetermined range, vertical distortion is suppressed in the first display device 100A, and trapezoidal distortion mainly consisting of horizontal distortion occurs.

[0039] Figure 4 is a hypothetical diagram illustrating the virtual image AA formed on the eye EY side and the original image BB displayed on the display element 11 when no correction is made by the display control device 88. In Figure 4, the solid line shows the virtual image AA corrected in the vertical direction by the imaging optical system 20 of the first display device 100A, and the dashed line shows the display area SB of an ideal rectangular image corresponding to the original image BB.

[0040] Before distortion correction is performed by the display control device 88 (described later), the virtual image AA formed by the first display device 100A has a length D1 in the left-right direction corresponding to the second direction in one of the vertical directions corresponding to the first direction of the image AA1 corresponding to the virtual image AA, which is shorter than the length D2 in the left-right direction in the other vertical direction of the image AA1. For example, if the shape of the image BB before correction is a rectangle, then due to a biased correction with respect to direction during projection by the imaging optical system 20 of the first display device 100A, the corrected image AA1 becomes trapezoidal. In the example in Figure 4, the upper side or the +Y direction (corresponding to the -y direction) is one side, and the lower side or the -Y direction (corresponding to the +y direction) is the other side. The display control device 88 performs the opposite distortion correction corresponding to this trapezoidal image AA1, making the final corrected virtual image the same rectangle as the original image BB.

[0041] Figure 5 illustrates the distortion state of the virtual image AA formed by correcting vertical distortion in the imaging optical system 20 of the first display device 100A. As shown in Figure 5, the virtual image AA includes a corrected first corrected region SP1 obtained by correcting the first region AP1 of the original image BB, and a corrected second corrected region SP2 obtained by correcting the second region AP2 which is aligned horizontally with the first region AP1 of the uncorrected image. The first and second regions AP1 and AP2 are unit regions with a certain extent.

[0042] As shown in the figure regions BR1 and BR2 of Figure 5, the ranges of the first region AP1 and the second region AP2 can be set as appropriate, and the ranges of the first correction region SP1 and the second correction region SP2 are determined corresponding to the first region AP1 and the second region AP2. The first region AP1 is a reference region set once for each vertical column or horizontal row. In the example of figure region BR1 in Figure 5, the first region AP1 and the second region AP2 correspond to one division when the entire pixel of image BB corresponding to the virtual image AA is divided vertically and horizontally into 10 sections. In this case, if the original image BB has 1920 horizontal pixels and 1080 vertical pixels, one section will be 192 × 108 pixels. In the example of figure region BR2 in Figure 5, the first region AP1 and the second region AP2 correspond to one division when the entire pixel of image BB corresponding to the virtual image AA is divided vertically into 10 sections and horizontally into 2 sections. In this case, if the original image BB has a width of 1920 pixels and a height of 1080 pixels, then one section will be 960 x 108 pixels.

[0043] The number of images in the left-right direction of the corrected first correction region SP1 is within ±10% of the number of pixels in the left-right direction of the corrected second correction region SP2. In this embodiment, with respect to the trapezoidal shape of the virtual image AA, the distortion interval of the lateral second correction region SP2 changes substantially constant within a range of ±20 pixels relative to the average value of the lateral lengths of the first and second correction regions SP1 and SP2. Furthermore, the lateral distortion changes substantially constant within a range of ±20 pixels in the vertical direction of the virtual image AA. The distortion inherent in the imaging optical system 20 of the first display device 100A is corrected by the display control device 88, which will be described later, to cancel out these distortions.

[0044] Figure 6 is a conceptual diagram illustrating a specific example of the distortion state of the corrected virtual image AA formed by the imaging optical system 20 of the first display device 100A. As shown in the figure, when the entire pixel of the original image BB corresponding to the virtual image AA is divided vertically and horizontally into 10 sections, the distortion of the virtual image AA in the horizontal direction is such that the interval A is in the first and second correction regions SP1 and SP2. i,j This occurs at [location]. The symbol i indicates the column number, and the symbol j indicates the row number. Interval A i,jmeans the dimension after distortion.

[0045] Interval A i,j is arranged at equal intervals so as to satisfy the following formula. Assuming the upper part of the screen is the 5th row, the first and second correction regions SP1 and SP2 are at an interval of A -5,+5 ~A +5,+5 and are arranged in a row. At this time, assuming the length of the 5th row at the upper part of the screen is A’5, it should satisfy the following formula. A’5 / 10 - 20pix ≤ A i,5 ≤ A’5 / 10 + 20pix i = -5~+5 Here, the value 20pix means 20 pixels.

[0046] In the above example, the 5th row was described, but for the entire distorted shape, it should satisfy the following formula. A’ j -20pix ≤ A i,j ≤ A’ j +20pix i = -5~+5

[0047] As shown in FIG. 6, for all the pixels of the original image BB corresponding to the virtual image AA, when the vertical and horizontal directions are divided into 10 equal parts, let the coordinates of the virtual image AA during distortion be B i,j . At this time, when paying attention to the distorted coordinates of the leftmost -5th column, B -5,+5 -B -5,-5 = B’ -5 . That is, the value B’ -5 is the horizontal length between the coordinate B -5,+5 and the coordinate B -5,-5 , that is, the dimensional difference due to distortion. At this time, the horizontal length B -5,j -B -5,j-1 between the coordinates of the adjacent rows in the -5th column satisfies the following formula. B’ -5 / 10 - 20pix ≤ B -5,j -B -5,j-1 ≤ B’ -5 +20pix j = -5~+5

[0048] In the above example, the 5th column was described, but for the entire distorted shape, it should satisfy the following formula. B' i / 10-20 pixels ≤ B i,j -B i,j-1 ≦B' i +20 pixels j = -5 to +5

[0049] Figure 7 illustrates the relationship between the range of input and output angles of the image light ML of the see-through mirror 23 and the horizontal or vertical field of view. Area CR1 in Figure 7 is the screen position where the principal ray angle of the imaging optical system 20 of the first display device 100A was investigated, and corresponds to the field of view of the screen or virtual image. Area CR2 in Figure 7 shows the difference between the horizontal incident and output principal ray angles of the see-through mirror 23, plotting the difference in the input and output angles of the see-through mirror 23 with respect to the horizontal field of view. Area CR3 in Figure 7 shows the difference between the vertical incident and output principal ray angles of the see-through mirror 23, plotting the difference in the input and output angles of the see-through mirror 23 with respect to the vertical field of view. In areas CR2 and CR3 of Figure 7, solid lines represent plots of the apparatus of this embodiment, and dashed lines represent plots of conventional apparatuses. Regarding the approximation lines in the figures, the coefficient of determination R is used as an index of the linearity of the plots. 2 In this case, the coefficient of determination R 2 The closer R approaches 1, the higher the linearity and the better the fitting. (Coefficient of determination R) 2 By making the value approach 1, the difference between the input and output angles of the see-through mirror 23 changes constantly, and the vertical spacing changes constantly. This makes it possible to adjust the shape of the intermediate image to eliminate vertical distortion.

[0050] In this embodiment, the intermediate image shape is controlled by controlling the principal ray angle of the vertical field-of-view rays and the principal ray of the horizontal field-of-view rays to the see-through mirror 23. As shown in areas CR2 and CR3 of Figure 7, the horizontal spacing of the distortion changes constantly, suppressing horizontal displacement relative to the vertical direction. As shown in areas CR2 and CR3 of Figure 7, in conventional devices, the difference between the input and output angles of the see-through mirror 23 was nonlinear, whereas in the device of this embodiment, it is linear. This makes it possible to prevent vertical distortion from occurring in the virtual image AA in the first display device 100A of this embodiment. By sufficiently suppressing vertical distortion, it becomes unnecessary to allocate frame buffer memory when distortion correction is performed by the display control device 88, which will be described later. Therefore, the computational load on the display control device 88 can be reduced when correcting distortion generated in the imaging optical system 20. In this way, by greatly reducing the burden of circuit correction in the display control device 88, power consumption can be made even lower than in conventional devices.

[0051] In conventional devices, the see-through mirror 23 is positioned at an angle of approximately 30° to 40° with respect to the horizontal axis, causing vertical distortion. Therefore, when correcting distortion in the circuit, it is necessary to secure a predetermined frame buffer in the correction process of the display control device 88, which will be described later. This presents problems such as requiring a large amount of power for memory retention (non-volatile memory) and necessitating the development of a dedicated IC that can maintain a frame buffer of a predetermined pixel size while simultaneously correcting distortion.

[0052] In this embodiment, by increasing the tilt angle of the see-through mirror 23, that is, by increasing the arrangement angle of the see-through mirror 23, eccentric aberration can be suppressed, thereby improving optical performance. Furthermore, by controlling the reflection angle of light rays at the see-through mirror 23 that forms the intermediate image, vertical distortion that occurred in conventional devices can be improved. As a result, there is no need to maintain a frame buffer, so the display control device 88 can handle distortion correction with an inexpensive general-purpose FPGA or other integrated circuit, and power consumption can be reduced.

[0053] Referring to Figure 8, the circuit system 70 of the HMD200, or head-mounted display device 100, will be described. The HMD200's circuit system 70 includes a display control device 88, a pair of display elements 11, and a user terminal circuit 91. One of the display elements 11 is incorporated into the first display device 100A, and the other display element 11 is incorporated into the second display device 100B. The display control device 88 functions as a control unit. In the illustrated example, the display control device 88 is shown as being incorporated into the first display device 100A, but it may be independent of the first display device 100A and the second display device 100B. Note that a combination of either the first display device 100A or the second display device 100B and the display control device 88 is also called a head-mounted display device 100 and displays a virtual image for one eye.

[0054] The display control device 88 includes an arithmetic processing unit 81a, a storage device 81m, and a data communication interface 81c.

[0055] The storage device 81m stores programs that cause the first display device 100A and the second display device 100B to perform display operations. The storage device 81m also stores images acquired from the user terminal 90, which is an information terminal, and images generated by the arithmetic processing unit 81a. The storage device 81m also has a frame memory 83, which stores image data to be output to the display element 11 corresponding to the image data generated by the arithmetic processing unit 81a. The image data output from the display control device 88 to the pair of display elements 11 may be the same or different. The processing on one of the first display devices 100A will be described below.

[0056] Furthermore, the storage device 81m has a non-volatile memory 87, which stores various data such as parameters used in the image correction calculations described later.

[0057] The display control device 88 causes the display element 11 to perform display operations. The display element 11 includes an accessory circuit 85 that incorporates a scanning driver and a data driver around the display surface 11a. When displaying each frame image, the display control device 88 outputs data signals corresponding to the image data stored in the frame memory 83, along with timing signals, etc., to the accessory circuit 85 in units of scan lines. The accessory circuit 85 rewrites the display state of the display surface 11a according to the data signals etc. input from the frame memory 83. The display control device 88 outputs one line at a time for the image data and displays it on the display surface 11a of the display element 11 in the scanning direction corresponding to the x direction in Figure 2. The x direction in Figure 2 corresponds to the horizontal or left-right direction of the display surface 11a. The image data stored in the frame memory 83 increases by one line at a time for each frame of image, and after all the lines of the image have been displayed on the display element 11, it is reset, and the image data for the next frame is stored in the same way.

[0058] The display control device 88 receives display data corresponding to image data from the user terminal circuit 91 via the data communication interface 81c. The display control device 88 applies distortion correction processing to the display data or image data acquired from the user terminal circuit 91. The display control device 88 outputs the processed display data, which is the image data stored in the frame memory 83, to the display element 11 via the data communication interface 81c.

[0059] The imaging optical system 20 shown in Figure 2 is an off-axis optical system OS, and as previously described, the occurrence of vertical distortion is suppressed, resulting in trapezoidal distortion, or distortion. To compensate for this distortion, it is necessary to form a distortion-corrected non-rectangular contour image on the display surface 11a of the display element 11.

[0060] Figure 9 illustrates the coordinate system before and after distortion transformation in the display control device 88. Figure region DR1 in Figure 9 represents the coordinate system in the normal state before distortion transformation and shows the initial image IM0. Figure region DR2 in Figure 9 represents the coordinate system in the display state on the display element 11 after distortion transformation and shows the corrected image IM1. The initial image IM0 is the original display image before distortion correction in the display control device 88. The corrected image IM1 is the display image after distortion correction in the display control device 88. The corrected image IM1 is an image in which the distortion generated in the imaging optical system 20 has been corrected so that the virtual image formed on the eye side EY becomes rectangular.

[0061] The display control device 88 performs various image processing operations, including calculations to correct distortion or curvature, so that the initial rectangular contour image IM0 corresponding to the input signal from the user terminal circuit 91 shown in Figure 9 becomes a corrected non-rectangular contour image IM1 to be displayed on the display surface 11a.

[0062] The processing unit 81a acquires video or images from the user terminal 90 and stores them in the storage device 81m. The images are display images shown on the head-mounted display device 100, specifically the initial image IM0 shown in Figure 9.

[0063] As shown in Figure 8, the arithmetic processing unit 81a has a correction unit 86 for distortion correction. The correction unit 86 is a correction circuit that generates a distorted image that cancels out the trapezoidal screen distortion that occurs in the imaging optical system 20 of the first display device 100A and displays it on the display element 11. The correction unit 86 generates a distorted image with simple calculation processing and allows adjustment of the distortion state by adjusting parameters. The correction unit 86 is implemented, for example, on a general-purpose FPGA together with other circuits that constitute the display control device 88.

[0064] The display control device 88 corrects the image so that the length D1 in the left-right direction of one side of the image in the vertical direction, as shown in the diagram area DR2 of Figure 9, is shorter than the length D2 in the left-right direction of the other side of the image in the vertical direction. As a result, for example, if the shape of the image before correction is rectangular, the corrected image displayed on the display element 11 will have an inverted trapezoidal shape that is wider at the top. In the example of Figure 9, the top side or the +y direction side is the other side, and the bottom side or the -y direction side is the one side. Since the virtual image before correction displayed by the imaging optical system 20 of the first display device 100A of this embodiment has a trapezoidal shape, the correction unit 86 performs the inverse distortion correction shown in the diagram area DR2 of Figure 9, so that the corrected virtual image becomes rectangular.

[0065] In the image correction performed by the display control device 88, similar to Figure 5, the corrected image includes a corrected first correction region SP1' (corresponding to the first correction region SP1 in Figure 5) obtained by correcting the first region AP1 of the image before correction, and a corrected second correction region SP2' (corresponding to the second correction region SP2 in Figure 5) obtained by correcting the second region AP2, which is aligned horizontally with the first region AP1 of the image before correction. The number of pixels in the horizontal direction of the corrected first correction region SP1' is within ±10% of the number of pixels in the horizontal direction of the corrected second correction region SP2'. As will be described in detail later, the relay system of the imaging optical system 20 inverts the image vertically and horizontally after passing through the optical system. However, whether or not the image displayed on the display screen of the display element 11 is rotated depends on the arrangement direction of the display element 11 and can be changed as appropriate according to the arrangement direction.

[0066] In the arithmetic processing unit 81a, the correction unit 86 includes a coordinate transformation unit 8a and a grayscale transformation unit 8b.

[0067] The coordinate transformation unit 8a performs a correction process that distorts the initial image IM0 as an image processing step, and stores the distorted corrected image IM1 (hereinafter also referred to as the distorted initial image IM0) shown in Figure 9 in the storage device 81m. The coordinate transformation unit 8a corrects the initial image IM0 by distorting it so as to invert the distortion caused by the imaging optical system 20, etc., shown in Figure 2. In other words, the coordinate transformation unit 8a corrects the initial image IM0 so as to cancel out the distortion of the imaging optical system 20.

[0068] The tone conversion unit 8b converts the tone or brightness of each pixel corresponding to the coordinates corrected for distortion by the coordinate transformation unit 8a. In other words, the tone conversion unit 8b adjusts the tone of the distortion-corrected image IM1 by interpolation.

[0069] The following describes the processing in the coordinate transformation unit 8a. The xy coordinate system, in which the center of the video or image before distortion transformation is the origin and the size of one pixel is 1, is transformed into the UV coordinate system after distortion transformation as shown in the following equation.

[0070] When y is the vertical coordinate of the initial image IM0 before correction, x is the horizontal coordinate of the initial image IM0 before correction, U is the horizontal coordinate of the corrected image IM1 after correction, a is the coefficient corresponding to the horizontal length of the corrected image IM1 after correction, and b is the coefficient corresponding to the angle between the vertical side and the horizontal side of the corrected image IM1 after correction, the coordinate transformation unit 8a of the display control device 88 corrects the initial image IM0 so that it satisfies the following equation. U = ax + bxy The coefficient 'a' corresponds to the relative ratio of the original lengths in the left-right direction. The coefficient 'b' corresponds to the slope of opposite sides of the trapezoid that are not parallel to each other, i.e., the slope of the legs or meridians of the trapezoid.

[0071] Furthermore, if V is the vertical coordinate of the corrected image IM1, then the following equation is satisfied. V=y

[0072] Furthermore, if c is a coefficient corresponding to the distortion between one tilt in the left-right direction and the other tilt in the left-right direction of the corrected image IM1 after correction, the coordinate transformation unit 8a of the display control device 88 corrects the image so that it satisfies the following equation. U = ax + bxy + cy The coefficient c corresponds to the imbalance in the slopes of opposite sides of the trapezoid that are not parallel to each other, that is, the imbalance in the slopes of the legs or meridians of the trapezoid.

[0073] In the above, coefficients a, b, and c are parameters that can be rewritten.

[0074] Figure 10 is a block diagram partially illustrating the display control device 88 and display element 11 of the first display device 100A. In the figure, the coordinate transformation unit 8a of the correction unit 86 receives the image signal from the first display device 100A for the right eye and the image signal from the second display device 100B for the left eye, and performs distortion transformation processing on each image signal.

[0075] As shown in Figure 10, the coefficients a, b, and c are recorded in a non-volatile memory 87 in the storage device 81m that can be rewritten from the outside. The correction unit 86 obtains the coefficient values ​​recorded in the non-volatile memory 87 and performs distortion calculations. Specifically, the coordinate transformation unit 8a of the correction unit 86 receives input signals J1 and J2 for the image signal of the first display device 100A and the image signal of the second display device 100B, respectively. Calculation processing is performed for each input signal J1 and J2 using different coefficients a1, b1, c1 and a2, b2, c2, and two output signals K1 and K2 are output with different distortion states. Note that the correction unit 86 may output two output signals with different distortion states using only one input system instead of two.

[0076] The non-volatile memory 87 stores the values ​​of coefficients a1, b1, c1 for the right eye and a2, b2, c2 for the left eye. The correction unit 86 retrieves these coefficients during distortion processing and performs calculations. This allows the right and left eye images to be output with different distortion states. The conversion coefficients recorded in the non-volatile memory 87 can be accessed and rewritten from an external source using a serial communication method such as I2C. The correction unit 86 also performs interface conversion to convert the image signal to one suitable for the display element 11. An example of a correction unit 86 used in practice is an FPGA, specifically the Lattice LIFCL-17-8MG121C. This FPGA is a high-speed transmission chip with a small footprint of 6mm x 6mm and low power consumption. However, its logical operation capabilities are limited, and it can only perform simple image processing. Therefore, the distortion conversion process needs to be implemented using a simple calculation formula as described above.

[0077] As shown in Figure 9, the distortion transformation is, for example, an inverse transformation. In the example in Figure 9, the upper side of the display element 11 is positioned downwards with respect to the coordinates of the imaging optical system 20, i.e., in the -Y direction, in an inverse configuration. The image displayed on the display screen of the display element 11 is displayed with only trapezoidal correction applied, and appears to be displayed as a trapezoid with a shorter top side with respect to the coordinates of the imaging optical system 20. In the case of the inverse configuration, the image that has passed through the imaging optical system 20 is inverted vertically and horizontally by the relay system, and the virtual image seen by the wearer US is trapezoidally corrected, while the orientation of the displayed content matches that of the original image. Note that the distortion transformation may also be a forward transformation. In this case, the upper side of the display element 11 is positioned upwards with respect to the coordinates of the imaging optical system 20, i.e., in the +Y direction, in a forward configuration, and the image displayed on the display screen of the display element 11 is displayed with trapezoidal correction applied and inverted vertically and horizontally with respect to the original image. The image, after passing through the imaging optical system 20, is inverted vertically and horizontally by the relay optical system. The virtual image seen by the wearer US is trapezoidally corrected, and the orientation of the displayed content is reversed compared to the original image.

[0078] Table 1 shows specific examples of parameters, including representative values ​​and fixed-point bit counts used during calculations. The conversion formula in this embodiment minimizes the number of multiplications to four, allowing for coordinate transformation with a very simple formula. This enables distortion transformation processing with a smaller, lower-power, and lower-cost correction circuit.

[0079] [Table 1] TIFF2023151368000002.tif90167<Representative Value> a = 0.84788 b = -0.00028791 c = -0.0055419 <Numerical Range> 0.7 ≤ a ≤ 1 -0.0005≦b≦0.0005 -0.2 ≤ c ≤ 0.2 <Functional request> a) The coefficients a, b, and c in the coordinate transformation calculation formula are common to RGB and can be set individually for the left and right sides. b) Coefficients a, b, and c can be directly adjusted using commands from the correction unit 86. They are used for parameter adjustment after the imaging optical system 20 is assembled, regardless of whether there is data in the non-volatile memory 87. c) The coefficients a, b, and c are parameters stored in the non-volatile memory 87. The data is stored in the non-volatile memory 87 and is used after parameter adjustment for each display element 11. d) The coefficients a, b, and c used in the conversion can be checked from the user terminal 90 via serial communication. e) You can choose between displaying the distortion-converted version and displaying the original version.

[0080] Figure 11 shows the distortion state when each parameter during image coordinate transformation is adjusted in areas ER1 to ER4. Depending on the variation in the lenses of the imaging optical system 20, the distortion state changes, so by adjusting coefficients a, b, and c, the change in distortion due to variations in the imaging optical system 20 can be corrected.

[0081] Furthermore, when the coordinate transformation unit 8a transforms a color image, it transforms the image so that the image containing each wavelength is displayed on the display element 11 as follows.

[0082] Specifically, as shown in Figure 12, the display element 11 displays a first image IMG containing a first wavelength, a second image IMB containing a second wavelength shorter than the first wavelength, and a third image IMR containing a third wavelength longer than the first wavelength. In this case, the horizontal length E1 in the first image IMG is the length between the horizontal length E2 in the second image IMB and the horizontal length E3 in the third image IMR, and the vertical length F1 in the first image IMG is the same as the vertical length F2 in the second image IMB and the vertical length F3 in the third image IMR. This makes it possible to improve the display accuracy of color images by changing the correction amount for each wavelength band.

[0083] The following describes the processing performed by the grayscale conversion unit 8b. In the grayscale conversion unit 8b, for example, linear interpolation is performed.

[0084] After the coordinate values ​​have been derived after distortion transformation, interpolation is necessary to derive the grayscale values ​​of each pixel in the display element 11. The simplest method for this is nearest neighbor interpolation. This method uses the grayscale value of the closest coordinate value as is, but when displaying lines or characters, jagged edges or character distortion are likely to occur. On the other hand, if cubic spline interpolation is used by taking the grayscale values ​​of the four nearest points, a smoother image can be displayed, but the calculation process becomes complex. In this embodiment, N-division linear interpolation is performed, which is based on linear interpolation and can be performed with simple calculations when deriving grayscale values.

[0085] The calculation method used during linear interpolation is shown below, with reference to Figure 13. Figure region FR1 in Figure 13 is a diagram that explains the relationship between the pixel center coordinate PM and the transformed coordinate TM in the i-th row pixel PE of the display element 11. In Figure 13, the value Up i,j This indicates the x-coordinate, and the value Vp i,j This indicates the y-coordinate, and the value Rp i,jThis indicates the gradation value of red, and the value Gp i,j This indicates the green grayscale value, value Bp i,j This indicates the blue gradation value. Interpolation calculations are performed for each line and each wavelength.

[0086] Figure 13, area FR2, is a diagram illustrating the grayscale values ​​corresponding to the pixel center coordinates PM and the transformed coordinates TM in, for example, the green image. As shown in Figure 13, area FR2, the two points B after distortion transformation. i,j ,B i,j+1 Grayscale value Gb i,j ,Gb i,j+1 and coordinate B i,j (Ub i,j ,Vb i,j ), B i,j+1 (Ub i,j+1 ,Vb i,j+1 ) and the pixel P of the display element 11 located between the two points i,j Regarding the grayscale value Gp i,j and coordinates (Up i,j ,Vp i,j ) If so, the pixel P of the display element 11 i,j The grayscale value Gp i,j It is calculated as follows: Gp i,j =Gb i,j +α(Gb i,j+1 -Gb i,j ) However, the value α is less than or equal to 1.

[0087] As described above, the correction unit 86, as a distortion generation circuit, can achieve distortion processing and high image quality with simpler processing, contributing to circuit miniaturization, power consumption reduction, and cost reduction.

[0088] The user terminal circuit 91 is incorporated into the user terminal 90 and includes a main control unit 91a, a storage device 91m, a data communication interface 91c, a mobile wireless communication device 91t, and a user interface device 91i. The user terminal circuit 91 can communicate with various devices such as external servers via a communication network (not shown) using the mobile wireless communication device 91t. The storage device 91m stores a basic program for operating the user terminal circuit 91, and also stores multiple application software programs that run on this basic program, such as a video viewer and a web browser. The user terminal circuit 91 operates in response to requests from the user interface device 91i operated by the user, outputting videos and still images stored in the storage device 91m in a predetermined format to the display control device 88, or acquiring videos and still images corresponding to various content via the mobile wireless communication device 91t, and outputting the acquired display data to the display control device 88 in a predetermined format. In the above description, the display control device 88 performs distortion correction processing on the display data input from the user terminal circuit 91, but the user terminal circuit 91 may also perform distortion correction processing on the display data.

[0089] The image processing to be displayed on the display surface 11a of the display element 11 will be explained below with reference to Figure 14.

[0090] First, in the display control device 88 shown in Figure 8, the arithmetic processing unit 81a acquires the initial image IM0 as display data from the user terminal 90 via the data communication interface 81c and stores it in the storage device 81m (step S11). The user terminal circuit 91 outputs the video or still image stored in the storage device 91m to the display control device 88 in a predetermined format, or outputs the video or still image acquired via the mobile wireless communication device 91t to the display control device 88 in a predetermined format.

[0091] Next, in the display control device 88, the arithmetic processing unit 81a, acting as the coordinate transformation unit 8a of the correction unit 86, performs distortion correction on the initial image IM0 by coordinate transformation (step S12). The coordinate transformation unit 8a performs coordinate transformation on the initial image IM0 in order to cancel the distortion aberration of the imaging optical system 20.

[0092] Next, in the display control device 88, the arithmetic processing unit 81a, acting as the grayscale conversion unit 8b of the correction unit 86, performs grayscale adjustment on the initial image IM0 (step S13). The grayscale conversion unit 8b performs image processing, such as grayscale adjustment, on the image after the coordinate transformation in step S12 to create a corrected image IM1. The corrected image IM1 is recorded in the frame memory 83 of the storage device 81m for each frame as needed.

[0093] Finally, in the display control device 88, the arithmetic processing unit 81a reads the image data of the corrected image IM1 line by line from the frame memory 83 of the storage device 81m, and displays it on the display surface 11a of the display element 11 one scan line SL at a time, as shown in Figure 8 (step S14). The trapezoidal corrected image IM1 displayed on the display element 11 is perceived as a rectangular image in the virtual image formed by the imaging optical system 20. In this embodiment, the imaging optical system 20 creates an intermediate image in between, so the virtual image is inverted vertically and horizontally with respect to the initial image IM0.

[0094] The head-mounted display device 100 of the first embodiment described above includes a control unit 88 that corrects the image, a display element 11 that displays the image corrected by the control unit 88, a first optical member 22 into which image light corresponding to the image is incident, and a reflecting member 23 that reflects the image light from the first optical member 22 and projects a virtual image corresponding to the image. The first optical member 22 and the reflecting member 23 correct distortion in a first direction related to the virtual image, and the control unit 88 corrects the image according to distortion in a second direction that intersects with the first direction related to the virtual image.

[0095] In the head-mounted display device 100 described above, the first optical member 22 and the reflective member 23 correct distortion in a first direction related to the virtual image, and the control unit 88 corrects the image according to distortion in a second direction related to the virtual image. As a result, the control unit 88 does not need to process distortion correction according to distortion in the first direction related to the virtual image, and only needs to correct distortion in one direction, thus reducing the load on the control unit 88 for image distortion correction.

[0096] [Second Embodiment] The following describes a head-mounted display device according to a second embodiment of the present invention. The head-mounted display device of the second embodiment is a modified version of the head-mounted display device of the first embodiment, and the common parts will not be described.

[0097] Figure 15 is a side cross-sectional view illustrating the internal structure of the head-mounted display device 100 of the second embodiment. As shown in Figure 15, in the head-mounted display device 100 of the second embodiment, the projection lens 21 in the imaging optical system 20 of the first display device 100A is composed of two lenses. Specifically, the projection lens 21 includes a first lens 21o and a second lens 21p. In the second embodiment, the concave second lens 21p, which had a color-correcting function in the projection lens 21 of the first embodiment shown in Figure 2, is omitted. The angles and arrangement of the optical elements of the imaging optical system 20 are the same as in the first embodiment. The first lens 21o and the second lens 21p are formed of, for example, resin, but can also be made of glass. In the second embodiment, it is preferable that the first lens 21o and the second lens 21p are formed of a resin with a moderate refractive index and a high Abbe number, such as a cyclic olefin copolymer (COC) or a ring-opening polymerization hydrogenated resin (COP).

[0098] (Example 2) The following shows specific numerical examples for each variable. θ1 = 36° (θ1' = 62°) θ2 = 52° (θ2' = 70°) θ3 = 10° θ4 = -4° L1 = 35 mm L2 = 46mm L3 = 26mm Ln=21mm L1cos(θ3+θ1) / L3cosθ4=0.93

[0099] Figure 16 illustrates the relationship between the range of input and output angles of the image light ML of the see-through mirror 23 and the vertical field of view. Region GR1 in Figure 16 is the screen position where the principal ray angle of the imaging optical system 20 of the first display device 100A was investigated, and corresponds to the field of view of the screen or virtual image. Region GR2 in Figure 16 shows the difference between the incident and output principal ray angles of the see-through mirror 23 in the vertical direction, and is a plot of the difference between the input and output angles of the see-through mirror 23 and the vertical field of view. In region GR2 of Figure 16, the solid line shows the plot of the apparatus of this embodiment, and the dashed line shows the plot of a conventional apparatus. Figure 17 is a conceptual diagram illustrating a specific example of the distortion state of the virtual image AA formed by the imaging optical system 20 of the first display device 100A of the second embodiment.

[0100] In this embodiment, by controlling the principal ray angle of the vertical angle-of-view light rays to the see-through mirror 23, the intermediate image shape can be controlled so that vertical distortion does not occur in the virtual image AA in the first display device 100A of this embodiment, as shown in the diagram area GR2 of Figure 16 and Figure 17. By suppressing vertical distortion, the need to allocate frame buffer memory when the display control device 88 performs distortion correction is eliminated. This reduces the computational load on the display control device 88 when correcting distortion generated in the imaging optical system 20. In this way, the burden of circuit correction on the display control device 88 is greatly reduced, and power consumption can be made lower than that of conventional devices.

[0101] Furthermore, as shown in Figure 17, the imaging optical system 20 of the second embodiment does not sufficiently suppress the lateral spacing of distortion and the lateral displacement relative to the vertical direction compared to the imaging optical system 20 of the first embodiment. Therefore, in the first and second correction regions SP1 and SP2 of the image, even in the same row, the spacing A i,j The horizontal length B' between adjacent rows differs, even within the same column. iThey are different.

[0102] [Differentiations and other variations] Although the present invention has been described in reference to the embodiments described above, the present invention is not limited to the embodiments described above, and can be implemented in various forms without departing from the spirit thereof, for example, the following modifications are also possible.

[0103] The first display device 100A may be configured to include an imaging optical system that is asymmetric in the lateral direction, so that the distortion correction is asymmetric in the lateral direction, i.e., in the X direction. In this case, the first optical element and the reflective element correct the left-right distortion related to the virtual image, and the control unit corrects the image according to the vertical distortion.

[0104] In the first display device 100A, the projection lens 21 is configured with three or two lenses, but it may also be configured with one or four or more lenses. Furthermore, the first display device 100A may be configured without a projection lens 21.

[0105] A dimming device can be attached to the exterior side of the see-through mirror 23 to control the light transmitted through the see-through mirror 23. The dimming device adjusts the transmittance electrically, for example. A mirror liquid crystal, an electronic shade, etc., can be used as the dimming device. The dimming device may also adjust the transmittance according to the ambient light intensity.

[0106] In the first display device 100A, a wedge-shaped optical element 28 that improves the imaging state may be provided between the prism mirror 22 and the see-through mirror 23. For example, the refractive index of the wedge-shaped optical element 28 can be made different from that of the prism mirror 22, and by adjusting the degree of refraction and dispersion between the wedge-shaped optical element 28 and the prism mirror 22, it becomes easier to achieve achromaticity.

[0107] In a specific embodiment, the head-mounted display device comprises a control unit for correcting an image, a display element for displaying the image corrected by the control unit, a first optical member into which image light corresponding to the image is incident, and a reflective member that reflects the image light from the first optical member and projects a virtual image corresponding to the image. The first optical member and the reflective member correct distortion in a first direction related to the virtual image, and the control unit corrects the image according to distortion in a second direction intersecting the first direction related to the virtual image. Here, distortion in the first direction related to the virtual image is eliminated in the head-mounted display device of the present invention by correction by the first optical member and the reflective member. Furthermore, any remaining distortion inherent in the virtual image is distortion caused by the first optical member and the reflective member, and is canceled out by distortion correction in the control unit, making it invisible to the eye.

[0108] In the above-described head-mounted display device, the first optical element and the reflective element correct distortion in a first direction related to the virtual image, and the control unit corrects the image according to distortion in a second direction related to the virtual image. As a result, the control unit does not need to process distortion correction according to distortion in the first direction related to the virtual image, and only needs to correct distortion in one direction, thus reducing the load on the control unit for image distortion correction.

[0109] In specific terms, the second direction corresponds to the scanning direction of the display element.

[0110] In specific terms, the control unit corrects the image so that the length of the second direction in one of the first directions of the image is shorter than the length of the second direction in the other of the first directions of the image. In this case, for example, if the shape of the image before correction is a rectangle, the corrected image displayed on the display element will have a trapezoidal shape.

[0111] In specific terms, if we let y be the coordinate in the first direction of the uncorrected image, x be the coordinate in the second direction of the uncorrected image, U be the coordinate in the second direction of the corrected image, a be the coefficient corresponding to the length in the second direction of the corrected image, and b be the coefficient corresponding to the angle between the side in the first direction and the side in the second direction of the corrected image, the control unit corrects the image so that it satisfies the following equation. U = ax + bxy

[0112] In specific terms, if c is a coefficient corresponding to the distortion between the tilt in one of the second directions in the corrected image and the tilt in the other of the second directions in the corrected image, the control unit corrects the image so that it satisfies the following equation. U = ax + bxy + cy

[0113] In specific terms, coefficients a, b, and c are rewritable parameters.

[0114] In specific terms, the display element displays a first image containing a first wavelength, a second image containing a second wavelength shorter than the first wavelength, and a third image containing a third wavelength longer than the first wavelength. The length in the second direction in the first image is the length between the length in the second direction in the second image and the length in the second direction in the third image, and the length in the first direction in the first image is the same as the length in the first direction in the second image and the length in the first direction in the third image. In this case, the display accuracy of the color image can be improved by changing the correction amount for each wavelength band.

[0115] In specific terms, the corrected image includes a first corrected area obtained by correcting the first area of ​​the uncorrected image, and a second corrected area obtained by correcting the second area of ​​the uncorrected image that is aligned with the first area in the second direction. The number of images in the second direction of the first corrected area is within ±10% of the number of pixels in the second direction of the second corrected area.

[0116] In specific aspects, the reflective member is positioned such that the angle θ1 formed by the image light incident from the first optical member to the reflective member and the image light reflected by the reflective member to the pupil position is 10° ≤ θ1 ≤ 70°. In this case, by setting the angle θ1 within the above range and making the arrangement angle of the reflective member relatively large, eccentric aberration can be suppressed, and optical performance can be improved. Furthermore, since the distance between the reflective member and the first optical member can be made relatively small, the size of the optical system can be reduced in the thickness direction, i.e., in the lateral cross-sectional direction.

[0117] In specific aspects, the first optical member has an incident portion into which image light from a display element is incident, an outgoing portion into which the incident image light is emitted toward a reflecting member, and a reflecting portion into which the image light incident from the incident portion is reflected toward the outgoing portion.

[0118] In specific aspects, the first optical element is positioned such that the angle θ2 formed by the image light incident on the reflecting element and the image light reflected by the reflecting element is 30° ≤ θ2 ≤ 90°. In this case, by setting the angle θ2 within the above range and making the arrangement angle of the first optical element relatively large, the size of the optical system can be reduced in the height direction, i.e., the vertical or longitudinal direction.

[0119] In more specific terms, if θ3 is the angle between the horizontal direction and the image light reflected by the reflecting member to the pupil's position, L1 is the optical path length of the image light incident from the first optical member to the reflecting member, and L2 is the optical path length of the image light reflected by the reflecting member to the pupil's position, then the distance Ln from the pupil to the first optical member in the horizontal direction satisfies the following equation. Ln = L2cosθ3 - L1cos(θ3 + θ1)

[0120] In specific terms, the display element is positioned such that the angle θ4 formed by the horizontal direction and the image light incident on the reflective portion of the first optical element is θ4 ≤ 10°.

[0121] In specific aspects, the system includes a second optical element positioned between the first optical element and the display element.

[0122] In specific aspects, the device comprises a display element, a first optical element, a reflective element, a support member that supports the display element, and a mounting member connected to the support member and worn on the user's head. [Explanation of Symbols]

[0123] 8a... Coordinate transformation unit, 8b... Grayscale conversion unit, 11... Display element, 11a... Display surface, 12... Projection optical system, 20... Imaging optical system, 21... Projection lens, 21o, 21p, 21q... Lenses, 22... Prism mirror, 22a... Incident surface, 22b... Internal reflection surface, 22c... Outgoing surface, 23... See-through mirror, 51... Case, 70... Circuitry system, 81a... Processing unit, 81c... Data communication interface, 81m... Memory device, 83... Frame memory, 85... Ancillary circuitry, 86... Correction unit, 87... Non-volatile memory, 88... Display control Your device, 90...User terminal, 91...User terminal circuit, 100...Head-mounted display device, 100A...First display device, 100B...Second display device, 100C...Support device, 102...Display drive unit, 103...External component, AA...Virtual image, AP1...First region, AP2...Second region, AX...Optical axis, AX1,AX2,AX3...Optical axis portion, ER...Eye ring, EY...Eye, IM0...Initial image, IM1...Corrected image, ML...Image light, OL...Outer light, OS...Optical system, PP...Pupil position, SP1...First correction region, SP2...Second correction region

Claims

1. A control unit that corrects the image; a display element that displays the image corrected by the control unit; a first optical member onto which image light corresponding to the image is incident; a reflecting member that reflects the image light from the first optical member and projects a virtual image corresponding to the image, the first optical member and the reflecting member correct distortion of the virtual image in a first direction; The control unit corrects the image according to distortion in a second direction intersecting the first direction regarding the virtual image.

2. The head mounted display device according to claim 1 , wherein the second direction corresponds to a scanning direction of the display element.

3. The head-mounted display device of claim 2, wherein the control unit corrects the image so that a length in the second direction at one of the first directions in the image is shorter than a length in the second direction at the other of the first directions in the image.

4. If the coordinate in the first direction of the image before correction is y, the coordinate in the second direction of the image before correction is x, the coordinate in the second direction of the image after correction is U, a coefficient corresponding to the length in the second direction of the image after correction is a coefficient corresponding to the angle between the side in the first direction and the side in the second direction of the image after correction is b, The head-mounted display device according to claim 2 , wherein the control unit corrects the image so as to satisfy the following formula: U = ax + bxy

5. When a coefficient corresponding to a distortion of one tilt in the second direction in the image after correction and a distortion of the other tilt in the second direction in the image after correction is c, The head-mounted display device according to claim 4 , wherein the control unit corrects the image so as to satisfy the following formula: U=ax+bxy+cy

6. The head mounted display device according to claim 5 , wherein the coefficient a, the coefficient b, and the coefficient c are rewritable parameters.

7. the display element displays a first image including a first wavelength, a second image including a second wavelength that is shorter than the first wavelength, and a third image including a third wavelength that is longer than the first wavelength; a length in the second direction of the first image is a length between a length in the second direction of the second image and a length in the second direction of the third image, A head-mounted display device as described in any one of claims 1 to 6, wherein the length of the first direction in the first image is the same as the length of the first direction in the second image and the length of the first direction in the third image.

8. the corrected image includes a first corrected area obtained by correcting a first area of ​​the image before correction, and a second corrected area obtained by correcting a second area aligned with the first area of ​​the image before correction in the second direction, A head-mounted display device according to any one of claims 1 to 7, wherein the number of pixels in the second direction of the first correction area after correction is within ±10% of the number of pixels in the second direction of the second correction area after correction.

9. A head-mounted display device as described in any one of claims 1 to 8, wherein the reflective member is arranged so that an angle θ1 between the image light incident on the reflective member from the first optical member and the image light reflected by the reflective member to the pupil position is 10°≦θ1≦70°.

10. A head-mounted display device as described in any one of claims 1 to 9, wherein the first optical member has an incident portion where the image light from the display element is incident, an exit portion where the incident image light is emitted toward the reflecting member, and a reflection portion where the image light incident from the incident portion is reflected toward the exit portion.

11. The head-mounted display device of claim 10, wherein the first optical member is arranged so that an angle θ2 between the image light incident on the reflecting portion and the image light reflected by the reflecting portion is 30°≦θ2≦90°.

12. A head-mounted display device as described in any one of claims 10 and 11, wherein, when the angle between the horizontal direction and the image light reflected by the reflecting member to the pupil position is θ3, the optical path length of the image light incident from the first optical member to the reflecting member is L1, and the optical path length of the image light reflected by the reflecting member to the pupil position is L2, the distance Ln from the pupil to the first optical member in the horizontal direction satisfies the following formula. Ln=L2cosθ3−L1cos(θ3+θ1)

13. The display element is arranged so that an angle θ4 between the horizontal direction and the image light incident on the reflecting portion of the first optical member is θ4≦10°. A head-mounted display device as described in any one of claims 10 to 12.

14. The head-mounted display device according to any one of claims 1 to 13, further comprising a second optical member disposed between the first optical member and the display element.

15. a support member that supports the display element, the first optical member, and the reflecting member; The head-mounted display device according to any one of claims 1 to 14, further comprising: a mounting member connected to the support member and mounted on a user's head.

Images