Head-mounted display device

Abstract

Head-mounted display device. Reduces the load on the control section in relation to correction of distortion of an image. A head-mounted display device (100) has a control section (88) that corrects an image, a display element (11) that displays the image after correction by the control section (88), a first optical member (22) through which image light corresponding to the image is incident, and a reflection member (23) that reflects the image light from the first optical member (22) to project a virtual image corresponding to the image, the first optical member (22) and the reflection member (23) correct distortion in a first direction related to the virtual image, and the control section (88) corrects the image in accordance with distortion in a second direction that intersects the first direction and that is related to the virtual image.

Description

Technical Field

[0001] This invention relates to a head-mounted display device capable of observing virtual images. Background Technology

[0002] The following apparatus is disclosed: In a display device for virtual images, a drive circuit that operates an image display element serves as a processing unit to perform distortion correction on an input image displayed on the image display element based on distortion in the vertical and horizontal directions, so that the image display element displays the image after distortion correction, wherein the distortion correction imparts distortion in the opposite direction to the distortion generated by the optical system used to observe the image displayed on the image display element (Patent Document 1).

[0003] Patent Document 1: Japanese Patent Application Publication No. 2008-058703

[0004] In the device described in Patent Document 1, the processing unit performs inverse correction corresponding to the vertical and horizontal distortions of the image, i.e., longitudinal and lateral distortions. Therefore, the burden on the processing unit increases. Summary of the Invention

[0005] One aspect of the present invention is a head-worn display device comprising: a control unit that corrects an image; a display element that displays the image corrected by the control unit; a first optical component that receives image light corresponding to the image; and a reflective component that reflects the image light from the first optical component and projects a virtual image corresponding to the image. The first optical component and the reflective component correct distortion in a first direction related to the virtual image, and the control unit corrects the image based on distortion in a second direction intersecting the first direction related to the virtual image. Attached Figure Description

[0006] Figure 1 This is a perspective view illustrating the wearing state of the head-worn display device according to the first embodiment.

[0007] Figure 2 This is a side sectional view illustrating the internal structure of a head-mounted display device.

[0008] Figure 3 This is a schematic diagram illustrating an example of the configuration of the optical system of a head-mounted display device.

[0009] Figure 4 It is a diagram illustrating the virtual image formed at the eye's side and the original image displayed on the display element.

[0010] Figure 5 This diagram illustrates the distortion state of the virtual image formed by correcting vertical distortion in the optical system of a head-worn display device.

[0011] Figure 6This is a schematic diagram illustrating a specific example of the distortion state of the corrected virtual image formed in the optical system of a head-mounted display device.

[0012] Figure 7 It is a diagram illustrating the relationship between the range of incident and exit angles of the image light from the reflecting component and the horizontal or vertical field of view.

[0013] Figure 8 This is a block diagram illustrating the circuit structure of a head-mounted display device.

[0014] Figure 9 It is a diagram illustrating the coordinate system before and after distortion transformation in the control unit.

[0015] Figure 10 This is a block diagram that partially illustrates the display control device and display elements of a head-mounted display device.

[0016] Figure 11 This is a diagram showing the distortion state when the parameters of the image are adjusted during coordinate transformation.

[0017] Figure 12 This diagram illustrates the correction of chromatic aberration in the optical system of a head-mounted display device.

[0018] Figure 13 This is a diagram illustrating linear interpolation used for grayscale adjustment of an image.

[0019] Figure 14 This is a flowchart illustrating image processing in a head-mounted display device.

[0020] Figure 15 This is a side sectional view illustrating the internal structure of the head-worn display device according to the second embodiment.

[0021] Figure 16 This is a diagram illustrating the relationship between the range of the incident and exit angles of image light from the reflective component of the head-worn display device of the second embodiment and the longitudinal field of view.

[0022] Figure 17 This is a schematic diagram illustrating a specific example of the distortion state of the virtual image formed in the optical system of the head-worn display device in the second embodiment.

[0023] Label Explanation

[0024] 8a: Coordinate transformation unit; 8b: Grayscale conversion unit; 11: Display element; 11a: Display surface; 12: Projection optical system; 20: Imaging optical system; 21: Projection lens; 21o, 21p, 21q: Lenses; 22: Prism mirror; 22a: Incident surface; 22b: Internal reflection surface; 22c: Exit surface; 23: Lens; 51: Housing; 70: Circuit system; 81a: Processing unit; 81c: Data communication interface; 81m: Storage device; 83: Frame memory; 85: Auxiliary circuit; 86: Correction unit; 87: Non-volatile memory; 88: Display control unit; 90: User terminal; 91: User terminal circuit; 100: Head-mounted display device; 100A: First display device; 100B: Second display device; 100C: Support device; 102: Display driving unit; 103: Appearance component; AA: Virtual image; AP1: First area; AP2: Second area; AX: Optical axis; AX1, AX2, AX3: Optical axis section; ER: Visual circle; EY: Eye; IM0: Initial image; IM1: Modified image; ML: Image light; OL: Ambient light; OS: Optical system; PP: Pupil position; SP1: First correction area; SP2: Second correction area. Detailed Implementation

[0025] [First Implementation]

[0026] The following is for reference Figures 1-3 The first embodiment of the present invention will be described below.

[0027] Figure 1 This diagram illustrates the wearing state of a head-mounted display (hereinafter also referred to as HMD) 200, which enables an observer or wearer US wearing the head-mounted display to recognize an image as a virtual image. Figure 1 In this system, X, Y, and Z are vertical coordinates. The +X direction corresponds to the horizontal alignment of the EY pairs of the observer or wearer US wearing the HMD 200 or head-mounted display device 100. The +Y direction corresponds to the upward direction perpendicular to the horizontal alignment of the EY pairs of the wearer US. The +Z direction corresponds to the forward or frontal direction for the wearer US. The ±Y directions are parallel to the vertical axis or the vertical direction.

[0028] The HMD 200 includes a first display device 100A for the right eye, a second display device 100B for the left eye, a pair of temple-shaped support devices 100C supporting the display devices 100A and 100B, and a user terminal 90 serving as an information terminal. The first display device 100A consists of a display driving unit 102 located at the top and an appearance component 103 that covers the eyes in a lens-like shape. The second display device 100B also consists of a display driving unit 102 located at the top and an appearance component 103 that covers the eyes in a lens-like shape. The support device 100C is a wearing component worn on the head of the wearer US, supporting the upper end of the appearance component 103 via the display driving unit 102. The first display device 100A and the second display device 100B are optically reversed; detailed description of the second display device 100B is omitted.

[0029] Figure 2 This is a side sectional view illustrating the optical structure of the first display device 100A. The first display device 100A includes a display element 11, an imaging optical system 20, and a display control device 88. The imaging optical system 20 includes a projection lens 21, a prism reflector 22, and a lens 23. The projection lens 21 and the prism reflector 22 in the imaging optical system 20 correspond to a second optical component and a first optical component, respectively, into which image light or image light ML corresponding to an image or picture is incident. The lens 23 corresponds to a reflective component that reflects the image light ML emitted from the aforementioned optical components toward the eye EY or the pupil position PP. Furthermore, the projection lens 21 and the prism reflector 22 in the imaging optical system 20... Figure 1 Corresponding to the display driver 102 shown, the perspective lens 23 and Figure 1 Corresponding to the exterior component 103 shown. The component consisting of the combined display element 11, projection lens 21, and prism reflector 22 is called the projection optical system 12, and they are fixed within the housing 51 in a mutually aligned state. The housing 51 is an outer shell or support component, formed of a light-shielding material, and supports the display control device 88 that enables the display element 11 to operate. The housing 51 has an opening 51a. The opening 51a allows the projection optical system 12 to emit image light ML outward.

[0030] The first display device 100A corrects distortion in a first direction related to the virtual image using a prism reflector 22 as a first optical component and a lens 23 as a reflective component. In this embodiment, for example, the prism reflector 22 and the lens 23 correct distortion in the first direction related to the virtual image, specifically the vertical direction. Furthermore, the first display device 100A corrects the image or picture by the display control device 88 based on distortion in a second direction that intersects the first direction related to the virtual image. In this embodiment, for example, the display control device 88 corrects distortion in the horizontal direction present in the virtual image. Here, the second direction or horizontal direction corresponds to the scanning direction of the display element 11. Distortion in the first direction related to the virtual image is not present in the first display device 100A due to correction by the prism reflector 22 and the lens 23. Furthermore, the remaining distortion present in the virtual image, caused by the prism reflector 22 and the lens 23, is canceled out by distortion correction in the display control device 88 and cannot be seen by the eye. In addition, the projection lens 21 affects the distortion of the virtual image, and the above-mentioned correction is performed in the imaging optical system 20 of the first display device 100A.

[0031] Display element 11 is a self-emissive display device. For example, display element 11 is an organic EL (organic electroluminescence) display, forming a colored static or dynamic image on a two-dimensional display surface 11a. Display element 11 is arranged along an xy-plane that is slightly tilted relative to the x-axis. Display element 11 is driven by a display control device 88, which serves as a control unit, to perform display operations. Furthermore, in Figure 2 In this example, the display element 11 is configured in a reverse orientation, with its upper +y direction facing downwards relative to the coordinates of the imaging optical system 20, i.e., in the -Y direction. In this case, due to the inverted nature of the imaging optical system 20, the orientation of the content displayed by the display element 11 is consistent with the original image before correction performed by the display control device 88.

[0032] Display element 11 is not limited to organic EL displays; for example, it can be replaced by display devices using inorganic EL, organic LEDs, LED arrays, laser arrays, quantum dot light-emitting elements, etc. Display element 11 is not limited to self-emissive image light generating devices; it can also be composed of LCDs and other light modulation elements, forming an image by illuminating the light modulation element with a light source such as a backlight. As display element 11, LCDs can also be replaced by LCOS (Liquid crystal on silicon, LCoS is a registered trademark), digital micromirror devices, etc.

[0033] The projection lens 21 in the imaging optical system 20 includes a first lens 21o, a second lens 21p, and a third lens 21q. The first lens 21o has an incident surface 21a and an exit surface 21b, the second lens 21p has an incident surface 21c and an exit surface 21d, and the third lens 21q has an incident surface 21e and an exit surface 21f. The projection lens 21 receives image light ML emitted from the display element 11 and directs it onto the prism reflector 22. The projection lens 21 converges the image light ML emitted from the display element 11 into a near-parallel beam. The prism reflector 22 has an incident surface 22a corresponding to the incident portion, an inner reflecting surface 22b corresponding to the reflecting portion, and an exit surface 22c corresponding to the exit portion. The prism reflector 22 emits the frontally incident image light ML in a manner that folds back in a direction inclined relative to the direction that reverses the incident direction (the direction of the light source observed from the prism reflector 22). The perspective mirror 23 has a reflecting surface 23a and an outer surface 23o. The perspective mirror 23 magnifies the intermediate image formed on the light-emitting side of the prism reflector 22.

[0034] The imaging optical system 20 is an off-axis optical system OS because the lens 23 is a concave mirror or the like. In this embodiment, the projection lens 21, the prism mirror 22, and the lens 23 are arranged non-axially symmetrically, having non-axially symmetrical optical surfaces. The imaging optical system 20 is an off-axis optical system OS because, among the optical elements 21, 22, and 23 constituting the imaging optical system 20, the light path is bent as a whole before and after the light is incident on multiple reflecting or refracting surfaces. In this imaging optical system 20, i.e., the off-axis optical system OS, the optical axis AX is bent such that it extends along an off-axis plane (a plane parallel to the YZ plane) corresponding to the paper surface. In this imaging optical system 20, the optical axis AX is bent within an off-axis plane parallel to the YZ plane, and thus, the optical elements 21, 22, and 23 are arranged along this off-axis plane. The optical axis AX of the imaging optical system 20 comprises optical axis portions AX1, AX2, and AX3, which are arranged along an off-axis surface (parallel to the YZ plane) that serves as a longitudinally extending reference plane and are inclined relative to each other before and after the reflecting surface. The overall optical axis AX extends along the path of the principal ray emanating from the center of the display element 11 and passes through the center of the viewing circle ER or pupil, corresponding to the viewpoint. When viewed in a cross-section parallel to the YZ plane, the optical axis AX is arranged in a Z-shape using multiple optical axis portions AX1, AX2, and AX3. That is, on the off-axis surface parallel to the YZ plane, the light path P1 from the projection lens 21 to the inner reflecting surface 22b, the light path P2 from the inner reflecting surface 22b to the lens 23, and the light path P3 from the lens 23 to the pupil position PP are arranged in a Z-shape, folding back in two stages. The imaging optical system 20 is arranged longitudinally. Correspondingly, the off-axis surface (the surface parallel to the YZ plane) which serves as the reference plane extends parallel to the longitudinal Y direction. In this case, the optical elements 21, 22, and 23 constituting the first display device 100A are arranged by changing their height position in the longitudinal direction, which can prevent the width of the first display device 100A from increasing.

[0035] In the imaging optical system 20, the optical path P1 from the projection lens 21 to the inner reflecting surface 22b extends slightly upwards or nearly parallel to the Z direction relative to the viewpoint. That is, in optical path P1, the optical axis portion AX1 extends in a direction close to the Z-axis, between the -Z and +Y directions. Alternatively, the optical axis portion AX1 can also extend in a direction close to the Z-axis, between the -Z and -Y directions. The optical path P2 from the inner reflecting surface 22b to the lens 23 extends forward and downwards. That is, in optical path P2, the optical axis portion AX2 extends in a direction nearly intermediate between the +Z and -Y directions. With the horizontal plane (XZ plane) as a reference, the tilt of optical path P2 is greater than that of optical path P1. The optical path P3 from the lens 23 to the pupil position PP extends in an upward direction slightly tilted backwards or nearly parallel to the Z direction. In the illustrated example, the optical axis portion AX3 becomes approximately -10° when the downward direction towards the +Z direction is set negative. That is, the emitted optical axis EX, which extends from the optical axis portion AX3, extends downward at approximately 10° relative to the central axis HX, which is parallel to the +Z direction in front. This is because a person's line of sight is stable when their head is slightly lowered, tilted downward at approximately 10° in the horizontal direction. Furthermore, the central axis HX extending horizontally relative to the pupil position PP is assumed to be in the case where the wearer US wearing the first display device 100A is in an upright, relaxed posture, looking directly at the horizontal direction or a horizontal line.

[0036] The incident surface 21a and the exit surface 21b of the first lens 21o constituting the projection lens 21 are asymmetrical along the optical axis AX in the longitudinal direction parallel to the YZ plane and intersecting the optical axis AX, but symmetrical along the optical axis AX in the transverse or X direction. The incident surface 21c and the exit surface 21d of the second lens 21p constituting the projection lens 21 are asymmetrical along the optical axis AX in the longitudinal direction parallel to the YZ plane and intersecting the optical axis AX, but symmetrical along the optical axis AX in the transverse or X direction. The incident surface 21e and the exit surface 21f of the third lens 21q constituting the projection lens 21 are asymmetrical along the optical axis AX in the longitudinal direction parallel to the YZ plane and intersecting the optical axis AX, but symmetrical along the optical axis AX in the transverse or X direction. The first lens 21o, the second lens 21p, and the third lens 21q are, for example, formed of resin, but may also be made of glass. Furthermore, preferably, the first lens 21o and the third lens 21q are formed of resins with moderate refractive index and high Abbe number, such as cyclic olefin copolymers (COC) and ring-opening polymerized hydrogenated resins (COP), while the second lens 21p is formed of resin with high refractive index and low Abbe number. By using a concave lens with high refractive index and low Abbe number as the second lens 21p, chromatic aberration in the imaging optical system 20 can be suppressed as an achromatic lens, significantly improving optical performance. The incident surface 21a and the exit surface 21b of the first lens 21o are, for example, freeform surfaces. The incident surface 21a and the exit surface 21b are not limited to freeform surfaces and may also be aspherical. The incident surface 21c and the exit surface 21d of the second lens 21p are, for example, freeform surfaces. The incident surface 21c and the exit surface 21d are not limited to freeform surfaces and may also be aspherical. The incident surface 21e and the exit surface 21f of the third lens 21q are, for example, freeform surfaces. The incident surface 21e and the exit surface 21f are not limited to freeform surfaces, but can also be aspherical surfaces. By making the incident surfaces 21a, 21c, 21e or the exit surfaces 21b, 21d, 21f freeform or aspherical surfaces, aberration reduction can be achieved. In particular, when using freeform surfaces, it is easy to improve the optical performance of the eccentric system. Therefore, it is easy to reduce the aberrations of the imaging optical system 20, which is a non-coaxial off-axis optical system OS. (Detailed illustration omitted) Antireflective coatings are formed on the incident surfaces 21a, 21c, 21e and the exit surfaces 21b, 21d, 21f.

[0037] The prism reflector 22 is an optical component with refractive and reflective functions, refracting and reflecting the image light ML from the projection lens 21. This refractive and reflective function combines the functions of a reflector and a lens. Specifically, the prism reflector 22 causes the image light ML to be incident on the incident surface 22a, then totally internally reflected by the inner reflecting surface 22b in a non-frontal direction, and finally exiting the incident image light ML through the exit surface 22c. The incident surface 22a and the exit surface 22c are curved optical surfaces, which helps to improve resolution compared to cases where they are merely reflecting surfaces or are planar. The optical surfaces constituting the prism reflector 22—the incident surface 22a, the inner reflecting surface 22b, and the exit surface 22c—are asymmetrical along the optical axis AX in the longitudinal direction, parallel to the YZ plane and intersecting the optical axis AX, but symmetrical in the transverse or X direction, also across the optical axis AX. The prism reflector 22 is made of resin, for example, but can also be made of glass. Furthermore, the prism reflector 22 is preferably formed of a resin with a moderate refractive index and a high Abbe number, such as a cyclic olefin copolymer (COC) or a ring-opening polymerized hydrogenated resin (COP). The refractive index of the main body of the prism reflector 22 is also set to a value that achieves total internal reflection on the inner surface, with reference to the reflection angle of the image light ML. The optical surfaces of the prism reflector 22, namely the incident surface 22a, the inner reflecting surface 22b, and the exit surface 22c, are, for example, freeform surfaces. The incident surface 22a, the inner reflecting surface 22b, and the exit surface 22c are not limited to freeform surfaces and may also be aspherical surfaces. In the prism reflector 22, by making the optical surfaces 22a, 22b, and 22c freeform or aspherical surfaces, aberrations can be reduced, and the optical performance of the eccentric system can be easily improved, especially when using freeform surfaces. Regarding the inner reflecting surface 22b, it is not limited to reflecting the image light ML by total internal reflection; it may also be a reflecting surface composed of a metal film or a dielectric multilayer film. In this case, a reflective film consisting of a single layer or multiple layers of metal such as Al or Ag is formed on the inner reflective surface 22b by means of vapor deposition or the like, or a sheet-like reflective film made of metal is attached. (Detailed illustration omitted) Anti-reflective films are formed on the incident surface 22a and the exit surface 22c.

[0038] The perspective mirror 23 is a curved, plate-shaped reflective optical component that functions as a concave mirror, reflecting the image light ML from the prism mirror 22. That is, the perspective mirror 23 reflects the image light ML from the prism mirror 22, which is disposed in the exit area of ​​the projection optical system 12, toward the pupil position PP. The perspective mirror 23 covers the pupil position PP, where the eye EY or pupil is located, and has a concave shape toward the pupil position PP and a convex shape toward the outside. The perspective mirror 23 is a concave transmission mirror covering the entire effective area of ​​the image in the field of view. The perspective mirror 23 is a collimator with a converging function, converging the principal rays of the image light ML emitted from each point on the display surface 11a—that is, the principal rays of the image light ML temporarily expanded by imaging near the exit side of the prism mirror 22 of the projection optical system 12—to the pupil position PP. The perspective mirror 23 is a reflective plate having a structure in which a translucent reflective film 23c is formed on the front or back of the plate-shaped body 23b. The reflecting surface 23a of the lens 23 is asymmetrical along the optical axis AX in the longitudinal direction, which is parallel to the YZ plane and intersects the optical axis AX, but symmetrical along the optical axis AX in the transverse or X direction. The reflecting surface 23a of the lens 23 is, for example, a freeform surface. The reflecting surface 23a is not limited to a freeform surface; it can also be an aspherical surface. By making the lens 23 a freeform or aspherical surface, aberration reduction can be achieved, especially when using a freeform surface, which can reduce aberrations in the imaging optical system 20, which is an off-axis or non-coaxial optical system.

[0039] The lens 23 is a transmissive reflective element that allows a portion of the light to pass through during reflection. The reflective surface 23a or the reflective film 23c of the lens 23 is formed of a semi-transparent reflective layer. Thus, external light OL passes through the lens 23, allowing the outside world to pass through, and enabling the virtual image to overlap with the external image. In this case, if the plate 23b supporting the reflective film 23c is thinned to about a few millimeters or less, the magnification change of the external image can be suppressed to a small extent. From the viewpoint of ensuring the brightness of the image light ML and facilitating observation of the external image based on perspective, the reflectivity of the reflective film 23c for both the image light ML and the external light OL is preferably 10% or more and 50% or less within the envisioned incident angle range of the image light ML. The plate 23b, which serves as the substrate of the lens 23, is formed of resin, but can also be made of glass. The plate 23b is formed of the same material as the support plate 61 that supports it from the surrounding area, and has the same thickness as the support plate 61. The reflective film 23c is formed, for example, of a dielectric multilayer film, which consists of multiple dielectric layers with adjusted thickness. The reflective film 23c can also be a single-layer or multilayer film of metals such as Al or Ag with adjusted thickness. The reflective film 23c can be formed by lamination, but it can also be formed by attaching a sheet-like reflective film. An anti-reflective film is formed on the outer surface 23o of the plate-like body 23b.

[0040] The optical path is described as follows: Image light ML from display element 11 is incident on projection lens 21 and exits from projection lens 21 in a roughly collimated state. After passing through projection lens 21, image light ML is incident on prism reflector 22, refracted and passed through incident surface 22a, reflected by inner reflecting surface 22b with a high reflectivity of nearly 100%, and refracted again by exiting surface 22c. Image light ML from prism reflector 22 is incident on lens 23 and reflected by reflecting surface 23a with a reflectivity of about 50% or less. Image light ML reflected by lens 23 is incident on pupil position PP, which is the eye EY or pupil of the wearer US. External light OL after passing through lens 23 or the support plate 61 around it is also incident on pupil position PP. That is, the wearer US wearing the first display device 100A can observe the virtual image formed by image light ML in a manner that overlaps with the external image.

[0041] Figure 3 This is a schematic diagram illustrating an example configuration of the imaging optical system 20 of the first display device 100A. Figure 3 Figure AR1 shows an example of a configuration focusing on the reflection angle of the optical axis AX of the prism reflector 22 and the perspective mirror 23, while Figure AR2 shows an example of a configuration focusing on the tilt of the prism reflector 22 and the perspective mirror 23 relative to the horizontal axis or the Z-axis.

[0042] exist Figure 3 In region AR1 of the diagram, angle θ1 is the angle formed by the image light ML incident from the prism reflector 22 to the lens 23 and the image light ML reflected from the lens 23 to the pupil position PP. Angle θ2 is the angle formed by the image light ML incident on the reflecting part of the prism reflector 22, i.e., the inner reflecting surface 22b, and the image light ML reflected by the inner reflecting surface 22b. Angle θ3 is the angle formed by the horizontal direction or the Z-direction with the image light ML reflected from the lens 23 to the pupil position PP. Angle θ4 is the angle formed by the horizontal direction or the Z-direction with the image light ML incident on the inner reflecting surface 22b of the prism reflector 22. Angle θ4 is approximately the tilt of the image light ML exiting to the display element 11 relative to the horizontal direction. Label L1 is the optical path length of the image light ML incident from the inner reflecting surface 22b of the prism reflector 22 to the lens 23. Label L2 is the optical path length of the image light ML reflected from the lens 23 to the pupil position PP. The designation L3 represents the optical path length of the image light ML incident from the display element 11 onto the inner reflecting surface 22b of the prism reflector 22. Specifically, the optical path length L3 is the distance from the display surface 11a of the display element 11 to the inner reflecting surface 22b of the prism reflector 22. Alternatively, optical path lengths L1 to L3 can be approximated as distances.

[0043] In region AR1 of the figure, the angle θ4 is as shown in the following formula.

[0044] θ4=-θ3-θ1+θ2

[0045] Here, angle θ4 is set negative on the lower side and positive on the upper side relative to the horizontal axis or horizontal direction. Angle θ3 is set positive on the lower side relative to the horizontal axis. Angles θ1 and θ2 are absolute values.

[0046] The horizontal distance Ln from the pupil position PP to the prism reflector 22 satisfies the following formula.

[0047] Ln=L2cosθ3-L1cos(θ3+θ1)

[0048] Here, distance Ln is the horizontal distance from the pupil position PP to the prism reflector 22, also known as the eye-to-lens distance. Angle θ3, relative to the horizontal line of sight (the horizontal direction of the line of sight), determines the position of the image center or picture center. In audiovisual imaging, it is preferable that it is located below the horizontal line of sight. Specifically, in audiovisual imaging, it is better for angle θ3 to be located 5° to 20° below the horizontal axis. The value θ3+θ1 in the formula represents the angle of L1 relative to the horizontal axis.

[0049] Each variable satisfies the following range.

[0050] 10°≤θ1≤70°

[0051] 30°≤θ2≤90°

[0052] -10°≤θ4≤+10°

[0053] 15mm≤Ln≤30mm

[0054] 0.5≤L1cos(θ3+θ1) / L3cosθ4≤1.5

[0055] The perspective mirror 23 is set with an angle θ1 within the aforementioned range. This results in a relatively large configuration angle for the perspective mirror 23, which suppresses eccentricity aberration and improves optical performance. Furthermore, since the distance between the perspective mirror 23 and the prism mirror 22 can be minimized, the size of the imaging optical system 20 can be reduced in the thickness direction, i.e., the lateral section direction. Specifically, with an angle θ1 above the lower limit, the size of the imaging optical system 20 can be prevented from increasing on the upper or lower sides. Furthermore, with an angle θ1 below the upper limit, the layout of the imaging optical system 20 can be appropriately arranged.

[0056] The prism reflector 22 is configured such that the angle θ2 is within the aforementioned range. This results in a relatively large configuration angle for the prism reflector 22, enabling miniaturization of the imaging optical system 20 in the height direction (vertical direction) or longitudinal direction. Specifically, when the angle θ2 is above the lower limit, the layout of the imaging optical system 20 can be appropriately optimized and miniaturized, and eccentricity aberration can be suppressed. Furthermore, when the angle θ2 is below the upper limit, the layout of the imaging optical system 20 can be appropriately optimized and miniaturized.

[0057] The display element 11 is configured such that the angle θ4 is within the aforementioned range, thereby suppressing eccentricity aberration and improving optical performance. Specifically, when the angle θ4 is above the lower limit, the layout of the imaging optical system 20 can be appropriately arranged and eccentricity aberration can be suppressed. Furthermore, when the angle θ4 is below the upper limit, the imaging optical system 20 can be miniaturized, and eccentricity aberration can be suppressed. In particular, by tilting the angle θ4 downwards, eccentricity aberration can be further suppressed. Moreover, when the angle θ4 is close to zero, the imaging optical system 20 can be further miniaturized, and it also has the effect of simplifying the overall design and improving the smart appearance when worn by the wearer US.

[0058] The distance Ln from the pupil position PP to the prism reflector 22 is above the lower limit value, thus preventing the virtual image from being incomplete even when the wearer US wears glasses. Furthermore, the distance Ln is below the upper limit value, thereby enabling the miniaturization of the imaging optical system 20.

[0059] The value L1cos(θ3+θ1) / L3cosθ4 is above the lower limit, thus, for example, even if the projection lens 21 has multiple lenses, the layout of the imaging optical system 20 can be appropriate. Furthermore, the upper limit value of L1cos(θ3+θ1) / L3cosθ4 is, for example, a magnitude limit value in the thickness direction of the imaging optical system 20 when the projection lens 21 has approximately one lens.

[0060] exist Figure 3 In region AR2, the orientation angle of the perspective mirror 23 is set to θ1'. The orientation angle of the reflecting part of the prism mirror 22 is set to θ2'. Other angles and optical path lengths are the same as in region AR1.

[0061] In region AR2, the angle θ4 is shown in the following formula.

[0062] θ4=-θ3-2θ1'+2θ2'

[0063] Here, angle θ4 is relative to the horizontal axis or horizontal direction, with the lower side being negative and the upper side being positive. Angle θ3 is relative to the horizontal axis, with the lower side being positive. Angles θ1' and θ2' are relative to the horizontal axis, with the upper side being positive.

[0064] The horizontal distance Ln from the pupil position PP to the prism reflector 22 is shown in the following formula.

[0065] Ln=L2cosθ3-L1cos(180-θ3-2θ1')

[0066] The value 180-θ3-2θ1' in the formula represents the angle of L1 relative to the horizontal axis.

[0067] Each variable satisfies the following range.

[0068] 45°≤θ1'≤75°

[0069] -10°≤θ4≤+10°

[0070] 15mm≤Ln≤30mm

[0071] 0.5≤L1cos(180-θ3-2θ1') / L3cosθ4≤1.5

[0072] (Example 1)

[0073] The following are examples of specific numerical values ​​for each variable.

[0074] θ1 = 28° (θ1' = 63°)

[0075] θ2 = 52° (θ2' = 67°)

[0076] θ3 = 12°

[0077] θ4=-4°

[0078] L1 = 35mm

[0079] L2 = 52mm

[0080] L3 = 26mm

[0081] Ln = 26mm

[0082] L1cos(θ3+θ1) / L3cosθ4=1.01

[0083] The first display device 100A, by making the reflection angle θ1 or arrangement angle θ1' of the lens 23 relatively large, generates a tilt angle of the lens 23 compared to existing devices, thereby controlling the position of the intermediate image. That is, by controlling the principal ray angle of the longitudinal field-of-view rays (Y-direction field-of-view rays) directed to the lens 23, the shape of the intermediate image can be controlled. Furthermore, by making the variables of the imaging optical system 20 of the first display device 100A within the aforementioned range, distortions generated in the vertical or longitudinal direction corresponding to the first direction are suppressed. In existing devices, for example, the arrangement angle θ1' of the lens 23 is tilted approximately 30° to 40° relative to the horizontal axis. In this case, the image light ML is reflected higher. In existing devices, in order to eliminate the eccentric aberration generated in the lens 23, the distance between the lens 23 and the reflecting surface of the prism mirror 22 is made larger (making the distance between the intermediate image and the reflecting surface larger), thereby adjusting the magnification of the intermediate image, suppressing the curvature of the reflecting surface, and reducing aberrations generated on the reflecting surface. Therefore, there is a problem that the size of the imaging optical system 20 increases on the brow side of the wearer US. In the first display device 100A of this embodiment, the size on the brow side, i.e., the distance from the center of the pupil to the maximum light height of the imaging optical system 20, is approximately 20 mm, which is about 10 mm smaller than that of conventional devices. Furthermore, in conventional devices, since the incident angle θ4 to the prism reflector 22 is approximately 30°, the housing structure of the optical system becomes complex. However, in the first display device 100A of this embodiment, the angle θ4 is within ±10°, preferably about 5°. Therefore, the housing 51, as the outer shell, is not too complex, and the assemblability and robustness are improved.

[0084] As described above, in this embodiment, by raising the perspective lens 23 at an angle θ1 or a configuration angle θ1' within a predetermined range, longitudinal distortion can be suppressed in the first display device 100A, resulting in a trapezoidal distortion dominated by lateral distortion.

[0085] Figure 4 This is a hypothetical diagram illustrating the virtual image AA formed on the eye's EY side without correction by the display control device 88 and the original image BB displayed by the display element 11. Figure 4 In the image, the solid line shows the virtual image AA obtained by longitudinal correction in the imaging optical system 20 of the first display device 100A, and the dashed line shows the display area SB of the ideal rectangular image corresponding to the original image BB.

[0086] Regarding the virtual image AA formed in the first display device 100A, before distortion correction is performed by the display control device 88 (described later), the length D1 of the image AA1 corresponding to the virtual image AA in the left-right direction (corresponding to the second direction) on the one side of the first direction (vertical direction) is shorter than the length D2 of the other side of the vertical direction (left-right direction) of the image AA1. For example, the image BB before correction is rectangular, and by correcting the directional bias during projection by the imaging optical system 20 of the first display device 100A, the corrected image AA1 becomes trapezoidal. Figure 4 In the example, the upper side or +Y direction (equivalent to -y direction) is set as one side, and the lower side or -Y direction (equivalent to +y direction) is set as the other side. The display control device 88 performs inverse distortion correction corresponding to the trapezoidal image AA1, so that the final corrected virtual image is a rectangle identical to the original image BB.

[0087] Figure 5 This diagram illustrates the distortion state of the virtual image AA formed by correcting vertical distortion in the imaging optical system 20 of the first display device 100A. (See diagram for example.) Figure 5 As shown, the virtual image AA includes a corrected first corrected region SP1 obtained by correcting the first region AP1 of the original image BB, and a corrected second corrected region SP2 obtained by correcting the second region AP2, which is arranged in the left-right direction relative to the first region AP1 of the original image. The first region AP1 and the second region AP2 are unit regions with fixed expansion.

[0088] like Figure 5 As shown in the diagram regions BR1 and BR2, the ranges of regions AP1 and AP2 can be appropriately set, and the ranges of the first correction region SP1 and the second correction region SP2 are determined accordingly. Region AP1 is a reference region set as a vertical column or a horizontal row. Figure 5 In the example of region BR1, regions AP1 and AP2 correspond to one partition when the entire pixel area of ​​image BB corresponding to virtual image AA is divided into 10 parts vertically and horizontally. In this case, with the original image BB having a horizontal dimension of 1920 pixels and a vertical dimension of 1080 pixels, one partition is 192 × 108 pixels. Figure 5 In the example of region BR2, regions AP1 and AP2 correspond to one partition when the entire pixel of the image BB corresponding to the virtual image AA is divided vertically into 10 parts and horizontally into 2 parts. In this case, with the original image BB having a horizontal dimension of 1920 pixels and a vertical dimension of 1080 pixels, one partition is 960×108 pixels.

[0089] The number of pixels in the left-right direction of the corrected first correction region SP1 is within ±10% of the number of pixels in the left-right direction of the corrected second correction region SP2. In this embodiment, regarding the trapezoidal shape of the virtual image AA, the distortion interval of the second correction region SP2 in the lateral direction varies approximately constant within a range of ±20 pixels, relative to the average length of the lateral direction of the first correction region SP1 and the second correction region SP2. Furthermore, the lateral distortion varies approximately constant within a range of ±20 pixels in the vertical direction of the virtual image AA. The aforementioned distortions present in the imaging optical system 20 of the first display device 100A are corrected by the display control device 88, described later, to cancel these distortions.

[0090] Figure 6 This is a schematic diagram illustrating a specific example of the distortion state of the corrected virtual image AA formed in the imaging optical system 20 of the first display device 100A. As shown in the figure, when the pixels of the original image BB corresponding to the virtual image AA are divided into 10 parts horizontally and vertically, in the first correction region SP1 and the second correction region SP2, the distortion is divided at intervals A. i，j This produces a distortion of the horizontal virtual image AA. The label 'i' indicates the column number, and the label 'j' indicates the row number. Interval A i，j Indicates the size after distortion.

[0091] Interval A i，j They are arranged at equal intervals to satisfy the following formula. When the top of the screen is the 5th row, the first correction area SP1 and the second correction area SP2 are arranged at intervals of A. -5，+5 ~A +5，+5 The intervals are arranged. At this time, when the length of the 5th row at the top of the screen is A'5, the following formula is satisfied.

[0092] A'5 / 10-20pix≤A i，5 ≤A'5 / 10+20pix

[0093] i = -5 to +5

[0094] Here, the value 20pix means 20 pixels.

[0095] In the example above, line 5 was described, but regarding the entire distorted shape, the following equation is satisfied.

[0096] A' j -20pix≤A i，j ≤A' j +20pix

[0097] i = -5 to +5

[0098] like Figure 6 As shown, when the entire pixel of the original image BB corresponding to the virtual image AA is divided into 10 parts vertically and horizontally, the coordinates of the virtual image AA during distortion are set to B. i，j At this point, when focusing on the distorted coordinates of the leftmost -5th column, assume B... -5，+5 -B -5，-5 =B' -5 That is, the value B' -5 Coordinate B -5，+5 With coordinate B -5，-5 The horizontal length between them, that is, the dimensional difference caused by distortion. At this time, the horizontal length B between the coordinates of adjacent rows in column -5. -5，j -B -5，j-1 The following expression is satisfied.

[0099] B' -5 / 10-20pix≤B -5，j -B -5，j-1 ≤B' -5 +20pix

[0100] j = -5 to +5

[0101] In the example above, column 5 was described, but for the entire distorted shape, the following formula is satisfied.

[0102] B' i / 10-20pix≤B i，j -B i，j-1 ≤B' i +20pix

[0103] j = -5 to +5

[0104] Figure 7 This is a diagram illustrating the relationship between the range of the incident / exit angles of the image light ML of the perspective lens 23 and the horizontal or vertical field of view. Figure 7 The image region CR1 is the position of the screen obtained by investigating the principal ray angle of the imaging optical system 20 of the first display device 100A, and corresponds to the field of view angle of the screen or virtual image. Figure 7 The region CR2 in the figure shows the difference between the incident and outgoing principal ray angles in the lateral direction of the perspective lens 23. It is a figure obtained by drawing the difference between the incident / outgoing angle and the lateral field of view of the perspective lens 23. Figure 7 The region CR3 in the figure shows the difference between the longitudinal incident and exit principal ray angles of the perspective lens 23, which is obtained by plotting the difference between the incident / exit angles of the perspective lens 23 and the longitudinal field of view. Figure 7 In regions CR2 and CR3 of the diagram, solid lines represent diagrams of the apparatus of this embodiment, and dashed lines represent diagrams of conventional apparatuses. The linearity of the approximate lines in the diagram is set as the coefficient of determination R. 2In this case, the coefficient of determination R 2 The closer the coefficient of determination is to 1, the higher the linearity and the better the fit. This is achieved by adjusting the coefficient of determination R0. 2 As the angle approaches 1, the difference between the incident and exit angles of the lens 23 changes constantly, and the longitudinal spacing changes constantly. Therefore, the shape of the intermediate image can be adjusted to eliminate longitudinal distortion.

[0105] In this embodiment, the shape of the intermediate image is controlled by controlling the principal ray angle of the longitudinal field of view of the perspective lens 23 and the principal ray angle of the lateral field of view. Thus, as... Figure 7 As shown in regions CR2 and CR3 of the figure, the lateral spacing of the distortion changes constantly, suppressing the lateral deviation relative to the longitudinal direction. Figure 7 As shown in regions CR2 and CR3 of the diagram, in conventional devices, the difference between the incident and exit angles of the lens 23 is nonlinear, while in the device of this embodiment, it is linear. Therefore, in the first display device 100A of this embodiment, the virtual image AA can be made to not produce longitudinal distortion. By sufficiently suppressing longitudinal distortion, it is not necessary to ensure the frame buffer memory when performing distortion correction by the display control device 88 (described later). Therefore, the computational load in the display control device 88 can be reduced when correcting distortion generated by the imaging optical system 20. This significantly reduces the burden of circuit correction in the display control device 88, thereby further reducing power consumption compared to conventional devices.

[0106] In existing devices, the configuration angle of the lens 23 is tilted by approximately 30° to 40° relative to the horizontal axis, resulting in longitudinal distortion. Therefore, during distortion correction in the circuit, the predetermined frame buffer needs to be ensured in the correction process of the display control device 88 described later. Consequently, there are problems such as the need for high power consumption to maintain the memory (non-volatile memory) and the need to develop a dedicated IC that can maintain a predetermined number of frame buffers and perform distortion correction.

[0107] In this embodiment, the tilt angle of the perspective lens 23 is increased, thereby suppressing eccentricity aberration and improving optical performance. Furthermore, the reflection angle of light rays in the perspective lens 23 forming the intermediate image is controlled, thus improving longitudinal distortion produced by conventional devices. Consequently, a frame buffer is unnecessary; therefore, distortion correction can be handled in the display control device 88 using inexpensive general-purpose integrated circuits such as FPGAs, and power consumption can be reduced.

[0108] Reference Figure 8The circuit system 70 of the HMD 200, i.e., the head-mounted display device 100, will be described. As the circuit system 70, the HMD 200 includes a display control device 88, a pair of display elements 11, and a user terminal circuit 91. One display element 11 is assembled in the first display device 100A, and the other display element 11 is assembled in the second display device 100B. The display control device 88 functions as a control unit. In the illustrated example, the display control device 88 is shown as being assembled in the first display device 100A, but it can also be independent of the first display device 100A and the second display device 100B. Furthermore, the device that combines either the first display device 100A or the second display device 100B with the display control device 88 is referred to as the head-mounted display device 100, which displays a virtual image for a single eye.

[0109] The display control device 88 has an arithmetic processing unit 81a, a storage unit 81m, and a data communication interface 81c.

[0110] The storage device 81m stores a program that causes the first display device 100A and the second display device 100B to perform display operations. Furthermore, the storage device 81m stores images obtained from the user terminal 90, which serves as an information terminal, images generated by the arithmetic processing unit 81a, and the like. Additionally, the storage device 81m includes a frame memory 83, which stores image data corresponding to the image data generated by the arithmetic processing unit 81a for output to the display elements 11. The image data output from the display control unit 88 to the pair of display elements 11 can be the same or different. The processing of the first display device 100A will be described below.

[0111] In addition, the storage device 81m has a non-volatile memory 87, in which various data such as parameters for image correction operations described later are stored.

[0112] The display control device 88 causes the display element 11 to perform display operations. The display element 11 includes an auxiliary circuit 85, which has a built-in scan driver and data driver around the periphery of the display surface 11a. When displaying each frame of image data, the display control device 88 outputs data signals corresponding to the image data stored in the frame memory 83, along with timing signals, to the auxiliary circuit 85 in units of scan lines. The auxiliary circuit 85 rewrites the display state of the display surface 11a based on the data signals input from the frame memory 83. The display control device 88 outputs image data line by line, in conjunction with... Figure 2 The x-direction corresponds to the scanning direction and is displayed on the display surface 11a of the display element 11. Additionally, Figure 2The x-direction corresponds to the horizontal or left-right direction of the display surface 11a. The image data stored in the frame memory 83 is incremented by one line each time in a frame of image. After the entire image is displayed on the display element 11, it is reset, and the image data for the next frame is stored in the same way.

[0113] The display control device 88 receives display data corresponding to the image data from the user terminal circuit 91 via the data communication interface 81c. The display control device 88 performs distortion correction processing on the display data or image data obtained from the user terminal circuit 91. The display control device 88 outputs the processed display data, i.e., the image data, stored in the frame memory 83 to the display element 11 via the data communication interface 81c.

[0114] Figure 2 The imaging optical system 20 shown is an off-axis optical system OS, which, as described above, can suppress longitudinal distortion, i.e., trapezoidal distortion. To compensate for this distortion, a distortion-corrected non-rectangular outline image needs to be formed on the display surface 11a of the display element 11.

[0115] Figure 9 This is a diagram illustrating the coordinate system before and after the distortion transformation performed by the display control device 88. Figure 9 The region DR1 in the image is the coordinate system in its normal state before distortion transformation, showing the initial image IM0. Figure 9 The region DR2 in the diagram is a coordinate system representing the display state in the display element 11 after distortion correction, showing the modified image IM1. The initial image IM0 is the original display image before distortion correction performed by the display control device 88. The modified image IM1 is the display image after distortion correction performed by the display control device 88. The modified image IM1 is an image obtained by correcting the distortion generated by the imaging optical system 20 in a manner where the virtual image formed on the eye's EY side is formed into a rectangle.

[0116] The display control device 88 performs various image processing operations including arithmetic processing, in which the image is compared with that from... Figure 9 The initial image IM0 of the rectangular contour corresponding to the input signal of the user terminal circuit 91 shown is transformed into a modified image IM1 of the non-rectangular contour to be displayed on the display surface 11a, and distortion or aberration correction is performed in this way.

[0117] The processing unit 81a acquires images or pictures from the user terminal 90 and stores them in the storage device 81m. The image is a display image shown by the head-mounted display device 100, specifically... Figure 9 The initial image shown is IM0.

[0118] like Figure 8As shown, the arithmetic processing unit 81a includes a correction unit 86 for distortion correction. The correction unit 86 is a correction circuit that generates a distorted image that compensates for the trapezoidal image distortion generated in the imaging optical system 20 of the first display device 100A and displays it on the display element 11. The correction unit 86 generates the distorted image through simple calculation processing and can adjust the distortion state through parameter adjustment. The correction unit 86, along with other circuitry constituting the display control unit 88, is mounted, for example, in a general-purpose FPGA.

[0119] Display control device 88 to enable Figure 9 The image is corrected such that the length D1 of the left-right direction in one side of the vertical direction of the image shown in region DR2 is shorter than the length D2 of the left-right direction in the other side of the vertical direction. Thus, for example, the shape of the image before correction is rectangular, and the corrected image displayed by display element 11 becomes an inverted trapezoidal shape that is wider at the top. Figure 9 In this example, the upper side or the +y direction side is designated as the other side, and the lower side or the -y direction side is designated as the other side. The virtual image before correction displayed by the imaging optical system 20 of the first display device 100A in this embodiment is trapezoidal in shape; therefore, by the correction unit 86... Figure 9 The inverse distortion correction shown in region DR2 of the image results in a rectangular virtual image after correction.

[0120] In the image correction performed by the display control device 88, and Figure 5 Similarly, the corrected image includes the corrected first corrected region SP1' obtained by correcting the first region AP1 of the uncorrected image (and...). Figure 5 The corrected second correction region SP2' is obtained by correcting the first correction region SP1 (corresponding to the first correction region SP1) and the second region AP2 (arranged in the left-right direction with the first region AP1 of the image before correction). Figure 5 (Corresponding to the second correction area SP2). The number of pixels in the left and right directions of the corrected first correction area SP1' is within ±10% of the number of pixels in the left and right directions of the corrected second correction area SP2'. Furthermore, as detailed later, although the image is flipped vertically and horizontally via the relay system of the imaging optical system 20, the rotation of the image displayed on the display screen of the display element 11 varies depending on the arrangement orientation of the display element 11 and can be appropriately changed according to the arrangement orientation.

[0121] In the arithmetic processing device 81a, the correction unit 86 includes a coordinate transformation unit 8a and a grayscale transformation unit 8b.

[0122] As part of image processing, the coordinate transformation unit 8a performs correction processing to correct the distortion of the initial image IM0, and... Figure 9The distorted modified image IM1 (hereinafter also referred to as the distorted initial image IM0) shown is stored in the storage device 81m. The coordinate transformation unit 8a converts the initial image IM0 into an image that is aligned with the original image IM0. Figure 2 The distortion generated by the imaging optical system 20, etc., is reversed and corrected. That is, the coordinate transformation unit 8a corrects the initial image IM0 to compensate for the distortion of the imaging optical system 20.

[0123] The grayscale conversion unit 8b converts the grayscale or brightness of each pixel corresponding to the coordinates corrected by the coordinate conversion unit 8a. That is, the grayscale conversion unit 8b adjusts the grayscale of the modified image IM1 after distortion correction by interpolation.

[0124] The following explains the processing of the coordinate transformation unit 8a. After distortion transformation, the xy coordinate system, with the center of the image or image before distortion transformation as the origin and a size of 1 pixel as 1, is transformed into the UV coordinate system as follows:

[0125] When the coordinates of the initial image IM0 before correction are set to y in the vertical direction, the coordinates of the initial image IM0 before correction are set to x in the horizontal direction, the coordinates of the modified image IM1 after correction are set to U in the horizontal direction, the coefficient corresponding to the length of the modified image IM1 after correction is set to a, and the coefficient corresponding to the angle formed by the vertical and horizontal edges of the modified image IM1 after correction is set to b, the coordinate transformation unit 8a of the display control device 88 corrects the initial image IM0 in a manner that satisfies the following formula.

[0126] U = ax + bxy

[0127] Additionally, coefficient 'a' corresponds to the relative lengths of the original lengths in the left and right directions. Furthermore, coefficient 'b' corresponds to the gradient of the non-parallel opposite sides of the trapezoid, i.e., the gradient of the waist or meridian of the trapezoid.

[0128] Furthermore, when the vertical coordinates of the corrected modified image IM1 are set to V, the following formula is satisfied.

[0129] V = y

[0130] Furthermore, when the coefficient corresponding to the distortion of the tilt of one side of the corrected modified image IM1 in the left and right directions and the tilt of the other side of the corrected modified image IM1 in the left and right directions is set to c, the coordinate transformation unit 8a of the display control device 88 corrects the image in a manner that satisfies the following formula.

[0131] U = ax + bxy + cy

[0132] In addition, the coefficient c corresponds to the imbalance of the inclination of the non-parallel opposite sides of the trapezoid, that is, the imbalance of the inclination of the waist or meridian of the trapezoid.

[0133] The coefficients a, b, and c above are parameters that can be rewritten.

[0134] Figure 10 This is a block diagram that partially illustrates the display control device 88 and display element 11 of the first display device 100A. In the coordinate transformation unit 8a of the correction unit 86 in the figure, the image signal of the first display device 100A for the right eye and the image signal of the second display device 100B for the left eye are received, and distortion transformation processing is performed on each image signal.

[0135] like Figure 10 As shown, coefficients a, b, and c are recorded in a non-volatile memory 87 that can be externally rewritten in the storage device 81m. The correction unit 86 obtains the coefficient values ​​recorded in the non-volatile memory 87 and performs distortion calculations. Specifically, the coordinate transformation unit 8a of the correction unit 86 receives input signals J1 and J2, respectively, for the image signals used by the first display device 100A and the second display device 100B. For each input signal J1 and J2, different coefficients a1, b1, c1 and a2, b2, c2 are used for calculation processing, and two output signals K1 and K2 are output with different distortion states. Alternatively, the correction unit 86 may also treat the input system as a single system instead of two separate systems and output two output signals with different distortion states.

[0136] The non-volatile memory 87 stores the values ​​of coefficients a1, b1, c1 for the right eye and coefficients a2, b2, c2 for the left eye. During distortion processing in the correction unit 86, these coefficients are obtained and calculated. Thus, the images for the right and left eyes can be output with different distortion states. The conversion coefficients recorded in the non-volatile memory 87 can be accessed externally via serial communication methods such as I2C and can be rewritten. Furthermore, the correction unit 86 performs interface conversion to convert the image signal to a signal suitable for the display element 11. For example, an FPGA can be used as the correction unit 86 in practical applications; specifically, the LIFCL-17-8MG121C from Lattice Corporation can be used. While this FPGA is a high-speed transmission chip, it occupies a small space of 6mm × 6mm, offering the advantage of low power consumption. However, due to its small scale for logical operations, it can only perform simple image processing. Therefore, the distortion conversion processing needs to be implemented using simple calculation formulas as described above.

[0137] like Figure 9 As shown, distortion transformation is, for example, an inverse transformation. In Figure 9In one example, the display element 11 is configured in a reverse configuration, with its upper side facing downwards relative to the imaging optical system 20 (i.e., in the -Y direction). The image displayed on the display element 11 is only keystone corrected, appearing as a trapezoid with the shorter upper side relative to the imaging optical system 20. In the reverse configuration, the image from the imaging optical system 20 is flipped vertically and horizontally via a relay system, resulting in a keystone-corrected virtual image seen by the wearer US, and the orientation of the displayed content is consistent with the original image. Alternatively, the distortion conversion can be a forward conversion. In this case, the display element 11 is configured in a forward configuration, with its upper side facing upwards relative to the imaging optical system 20 (i.e., in the +Y direction). The image displayed on the display element 11 is keystone corrected relative to the original image and displayed in a reversed vertically and horizontally manner. The image from the imaging optical system 20 is flipped vertically and horizontally via a relay optical system, resulting in a keystone-corrected virtual image seen by the wearer US, and the orientation of the displayed content is opposite to the original image.

[0138] Table 1 shows specific examples of parameters, illustrating the representative values ​​and fixed-point numbers used in the calculation. In the transformation formula of this embodiment, the number of multiplications is minimized to four, enabling coordinate transformation with a very simple formula. This allows for distortion conversion processing using a smaller, lower-power, and lower-cost correction circuit.

[0139] [Table 1]

[0140]

[0141] <Representative Value>

[0142] a = 0.84788

[0143] b = -0.00028791

[0144] c = -0.0055419

[0145] <Numerical range>

[0146] 0.7≤a≤1

[0147] -0.0005≤b≤0.0005

[0148] -0.2≤c≤0.2

[0149] <Functional Requirements>

[0150] a) The coefficients a, b, and c of the coordinate transformation calculation formula can be set separately for the left and right sides in a shared RGB manner.

[0151] (b) Coefficients a, b, and c can be directly adjusted according to the instructions of the correction unit 86. Regardless of the presence or absence of data in the non-volatile memory 87, they are used for parameter adjustment of each display element 11 after assembling the imaging optical system 20.

[0152] c) The coefficients a, b, and c are parameters stored in non-volatile memory 87. Non-volatile memory 87 contains data that is used after the parameters of each display element 11 are adjusted.

[0153] d) The coefficients a, b, and c used in the conversion can be confirmed from the user terminal 90 via serial communication.

[0154] e) Allows you to choose between distortion conversion and the original display.

[0155] Figure 11 The regions ER1 to ER4 in the diagram show the distortion state when the parameters of the image coordinate transformation are adjusted. The distortion state varies depending on the deviation of the lens of the imaging optical system 20. Therefore, by adjusting the coefficients a, b, and c, the distortion changes caused by the deviation of the imaging optical system 20 can be corrected.

[0156] Furthermore, in the coordinate transformation unit 8a, when converting a color image, it is converted into an image containing each wavelength and displayed on the display element 11 as follows.

[0157] Specifically, such as Figure 12 As shown, display element 11 displays a first image IMG containing a first wavelength, a second image IMB containing a second wavelength shorter than the first wavelength, and a third image IMR containing a third wavelength longer than the first wavelength. In this case, the left-right length E1 in the first image IMG is the length between the left-right length E2 in the second image IMB and the left-right length E3 in the third image IMR, and the up-down length F1 in the first image IMG is the same as the up-down length F2 in the second image IMB and the up-down length F3 in the third image IMR. Therefore, by changing the correction amount for each band, the display accuracy of the color image can be improved.

[0158] The processing in the grayscale conversion unit 8b will be described below. For example, linear interpolation is performed in the grayscale conversion unit 8b.

[0159] To export the grayscale values ​​of each pixel of display element 11 after the coordinate values ​​have been derived following distortion transformation, interpolation processing is required. The simplest method is nearest-neighbor interpolation. However, this method directly uses the grayscale value of the nearest coordinate, which can easily produce jagged edges and character distortion when displaying lines or characters. On the other hand, using cubic spline interpolation with the grayscale values ​​of four nearby points can display a smoother image, but the computation becomes more complex. In this embodiment, based on linear interpolation, N-segment linear interpolation is implemented, which allows for simple computation when exporting grayscale values.

[0160] The following is for reference Figure 13 This illustrates the calculation method used in linear interpolation. Figure 13 The diagram FR1 illustrates the relationship between the pixel center coordinates PM and the transformed coordinates TM in pixel PE of row i of display element 11. Figure 13 In the middle, the value Up i，j Represents the x-coordinate, value Vp i，j Represents the y-coordinate, value Rp i，j Gp represents the grayscale value of red. i，j This represents the grayscale value of green, value Bp. i，j This represents the grayscale value of blue. Interpolation is performed for each row and each wavelength.

[0161] Figure 13 For example, the image region FR2 illustrates the pixel center coordinates PM in the green image and the corresponding grayscale values ​​at the transformed coordinates TM. Figure 13 As shown in region FR2 of the image, for point B after distortion transformation i，j B i，j+1 grayscale value Gb i，j , Gb i，j+1 and coordinates B i，j (Ub i，j Vb i，j B i，j+1 (Ub i，j+1 Vb i，j+1 Regarding pixel P of display element 11 located between 2 points. i，j When set to grayscale value Gp i，j and coordinates (Up) i , j Vp i，j When ), the pixel P of display element 11 i，j grayscale value Gp i，j The following calculations are performed.

[0162] Gp i，j =Gb i，j +α(Gbi，j+1 -Gb i，j )

[0163] Among them, the value α is less than 1.

[0164] As described above, the correction unit 86, which is a distortion generation circuit, can achieve distortion processing and higher image quality through simpler processing, which helps to reduce circuit miniaturization, power consumption, and cost.

[0165] User terminal circuit 91 is assembled in user terminal 90 and includes main control device 91a, storage device 91m, data communication interface 91c, mobile wireless communication device 91t, and user interface device 91i. User terminal circuit 91 can communicate with various external devices such as servers via a communication network (not shown) using mobile wireless communication device 91t. Storage device 91m stores a basic program that enables user terminal circuit 91 to operate. Application software that operates on this basic program may include, for example, multiple application software such as a viewer for reproducing dynamic images and a web browser. User terminal circuit 91 operates in response to requests from user interface device 91i operated by the user. It outputs dynamic and static images stored in storage device 91m in association with application software to display control device 88 in a predetermined format, or obtains dynamic and static images corresponding to various content via mobile wireless communication device 91t and outputs the obtained display data to display control device 88 in a predetermined format. The above assumes that the display control device 88 performs distortion correction processing on the display data input from the user terminal circuit 91, but it is also possible that the user terminal circuit 91 performs distortion correction processing on the display data.

[0166] The following is for reference Figure 14 The image processing for displaying the display surface 11a of the display element 11 will be described.

[0167] First of all, Figure 8 In the display control device 88 shown, the arithmetic processing unit 81a obtains the initial image IM0 from the user terminal 90 via the data communication interface 81c as display data and stores it in the storage device 81m (step S11). The user terminal circuit 91 outputs the dynamic and static images stored in the storage device 91m to the display control device 88 in a predetermined format, or outputs the dynamic and static images obtained via the wireless communication device 91t of the mobile body to the display control device 88 in a predetermined format.

[0168] Next, in the display control device 88, the coordinate transformation unit 8a, which is the correction unit 86, performs distortion correction on the initial image IM0 through coordinate transformation (step S12). The coordinate transformation unit 8a performs coordinate transformation on the initial image IM0, thereby eliminating the distortion aberrations of the imaging optical system 20.

[0169] Next, in the display control device 88, the grayscale conversion unit 8b, which is the correction unit 86, and the arithmetic processing unit 81a perform grayscale adjustment on the initial image IM0 (step S13). The grayscale conversion unit 8b performs image processing such as grayscale adjustment on the image after coordinate transformation in step S12 to generate a modified image IM1. The modified image IM1 is recorded into the frame memory 83 of the storage device 81m every frame.

[0170] Finally, in the display control device 88, the arithmetic processing device 81a reads the image data of the modified image IM1 from the frame memory 83 of the storage device 81m line by line, such as Figure 8 As shown, one scan line SL is displayed on each display surface 11a of the display element 11 (step S14). The trapezoidal modified image IM1 displayed by the display element 11 is seen as a rectangular image in the virtual image formed by the imaging optical system 20. In this embodiment, the imaging optical system 20 generates an intermediate image in the middle, therefore, the virtual image is flipped vertically and horizontally relative to the initial image IM0.

[0171] The head-worn display device 100 of the first embodiment described above includes: a control unit 88 that corrects an image; a display element 11 that displays the image corrected by the control unit 88; a first optical component 22 that receives image light corresponding to the image; and a reflective component 23 that reflects the image light from the first optical component 22 and projects a virtual image corresponding to the image. The first optical component 22 and the reflective component 23 correct distortion in a first direction related to the virtual image, and the control unit 88 corrects the image based on distortion in a second direction that intersects the first direction and is related to the virtual image.

[0172] In the aforementioned head-mounted display device 100, the first optical component 22 and the reflective component 23 correct distortion in a first direction related to the virtual image, and the control unit 88 corrects the image based on distortion in a second direction related to the virtual image. Therefore, the control unit 88 does not need to perform distortion correction processing corresponding to the distortion in the first direction related to the virtual image; it only needs to correct distortion in one direction. Thus, the load on the control unit 88 related to image distortion correction can be reduced.

[0173] [Second Implementation]

[0174] The head-worn display device according to the second embodiment of the present invention will be described below. Furthermore, the head-worn display device of the second embodiment partially modifies the head-worn display device of the first embodiment; descriptions of common parts are omitted.

[0175] Figure 15 This is a side sectional view illustrating the internal structure of the head-worn display device 100 according to the second embodiment. (See attached image.) Figure 15 As shown, in the head-worn display device 100 of the second embodiment, in the imaging optical system 20 of the first display device 100A, the projection lens 21 is composed of two lenses. Specifically, the projection lens 21 includes a first lens 21o and a second lens 21p. In the second embodiment, [the following is omitted] Figure 2 The projection lens 21 of the first embodiment shown has a concave second lens 21p with achromatic function. The angles and arrangements of the optical elements of the imaging optical system 20 are the same as in the first embodiment. The first lens 21o and the second lens 21p are formed of resin, for example, but they may also be made of glass. In addition, in the second embodiment, the first lens 21o and the second lens 21p are preferably formed of resins with moderate refractive index and high Abbe number, such as cyclic olefin copolymers (COC) and ring-opening polymerized hydrogenated resins (COP).

[0176] (Example 2)

[0177] The following are examples of specific numerical values ​​for each variable.

[0178] θ1 = 36° (θ1' = 62°)

[0179] θ2 = 52° (θ2' = 70°)

[0180] θ3=10°

[0181] θ4=-4°

[0182] L1 = 35mm

[0183] L2 = 46mm

[0184] L3 = 26mm

[0185] Ln=21mm

[0186] L1cos(θ3+θ1) / L3cosθ4=0.93

[0187] Figure 16 This is a diagram illustrating the relationship between the range of the incident / exit angles of the image light ML of the perspective lens (synthesizer) 23 and the longitudinal field of view. Figure 16The image region GR1 is the position of the screen obtained by investigating the principal ray angle of the imaging optical system 20 of the first display device 100A, and corresponds to the field of view angle of the screen or virtual image. Figure 16 The region GR2 in the diagram shows the difference between the longitudinal incident and exit principal ray angles of the perspective lens 23. It is a diagram obtained by plotting the difference between the incident / exit angles of the perspective lens 23 and the longitudinal field of view. Figure 16 In the GR2 area of ​​the figure, the solid line shows the diagram of the device of this embodiment, and the dashed line shows the diagram of the conventional device. Figure 17 This is a schematic diagram illustrating a specific example of the distortion state of the virtual image AA formed in the imaging optical system 20 of the first display device 100A in the second embodiment.

[0188] In this embodiment, the shape of the intermediate image is controlled by controlling the principal ray angle of the longitudinal field of view of the perspective lens 23, thereby, as Figure 16 The map region GR2 and Figure 17 As shown, in the first display device 100A of this embodiment, the virtual image AA is prevented from producing longitudinal distortion. By suppressing longitudinal distortion, it is not necessary to ensure the frame buffer memory when the display control device 88 performs distortion correction. As a result, the computational load on the display control device 88 can be reduced when correcting distortion generated by the imaging optical system 20. This significantly reduces the burden of circuit correction in the display control device 88, thereby reducing power consumption compared to conventional devices.

[0189] In addition, such as Figure 17 As shown, in the imaging optical system 20 of the second embodiment, compared with the imaging optical system 20 of the first embodiment, the lateral spacing of the distortion and the lateral deviation of the lateral relative to the longitudinal direction are not sufficiently suppressed. Therefore, in the first correction region SP1 and the second correction region SP2 of the image, even in the same row, the spacing A i，j They are also different; even within the same column, the horizontal length B' between the coordinates of adjacent rows is different. i They are different.

[0190] [Variations and others]

[0191] The present invention has been described above in conjunction with the embodiments, but the present invention is not limited to the embodiments described above. It can be implemented in various ways without departing from its spirit, for example, the following modifications can also be made.

[0192] In the first display device 100A, it may also be configured to include an imaging optical system that is asymmetrical in the lateral direction, with distortion correction being asymmetrical in the lateral direction, i.e., the X direction. In this case, the first optical component and the reflective component correct the distortion in the left-right direction related to the virtual image, and the control unit corrects the image based on the distortion in the vertical direction.

[0193] In the first display device 100A, the projection lens 21 is a lens structure with 3 or 2 lenses, but it can also be a lens structure with 1 or more lenses. Alternatively, the first display device 100A can be configured without the projection lens 21.

[0194] A dimming device that adjusts the light transmitted through the lens 23 by limiting its transmission can also be installed on the outer side of the lens 23. The dimming device can, for example, adjust the transmittance electrically. Mirrored liquid crystals, electronic light shields, etc., can be used as dimming devices. The dimming device can also adjust the transmittance according to the external illuminance.

[0195] In the first display device 100A, a wedge-shaped optical element 28 for improving the imaging state can also be provided between the prism reflector 22 and the lens 23. For example, by making the refractive index of the wedge-shaped optical element 28 different from that of the prism reflector 22, the degree of refraction and dispersion can be adjusted between the wedge-shaped optical element 28 and the prism reflector 22, thereby easily achieving achromatic aberration.

[0196] The head-worn display device in a specific embodiment includes: a control unit that corrects an image; a display element that displays the image corrected by the control unit; a first optical component that receives image light corresponding to the image; and a reflective component that reflects the image light from the first optical component and projects a virtual image corresponding to the image. The first optical component and the reflective component correct distortion in a first direction related to the virtual image, and the control unit corrects the image based on distortion in a second direction intersecting the first direction related to the virtual image. Here, the distortion in the first direction related to the virtual image is not present in the head-worn display device of the present invention due to the correction performed by the first optical component and the reflective component. Furthermore, the remaining distortion present in the virtual image is caused by distortion generated by the first optical component and the reflective component, and is canceled out by distortion correction in the control unit, making it invisible to the eye.

[0197] In the aforementioned head-mounted display device, the first optical component and the reflective component correct distortion in a first direction related to the virtual image, and the control unit corrects the image based on distortion in a second direction related to the virtual image. Therefore, the control unit does not need to perform distortion correction processing corresponding to the distortion in the first direction related to the virtual image; it only needs to correct distortion in one direction. This reduces the load on the control unit related to image distortion correction.

[0198] In a specific aspect, the second direction is the direction corresponding to the scanning direction of the display element.

[0199] In a specific aspect, the control unit corrects the image in such a way that the length of the second direction in one of the first directions of the image is shorter than the length of the second direction in the other of the first direction of the image. In this case, for example, the shape of the image before correction is rectangular, and the corrected image displayed by the display element is trapezoidal.

[0200] In a specific aspect, if the coordinates of the first direction of the image before correction are y, the coordinates of the second direction of the image before correction are x, the coordinates of the second direction of the image after correction are U, the coefficient corresponding to the length of the second direction of the image after correction is a, and the coefficient corresponding to the angle formed by the edge of the first direction and the edge of the second direction in the image after correction is b, the control unit corrects the image in a manner that satisfies the following formula: U=ax+bxy.

[0201] In a specific aspect, if the coefficient corresponding to the distortion of the tilt of one side of the second direction in the corrected image and the tilt of the other side of the second direction in the corrected image is c, the control unit corrects the image in a manner that satisfies the following formula: U=ax+bxy+cy.

[0202] In specific terms, coefficients a, b, and c are parameters that can be rewritten.

[0203] In a specific embodiment, the display element displays a first image containing a first wavelength, a second image containing a second wavelength shorter than the first wavelength, and a third image containing a third wavelength longer than the first wavelength. The length in the second direction of the first image is the length between the length in the second direction of the second image and the length in the second direction of the third image, and the length in the first direction of the first image is the same as the length in the first direction of the second image and the length in the first direction of the third image. In this case, by changing the correction amount for each band, the display accuracy of the color image can be improved.

[0204] In a specific aspect, the corrected image includes a corrected first corrected region obtained by correcting a first region of the image before correction, and a corrected second corrected region obtained by correcting a second region of the image before correction that is parallel to the first region in a second direction. The number of pixels in the second direction of the corrected first corrected region is the number of pixels within ±10% of the number of pixels in the second direction of the corrected second corrected region.

[0205] Specifically, the reflecting element is configured such that the angle θ1 formed by the image light incident from the first optical element to the reflecting element and the image light reflected by the reflecting element towards the pupil is 10°≤θ1≤70°. In this case, by making the angle θ1 within the above range, the placement angle of the reflecting element is relatively large, which can suppress eccentric aberration and improve optical performance. Furthermore, the distance between the reflecting element and the first optical element can be made relatively small, thus enabling miniaturization of the optical system in the thickness direction, i.e., the lateral section direction.

[0206] In a specific aspect, the first optical component includes: an incident portion into which image light from a display element is incident; an exit portion into which the incident image light is exited toward a reflective component; and a reflective portion into which image light incident from the incident portion is reflected toward the exit portion.

[0207] In a specific aspect, the first optical component is configured such that the angle θ2 formed by the image light incident on the reflector and the image light reflected by the reflector is 30°≤θ2≤90°. In this case, by making the angle θ2 within the above range, the arrangement angle of the first optical component is relatively large, and the size of the optical system can be miniaturized in the height direction, i.e., the vertical direction or the longitudinal direction.

[0208] In a specific context, if the angle between the horizontal direction and the position of the image light reflected by the reflecting component toward the pupil is θ3, the optical path length of the image light incident from the first optical component to the reflecting component is L1, and the optical path length of the image light reflected by the reflecting component to the position of the pupil is L2, then the horizontal distance Ln from the pupil to the first optical component satisfies the following formula: Ln=L2cosθ3-L1cos(θ3+θ1).

[0209] In a specific aspect, the display element is configured such that the angle θ4 formed between the horizontal direction and the image light incident on the reflective portion of the first optical component is θ4≤10°.

[0210] In a specific aspect, the head-worn display device has a second optical component disposed between the first optical component and the display element.

[0211] In a specific aspect, the head-worn display device includes: a support member that supports a display element, a first optical element, and a reflective element; and a wearing member that is connected to the support member and worn on the user's head.

Claims

1. A head-worn display device, comprising: Control unit, its image correction; A display element displays the image corrected by the control unit; A first optical component, which provides incident image light corresponding to the image; and A reflective component reflects the image light from the first optical component and projects a virtual image corresponding to the image. The first optical component and the reflective component correct the distortion in the first direction associated with the virtual image. The control unit corrects the image based on the distortion in a second direction that intersects the first direction and is related to the virtual image. The second direction is the direction corresponding to the scanning direction of the display element. Let y be the coordinate of the image before correction in the first direction, x be the coordinate of the image before correction in the second direction, U be the coordinate of the image after correction in the second direction, a be the coefficient corresponding to the length of the image after correction in the second direction, b be the coefficient corresponding to the angle formed by the edge in the first direction and the edge in the second direction in the image after correction, and c be the coefficient corresponding to the distortion of the tilt of one side of the second direction and the tilt of the other side of the second direction in the image after correction. The coefficient c includes the case where it is 0. The control unit corrects the image in a manner that satisfies the following formula. U = ax + bxy + cy.

2. The head-mounted display device according to claim 1, wherein, The control unit corrects the image in such a way that the length in the second direction of one of the first directions in the image is shorter than the length in the second direction of the other of the first direction in the image.

3. The head-mounted display device according to claim 1, wherein, The coefficients a, b, and c are rewriteable parameters.

4. The head-mounted display device according to any one of claims 1 to 3, wherein, The display element displays a first image containing a first wavelength, a second image containing a second wavelength shorter than the first wavelength, and a third image containing a third wavelength longer than the first wavelength. The length in the second direction in the first image is the length between the length in the second direction in the second image and the length in the second direction in the third image. The length in the first direction of the first image is the same as the length in the first direction of the second image and the length in the first direction of the third image.

5. The head-mounted display device according to any one of claims 1 to 3, wherein, The corrected image includes a corrected first corrected region obtained by correcting a first region of the uncorrected image, and a corrected second corrected region obtained by correcting a second region of the uncorrected image that is parallel to the first region in the second direction. The number of pixels in the second direction of the first corrected region is the number of pixels within ±10% of the number of pixels in the second direction of the second corrected region.

6. The head-mounted display device according to any one of claims 1 to 3, wherein, The reflective component is configured such that the angle θ1 formed by the image light incident from the first optical component onto the reflective component and the image light reflected by the reflective component toward the pupil is 10°≤θ1≤70°.

7. The head-mounted display device according to any one of claims 1 to 3, wherein, The first optical component includes: an incident portion for receiving image light from the display element; an exit portion for receiving the incident image light toward the reflective component; and a reflective portion for reflecting the image light incident from the incident portion toward the exit portion.

8. The head-mounted display device according to claim 7, wherein, The first optical component is configured such that the angle θ2 formed by the image light incident on the reflective portion and the image light reflected by the reflective portion is 30°≤θ2≤90°.

9. The head-mounted display device according to claim 7, wherein, Given that the angle formed by the image light reflected from the reflective component towards the pupil in the horizontal direction is θ3, the optical path length of the image light incident from the first optical component to the reflective component is L1, and the optical path length of the image light reflected from the reflective component to the pupil is L2, the horizontal distance Ln from the pupil to the first optical component satisfies the following formula. Ln=L2cosθ3-L1cos(θ3+θ1).

10. The head-mounted display device according to claim 7, wherein, The display element is configured such that the angle θ4 between the horizontal direction and the image light incident on the reflective portion of the first optical component is θ4≤10°.

11. The head-mounted display device according to any one of claims 1 to 3, wherein, The head-worn display device has a second optical component disposed between the first optical component and the display element.

12. The head-mounted display device according to any one of claims 1 to 3, wherein, The head-mounted display device has: Support member, which supports the display element, the first optical component, and the reflective component; and A wearable component, which is connected to the support component, is worn on the user's head.

Images