Eye movement tracking device and head-mounted display device

Abstract

The invention relates to the technical field of optical display, and particularly provides an eye movement tracking device and a head-mounted display device.The eye movement tracking device comprises a light source, a hot mirror, an optical waveguide structure and a camera module; the light source is used for emitting light to eyeballs of a user, and the light is reflected by the eyeballs to form first light emitted to the hot mirror; the hot mirror is arranged in the optical waveguide structure, the hot mirror is used for reflecting the first light into second light, and the second light is configured to be emitted to a camera module through the optical waveguide structure; the camera module is used for receiving the second light. Through the hot mirror arranged in the center of the optical waveguide structure, light from eyeballs can be reflected and guided into the camera module. By means of the design, the installation mode of a traditional infrared camera is changed, the infrared camera can be installed in an area far away from human eyes, and therefore the shielding problem is solved.

Description

Technical Field

[0001] The present invention relates to the field of optical display technology, and specifically provides an eye tracking device and a head-mounted display device. Background Art

[0002] Eye tracking technology is a scientific application technology. First, it tracks according to the changes in the characteristics of the eye and its surrounding areas. Second, it tracks according to the changes in the angle of the iris. Third, it actively projects infrared light beams or other light beams to the iris to extract features.

[0003] Eye tracking technology has been applied in many fields, including medical research, VR / AR, etc. Eye tracking can achieve dynamic rendering and optimize visual experience in VR. Eye tracking technology in virtual reality (VR) can bring image quality optimization, user experience innovation, more accurate research methods and more adaptable training programs.

[0004] With the application of the reentrant pancake optical solution in VR / AR, the eye tracking technology has the following problems: 1. Due to the short pupil distance of the pancake, the human eye captured by the infrared camera will be blocked by the eyelashes, affecting the shooting effect; 2. The infrared camera is set at the edge of the screen, mainly using large-angle light to pass through the edge of the lens or close to the edge of the lens, and then received by the infrared camera. This solution is prone to the lens edge blocking the camera's picture due to poor camera installation accuracy. In addition, the space of the optical path design of the reentrant pancake solution is extremely compressed, and the original eye tracking optical solution is difficult to apply. Therefore, the current reentrant pancake eye tracking urgently needs an optical solution that can solve or partially solve the human eye occlusion and lens occlusion. Summary of the invention

[0005] In order to overcome the above defects, the present invention proposes an eye tracking device and a head-mounted display device. Since the infrared camera is no longer placed on the side of the lens barrel close to the human eye, occlusion of the human eye and lens is avoided.

[0006] In a first aspect, the present invention provides an eye tracking device, comprising: a light source, a thermal mirror, an optical waveguide structure and a camera module; wherein:

[0007] The light source is used to emit light toward the eyeball of the user, and the light is reflected by the eyeball to form a first light beam directed toward the thermal mirror;

[0008] The heat mirror is arranged in the optical waveguide structure, and the heat mirror is used to reflect the first light into a second light, and the second light is configured to be emitted to the camera module via the optical waveguide structure;

[0009] The light incident side of the camera module faces the thermal mirror, and the camera module is used to receive the second light.

[0010] Furthermore, the camera module comprises: a camera and at least one first lens; wherein, along the light emitting direction away from the thermal mirror, the at least one first lens and the camera are arranged in sequence.

[0011] Furthermore, the center of the heat mirror, the center of the optical waveguide structure and the center of the human eye are located on the same straight line.

[0012] Furthermore, the device also includes: a reflector for changing the propagation direction of the second light; wherein, along the light emitting direction away from the thermal mirror, the reflector, at least one first lens and the camera are arranged in sequence.

[0013] Furthermore, the light source includes an infrared light source for emitting infrared light in the range of 800 to 1100 nm; and the camera module is an infrared camera module.

[0014] In a second aspect, the present invention provides a head mounted display device, comprising:

[0015] Display module, capable of displaying images;

[0016] A lens group, comprising a plurality of lenses, wherein the lens group can adjust the clarity of the image projected by the display module into the human eye; and

[0017] According to the eye tracking device of the first aspect, the optical waveguide structure of the eye tracking device is located between any two adjacent lenses in the lens group; or

[0018] The optical waveguide structure is located between the lens group and the display module.

[0019] Furthermore, the light source of the eye tracking device is arranged on a side of the lens in the lens group that is adjacent to the user's eye and close to the user's eye.

[0020] Furthermore, the optical waveguide structure is arranged perpendicular to a line connecting the centers of all lenses of the lens group.

[0021] Furthermore, the angle between the heat mirror and the line connecting the centers of all lenses of the lens group is 44.5°-45.5°.

[0022] Furthermore, the lens group includes: a second lens and a third lens, wherein the third lens, the optical waveguide structure, the second lens and the light source are arranged in sequence along a direction away from the display module.

[0023] Further, the lens group is a pancake display lens group, and the lens group further comprises: a semi-transmissive and semi-reflective film, a phase delay film and a reflective polarizing film, wherein the phase delay film and the reflective polarizing film are stacked and arranged on a side of the second lens away from the display module or a side of the optical waveguide structure away from the display module in a direction away from the display module;

[0024] The semi-transmissive and semi-reflective film is arranged on a side of the third lens close to the display module.

[0025] Furthermore, the camera module includes: a camera and a first lens.

[0026] Furthermore, the total focal length f of the optical system formed by the display module and the lens group is 29 mm to 29.5 mm, and the total focal length f' of the optical system formed by the eye tracking device is 18 mm to 18.5 mm.

[0027] Further, the second lens includes a first surface close to the display module and a second surface away from the display module, the optical waveguide structure includes a third surface close to the display module and a fourth surface away from the display module and a seventh surface facing the first lens, the third lens includes a fifth surface close to the display module and a sixth surface away from the display module, and the first lens includes an eighth surface facing the optical waveguide structure and a ninth surface away from the optical waveguide structure; wherein,

[0028] The first surface is aspherical, and the second surface is a plane; the radius of curvature of the first surface is -72mm to -73mm; the thickness of the second lens is 5.5 to 6.5mm, the refractive index is 1.54, and the focal length f1 is 133.5mm to 134mm;

[0029] The thickness of the optical waveguide structure is 3mm to 6.5mm, and the material is resin with a refractive index of 1.53;

[0030] The hot mirror is an aspherical mirror with a curvature radius of 43mm to 44mm and a focal length f2 of -14mm to 13.5mm;

[0031] The distance between the fourth surface and the first surface is 0.1 mm; the distance between the third surface and the sixth surface is 0.1 mm; the distance between the fifth surface and the display module is 6 mm to 6.5 mm, the distance between the eighth surface and the seventh surface is 0.5 mm to 1 mm, and the distance between the eighth surface and the center of the thermal mirror is 34.5 mm to 35.5 mm;

[0032] The first lens is a double-sided aspherical mirror, the curvature radius of the eighth surface is -19.5mm to 20.5mm; the curvature radius of the ninth surface is 24.5mm to 25.5mm; the focal length f3 of the first lens is 21mm to 21.5mm, the thickness is 3mm to 4mm, the material is resin, the refractive index is 1.53, and the distance between the ninth surface and the camera is 3mm to 5mm.

[0033] Furthermore, the relationship between the focal lengths of the second lens, the third lens and the thermal mirror is:

[0034] -1.5 <f3 / f2<-1;

[0035] -16.5 <f2*f3 / f’<-16;

[0036] 7 <f1 / f’<7.5。

[0037] Furthermore, the lens group includes: a fourth lens, a fifth lens and a sixth lens, wherein the sixth lens, the optical waveguide structure, the fifth lens, the fourth lens and the light source are arranged in sequence along a direction away from the display module.

[0038] The lens group is a pancake display lens group, and the lens group further includes: a semi-transmissive and semi-reflective film, a phase delay film and a reflective polarizing film; wherein,

[0039] The phase delay film and the reflective polarizing film are stacked and arranged on the side of the fourth lens away from the display module or the side of the optical waveguide structure away from the display module in a direction away from the display module;

[0040] The semi-transmissive and semi-reflective film is arranged on the side of the sixth lens close to the display module.

[0041] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0042] In the technical solution of the present invention, the reflected light from the human eye is reflected by the thermal mirror, which transforms the incident light by nearly 90 degrees. The light beam passing through the thermal mirror then enters the optical waveguide structure, which is responsible for guiding the light beam to the target area and can effectively control the propagation direction of the light beam. Finally, the camera module is responsible for capturing the infrared light passing through the optical waveguide structure, thereby realizing the accurate recording and tracking of the trajectory of human eye activities.

[0043] Compared with the existing devices, the human eye tracking device of the present invention eliminates blind spots in shooting, and effectively improves the fidelity and accuracy of shooting during the tracking process.

[0044] The VR optical solution applicable to the present invention can be either a traditional straight-through solution or a return pancake solution, and has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which:

[0046] Figure 1 is a schematic diagram of the principle of an eye tracking device according to an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram of the main structure of a head mounted display device including a second lens and a third lens according to an embodiment of the present invention;

[0048] Figure 3 is the MTF & spatial frequency 20lp curve of the infrared system composed of the eye tracking device according to one embodiment of the present invention;

[0049] Figure 4 is an infrared system MTF & spatial frequency 40lp curve according to an embodiment of the present invention;

[0050] Figure 5 is a field curvature test result of an infrared system composed of an eye tracking device according to an embodiment of the present invention;

[0051] Figure 6 is a schematic diagram of spectral data of a reflective thermal mirror of an optical waveguide structure according to an embodiment of the present invention;

[0052] Figure 7 is a schematic structural diagram of an optical waveguide structure located between the lens group and the display module according to an embodiment of the present invention;

[0053] Figure 8 is a schematic diagram of a three-piece structure of a straight-through optical solution according to an embodiment of the present invention;

[0054] Fig. 9 is a schematic diagram of a three-piece structure of a folded pancake optical solution according to an embodiment of the present invention;

[0055] Fig.10 is a schematic diagram of a structure provided with a reflector according to an embodiment of the present invention;

[0056] Fig.11 is a schematic diagram of an optical scheme of a field angle inscribed module according to an embodiment of the present invention;

[0057] Fig.12 is a schematic structural diagram of Comparative Example 1 according to the present invention;

[0058] Fig.13is a schematic structural diagram of Comparative Example 2 according to the present invention;

[0059] Fig.14 It is a structural schematic diagram of Comparative Example 3 according to the present invention.

[0060] Reference numerals list :

[0061] 1 eyeball; 2 second surface; 3 second lens; 4 light source; 5 optical waveguide structure; 6 thermal mirror; 7 third lens; 8 display module; 9 first lens; 10 camera; 11 fourth lens; 12 fifth lens; 13 sixth lens; 14 reflector; 15 seventh surface. DETAILED DESCRIPTION

[0062] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0063] The present invention discloses an eye tracking device, referring to Figure 1 , including: a light source 4, a thermal mirror 6, an optical waveguide structure 5 and a camera module; wherein,

[0064] The light source 4 is used to emit light to the eyeball 1 of the user, and the light is reflected by the eyeball 1 to form a first light emitted to the thermal mirror 6;

[0065] The heat mirror 6 is disposed in the optical waveguide structure 5, and the heat mirror 6 is used to reflect the first light into a second light, and the second light is configured to be emitted to the camera module via the optical waveguide structure 5;

[0066] The light incident side of the camera module faces the thermal mirror 6 , and the camera module is used to receive the second light.

[0067] The core components of the device are described below.

[0068] 1. Light source

[0069] The light emitted by the light source 4 is a detection light, which is used to detect the movement track and activity range of the human eye. Usually, the light emitted by the light source 4 is infrared light.

[0070] Light source 4 can be an infrared fill light.

[0071] The light source 4 emits light to the eyeball 1 of the user, and the light is reflected by the eyeball 1 to form a first light beam directed to the thermal mirror 6 , wherein the first light beam contains a reflected light spot generated on the cornea of ​​the human eye.

[0072] 2. Thermal mirror

[0073] The heat mirror 6 allows visible light to pass through and reflects infrared light. The first light is reflected after passing through the surface of the heat mirror 6, thus changing the propagation direction of the light, and the reflected light is the second light.

[0074] 3. Optical waveguide structure

[0075] The material of the optical waveguide structure 5 is resin, and it can be set to have a specific thickness according to the actual application requirements. The interior of this structure is designed to place the thermal mirror 6. When the first light contacts the surface of the thermal mirror 6, the light will be totally reflected. Then, the reflected light is emitted along the path of the optical waveguide structure 5. The optical waveguide structure 5 can guide the propagation of light waves therein to ensure that the light can move along a predetermined path. In addition, the use of the optical waveguide structure 5 also avoids the optical path of the infrared light in the eye tracking device from interfering with the optical path of the original optical system, thereby ensuring the stability and accuracy of the entire system.

[0076] 4. Camera module

[0077] The camera module receives the second light and then forms an image to achieve tracking and positioning of the human eye.

[0078] In one embodiment, still referring to Figure 1 The camera module includes: a camera 10 and at least one first lens 9; wherein, along the light emitting direction away from the heat mirror 6, the at least one first lens 9 and the camera 10 are arranged in sequence. The number of first lenses 9 is at least one, but according to actual application requirements and design considerations, one or more first lenses 9 can be selected. Regardless of the number of one or more, the function of the first lens 9 is to converge the second light to effectively converge and guide the second light from different directions to the camera 10, Figure 1 A first lens 9 is used as an example. When the human eyeball rotates and moves, the first lens significantly improves the collection efficiency of the detection light, so that the accuracy of the eyeball tracking function in the camera imaging is effectively improved.

[0079] The second light is received by the lens of the camera 10, which focuses the light and then allows the light to enter the camera through the lens. The light from different directions is recorded by the photosensitive element, i.e., the image sensor, in the camera 10, and the image is formed by the image processor in the camera 10. In this way, the camera 10 obtains the movement trajectory of the human eye by collecting the reflected light spots generated on the cornea of ​​the human eye contained in the second light. Finally, by analyzing the reflected light spots in the image, the camera 10 can track and record the movement path of the human eye.

[0080] The eye tracking device involved in the present invention can effectively reflect and guide light from different parts and angles of the eyeball to the infrared camera of the camera module through the thermal mirror 6 set at the center of the optical waveguide structure 5. This innovative design changes the installation method of traditional infrared cameras, allowing the infrared camera to be installed in an area far away from the human eye, thereby effectively solving the occlusion problem that may be caused by improper equipment position. In addition, the light reflected by the thermal mirror 6 can not only be transmitted directly through the optical waveguide structure 5, but also undergo multiple total reflections inside the optical waveguide structure 5, and finally be exported through the optical waveguide structure 5 and received by the camera. Such a design greatly improves the efficiency of light transmission and the stability of the system.

[0081] In one embodiment, a heat mirror having a film coated on its surface for reflecting infrared light is bonded to the optical waveguide by gluing.

[0082] In one embodiment, the center of the heat mirror 6 , the center of the optical waveguide structure 5 , and the center of the human eye are located on the same straight line.

[0083] from Figure 1 It can be seen that the present invention can realize direct shooting of the human eye. The center of the thermal mirror 6 and the center of the human eye are located in the same horizontal plane. The reflected light from the human eye is reflected by the thermal mirror 6. The thermal mirror 6 transforms the incident light by nearly 90 degrees, thereby directing the infrared light from the original horizontal propagation path to the vertical propagation path. The horizontal direction is also used to arrange the imaging display device, and the optical waveguide structure guides the light path and suppresses the disorderly emission of light.

[0084] In one embodiment, the light source includes an infrared light source for emitting infrared light in the range of 800 to 1100 nm; and the camera module is an infrared camera module.

[0085] Reference Figure 2 , Figure 7 and Figure 8-Figure 10 The present invention also provides a head mounted display device, comprising:

[0086] The display module 8 can display images and can be an OLED.

[0087] A lens group, comprising a plurality of lenses, wherein the lens group can adjust the clarity of the image projected by the display module into the human eye; and

[0088] Reference Figure 2 According to the eye tracking device, the optical waveguide structure 5 of the eye tracking device is located between any two adjacent lenses in the lens group; or

[0089] Reference Figure 7, the optical waveguide structure 5 is located between the lens group and the display module 8, that is, the optical waveguide structure 5 is located on the side of the lens adjacent to the display module 8 in the lens group close to the display module 8. From the human eyeball 1 to the display module 8, the second lens 3, the third lens 7 and the optical waveguide structure 5 are arranged in sequence.

[0090] In one embodiment, the light source 4 of the eye tracking device is arranged on a side of the lens of the lens group adjacent to the user's eye, which is close to the eyeball 1 of the user's eye.

[0091] In one embodiment, the optical waveguide structure is arranged perpendicular to a line connecting the centers of all lenses of the lens group. The longitudinal center line of the optical waveguide structure is perpendicular to a line connecting the centers of all lenses of the lens group.

[0092] In one embodiment, the optical waveguide structure is square, with the long side in the vertical direction.

[0093] In one embodiment, the angle between the heat mirror and the line connecting the centers of all lenses of the lens group is 44.5°-45.5°.

[0094] When all lenses of the lens group are arranged horizontally, the angle between the thermal mirror and the vertical direction is 44.5°-45.5°.

[0095] In one embodiment, the angle between the heat mirror and the line connecting the centers of all lenses of the lens group is 45°.

[0096] When all lenses of the lens group are arranged horizontally, the angle between the hot mirror and the vertical direction is 45°.

[0097] In one embodiment, referring to Figure 2 The lens group includes: a second lens 3 and a third lens 7, wherein along the direction away from the display module 8, the third lens 7, the optical waveguide structure 5, the second lens 3 and the light source 4 are arranged in sequence.

[0098] The VR optical solution applicable to the lens group can be either a traditional straight-through solution or a fold-back pancake solution.

[0099] In one embodiment, when the eye tracking device uses a VR light system with a reentrant pancake solution, the lens group is a pancake display lens group, and the lens group also includes: a semi-transparent and semi-reflective film, a phase delay film, and a reflective polarizing film.

[0100] The semi-transmissive and semi-reflective film, the phase delay film and the reflective polarizing film enable the optical path between the lens group and the display module 8 to form a pancake folded optical path.

[0101] Wherein, the phase delay film and the reflective polarizing film are stacked and arranged on the side of the second lens away from the display module or the side of the optical waveguide structure away from the display module in a direction away from the display module;

[0102] The semi-transmissive and semi-reflective film is arranged on a side of the third lens close to the display module.

[0103] In one embodiment, the phase retardation film is a quarter wave plate.

[0104] In the present invention, the optical waveguide structure 5 not only assumes the important responsibility of accurately guiding the optical path, but also provides a surface for film material to be attached. The film material can be attached to the outer surface of the optical waveguide structure 5, so that the plane of the optical waveguide structure 5 can be used as the film-attaching surface of the Pancake lens. In this way, the Pancake lens does not need to reserve a separate film-attaching plane, which is equivalent to adding an additional surface that can be coated. This design significantly improves the flexibility and freedom of the entire design process, and can perform optical design and layout more freely.

[0105] In one embodiment, the film assembly attached to the side of the second lens away from the display module or the side of the optical waveguide structure away from the display module can also be a four-in-one film, including AR, QWP, reflective polarizer (IQPS / IQPE) and absorptive polarizing film POL. AR is an anti-reflection layer or anti-reflection film, and its main function is to reduce the reflection of light on the surface of the display panel, improve the clarity and brightness of the display image, reduce glare, and thus enhance the user's visual experience. QWP is a quarter-wave plate, and its main function is to modulate the phase of the light wave so that the light wave passing through it produces a phase delay of π / 2 (i.e., 90 degrees) in a specific direction. The reflective polarizer uses a multilayer film to reflect light, so that the phase difference between the reflected light and the transmitted light is 180 degrees, thereby achieving a polarizing effect. The absorptive polarizer absorbs part of the absorbed light, so that the phase difference between the transmitted light and the absorbed light is 90 degrees, thereby achieving a polarizing effect.

[0106] In one embodiment, the second lens is flat on one side close to the human eye and convex on the other side, and the four-in-one film is arranged on the plane of the second lens. AR, absorption polarizing film POL, reflective polarizer (IQPS / IQPE) and QWP are stacked in sequence in the direction away from the human eye.

[0107] In one embodiment, the second lens has a convex surface on one side close to the human eye and a flat surface on the other side, and the four-in-one film is disposed on the flat surface of the second lens. AR, QWP, reflective polarizer (IQPS / IQPE) and POL are stacked in sequence in a direction away from the human eye.

[0108] In one embodiment, the camera module includes: a camera 10 and a first lens 9.

[0109] In one embodiment, if Figure 2 The optical system, the second lens 3 includes a first surface close to the display module 8 and a second surface 2 away from the display module 8, the optical waveguide structure 5 includes a third surface close to the display module 8 and a fourth surface away from the display module 8 and a seventh surface 15 facing the first lens 9, the third lens 7 includes a fifth surface close to the display module 8 and a sixth surface away from the display module, the first lens 9 includes an eighth surface facing the optical waveguide structure 5 and a ninth surface away from the optical waveguide structure 5; wherein,

[0110] The second lens 3 and the third lens 7 are two identical lenses, both providing a positive focal length. The side of the second lens 3 near the eye is a plane and can be used as a film-sticking surface. The side of the second lens 3 near the screen is an aspherical surface. The lens material can be plastic or glass.

[0111] The radius of curvature of the first surface is -72 mm to -73 mm; the thickness of the second lens is 5.5 mm to 6.5 mm, the refractive index is 1.54, and the focal length f1 is 133.5 mm to 134 mm.

[0112] The thickness of the optical waveguide structure is 3mm to 6.5mm, and the material is resin with a refractive index of 1.53;

[0113] The hot mirror is an aspherical mirror with a curvature radius of 43mm to 44mm and a focal length f2 of -14mm to 13.5mm;

[0114] The distance between the fourth surface and the first surface is 0.1 mm; the distance between the third surface and the sixth surface is 0.1 mm. Figure 2 In the figure, the right surface vertex of the second lens 3 and the left surface of the optical waveguide structure 5 appear to be attached together, and the left surface vertex of the third lens 3 and the right surface of the optical waveguide structure 5 appear to be attached together, but there can actually be a small gap between them to ensure that the overall optical system has better imaging quality. The distance between the fifth surface and the display module is 6mm to 6.5mm, the distance between the eighth surface and the seventh surface is 0.5mm to 1mm, and the distance between the eighth surface and the center of the hot mirror is 34.5mm to 35.5mm.

[0115] The first lens is a double-sided aspherical mirror, the curvature radius of the eighth surface is -19.5mm to 20.5mm; the curvature radius of the ninth surface is 24.5mm to 25.5mm; the focal length f3 of the first lens is 21mm to 21.5mm, the thickness is 3mm to 4mm, the material is resin, the refractive index is 1.53, and the distance between the ninth surface and the camera is 3mm to 5mm.

[0116] In one embodiment, the relationship between the focal lengths of the second lens 3, the third lens 7 and the thermal mirror 6 is:

[0117] -1.5 <f3 / f2<-1;

[0118] -16.5 <f2*f3 / f’<-16;

[0119] 7 <f1 / f’<7.5。

[0120] Combine the following Figure 2 The application process of the present invention is described with reference to the embodiments of the present invention.

[0121] The head mounted display device of the present invention has two optical paths, one is a Pancake optical path and the other is an infrared light path.

[0122] The light source 4 emits infrared light with a wavelength in the range of 800 to 1100 nm. The infrared light is reflected by the human eye 1, passes through the composite film material (including at least a phase delay film and a reflective polarizing film) and the second lens 3, is reflected by the reflective surface of the heat mirror 6, and is totally reflected from the optical waveguide structure 6 at a specific angle to propagate to the first lens 9. After being refracted by the first lens 9, the infrared light converges at the infrared camera 10, is collected by the sensor of the camera 10, and is photographed.

[0123] In one embodiment, the total focal length f of the optical system composed of the display module and the lens group is 29 mm to 29.5 mm. The optical system composed of the display module and the lens group is a folded optical path of a pancake optical path, which is used for imaging. For example, in a virtual reality (VR) device, the optical system composed of the display module and the lens group can effectively display high-quality virtual reality images to the user's eyes, thereby providing the user with an immersive visual experience.

[0124] The total focal length f' of the optical system formed by the eye tracking device is 18 mm to 18.5 mm.

[0125] This optical path is another optical path independent of the above-mentioned pancake folding optical path, that is, the infrared optical path, which is specifically used to record and track the activity trajectory of the human eye. This optical path is composed of a light source, a thermal mirror, an optical waveguide structure and a camera module. After the infrared light beam emitted by the light source is reflected by the human eye, it passes through the thermal mirror. The function of the thermal mirror is to ensure that only infrared light of a specific wavelength can pass through. The light beam passing through the thermal mirror then enters the optical waveguide structure, which is responsible for guiding the light beam to the target area and can effectively control the propagation direction of the light beam. Finally, the camera module is responsible for capturing the infrared light passing through the optical waveguide structure, thereby realizing the accurate recording and tracking of the activity trajectory of the human eye.

[0126] by Figure 2Taking the optical system as an example, the total focal length f of the optical system composed of the display module and the lens group and the total focal length f' of the optical system composed of the eye tracking device are described. The total focal length f is the focal length of the propagation optical path of the visible light composed of the second lens 3, the third lens 7, the display module 8, the semi-transparent and semi-reflective film, the phase delay film and the reflective polarizing film, and the total focal length f' is the focal length of the propagation optical path of the infrared light composed of the light source 4, the thermal mirror 6, the optical waveguide structure 5, the first lens 9 and the camera 10.

[0127] The optical system after the above optical parameters are optimized has a better image display effect. The test results are as follows:

[0128] 1. Figure 3 is the MTF & spatial frequency 20lp curve of the infrared system composed of the eye tracking device of the present invention, Figure 4 This is the MTF & spatial frequency 40lp curve of the infrared system. MTF, or Modulation Transfer Function, is an important indicator for measuring the performance of an optical system. Figure 3 and Figure 4 In the MTF curve, the horizontal axis represents the spatial frequency and the vertical axis represents the transfer function (i.e., contrast). The MTF value of an optical system can illustrate the imaging quality of the system, especially its performance in terms of resolution and contrast. The MTF value reflects the degree of contrast attenuation after the optical system images the sinusoidal intensity distribution function of different frequencies. Figure 3 and Figure 4 The test results show that Figure 2 The MTF of the infrared optical system of the embodiment is not less than 0.5 at 20lp / mm and not less than 0.3 at 40lp / mm, which means that the higher the resolution of the infrared system of the present invention, the smaller the details can be transmitted while maintaining a higher contrast.

[0129] 2. Figure 5 This is the field curvature test result of the infrared system composed of an eye tracking device. Field curvature is an optical parameter that describes the degree of image curvature. Field curvature will affect the imaging quality of the optical system in the entire field of view. If the field curvature is too large, the clarity and flatness of the image will be affected in the entire field of view. Figure 5 It can be seen from the test data that the field curvature of the infrared system of the present invention does not exceed 0.3 as a whole.

[0130] 3. The imaging spot diameter of the infrared system formed by the eye tracking device of the present invention is less than 10um.

[0131] 4. Achieved results:

[0132] Reference Figure 6The infrared system composed of the eye tracking device has an overall transmittance of infrared light of >90%, which can reduce the power of the light source and reduce power consumption.

[0133] The camera can shoot directly at the human eye, eliminating blind spots and improving the fidelity and accuracy of the tracking process.

[0134] At a higher line pair of 40lp / mm, it still has high resolution and can clearly capture subtle changes in the eyes.

[0135] In one embodiment, referring to Figure 8 The lens group includes: a fourth lens 11, a fifth lens 12 and a sixth lens 13, wherein along the direction away from the display module 8, the sixth lens 13, the optical waveguide structure 5, the fifth lens 12, the fourth lens 11 and the light source 4 are arranged in sequence.

[0136] Figure 8 A three-piece optical system showing a straight-through optical solution.

[0137] The present invention is applicable not only to straight-through optical solutions but also to folded pancake optical solutions.

[0138] In one embodiment, referring to Fig. 9 , this embodiment shows a folded pancake optical solution.

[0139] Specifically, the lens group is a pancake display lens group, and the lens group further includes:

[0140] Semi-transmissive and semi-reflective films, phase-delay films and reflective polarizing films; wherein,

[0141] The phase delay film and the reflective polarizing film are stacked and arranged on the side of the fourth lens away from the display module or the side of the optical waveguide structure away from the display module in a direction away from the display module;

[0142] The semi-transmissive and semi-reflective film is arranged on the side of the sixth lens close to the display module.

[0143] The semi-transmissive and semi-reflective film, phase delay film and reflective polarizing film play a role in realizing the pancake folded light path, and do not affect the propagation path of the infrared light of the eye tracking device.

[0144] In one embodiment, referring to Fig.10 The device also includes: a reflector 14 for changing the propagation direction of the second light; wherein, along the light emitting direction away from the heat mirror 6, the reflector 14, at least one first lens 9 and the camera 10 are arranged in sequence. Figure 8 A first lens 9 and a camera 10 are taken as an example for description.

[0145] In this embodiment, the reflection characteristics of the reflector are utilized to change the propagation path of the light emitted by the thermal mirror. Fig.10 As shown, the light originally directed vertically upward is directed to the horizontal direction, and this change is achieved by precisely adjusting the tilt angle of the reflector. In this way, the direction in which the light enters the first lens 9 and the camera 10 can be controlled, so that the camera module can be placed in a position more suitable for the overall assembly of the device.

[0146] In this embodiment, the optical path is optimized by a reflector, and the first lens 9 and the camera 10 are adjusted to a corner position more suitable for assembly. Such a design significantly improves the space utilization of the device.

[0147] The Pancake solution performs better in terms of lightness, image quality, and diopter adjustment. However, the optical solution of Pancake and other field-of-view modules has limited space and cannot accommodate infrared cameras, let alone realize the frontal shooting of moving eyes. Fig.11 , which illustrates the optical scheme of the module inscribed in the field of view angle. It can be seen that the display module is a square area, the image circle is a circular area, and the space between the two, such as the area encircled by the dotted line, can be used to place the camera module or infrared light source.

[0148] The solution of the present invention changes the propagation path of infrared light by adding a thermal mirror or a fully reflective waveguide, i.e., an optical waveguide structure, and places the lens and the infrared camera on the side wall of the lens barrel to avoid eyelashes, corners of the eye, etc. from blocking the eyeball, thereby achieving positive shooting of the eye movement part, improving the fidelity and accuracy of the eye movement tracking process captured by the infrared camera, and improving the accuracy of software rendering of the gaze point area, thereby improving the imaging quality and the visual and human-computer interaction experience.

[0149] In view of the small pupil distance of the existing pancake optical solution, the high-precision optical system for VR eye tracking proposed in this invention can realize the direct shooting of the eyes, solve the problem of eyelash occlusion when the infrared camera is set near the eye, and the image occlusion problem caused by poor camera installation accuracy when the infrared camera is set near the screen. At the same time, through special optical design and optical parameter setting, the imaging quality can be improved, ultra-thin VR can be achieved, and the manufacturing cost can be further reduced.

[0150] The effect of the present invention in solving the above-mentioned occlusion problem by photographing the human eye directly is described below in conjunction with a comparative example.

[0151] Comparative Example 1

[0152] Reference Fig.12 The light source 4 for emitting infrared light is placed near the lens of the human eye and close to the human eye side.

[0153] The camera 10 is placed near the display module 8 to receive infrared light reflected by the human eye. Fig. 9 8 shown in the figure. When propagating along this infrared light path, there is a significant problem, that is, there is a blind spot in some active areas of the human eye (as shown by the circled part in the figure). In this area, the camera cannot effectively collect the infrared light signal reflected back by the eyeball. This limitation leads to the inability to capture the feedback information of the eyeball in some areas during the eye tracking process, thereby affecting the continuity and accuracy of the tracking. This phenomenon not only reduces the tracking effect, but also makes it difficult to achieve the ideal level of tracking accuracy.

[0154] Comparative Example 2

[0155] Reference Fig.13 , the light source 4 is placed near the display module 8, specifically, below the display module 8. The camera 10 is placed near the lens of the human eye, close to the human eye side, more specifically, above the lens. During the propagation of this infrared light path, there is still a situation where some areas below the human eye cannot capture the feedback information of the eyeball during the human eye tracking process (as shown by the circled part in the figure).

[0156] Comparative Example 3

[0157] Reference Fig.14 , the light source 4 and the camera 10 are located in the same vertical direction, and both are located near the lens of the human eye and close to the human eye. The light source 4 is above the camera 10. When the infrared light propagates along this path, in the area below the human eye, during the process of eye tracking, the feedback information of the eyeball cannot be captured (as shown in the circled part in the figure).

[0158] In the existing optical systems reflected in Examples 1-3, the infrared camera is placed in the lens barrel close to the top of the human eye, which affects the pupil distance and has certain blind spots such as eyelashes and eye corners when shooting, so the entire eye cannot be photographed, resulting in image distortion. The present invention no longer uses heavy components such as splitter prisms, making the optical machine lighter and thinner, and the design is no longer limited to the size of the optical machine.

[0159] The present invention optimizes Pancake and other field-of-view incision modules and introduces eye tracking technology. By constructing an infrared shooting optical system, combined with the use of thermal mirrors and optical waveguide structures, the infrared camera can shoot the user's eyes from the front. This design not only improves the fidelity of the eye tracking process, but also significantly improves the tracking accuracy, providing users with a more accurate and natural interactive experience.

[0160] The eye tracking device involved in the present invention has brought significant progress to virtual reality technology, especially in dynamic gaze point rendering. The device significantly enhances the user's visual experience by optimizing the image quality of the most sensitive area on the retina. The application of the technology of the present invention makes the images in virtual reality clearer and richer in details, thus providing users with a more immersive and realistic visual experience.

[0161] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0162] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An eye tracking device, characterized in that: include: Light source, thermal mirror, optical waveguide structure and camera module; wherein, The light source is used to emit light toward the eyeball of the user, and the light is reflected by the eyeball to form a first light beam directed toward the thermal mirror; The heat mirror is arranged in the optical waveguide structure, and the heat mirror is used to reflect the first light into a second light, and the second light is configured to be emitted to the camera module via the optical waveguide structure; The light incident side of the camera module faces the thermal mirror, and the camera module is used to receive the second light.

2. The device according to claim 1, characterized in that The camera module comprises: a camera and at least one first lens; wherein, along the light emitting direction away from the thermal mirror, the at least one first lens and the camera are arranged in sequence.

3. The device according to claim 2, characterized in that The device also includes: a reflector for changing the propagation direction of the second light; wherein, along the light exiting direction away from the thermal mirror, the reflector, at least one first lens and the camera are arranged in sequence.

4. The device according to claim 1, characterized in that The camera module is an infrared camera module.

5. A head mounted display device, characterized in that: include: Display module, capable of displaying images; a lens assembly, comprising a plurality of lenses; and The eye tracking device according to any one of claims 1 to 4, wherein the optical waveguide structure of the eye tracking device is located between any two adjacent lenses in the lens group; or The optical waveguide structure is located between the lens group and the display module.

6. The device according to claim 5, characterized in that The light source of the eye tracking device is arranged on a side of the lens in the lens group that is adjacent to the user's eye and close to the user's eye.

7. The device according to claim 5, characterized in that The optical waveguide structure is arranged perpendicular to a line connecting the centers of all lenses of the lens group.

8. The device according to claim 5, characterized in that The angle between the heat mirror and the line connecting the centers of all lenses of the lens group is 44.5°-45.5°.

9. The device according to any one of claims 5 to 8, characterized in that: The lens group includes: a second lens and a third lens, wherein the third lens, the optical waveguide structure, the second lens and the light source are arranged in sequence along a direction away from the display module.

10. The device according to claim 9, characterized in that The lens group is a pancake display lens group, and the lens group further comprises: a semi-transmissive and semi-reflective film, a phase delay film and a reflective polarizing film, wherein the phase delay film and the reflective polarizing film are stacked and arranged on a side of the second lens away from the display module or a side of the optical waveguide structure away from the display module in a direction away from the display module; The semi-transmissive and semi-reflective film is arranged on a side of the third lens close to the display module.

11. The device according to claim 10, characterized in that The total focal length f of the optical system formed by the display module and the lens group is 29 to 29.5 mm, and the total focal length f' of the optical system formed by the eye tracking device is 18 to 18.5 mm.

12. The device according to claim 11, characterized in that The second lens includes a first surface close to the display module and a second surface away from the display module, the optical waveguide structure includes a third surface close to the display module, a fourth surface away from the display module, and a seventh surface facing the first lens, the third lens includes a fifth surface close to the display module and a sixth surface away from the display module, and the first lens includes an eighth surface facing the optical waveguide structure and a ninth surface away from the optical waveguide structure; wherein, The first surface is aspherical, and the second surface is a plane; the radius of curvature of the first surface is -72mm to -73mm; the thickness of the second lens is 5.5mm to 6.5mm, the refractive index is 1.54, and the focal length f1 is 133.5mm to 134mm; The thickness of the optical waveguide structure is 3mm to 6.5mm, and the material is resin with a refractive index of 1.53; The hot mirror is an aspherical mirror with a curvature radius of 43mm to 44mm and a focal length f2 of -14mm to 13.5mm; The distance between the fourth surface and the first surface is 0.1 mm; the distance between the third surface and the sixth surface is 0.1 mm; the distance between the fifth surface and the display module is 6 to 6.5 mm, the distance between the eighth surface and the seventh surface is 0.5 to 1 mm, and the distance between the eighth surface and the center of the thermal mirror is 34.5 to 35.5 mm; The first lens is a double-sided aspherical mirror, the curvature radius of the eighth surface is -19.5mm to 20.5mm; the curvature radius of the ninth surface is 24.5mm to 25.5mm; the focal length f3 of the first lens is 21mm to 21.5mm, the thickness is 3mm to 4mm, the material is resin, the refractive index is 1.53, and the distance between the ninth surface and the camera is 3mm to 5mm.

13. The device according to claim 12, characterized in that The relationship between the focal lengths of the second lens, the third lens and the thermal mirror is: -1.5 <f3 / f2<-1; -16.5 <f2*f3 / f’<-16; 7 <f1 / f’<7.5。 14. The device according to any one of claims 5 to 8, characterized in that: The lens group includes: a fourth lens, a fifth lens and a sixth lens, wherein the sixth lens, the optical waveguide structure, the fifth lens, the fourth lens and the light source are arranged in sequence along a direction away from the display module.

15. The device according to claim 14, characterized in that The lens group is a pancake display lens group, and the lens group further includes: a semi-transmissive and semi-reflective film, a phase delay film and a reflective polarizing film; wherein, The phase delay film and the reflective polarizing film are stacked and arranged on the side of the fourth lens away from the display module or the side of the optical waveguide structure away from the display module in a direction away from the display module; The semi-transmissive and semi-reflective film is arranged on the side of the sixth lens close to the display module.

Images