Eyeball tracking system, eyeball tracking method and intelligent head-mounted device

Abstract

The embodiment of the invention provides an eyeball tracking system, an eyeball tracking method and intelligent head-mounted equipment. The eyeball tracking system is applied to the intelligent head-mounted equipment and comprises a projector, a receiving camera and a processor, the number of the projectors is at least one, each projector comprises a light source and an optical diffraction element, the light source is obliquely arranged, a set inclination angle theta is formed between the light source and a plane where eyeballs of a wearer are located, 25 degrees < = theta < = 35 degrees, and the optical diffraction element is located on a light emitting path of the light source. The light field module is used for forming a uniform light field covering a specific eye area at a set distance from eyeballs of a wearer; the receiving camera and the projector are correspondingly arranged, and the receiving camera is used for acquiring an eye image which is reflected by an eyeball and is formed by the optical diffraction element in a uniform light field; and the processor is used for determining pupil center position information of the wearer according to the eye image so as to obtain a fixation point of the eyes of the wearer.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of optical technology. More specifically, the embodiments of the present application relate to an eye tracking system, an eye tracking method and an intelligent head-mounted device. Background Art

[0002] Eye tracking technology is a technology that determines the user's line of sight and gaze point by capturing and analyzing eye movements. Although traditional eye tracking methods, such as pupil-corneal reflection method, can achieve eye tracking to a certain extent, they are easily disturbed in complex ambient light, affecting the extraction and judgment of light spots, resulting in inaccurate line of sight and gaze point estimation. In addition, this method usually requires the use of multiple light sources, which is not only costly but also occupies a large space.

[0003] In order to improve the above problems, eye tracking methods based on image recognition have emerged on the market. However, due to the off-axis light source relative to the eyeball, the traditional eye tracking methods based on image recognition have poor light field uniformity, which affects the collected image information and thus affects the eye tracking effect. Summary of the invention

[0004] The purpose of this application is to provide a new technical solution for an eye tracking system, an eye tracking method and an intelligent head-mounted device.

[0005] In a first aspect, the present application provides an eye tracking system, which is applied to a smart head mounted device, and the eye tracking system comprises:

[0006] At least one projector, including a light source and an optical diffraction element, wherein the light source is tilted to form a set tilt angle θ with the plane where the wearer's eyeball is located, 25°≤θ≤35°, and the optical diffraction element is located on the light output path of the light source, and is used to form a uniform light field covering a specific area of ​​the eye at a set distance from the wearer's eyeball;

[0007] A receiving camera, which is arranged corresponding to the projector and is used to obtain an eye image reflected by the eyeball and in the uniform light field formed by the optical diffraction element;

[0008] The processor is used to determine the pupil center position information of the wearer according to the eye image, and then obtain the gaze point of the wearer's eyes.

[0009] Optionally, the set inclination angle θ is 30°.

[0010] Optionally, a collimating element is arranged between the light source and the optical diffraction element.

[0011] Optionally, the light source is a light emitting diode (LED), an edge emitting laser (EEL) or a vertical cavity surface emitting laser (Vcsel).

[0012] Optionally, the light source is a vertical cavity surface emitting laser Vcsel, and the vertical cavity surface emitting laser Vcsel includes a plurality of light emitting regions, each of which is configured to emit light independently.

[0013] Optionally, the optical diffraction element is any one of DOE, MOE and diffuser.

[0014] Optionally, the light source is disposed on a substrate, and the substrate is any one of a PCB substrate, a copper substrate, an aluminum substrate and a BT substrate.

[0015] Optionally, the projector further comprises a shell, the shell is cylindrical, and the substrate and the optical diffraction element are respectively arranged at two opposite ends of the shell;

[0016] The center of the light source and the center of the optical diffraction element are located on the same straight line.

[0017] In a second aspect, the present application provides an eye tracking method, which is applied to a smart head mounted device, and the eye tracking method comprises:

[0018] A uniform light field covering a specific area of ​​the eye is formed at a set distance from the eyeball of the wearer by at least one projector; wherein the projector comprises an inclined light source and an optical diffraction element located on the light output path of the light source, and a set inclination angle θ is formed between the light source and the plane where the eyeball of the wearer is located, 25°≤θ≤35°;

[0019] Acquire, by a receiving camera, an eye image reflected by the eyeball and in a uniform light field formed by the optical diffraction element;

[0020] The eye image is processed by a processor to determine the pupil center position information of the wearer, and then the gaze point of the wearer's eyes is obtained.

[0021] Optionally, the processing the eye image by a processor to determine the pupil center position information of the wearer, and then obtaining the gaze point of the wearer's eyes, includes:

[0022] Processing the received eye image by the processor, extracting the image feature area from the eye image, and then determining the position information of the pupil center of the wearer;

[0023] Calculating the wearer's sight line vector through a sight line estimation model according to the position information of the pupil center;

[0024] The gaze point of the wearer's eyes is obtained according to the sight line vector.

[0025] In a third aspect, the present application provides a smart head-mounted device, the smart head-mounted device comprising:

[0026] An eye tracking system as described in the first aspect.

[0027] Optionally, the eye tracking systems are provided in two, wherein one eye tracking system corresponds to the left eye of the user, and the other eye tracking system corresponds to the right eye of the user.

[0028] The beneficial effects of this application are:

[0029] The embodiment of the present application provides an eye tracking system, which uses a projector to illuminate the wearer's eyeball, and by arranging a light source and an optical diffraction element in the projector, and tilting the light source, a set tilt angle is formed between the light source and the plane where the wearer's eyeball is located, thereby forming a uniformly distributed light field at a certain distance from the eyeball. This setting can solve the brightness difference caused by the off-axis light source relative to the eyeball, eliminate the impact on the image acquisition end, and thus improve the effect of eye tracking.

[0030] Other features and advantages of the present specification will become apparent from the following detailed description of exemplary embodiments of the present specification with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the specification and, together with the description, serve to explain the principles of the specification.

[0032] Figure 1 One of the hardware structure diagrams of the eye tracking system provided in the embodiment of the present application;

[0033] Figure 2 The second schematic diagram of the hardware structure of the eye tracking system provided in the embodiment of the present application;

[0034] Figure 3 One of the structural schematic diagrams of the smart head mounted device provided in the embodiment of the present application;

[0035] Figure 4 The second structural diagram of the smart head mounted device provided in the embodiment of the present application;

[0036] Figure 5 A schematic diagram of the light source arrangement of the eye tracking system provided in an embodiment of the present application;

[0037] Figure 6 Light path diagrams of the projector provided in the embodiment of the present application at four different viewing angles;

[0038] Figure 7 One of the optical effect diagrams of the eye tracking system provided in an embodiment of the present application;

[0039] Figure 8 The second optical effect diagram of the eye tracking system provided in the embodiment of the present application;

[0040] Fig. 9 This is the third optical effect diagram of the eye tracking system provided in an embodiment of the present application.

[0041] Description of reference numerals:

[0042] 1. Projector; 11. Light source; 12. Optical diffraction element; 13. Collimation element; 14. Substrate; 15. Housing; 2. Receiving camera. DETAILED DESCRIPTION

[0043] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0044] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0045] Techniques and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the techniques and equipment should be considered part of the specification.

[0046] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0047] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0048] The eye tracking system, eye tracking method and smart head mounted device provided in the embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0049] According to an embodiment of the present application, an eye tracking system is provided, which is applied to a smart head-mounted device. Figure 1 , Figure 3 and Figure 4 The eye tracking system includes a projector 1, a receiving camera 2 and a processor ( Figure 1); wherein the projector 1 is set to be at least one, the projector 1 includes a light source 11 and an optical diffraction element 12, the light source 11 is tilted to form a set tilt angle θ with the plane where the wearer's eyeball is located, 25°≤θ≤35°, the optical diffraction element 12 is located on the light output path of the light source 11, and is used to form a uniform light field covering a specific area of ​​the eye at a set distance from the wearer's eyeball; the receiving camera 2 is correspondingly set to the projector 1, and is used to obtain an eye image reflected by the eyeball and in the uniform light field formed by the optical diffraction element 12; the processor is used to determine the wearer's pupil center position information based on the eye image, and then obtain the gaze point of the wearer's eye.

[0050] The eye tracking system provided in the embodiment of the present application can be applied to smart head-mounted devices, such as smart glasses or smart helmets, etc. The eye tracking system provided in the embodiment of the present application mainly includes a projector 1, a receiving camera 2 and a processor.

[0051] See also Figure 1 The projector 1 includes a light source 11 and an optical diffraction element 12.

[0052] The light source 11 is tilted, and a set tilt angle θ is formed between the light source 11 and the plane where the wearer's eyeball is located, and 25°≤θ≤35°, see Figure 5 The tilted light source design in this application is intended to optimize the light propagation path, reduce light interference, and facilitate the formation of a uniform light field at a set distance.

[0053] The optical diffraction element 12 is located on the light output path of the light source 11, and is used to convert the light emitted by the light source 11 into a light beam that covers a specific area of ​​the eye (e.g., 30 mm*15 mm, corresponding to the eye size, see Figure 5 ) of a uniform light field.

[0054] It should be noted that by changing the surface scattering and other characteristics of the optical diffraction element 12 through specific optical design parameters, it is possible to maintain the uniformity of the light field at different tilt angles.

[0055] See also Figure 3 and Figure 4 The receiving camera 2 and the projector 1 can be set correspondingly in quantity.

[0056] For example, two receiving cameras 2 are provided, and two projectors 1 are also provided.

[0057] The receiving camera 2 is used to obtain eye images reflected by the wearer's eyeballs in the uniform light field formed by the optical diffraction element 12. These reflected eye images will be used for subsequent pupil center positioning and line of sight estimation, all of which are to obtain the gaze point of the wearer's eyes, thereby realizing the eye tracking function.

[0058] When the eye tracking system provided in the embodiment of the present application is applied to a smart head mounted device, see Figure 3 and Figure 4 , the receiving camera 2 and the projector 1 can be arranged adjacent to each other.

[0059] Of course, the receiving camera 2 and the projector 1 are not necessarily adjacent to each other, and can be arranged at intervals, and their positions can be moved according to the design of the overall external unit of the smart head-mounted device.

[0060] In the eye tracking system provided in the embodiment of the present application, the processor is mainly used to determine the pupil center position information of the wearer based on the eye image obtained by the receiving camera 2, and then obtain the gaze point of the wearer's eyes.

[0061] Optionally, the processor of the present application may include a deep learning model to improve the accuracy and precision of line of sight estimation.

[0062] In the embodiment of the present application, the angle range of the tilted placement of the light source 11 in the projector 1 is designed to be 25° to 35°.

[0063] When the set inclination angle θ between the light source 11 and the plane where the wearer's eyeball is located is 25°, see Figure 7 , the light field formed is well uniform, but the right side is slightly brighter.

[0064] When the set inclination angle θ between the light source 11 and the plane where the wearer's eyeball is located is 30°, see Figure 8 , the uniformity of the light field is optimal, for example, the uniformity of the light field is very good within the range of 30mm*15mm.

[0065] When the set inclination angle θ between the light source 11 and the plane where the wearer's eyeball is located is 35°, see Fig. 9 , the light field uniformity is still good, but the left side is slightly brighter.

[0066] See also Figures 7 to 9 The results show that by adjusting the inclination angle θ of the light source 11 in the projector 1 relative to the plane where the eyeball is located, and then matching the optical diffraction element 12, a uniform light field covering a specific area of ​​the wearer's eye can be formed at a set distance, thereby improving the quality of the collected eye image information.

[0067] Among them, the uniform light field improves the quality of the collected eye image information, making the eye features more obvious and easier to identify, which is the basis for achieving high-precision eye tracking.

[0068] The eye tracking system provided in the embodiment of the present application is based on the design of the projector 1, and then obtains the eye image of the uniform light field reflected by the eyeball through the receiving camera 2, and the processor extracts the image feature area and locates the pupil center, so as to accurately obtain the gaze point of the wearer's eyes.

[0069] Compared with the traditional pupil-corneal reflection method using multiple light sources in combination, the eye tracking system provided by the embodiment of the present application has greatly reduced production costs and occupied space.

[0070] The eye tracking system of the present application has the characteristics of simple structure and small size, which can better adapt to the compact structure of smart head-mounted devices and facilitate integration and large-scale production.

[0071] The eye tracking system provided in the embodiment of the present application obtains the eye image of the uniform light field reflected by the wearer's eyeball through the receiving camera 2, and the processor extracts the image feature area and locates the position information of the wearer's pupil center. The pupil image under the uniform light field is clearer, which improves the accuracy of pupil center positioning.

[0072] It should be noted that the processor of the present application can further use the deep learning model to estimate the line of sight based on the pupil center position information to accurately obtain the gaze point of the wearer's eyes. This can achieve faster and more accurate eye tracking response, thereby improving the interactive experience of the smart head-mounted device.

[0073] The embodiment of the present application provides an eye tracking system, which uses the projector 1 to illuminate the eye of the wearer, and by arranging a light source 11 and an optical diffraction element 12 in the projector 1, and tilting the light source 11, a set tilt angle is formed between the light source 11 and the plane where the eye of the wearer is located, see Figure 5 , thus forming a uniformly distributed light field at a certain distance from the eyeball. This setting can solve the brightness difference caused by the off-axis light source relative to the eyeball, eliminate the impact on the image acquisition end, and thus improve the effect of eye tracking.

[0074] In some examples of this application, see Figure 5 and Figure 7 , the set inclination angle θ is 30°.

[0075] In the projector 1 provided in the present application, the light source 11 is tilted at an angle ranging from 25° to 35°. The preferred tilt angle θ is 30°. Figure 7When the set inclination angle θ between the light source 11 and the plane where the wearer's eyeball is located is 30°, the uniformity of the light field is optimal. For example, the uniformity of the light field is very good within the range of 30mm*15mm.

[0076] It should be noted that compared with the tilt angles of 25° and 35°, the light field uniformity at 30° is better, avoiding the problem of uneven brightness on the left and right sides.

[0077] The projector 1 provided in the present application is an optical axis deflection projector, and its design itself is to solve the problem of poor light field uniformity in traditional eye tracking technology. By tilting the light source 11 and matching the optical diffraction element 12, a uniformly distributed light field can be obtained at the user's eyes. This improvement helps to improve the quality of the image acquisition end, thereby improving the accuracy of eye tracking.

[0078] Since the uniformity of the light field has been significantly improved, the collected eye image information can be more accurate, which helps to improve the tracking accuracy of the eye tracking system, so that the entire eye tracking system can more accurately track the subtle changes in the user's eyes.

[0079] Compared with the traditional pupil-corneal reflection method, the technical solution of the present application has a greater cost advantage. It avoids the coordinated use of multiple light sources, thereby reducing hardware costs.

[0080] It should be noted that the technical solution of the present application is not only applicable to the field of eye tracking, but can also be extended to other application scenarios that require high-precision imaging and recognition, such as the field of iris recognition.

[0081] In some examples of this application, see Figure 2 A collimating element 13 is arranged between the light source 11 and the optical diffraction element 12 .

[0082] A collimating element 13 may also be added to the projector 1 of the present application. The main function of the collimating element 13 is to adjust the direction of the light emitted by the light source 11 to make it more parallel and uniform. This is relatively important for the subsequent optical diffraction element 12, because it needs to receive a stable and uniform light beam to produce the expected diffraction effect.

[0083] The collimating element 13 is, for example, a collimating mirror.

[0084] By providing the collimating element 13, the directionality and uniformity of the light emitted by the light source 11 can be significantly improved. This helps to form a more stable and uniform light beam on the optical diffraction element 12, thereby further improving the uniformity of the light field.

[0085] In the absence of the collimating element 13, the production cost of the eye tracking system is low, but the uniformity of the light field and the stability of the system may be affected to a certain extent, but there will be no significant impact on the tracking accuracy. After adding the collimating element 13, although the production cost of the eye tracking system has increased, the uniformity of the light field and the stability of the system have been significantly improved. Therefore, in practical applications, it is necessary to weigh whether to set a collimating element based on specific application scenarios and requirements.

[0086] In some examples of the present application, the light source 11 is a light emitting diode LED, an edge emitting laser EEL or a vertical cavity surface emitting laser Vcsel.

[0087] The projector 1 of the present application has relatively few restrictions on the light source 11 therein, and can be at least one of a light emitting diode LED, an edge emitting laser EEL, and a vertical cavity surface emitting laser Vcsel.

[0088] (1) Light-emitting diodes (LEDs) have the advantages of low cost, low power consumption, long life, and easy integration. Although their luminous efficiency is slightly lower than that of laser light sources, it is sufficient in many eye tracking applications.

[0089] (2) Edge emitting laser (EEL) has high luminous efficiency and good directivity, and can provide a stronger light beam.

[0090] (3) Vertical cavity surface emitting laser (VCSEL) is favored for its small size, high efficiency, good beam quality and easy modulation. It can provide high-power laser output in a small package and is very suitable for eye tracking systems.

[0091] Using laser light sources such as edge emitting lasers (EEL) or vertical cavity surface emitting lasers (VCSEL) can significantly improve the brightness and directionality of the projector 1, thereby enhancing the accuracy and stability of the entire eye tracking. This is especially important for application scenarios that require high precision and high stability.

[0092] As a low-cost light source option, LED helps reduce the overall cost of the entire eye tracking system. Light sources such as edge-emitting lasers (EELs) and vertical-cavity surface-emitting lasers (VCSELs) have small package sizes and high reliability, making them easy to integrate into eye tracking systems.

[0093] Specifically, in this application, when selecting the light source 11, it is necessary to comprehensively consider factors such as system performance requirements, cost-effectiveness, and integration difficulty. LED, EEL, and Vcsel each have their own advantages and disadvantages and are suitable for different application scenarios. For example, in an eye tracking system that requires high precision and high stability, Vcsel or EEL may be a better choice; while in cost-sensitive application scenarios, LED may have more advantages.

[0094] In the present application, the light source 11 can be, for example, a light emitting diode (LED), an edge emitting laser (EEL) or a vertical cavity surface emitting laser (Vcsel), and these light sources can be individually processed by semiconductor packaging, thereby significantly improving their reliability.

[0095] In some examples of the present application, the light source 11 is a vertical cavity surface emitting laser Vcsel, and the vertical cavity surface emitting laser Vcsel includes a plurality of light emitting regions, each of which is configured to emit light independently.

[0096] In this example of the present application, the light source 11 uses a vertical cavity surface emitting laser Vcsel, and the design of the vertical cavity surface emitting laser Vcsel is relatively unique, and it includes multiple light-emitting areas that can emit light independently. Each light-emitting area of ​​the vertical cavity surface emitting laser Vcsel can emit light independently, which means that they can be activated or extinguished as needed, or the light intensity can be adjusted. This flexibility provides better control capabilities for the eye tracking system, allowing the eye tracking system of the present application to accurately adjust the light field according to the position, movement or gaze direction of the eye.

[0097] Because the multiple light-emitting areas of Vcsel can be controlled independently, the eye tracking system can adjust the distribution of the light field as needed, thereby achieving higher light field uniformity. This is crucial to the accuracy of eye tracking because it ensures that a clear and consistent image is obtained no matter where the eyes are.

[0098] In this example of the present application, the optical diffraction element 12 (DOE) located behind Vcsel plays a role in beam shaping and homogenization. The optical diffraction element 12 can convert the laser beam emitted by Vcsel into a uniform light field suitable for eye tracking.

[0099] In some examples of the present application, the optical diffraction element 12 is any one of DOE, MOE and diffuser.

[0100] Among them, DOE is an optical diffraction element with a micron or nanostructured surface, which can diffract the light incident on the surface. DOE can convert the incident light into a light field with a specific pattern or distribution by controlling the diffraction and interference of light. In the eye tracking system of the present application, when the optical diffraction element 12 is a DOE, it can be used to optimize the uniformity of the light field to ensure stable and consistent lighting conditions within the range of eye activity. In addition, DOE can also achieve precise control of the shape and size of the light spot to meet the needs of high-precision eye tracking.

[0101] MOE is a micro-nanoscale optical diffraction element that can achieve precise control of light through microstructure design, such as focusing, deflection, etc. In the eye tracking system of this application, MOE can be used to further enhance the uniformity and stability of the light field.

[0102] The diffuser is a diffuser that evenly distributes the incident light within a certain range through scattering, thereby achieving light field homogenization. In the eye tracking system of the present application, the diffuser can be used to effectively diffuse the light emitted by the light source 11 to the eye activity area, ensuring that sufficient and uniform illumination can be obtained within the entire tracking range.

[0103] In an example of the present application, the optical diffraction element 12 is the above-mentioned diffuser, that is, a diffuser, and the diffuser is composed of multiple groups of microlens arrays, and the arrangement is random, wherein each group of microlens arrays is composed of a number of minimum unit structures; the radius of the circumscribed circle of each minimum unit structure is 20um to 30um, and the size of each minimum unit structure is uniform, and there is a slight disturbance in the height within a certain range.

[0104] In the above example, each group of microlens arrays contains many minimum unit structures. Although these minimum unit structures are small, each of them carries key design elements. The radius of their circumscribed circles is controlled between 20um and 30um, ensuring the consistency between the minimum unit structures. At the same time, the size of each minimum unit structure is uniform, but there are small disturbances in height within a certain range. This design plays a vital role in the uniformity of the light field.

[0105] When the light projected by the light source 11 reaches the diffuser, the light is refracted and diffracted by the microlens array, forming a uniformly distributed light field at a distance of 15 cm from the eye. This light field is not only uniform in brightness but also precise in range, providing ideal lighting conditions for the eye tracking system.

[0106] According to this example of the present application, whether DOE, MOE or diffuser is selected as the optical diffraction element 12, the uniformity of the light field can be improved to a certain extent. This is crucial for an eye tracking system because a uniform light field can ensure that the eyes can be accurately tracked at different positions and angles.

[0107] By flexibly selecting the type of optical diffraction element, the eye tracking system of the present application can adapt to different application scenarios and user needs. For example, in cost-sensitive applications, a lower-cost diffuser can be selected; while in situations where high precision and high stability are required, DOE or MOE can be given priority.

[0108] In some examples of this application, see Figure 3 and Figure 4 The light source 11 is disposed on a substrate 14, and the substrate 14 is any one of a PCB substrate, a copper substrate, an aluminum substrate and a BT substrate.

[0109] In this example of the present application, the light source 11 is disposed on a substrate 14, and the substrate 14 has a high degree of flexibility and can be selected from any one of a PCB substrate, a copper substrate, an aluminum substrate and a BT substrate.

[0110] The PCB (printed circuit board) substrate has good electrical conductivity, insulation and processability, and can easily realize the electrical connection between the light source 11 and other electronic components. In addition, the PCB substrate also has the advantages of low cost, mature technology and easy large-scale production.

[0111] The copper substrate is known for its excellent thermal conductivity. In the eye tracking system of the present application, the light source 11 may generate a certain amount of heat when working, and the copper substrate can effectively conduct the heat away to ensure the stable operation of the entire eye tracking system. At the same time, the copper substrate also has high strength and rigidity, and can provide additional mechanical support.

[0112] Aluminum substrates also have good thermal conductivity. However, compared with copper, aluminum has lower cost, lower density, and lighter weight, so it has more advantages in application scenarios with strict requirements on weight and cost. In addition, aluminum substrates also have good corrosion resistance and machinability.

[0113] The BT (Bismaleimide Triazine) substrate is a high-performance thermosetting resin substrate with excellent heat resistance, electrical insulation and mechanical strength. In the eye tracking system of the present application, the BT substrate can ensure the stable operation of the light source 11 in extreme environments while providing reliable electrical connection and mechanical support.

[0114] In some examples of this application, see Figure 3 and Figure 4 The projector 1 further includes a housing 15, which is cylindrical. The substrate 14 and the optical diffraction element 12 are disposed at opposite ends of the housing 15. The center of the light source 11 and the center of the optical diffraction element 12 are located on the same straight line.

[0115] In this example of the present application, the projector 1 comprises a cylindrical housing 15, see Figure 3 and Figure 4 The introduction of this housing 15 not only provides necessary protection and support for the internal components of the entire projector, but also realizes the spatial layout between different optical components.

[0116] See also Figure 3 The cylindrical design of the housing 15 forms a relatively closed space inside. In this space, the substrate 14 and the optical diffraction element 12 are respectively arranged at opposite ends of the housing 15. This layout not only ensures the precise alignment between the light source 11 and the optical diffraction element 12, but also effectively avoids interference from the external environment, thereby improving the stability and reliability of the entire eye tracking system.

[0117] When the collimating element 13 is introduced into the projector 1, see Figure 4 The collimating element 13 is located inside the housing 15 and between the light source 11 and the optical diffraction element 12 .

[0118] The housing 15 has two open ends, one end of which fixes the optical diffraction element 12, and the other end fixes the substrate 14 carrying the light source 11. The optical diffraction element 12 is parallel to the substrate 14. The center of the light source 11 and the center of the optical diffraction element 12 are located on the same straight line. When the projector 1 includes a collimating element 13, the center of the collimating element 13 is coaxially arranged with the center of the light source 11 and the optical diffraction element 12, and the collimating element 13 is arranged parallel to the optical diffraction element 12 and the substrate 14 carrying the light source 11.

[0119] The introduction of the housing 15 also facilitates the overall packaging and integration of the projector 1 .

[0120] Figure 6 FIG. 1 shows the optical path diagram of the projector 1 provided in the embodiment of the present application at four different viewing angles. Figure 6It can be seen from the four figures that when the light emitted from the light source 11 is projected to the wearer's eyes after passing through the optical diffraction element 12, a uniform light field can be presented within a range of 30mm*15mm at a distance of 15cm from the eyes (corresponding to an eye size of 30mm*15mm). In other words, the light distribution within the range of 30mm*15mm is relatively uniform.

[0121] The design calculation results of the optical diffraction element 12 of the projector 1 of the present application are shown in Table 1 below.

[0122] Table 1

[0123] Design Description Window efficiency (within 30mm) Transmittance Illuminance distribution at 15mm projection distance 37.93％ 59.82％

[0124] In the range of 30mm, the nine-point uniformity is 75.44%. See Table 2 below.

[0125] Table 2

[0126] 0.897 0.937 0.756 0.963 1.000 0.827 0.892 0.929 0.754

[0127] According to Table 1, the illumination distribution at a projection distance of 15mm. This data shows the illumination distribution when the projection distance is 15mm. This is an important parameter in the eye tracking system because it directly affects the accuracy of eye tracking. The more uniform the illumination distribution, the better the image quality and the higher the accuracy of eye tracking.

[0128] According to Table 1, the window efficiency is 37.93%. Window efficiency refers to the proportion of effective light passing through the optical diffraction element within a specific range (here, 30mm, which is the same as the eye size). The window efficiency of 37.93% means that 37.93% of the light is effectively used within the 30mm range. This ratio is relatively high for the eye tracking system and helps ensure that enough light enters the receiving camera, thereby improving the imaging quality and the accuracy of eye tracking.

[0129] According to Table 1, the transmittance is 59.82%. The transmittance refers to the proportion of light that can maintain its original intensity and direction after passing through the optical diffraction element 12. The transmittance of 59.82% indicates that 59.82% of the light can maintain its original intensity and direction after passing through the optical diffraction element 12, which is crucial for the accuracy of eye tracking.

[0130] Regarding the nine-point uniformity data, refer to Table 2: Within the range of 30mm, the nine-point uniformity data gives a specific uniformity distribution. These data are the illuminance values ​​measured at different positions and are used to evaluate the uniformity of the light field. From the data given in Table 2, it can be seen that the illuminance values ​​of the nine measurement points fluctuate between 0.754 and 1.000, and the overall uniformity is good. The nine-point uniformity is 75.44%, which means that within the range of 30mm, the uniformity of the light field reaches 75.44%, which is a relatively high level of uniformity. A light field with good uniformity can eliminate the brightness difference caused by the off-axis of the light source relative to the eyeball, thereby eliminating the impact on the image acquisition end and achieving high-resolution imaging.

[0131] High window efficiency and high transmittance help ensure that enough light enters the receiving camera 2, thereby improving the image quality and the accuracy of eye tracking. Good light field uniformity can eliminate the brightness difference caused by the off-axis light source relative to the eyeball, thereby eliminating the impact on the image acquisition end and achieving high-resolution imaging. This is crucial for the eye tracking system because it directly affects the accuracy and reliability of the eye tracking system.

[0132] The optical path constructed based on the short-axis optical path and the optical diffraction element 12 that can be used for uniform light can achieve a uniform light spot at the near focus; at the same time, the window efficiency is about 38%; the uniformity is more than 75%, and the transmittance is about 60%.

[0133] Optionally, the transmittance of the light field can be further enhanced and improved by adding an AR anti-reflection film or a polarizing film or other means to the optical diffraction element 12 in the projector 1.

[0134] The eye tracking system provided by the embodiment of the present application has an optical path design that can produce a uniform light spot at a close focus (e.g., 15 cm from the eye). This is critical for the eye tracking system because it allows the system to perform high-precision imaging and tracking near the user's eyes.

[0135] According to another embodiment of the present application, an eye tracking method is provided, which is applied to a smart head mounted device. The eye tracking method includes the following steps S100 to S300:

[0136] Step S100, forming a uniform light field covering a specific area of ​​the eye at a set distance from the wearer's eyeball by at least one projector 1; wherein the projector 1 comprises an inclined light source 11 and an optical diffraction element 12 located on a light output path of the light source 11, and a set inclination angle θ is formed between the light source 11 and the plane where the wearer's eyeball is located, 25°≤θ≤35°;

[0137] Step S200, acquiring an eye image of a uniform light field formed by the optical diffraction element 12 and reflected by the eyeball through the receiving camera 2;

[0138] Step S300: Process the eye image through a processor to determine the pupil center position information of the wearer, and then obtain the gaze point of the wearer's eyes.

[0139] In some examples of the present application, the processing of the eye image by a processor to determine the pupil center position information of the wearer, and then obtaining the gaze point of the wearer's eyes, includes the following steps S310 to 330:

[0140] Step S310: Processing the received eye image by the processor, extracting the image feature area from the eye image, and then determining the position information of the pupil center of the wearer;

[0141] Step S320, calculating the wearer's sight line vector through a sight line estimation model according to the position information of the pupil center;

[0142] Step S330: Obtain the gaze point of the wearer's eyes according to the sight line vector.

[0143] According to the embodiments provided by the present application, when the eye tracking method is applied to a smart head mounted device, the eye tracking accuracy can be improved. Specifically, by forming a uniform light field covering a specific area of ​​the eye at a set distance from the wearer's eye, the brightness difference caused by the off-axis light source relative to the eye can be reduced, thereby eliminating the impact on the image acquisition end and achieving high-resolution imaging.

[0144] Among them, by adopting deep learning methods for line of sight estimation, the recognition accuracy can be further improved and the accuracy of eye tracking can be ensured.

[0145] Compared with the traditional pupil-corneal reflection method, the eye tracking method provided in the embodiment of the present application is not easily disturbed by complex ambient light and can maintain a stable tracking effect under various lighting conditions.

[0146] High-precision eye tracking and iris recognition can enhance the interactive experience of the device, allowing the wearer to control the device more naturally and accurately through their eyes.

[0147] The uniform light field and stable tracking effect ensure that the wearer will not feel visual fatigue or discomfort when using the device for a long time.

[0148] The eye tracking method of the present application provides an efficient and accurate eye tracking solution for smart head-mounted devices.

[0149] According to another embodiment of the present application, a smart head-mounted device is provided, which includes the eye tracking system as described above.

[0150] In some examples of the present application, the eye tracking system is set to two, one of the eye tracking systems corresponds to the user's left eye, and the other eye tracking system corresponds to the user's right eye.

[0151] The smart head mounted device provided in the embodiment of the present application may be an AR device or a VR device.

[0152] In augmented reality (AR) and virtual reality (VR) devices, eye tracking is essential to providing an immersive experience. By tracking the direction and gaze of both eyes, the device can adjust the displayed content in real time to provide a more interactive and realistic experience.

[0153] The specific implementation methods of the eye tracking method and the smart head-mounted device in the embodiments of the present application can refer to the various embodiments of the eye tracking system mentioned above, and therefore have at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be repeated here one by one.

[0154] The above embodiments focus on the differences between the various embodiments. As long as the different optimization features between the various embodiments are not contradictory, they can be combined to form a better embodiment. Considering the simplicity of the text, they will not be repeated here.

[0155] Although some specific embodiments of the present application have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are only for illustration, not for limiting the scope of the present application. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present application. The scope of the present application is defined by the appended claims.

Claims

1. An eye tracking system, applied to a smart head-mounted device, characterized in that: include: At least one projector (1) comprises a light source (11) and an optical diffraction element (12), wherein the light source (11) is tilted to form a set tilt angle θ with the plane where the wearer's eyeball is located, 25°≤θ≤35°, and the optical diffraction element (12) is located on the light output path of the light source (11) and is used to form a uniform light field covering a specific area of ​​the eye at a set distance from the wearer's eyeball; a receiving camera (2), which is arranged corresponding to the projector (1) and is used to obtain an eye image reflected by the eyeball and in a uniform light field formed by the optical diffraction element (12); and The processor is used to determine the pupil center position information of the wearer according to the eye image, and then obtain the gaze point of the wearer's eyes.

2. The eye tracking system according to claim 1, characterized in that: The set inclination angle θ is 30°.

3. The eye tracking system according to claim 1, characterized in that: A collimating element (13) is provided between the light source (11) and the optical diffraction element (12).

4. The eye tracking system according to claim 1, characterized in that: The light source (11) is a light emitting diode (LED), an edge emitting laser (EEL) or a vertical cavity surface emitting laser (Vcsel).

5. The eye tracking system according to claim 1, wherein: The light source (11) is a vertical cavity surface emitting laser (Vcsel), and the vertical cavity surface emitting laser (Vcsel) comprises a plurality of light emitting regions, each of which is configured to be able to emit light independently.

6. The eye tracking system according to claim 1, characterized in that: The optical diffraction element (12) is any one of DOE, MOE and diffuser.

7. The eye tracking system according to claim 1, characterized in that: The light source (11) is arranged on a substrate (14), and the substrate (14) is any one of a PCB substrate, a copper substrate, an aluminum substrate and a BT substrate.

8. The eye tracking system according to claim 7, characterized in that: The projector (1) further comprises a housing (15), the housing (15) is cylindrical, the substrate (14) and the optical diffraction element (12) are respectively arranged at two opposite ends of the housing (15); The center of the light source (11) and the center of the optical diffraction element (12) are located on the same straight line.

9. An eye tracking method, applied to a smart head mounted device, characterized in that: include: A uniform light field covering a specific area of ​​the eye is formed at a set distance from the eyeball of the wearer by at least one projector (1); wherein the projector (1) comprises an inclined light source (11) and an optical diffraction element (12) located on a light output path of the light source (11); a set inclination angle θ is formed between the light source (11) and the plane where the eyeball of the wearer is located, 25°≤θ≤35°; Acquiring an eye image reflected by the eyeball and within a uniform light field formed by the optical diffraction element (12) through a receiving camera (2); The eye image is processed by a processor to determine the pupil center position information of the wearer, and then the gaze point of the wearer's eyes is obtained.

10. The eye tracking method according to claim 9, characterized in that: The processing of the eye image by a processor to determine the pupil center position information of the wearer, and then obtaining the gaze point of the wearer's eyes, includes: Processing the received eye image by the processor, extracting the image feature area from the eye image, and then determining the position information of the pupil center of the wearer; Calculating the wearer's sight line vector through a sight line estimation model according to the position information of the pupil center; The gaze point of the wearer's eyes is obtained according to the sight line vector.

11. A smart head mounted device, characterized in that: include: An eye tracking system as claimed in any one of claims 1 to 8.

12. The smart head mounted device according to claim 11, characterized in that: The eye tracking systems are set to two, one of which corresponds to the user's left eye, and the other corresponds to the user's right eye.

Images