Eye movement tracking method and device based on optical waveguide near-infrared luminous point

Abstract

The invention relates to the technical field of eye movement tracking, in particular to an eye movement tracking method and device based on an optical waveguide near-infrared luminous point, and the method comprises the steps: controlling a near-infrared light machine to generate illumination light in a near-infrared band, and guiding the illumination light to an in-coupling region of a near-infrared light waveguide lens; the illumination light is coupled into the near-infrared light waveguide lens through an in-coupling diffractive optical element arranged in the in-coupling area, so that the illumination light is transmitted in the near-infrared light waveguide lens along a predetermined propagation path and reaches the out-coupling area; in the out-coupling area, an out-coupling diffractive optical element is used for emitting the illumination light from the near-infrared light waveguide lens to form a plurality of near-infrared light-emitting points distributed on the near-infrared light waveguide lens, and the emitting direction of the near-infrared light-emitting points is made to point to the pupil area of the user, so that corresponding cornea reflection light spots are formed on the cornea surface of the user; an infrared eye movement camera is used for collecting an eye image containing a pupil area and a cornea reflection light spot; and performing feature extraction on the eye image to obtain an image coordinate of the pupil center and an image coordinate of the cornea reflection facula, establishing an eyeball geometric model based on a cornea reflection method, and performing calculation to obtain the sight line direction of the user. According to the invention, the problems that the frame is too wide, the view shielding is too large and the wearing comfort is insufficient when the eye movement glasses adopt a cornea reflex method can be solved.

Description

Technical Field

[0001] This invention relates to the field of eye-tracking technology, and in particular to an eye-tracking method and apparatus based on near-infrared emitting points of optical waveguides. Background Technology

[0002] In existing eye-tracking methods, especially glasses-based eye-tracking, corneal reflection is widely used. This method typically uses near-infrared illumination to create a reflected light spot on the cornea, and then combines this with features such as the pupil to perform eye-tracking calculations. To improve the observability and coverage of the reflected light spot, multiple near-infrared light sources are often placed on the lens frame in engineering implementations, and structural methods are used to ensure that the light sources illuminate the pupil area as much as possible. However, this approach easily leads to a wider lens frame, increased distance between the lens and the surface, and a significant conflict between visual field obstruction and wearing comfort. Summary of the Invention

[0003] In view of the above technical problems, the present invention provides an eye-tracking method and device based on near-infrared light-emitting points of optical waveguides, which aims to solve the problems of excessively wide frames, excessive field of vision obstruction, and insufficient wearing comfort when using corneal reflection method for eye-tracking glasses, which require multiple near-infrared light sources to be arranged on the frame and the frame to be pushed away from the face to ensure illumination coverage.

[0004] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0005] According to one aspect of the present invention, an eye-tracking method based on near-infrared emitting points of optical waveguides is proposed, the method comprising: The near-infrared optical engine is controlled to generate illumination light in the near-infrared band, and the illumination light is guided to the ingress coupling region of the near-infrared optical waveguide lens; The illumination light is coupled into the near-infrared waveguide lens by an ingress coupling diffraction optical element disposed in the ingress coupling region, so that the illumination light is transmitted along a predetermined propagation path in the near-infrared waveguide lens and reaches the outgress coupling region. In the out-coupling region, the illumination light is emitted from the near-infrared waveguide lens using an out-coupling diffraction optical element, forming multiple near-infrared emitting points distributed on the near-infrared waveguide lens, and the emission direction of the multiple near-infrared emitting points is directed towards the user's pupil area, so as to form a corresponding corneal reflective spot on the user's corneal surface. An infrared eye-tracking camera is used to capture an image of the eye, including the pupil area and the corneal reflective spot. Feature extraction is performed on the eye image to obtain the image coordinates of the pupil center and the image coordinates of the corneal reflective spot. Based on the corneal reflection method, an eyeball geometric model is established to calculate the user's gaze direction.

[0006] Furthermore, there are two near-infrared waveguide lenses, corresponding to the user's eyes, and each near-infrared waveguide lens has an input coupling area near the side of the eyeglass frame and an output coupling area near the bridge of the nose; the near-infrared optical engine is disposed in the side area of ​​the eyeglass frame and aligned with the corresponding input coupling area, so that the near-infrared optical engine is located outside the user's main line of sight but can still supply light to the near-infrared waveguide lenses; the infrared eye-tracking camera is disposed in the bridge of the eyeglass frame and faces the user's eyes.

[0007] Furthermore, the near-infrared waveguide lens includes a waveguide substrate forming a total internal reflection transmission channel, and a diffraction structure disposed on the surface, inner surface, or interlayer of the waveguide substrate; the out-coupled diffraction optical element includes multiple out-coupled sub-regions distributed along the surface of the near-infrared waveguide lens, each out-coupled sub-region being used to emit the illumination light and form multiple near-infrared emitting points, such that the multiple near-infrared emitting points are distributed in a predetermined pattern on the near-infrared waveguide lens and form multiple mutually distinguishable corneal reflective spots on the user's cornea; wherein, the predetermined pattern distribution matches the imaging field of view of the infrared eye-tracking camera and the reachable range of the pupil region, so that when the user's gaze rotates or the range of eye movement changes, the infrared eye-tracking camera can still simultaneously observe the center of the pupil and at least part of the corneal reflective spots.

[0008] Furthermore, the near-infrared waveguide lens is a curved waveguide lens adapted to the user's facial contours. The curved waveguide lens is a concentric curved waveguide structure formed by inner and outer curved surfaces with a common center of curvature, or a bent waveguide structure formed by an equivalent curvature distribution. In addition, the ingress coupling diffraction optical element and the egress coupling diffraction optical element have varying grating periods along the curvilinear coordinate direction of the curved waveguide lens to compensate for astigmatic propagation errors introduced by the curved waveguide. This ensures that the incident angle of the illumination light relative to the local surface normal remains consistent or approximately consistent during multiple total internal reflections when propagating within the curved waveguide lens, thereby reducing the outgoing direction drift of the multiple near-infrared emitting points and reducing the deformation expansion of the corneal reflected light spot.

[0009] Furthermore, obtaining the variable periodicity distribution of the incident-coupled diffractive optical element includes: A coordinate system is established with the curvature center or equivalent optical axis of the curved waveguide lens as a reference, and curve coordinates are defined in the coupling region along the surface unfolding direction. Multiple sampling positions are selected on the curve coordinates. For each sampling position, the local surface normal direction of the curved waveguide lens and the incident direction of the incident light emitted by the near-infrared optomechanism when it reaches the sampling position are determined. The target coupling condition is determined so that the incident light enters the curved waveguide lens after diffraction and propagates relative to the local surface normal at a predetermined propagation angle within the waveguide, thereby achieving an angle range that satisfies total internal reflection propagation within the waveguide and ensuring consistency of propagation characteristics in the meridional and sagittal directions. Under the conditions of predetermined diffraction order and predetermined waveguide refractive index, based on the phase matching relationship between the incident direction, the local surface normal and the target coupling condition, the local grating period corresponding to each sampling position is calculated and a periodic distribution curve of the grating period as a function of the curve coordinates is formed. The processing data of the ingress coupling diffraction optical element is generated based on the periodic distribution curve, so that after the illumination light is coupled into the curved waveguide lens through the ingress coupling diffraction optical element, the propagation angle of the light entering the waveguide at different sampling positions remains consistent, thereby reducing the focusing effect and coupling aberration introduced by the curved surface.

[0010] Furthermore, obtaining the variable periodicity distribution of the out-coupled diffractive optical element includes: Based on the geometric model of the curved waveguide lens, curve coordinates are defined in the out-coupling region and multiple sampling positions are selected; For each sampling position, determine the angle of incidence of the light ray incident on that sampling position within the waveguide relative to the local surface normal; The target emission conditions are determined based on the predetermined spatial relationship between the user's eye and the curved waveguide lens. The target emission conditions are used to direct the diffracted light rays toward the pupil area and cover a predetermined eye movement range so that an observable corneal reflective spot can still be formed on the cornea when there is a change in the position of the user's eye. Under predetermined diffraction order conditions, based on the phase matching relationship between the target emission conditions and the incident angle, the local grating period corresponding to each sampling position is obtained, and a periodic distribution curve of the grating period changing with the curve coordinate is formed; and the local grating period at the reference sampling position located near the equivalent optical axis is matched with the reference grating period of the input coupling diffraction optical element to form an approximately coaxial input coupling, waveguide propagation, and output coupling optical path.

[0011] Furthermore, determining the periodic distribution curve also includes: The periodic distribution curve is represented as a parameterized function containing multiple parameters to be optimized, and the analytical initial periodic distribution obtained based on the surface geometric model is used as the initial solution of the parameterized function. Construct a set of combinations describing the position distribution of the exit pupil of the near-infrared optomechanical system and the angle distribution of the illumination light, and for each combination in the set, calculate the waveguide in-waveguide propagation angle coupled into the curved waveguide lens by the input coupling diffraction optical element, thereby obtaining the coupling angle error relative to a predetermined reference propagation angle; Construct a set of combinations describing the possible positional distribution of the user's pupil and the directional distribution of the field of view, and for each combination in the set, calculate the exit angle and incident position of the out-coupled diffractive optical element, thereby obtaining the exit angle error or angular dispersion relative to a predetermined reference exit angle. With the goal of minimizing the combined evaluation index of the coupling angle error and the emission angle error, the parameters to be optimized are iteratively adjusted until the combined evaluation index reaches the convergence condition.

[0012] Furthermore, the input coupling diffraction optical element and the output coupling diffraction optical element are implemented in the form of holographic optical elements, and the fabrication of the holographic optical elements includes: A recording layer is formed by laying photosensitive holographic material on a planar recording substrate; Based on the periodic distribution curve, the target surface period of the recording layer at multiple curve coordinate positions is converted into the corresponding recording beam incident angle condition, and an interference recording optical path composed of a reference beam and an object beam is set accordingly. The reference beam is an approximately planar wavefront to characterize the equivalent wavefront of light propagating in the waveguide, and the object beam is an approximately cylindrical wavefront or a wavefront with curvature to characterize the equivalent wavefront of light propagating in external space. The object beam is converged or diverged by a cylindrical lens or an equivalent wavefront shaping device so that the interference fringes form the variable periodic distribution on the recording layer. After recording is completed, the formed holographic optical element is attached to the corresponding surface or interlayer of the curved waveguide lens to form the ingress coupling diffraction optical element and the egress coupling diffraction optical element; wherein, the curved waveguide lens is formed by bending or equivalent forming process of planar transparent waveguide sheet through mold, so that the curved waveguide lens maintains a predetermined curvature and reduces the period deviation caused by deformation after being assembled with the eyeglass frame in the wearing state; and, the egress coupling diffraction optical element has a varying diffraction efficiency distribution at different positions along the propagation direction of the illumination light in the curved waveguide lens.

[0013] Furthermore, the step of establishing an eyeball geometric model based on the corneal reflection method and calculating the line of sight direction includes: The pupil boundary is detected in the eye image, and the pupil center is obtained by boundary fitting; Detect multiple corneal reflective spots in the eye image and establish a correspondence between the corneal reflective spots and multiple near-infrared emitting points; Based on the spatial positions of the multiple near-infrared emitting points on the near-infrared waveguide lens, the emission directions of the multiple near-infrared emitting points, and the law of light reflection on the corneal surface, the spatial position of the corneal center is obtained by inverse calculation, and the three-dimensional position correction of the pupil center is performed in combination with the corneal refraction effect to obtain the optical axis of the eyeball. The eye's optical axis is fused with the calibrated visual axis offset to obtain the gaze direction. During calibration, a mapping relationship between the gaze direction and external reference coordinates is established under multiple fixation target conditions. When the eye-tracking device experiences changes in assembly stress, the near-infrared waveguide lens undergoes slight displacement, or the actual emission direction of multiple near-infrared emitting points deviates, the mapping relationship is updated and compensated based on the relative geometric relationship of the corneal reflected light spot.

[0014] According to another aspect of the present invention, an eye-tracking device based on a near-infrared emitting point of an optical waveguide is provided. The eye-tracking device includes an eyeglass frame, a near-infrared optical waveguide lens, a near-infrared optical engine optically coupled to the near-infrared optical waveguide lens, an infrared eye-tracking camera facing the wearer's eye, and a data processing unit electrically connected to the near-infrared optical engine and the infrared eye-tracking camera; the data processing unit is used to perform the method as described above.

[0015] The technical solution disclosed herein has the following beneficial effects: Compared to the solution of directly arranging multiple near-infrared light sources around the frame, this disclosure couples near-infrared light into the waveguide lens and emits it towards the eye area at a predetermined position on the lens as an equivalent light-emitting point. This provides illumination for the pupil area to form corneal reflective spots without significantly increasing the size of the frame, reducing visual field obstruction and wearing burden caused by widening the frame or raising the nose pads. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the eye-tracking device based on near-infrared light-emitting points of optical waveguides in the embodiments of this specification; Figure 2 This is a schematic diagram of the cross-sectional structure of the near-infrared waveguide lens in the embodiments of this specification; Figure 3 This is a flowchart of an eye-tracking method based on near-infrared emitting points of optical waveguides, as described in the embodiments of this specification. Figure 4 This is a schematic diagram of the path of light propagating in the waveguide substrate in the embodiments of this specification; Figure 5 This is a schematic diagram of light being coupled to the inner surface of a curved waveguide in the meridional plane in an embodiment of this specification. Detailed Implementation

[0017] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0018] Furthermore, the accompanying drawings are merely illustrative of this disclosure. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0019] In one embodiment, an eye-tracking device based on a near-infrared emitting point of an optical waveguide is provided. (See reference...) Figure 1 As shown, the eye-tracking device includes an eyeglass frame 12, a near-infrared waveguide lens 1, a near-infrared optical engine 13 optically coupled to the near-infrared waveguide lens 1, an infrared eye-tracking camera 14 facing the wearer's eyes, and a data processing unit 15 electrically connected to the near-infrared optical engine 13 and the infrared eye-tracking camera 14. During operation, the eye-tracking glasses are worn on the user's head. The near-infrared waveguide lens 1 is positioned in front of the user's eyes, and the infrared eye-tracking camera 14 is aimed at the user's eyes. There are two near-infrared waveguide lenses 1, corresponding to the user's eyes, and each near-infrared waveguide lens 1 has an input coupling area 2 near the side of the eyeglass frame and an output coupling area 3 near the bridge of the nose. The near-infrared optical engine 13 is positioned in the side area of ​​the eyeglass frame and aligned with the corresponding input coupling area 2, so that the near-infrared optical engine 13 is outside the user's main visual channel but can still supply light to the near-infrared waveguide lens 1. The infrared eye-tracking camera 14 is positioned in the bridge of the eyeglass frame and faces the user's eyes.

[0020] like Figure 2As shown, the incident light 4 output by the near-infrared optical engine 13 enters the near-infrared waveguide lens 1 through the coupling structure of the input coupling region 2. It is transmitted along a predetermined propagation path in the waveguide substrate to the output coupling region 3, and then emitted through diffraction and other means in the output coupling region 3 to form the output light 5, which forms distributed near-infrared light emission points. This allows the near-infrared illumination light to reach the pupil area in a structure that is closer to that of ordinary glasses and to form a reflected light spot on the cornea that can be used for calculation. This "side-to-side light supply - waveguide transmission - side emission" layout allows the near-infrared optical engine to avoid the main visual channel, while making the emission position closer to the center of the eye. This is beneficial for obtaining stable illumination coverage under conditions of small frame size and small visual field obstruction.

[0021] Additionally, it is worth noting that eye-tracking devices are mostly VR headsets, which differ from existing nearsighted and farsighted glasses. The lenses of VR headsets are essentially displays, so diffraction structures can be set inside the display (near-infrared waveguide lenses).

[0022] In one implementation, such as Figure 2 As shown, the data processing unit is used to execute an eye-tracking method based on near-infrared light-emitting points of optical waveguides to achieve eye tracking of the user. Both the near-infrared optical engine and the infrared eye-tracking camera are electrically connected to the data processing unit, enabling the data processing unit to output control signals to drive the near-infrared optical engine to complete illumination emission and timing synchronization, and to receive image data from the infrared eye-tracking camera and complete subsequent feature extraction and eye-tracking calculation. This process is typically executed cyclically in a continuous frame manner, thereby forming a gaze direction result updated over time. The same process can be executed separately for both eyes to obtain binocular eye-tracking data or fusion results. The method specifically includes the following steps S101-S105: In step S101, the near-infrared optical engine is controlled to generate illumination light in the near-infrared band and the illumination light is directed to the ingress coupling region of the near-infrared optical waveguide lens.

[0023] Specifically, the near-infrared optomechanism generates illumination light in the near-infrared band and guides the illumination light to the coupling region of the near-infrared waveguide mirror. The near-infrared optomechanism may include a near-infrared emitting light source and collimating / shaping optics to ensure that the output beam has a suitable angular distribution and beam cross-section for coupling. The process of guiding the illumination light into the coupling region can be achieved through a free-space optical path, a reflector / folder, or a directly attached coupling interface, ensuring that the illumination light is aligned with the coupling structure in space and meets the incident conditions for entering the waveguide in terms of angle, thereby providing stable incident light for subsequent coupling into the waveguide.

[0024] In step S102, the illumination light is coupled into the near-infrared waveguide lens by the ingress coupling diffraction optical element disposed in the ingress coupling region, so that the illumination light is transmitted along a predetermined propagation path in the near-infrared waveguide lens and reaches the outgress coupling region.

[0025] In this process, an ingress-coupled diffractive optical element positioned in the ingress coupling region couples the illumination light into a near-infrared waveguide mirror. This allows the illumination light to propagate along a predetermined path within the near-infrared waveguide mirror and reach the egress coupling region. The ingress-coupled diffractive optical element can be positioned on the waveguide surface and diffracts the incident light from the external space into a propagation angle within the waveguide that satisfies the total internal reflection propagation condition. This allows the light to undergo multiple reflections along a predetermined direction within the waveguide and propagate to the egress coupling region. When the waveguide has a curved surface structure and the ingress coupling structure is located on the curved surface, the local grating period satisfies the grating equation for the curved surface in relation to the incident angle, the waveguide refractive index, and the propagation angle within the waveguide. ; in, Let L be the local period that varies along the arc length coordinate L of the surface. Let be the angle between the external incident light and the local normal at the point of incidence. The refractive index of the waveguide material, The propagation angle within the waveguide (relative to the local normal at the point of incidence). For diffraction orders, The wavelength of radiation; by letting As L varies, phase matching can be achieved at different positions on the surface, and the desired result can be obtained more stably. This suppresses inconsistencies in coupling angles caused by curvature and improves coupling efficiency and propagation consistency.

[0026] In step S103, in the out-coupling region, the illumination light is emitted from the near-infrared waveguide lens using the out-coupling diffraction optical element to form multiple near-infrared emitting points distributed on the near-infrared waveguide lens, and the emission direction of the multiple near-infrared emitting points is directed towards the user's pupil area to form a corresponding corneal reflective spot on the user's corneal surface.

[0027] In the out-coupling region, an out-coupling diffractive optical element directs illumination light from the near-infrared waveguide lens, forming multiple near-infrared emitting points distributed on the lens. These emitting points are directed towards the user's pupil area, creating a corresponding corneal reflection spot on the corneal surface. The out-coupling diffractive optical element can be configured as multiple discrete or continuous outgoing sub-regions along the waveguide propagation direction, allowing light propagating within the waveguide to be successively emitted from different positions and manifested as multiple spatially distributed near-infrared emitting points on the lens. The outgoing direction of these emitting points is designed to cover the pupil area, thereby forming a corneal reflection spot observable by the camera and providing features for eye-tracking calculations.

[0028] In step S104, an infrared eye-tracking camera is used to acquire an eye image including the pupil area and corneal reflective spot.

[0029] This process utilizes an infrared eye-tracking camera to capture images of the eye, including the pupil area and corneal reflections. The infrared eye-tracking camera faces the eye to image; under near-infrared illumination, the corneal reflection appears as a bright feature in the image, and the pupil area exhibits segmentable boundary features under infrared imaging. The captured images may include dynamic factors such as eyelash occlusion, eyelid changes, and eye movement. Therefore, the acquisition process is typically coordinated with the emission timing of the near-infrared optomechanism to maintain stable contrast across consecutive frames and reduce the impact of ambient light interference on the corneal reflection.

[0030] In step S105, feature extraction is performed on the eye image to obtain the image coordinates of the pupil center and the image coordinates of the corneal reflection spot. Based on the corneal reflection method, an eyeball geometric model is established to calculate the user's gaze direction.

[0031] Feature extraction is performed on eye images to obtain the image coordinates of the pupil center and the corneal reflective spot. An eye geometry model is then built based on the corneal reflectivity method to calculate the user's gaze direction. Feature extraction may include pupil boundary detection and fitting to determine the pupil center position, and detection, filtering, and pairing of corneal reflective spots to obtain a set of corneal reflective spot coordinates. During geometric modeling and calculation, the laws of refraction and reflection can be used to calculate parameters such as the spatial coordinates of the corneal center and pupil center, and reconstruct the eye geometry model, thereby estimating the gaze direction. When multiple near-infrared emitting points exist on the lens, multiple corneal reflective spots can be formed to provide redundancy constraints. This allows for stable calculations based on the remaining spots even when some spots are obscured or contrast decreases. It also facilitates compensation and updates for systematic errors caused by wearing displacement or optical path deviation, thereby improving the robustness and consistency of continuous eye tracking.

[0032] In one embodiment, the near-infrared waveguide lens includes a waveguide substrate forming a total internal reflection transmission channel, and a diffraction structure disposed on the surface, inner surface, or interlayer of the waveguide substrate; the out-coupled diffraction optical element includes multiple out-coupled sub-regions distributed along the surface of the near-infrared waveguide lens, each out-coupled sub-region being used to emit illumination light and form multiple near-infrared emitting points, such that the multiple near-infrared emitting points are distributed in a predetermined pattern on the near-infrared waveguide lens and form multiple mutually distinguishable corneal reflective spots on the user's cornea; wherein, the predetermined pattern distribution matches the imaging field of view of the infrared eye-tracking camera and the reachable range of the pupil area, so that when the user's gaze moves or the range of eye movement changes, the infrared eye-tracking camera can still simultaneously observe the center of the pupil and at least part of the corneal reflective spots.

[0033] Furthermore, such as Figures 4-5 As shown, Figure 4 The path of light propagation in the waveguide substrate is shown. Figure 5The diagram illustrates the relationship between infrared light emitted from radiation source 6 and coupled to the inner surface 8 of a curved waveguide within the meridional plane, and the center of curvature 7. Radiation source 6 can be a near-infrared optomechanism 13 or a corresponding diffraction structure. The waveguide substrate can be defined by opposing inner and outer surfaces, creating a light propagation path. Illuminating light, after entering the waveguide substrate, undergoes multiple reflections between the inner and outer surfaces to satisfy total internal reflection conditions and propagates along the in-plane direction of the mirror, allowing light energy to be transmitted from the coupling position to the emission position within a relatively thin mirror structure. Within the meridional plane of the curved waveguide, the incident angle and reflection angle of the light relative to the waveguide surface normal exhibit a paired characteristic on the inner and outer surfaces, which can be expressed as: ; And satisfy: ; in, , , , Let be the angle of propagation of light relative to the local normal at different surfaces. , Let be the radius of curvature of the inner and outer surfaces of the waveguide. This relationship reflects the change in the normal direction of the curved waveguide with position. If a constant-period diffraction structure is used in the out-coupling region, each interaction with the diffraction structure easily forms a diffraction response relative to a fixed direction of the local normal. However, the normals at different positions are not parallel, thus producing a focusing-like effect in the meridional plane, causing the out-of-emission direction to drift with position. This makes the spatial distribution of the corneal reflection spot unstable with changes in wearing status or line of sight. Therefore, it is more suitable to divide the out-coupling diffraction optical element into multiple out-coupling sub-regions and design the local parameters of each sub-region differently, so that multiple near-infrared emitting points can both satisfy the predetermined pattern distribution and form a stable and distinguishable corneal reflection spot within the reachable range of the pupil region. Furthermore, the design of the diffraction structure in the curved surface coordinates can use a local period that varies along the arc length coordinate of the curved surface to satisfy phase matching. The basic matching relationship of the in-coupling or out-coupling can be expressed as: ; in, Let L be the local period that varies along the arc length coordinate L of the surface. Let be the angle between the external incident light and the local normal at the point of incidence. The refractive index of the waveguide material, The propagation angle within the waveguide is denoted by m, and the diffraction order is denoted by m. Where L is the radiation wavelength; and, to facilitate the arrangement and numbering of the coupling sub-regions within the lens surface, the arc length coordinate L of the curved surface can be correlated with rectangular coordinates, for example, within the meridional plane: ; ; This allows for geometric matching of the positional distribution of the outgoing coupling sub-region with the imaging field of view of the infrared eye-tracking camera. Furthermore, to ensure that the emission directions of multiple near-infrared emitting points cover the reachable range of the pupil region, the period of the outgoing coupling diffraction optical element can be designed to vary with L to control the emission angle and ensure that the emitted light falls within the finite-sized eye-tracking box. The periodic variation law within the meridional plane can be written as: ; In the sagittal plane, it can be written as ; in, Characterizing the equivalent imaging distance, The periodicity at the reference location represents the coupling period; as the equivalent imaging distance approaches infinity, the periodicity variation within the meridional plane can be expressed as: ; By employing different local periodicity and local efficiency distributions in different outgoing coupling sub-regions, the predetermined pattern distribution of multiple near-infrared emitting points on the lens can be matched with the reachable range of the pupil region. This allows the infrared eye-tracking camera to simultaneously observe the pupil center and at least part of the corneal reflective spot even when the user's gaze moves or the eye movement range changes. This multi-spot redundancy enhances the continuity and robustness of tracking.

[0034] Continue as Figure 4-5 As shown, the near-infrared waveguide lens is a curved waveguide lens adapted to the user's facial contours. The curved waveguide lens is a concentric curved waveguide structure formed by inner and outer curved surfaces with a common curvature center, or a bent waveguide structure formed by an equivalent curvature distribution. Furthermore, the input coupling diffraction optical element and the output coupling diffraction optical element have varying grating periods along the curved coordinate direction of the curved waveguide lens to compensate for astigmatic propagation errors introduced by the curved waveguide. This ensures that the incident angle of the illumination light relative to the local surface normal remains consistent or approximately consistent during multiple total internal reflections when propagating within the curved waveguide lens, thereby reducing the drift of the output direction of multiple near-infrared emitting points and reducing the deformation expansion of the corneal reflected light spot.

[0035] Among them, the near-infrared waveguide lens is a curved waveguide lens adapted to the user's facial contours. The curved waveguide lens is a concentric curved waveguide structure formed by inner and outer curved surfaces with a common curvature center, or a bent waveguide structure formed by an equivalent curvature distribution. Furthermore, the input coupling diffraction optical element and the output coupling diffraction optical element have varying grating periods along the curved coordinate direction of the curved waveguide lens to compensate for astigmatic propagation errors introduced by the curved waveguide. This ensures that the incident angle of the illumination light relative to the local surface normal remains consistent or approximately consistent during multiple total internal reflections when propagating within the curved waveguide lens, thereby reducing the drift of the output direction of multiple near-infrared emitting points and reducing the deformation and expansion of the corneal reflected light spot.

[0036] Furthermore, the concentric curved waveguide structure can be understood as a meniscus channel composed of cylindrical or approximately cylindrical surfaces with the same center of curvature on both the inner and outer surfaces. This allows light to maintain its incident angle characteristics with the local normal as a reference each time it is reflected from the surface when it propagates in the meridional plane. This gives the same ray propagating within the surface a "maintaining the incident angle with the local normal" rule. However, when a constant-period coupling structure is used, each coupling interaction will form a diffraction response with an approximately fixed direction around the local normal. Since the local normals at different positions on the surface are not parallel, the result will be a focusing-like effect in the meridional plane. This causes the outgoing direction to drift systematically with the curvilinear coordinates and expands the outgoing point corresponding to the same spatial direction into an angular dispersion within a certain range. This manifests on the cornea as stretching of the reflected light spot, ghosting-like expansion, or geometric deformation.

[0037] Furthermore, the typical manifestation of astigmatic propagation error is that the coupling / emission angle components of each ray are no longer consistent in the meridional plane, while the corresponding components are more consistent in the sagittal plane. This difference will be amplified in the case of "finite aperture beams" rather than single rays, making the emission angles or equivalent focal lengths corresponding to different rays inconsistent. Ultimately, this will make the shape and position of the light spot more prone to drift as the line of sight rotates, the range of eye movement changes, or the wearer is slightly displaced.

[0038] Furthermore, to ensure that the incident angle relative to the local normal remains consistent or approximately consistent during multiple total internal reflections in a curved waveguide, the curvilinear coordinates on the surface can be parameterized using arc length, and the position of this arc length can be expressed using angular coordinates related to the center of curvature. For example, the angular coordinates can be made to satisfy... ; in, L represents the angular coordinates of the surface position relative to the center of curvature, and L represents the arc length coordinates of the surface. This represents the radius of the curved surface; under this representation, the grating period changing with L is equivalent to the grating period changing with L. This variation allows for compensation of local phase-matching conditions at different curved surface positions, making the propagation angles of light entering or exiting the waveguide more consistent relative to the local normal at various positions, reducing the cumulative angular deviation caused by normal rotation. Furthermore, the curved waveguide lens can be a strictly concentric inner and outer curved surface, or a bent structure formed by an equivalent curvature distribution; for the latter, local arc segments can be approximated as concentric curved surfaces with varying periodicity rules set for each segment, ensuring the overall effect of "compensating for angular errors along the curve coordinates" is still met, resulting in a shape that better conforms to facial contours and optically more controllable propagation. To further suppress outgoing direction drift and stabilize the outgoing light direction, consistency constraints on the input and output coupling periods can be established at selected reference positions, ensuring the reference ray is symmetrical relative to the local normal at the input and output coupling points, for example, satisfying: , as well as ; in, This indicates the out-coupling period at the reference position. Indicates the in-coupling period at the reference position. Indicates the propagation angle within the waveguide at the reference position. This represents the reference emission angle. Under this constraint, the reference ray can form an approximately coaxial coupling-propagation-emission relationship on the curved surface, reducing systematic bias and providing a unified reference for the periodic changes along the curve coordinates. Furthermore, the variation period is not only used to meet the angle compensation of a single ray, but can also be optimized for "finite-size eye trackers" in eye-tracking applications. This results in smaller angular errors and more convergent angular dispersion of the emitted light under different combinations of pupil positions and different line-of-sight directions. Consequently, the reflected light spots formed by multiple near-infrared emitting points on the cornea maintain a more stable geometric shape and distinguishability over a larger eye-tracking range, reducing the center positioning error and matching failure probability caused by the expansion of light spot deformation.

[0039] In one embodiment, the ingress coupling diffraction optical element and the egress coupling diffraction optical element are implemented in the form of a holographic optical element, and the fabrication of the holographic optical element includes: A photosensitive holographic material is deposited on a planar recording substrate to form a recording layer. Based on the periodic distribution curve, the target surface period at multiple curve coordinate positions of the recording layer is converted into the corresponding incident angle conditions of the recording beam. An interferometric recording optical path consisting of a reference beam and an object beam is then set accordingly. The reference beam is an approximately planar wavefront to characterize the equivalent wavefront of light propagating within the waveguide, and the object beam is an approximately cylindrical wavefront or a wavefront with curvature to characterize the equivalent wavefront of light propagating in external space. The object beam is converged or diverged using a cylindrical lens or an equivalent wavefront shaping device to ensure that the interference fringes are aligned. A variable periodic distribution is formed on the recording layer. After recording is completed, the formed holographic optical element is attached to the corresponding surface or interlayer of the curved waveguide lens to form an ingress coupling diffraction optical element and an egress coupling diffraction optical element. The curved waveguide lens is formed by bending or equivalent forming of a planar transparent waveguide sheet through a mold, so that the curved waveguide lens maintains a predetermined curvature and reduces the periodic deviation caused by deformation after being assembled with the eyeglass frame in the wearing state. Furthermore, the egress coupling diffraction optical element has a varying diffraction efficiency distribution at different positions along the propagation direction of the illumination light within the curved waveguide lens.

[0040] When using holographic recording to realize the coupling element, the recording process can be completed on a planar tooling substrate, which facilitates the formation of a high-precision spatial frequency distribution and reduces the difficulty of surface processing. At the same time, surface propagation analysis usually describes the position of the coupling point in curve coordinates, while the exposure surface of holographic recording is in planar coordinates. Therefore, it is necessary to convert the curve coordinates into an axis vector of the planar recording coordinates to establish the recording scheme. That is, first use curve coordinates to describe the position change of the coupling point along the surface, and then map the position change to the planar coordinate direction of the recording layer to ensure that the spatial frequency change with position obtained by recording is consistent with the target periodic distribution curve.

[0041] Furthermore, the reference beam employs an approximate planar wavefront, enabling the spatial frequency formed within the recording layer to characterize the equivalent wavefront of approximately parallel propagation within the waveguide; the object beam employs an approximate cylindrical wavefront or a wavefront with curvature, capable of characterizing the equivalent wavefront of light propagating in external space, and through cylindrical lenses, achieves different convergence / divergence shaping of the meridional and sagittal planes, thereby allowing the recorded fringes to exhibit significant changes in one direction while remaining relatively stable or changing as required in another direction, in order to match the differences in propagation characteristics of the meridional and sagittal planes in the curved waveguide.

[0042] Furthermore, after the curved waveguide lens is bent from a flat sheet, the normal direction of its inner and outer surfaces changes with the position. If the holographic structure after bonding experiences adhesion stress or secondary deformation, the spatial frequency will shift. Therefore, maintaining the curvature by molding and controlling the deformation in the assembly state helps to reduce the influence of periodic deviation on the coupling angle and emission angle.

[0043] Furthermore, the variation of the out-coupling diffraction efficiency along the propagation direction can be used to compensate for the energy attenuation caused by the gradual outgoing of optical power within the waveguide, making the outgoing brightness at each position more consistent. This results in a more balanced intensity among the multiple near-infrared emitting points, improving the contrast consistency of the corneal reflective spot in imaging and reducing threshold drift and detection instability caused by local overbrightness or underbrightness.

[0044] As an explanation, obtaining the variable periodicity distribution of the input-coupled diffractive optical element includes: establishing a coordinate system with the curvature center or equivalent optical axis of the curved waveguide mirror as a reference, and defining curvilinear coordinates along the surface unfolding direction in the input coupling region; selecting multiple sampling positions on the curvilinear coordinates, and determining the local surface normal direction of the curved waveguide mirror and the incident direction of the incident light emitted from the near-infrared optomechanism when it reaches that sampling position for each sampling position; determining the target coupling condition, which is used to ensure that the incident light enters the curved waveguide mirror after diffraction and propagates relative to the local surface normal at a predetermined propagation angle within the waveguide, thereby achieving total reflection within the waveguide. The angular range of the light propagation is determined, and the propagation characteristics in the meridional and sagittal directions are consistent. Under predetermined diffraction order and predetermined waveguide refractive index, based on the phase matching relationship between the incident direction, local surface normal, and target coupling conditions, the local grating period corresponding to each sampling position is calculated, and a periodic distribution curve of the grating period as a function of the curve coordinates is formed. The processing data of the incident coupling diffraction optical element is generated according to the periodic distribution curve, so that after the illumination light is coupled into the curved waveguide lens by the incident coupling diffraction optical element, the propagation angle of the light entering the waveguide at different sampling positions remains consistent, thereby reducing the focusing effect and coupling aberration introduced by the curved surface.

[0045] Furthermore, the coupling position is described using curvilinear coordinates, which can directly reflect the true geometric distance of the coupling point along the unfolding direction of the surface. This makes it easy to connect the "sampling position - local normal - incident direction - target propagation angle" into a calculable relationship chain, and to give each sampling position a unique local period. The core of this approach is to ensure that the light rays at different positions obtain consistent propagation angle characteristics after entering the waveguide, so that the beam does not generate additional angular dispersion due to different positions when propagating in the curved channel, thereby suppressing beam spot expansion and astigmatic errors caused by curvature.

[0046] Furthermore, the incident holographic structure can achieve local periodic changes on the recording layer by changing the recording incident angle condition at each position, that is, the periodic distribution curve is equivalently converted into the "recording angle distribution curve", so that the final holographic structure is consistent with the incident periodic distribution of the target in spatial frequency. Thus, in engineering, the "variable periodic diffraction structure" is converted into a feasible process path of "variable angle interference exposure".

[0047] As an explanation, obtaining the variable periodic distribution of the out-coupled diffractive optical element includes: defining curvilinear coordinates in the out-coupled region based on the geometric model of the curved waveguide lens and selecting multiple sampling positions; determining the incident angle of the light rays incident on that sampling position within the waveguide relative to the local surface normal for each sampling position; determining the target emission conditions based on the predetermined spatial relationship between the user's eye and the curved waveguide lens, the target emission conditions being used to direct the diffracted light rays towards the pupil region and cover a predetermined eye movement range, so that an observable corneal reflection spot can still be formed on the cornea even when there is a change in the position of the user's eye; under the predetermined diffraction order conditions, obtaining the local grating period corresponding to each sampling position based on the phase matching relationship between the target emission conditions and the incident angle, and forming a periodic distribution curve of the grating period varying with the curvilinear coordinates; and matching the local grating period at the reference sampling position located near the equivalent optical axis with the reference grating period of the in-coupled diffractive optical element to form an approximately coaxial in-coupled, waveguide propagation, and out-coupled optical path.

[0048] Furthermore, the outgoing coupling design takes "waveguide incident angle - target outgoing direction - eye movement range coverage" as input and calculates the local period for each sampling position, so that the light emitted from each position can fall into the reachable range of the pupil area, thereby enabling multiple outgoing points to jointly form an illumination coverage that is friendly to the eye movement range; at the same time, the period matching of the reference sampling position is equivalent to establishing a geometric benchmark within the system, so that the coupling and outgoing of the light near the center maintain an approximately symmetrical relationship, reducing the systematic bias and providing a unified reference for the period changes of the two sides.

[0049] Furthermore, when using a holographic structure to achieve out-coupling, the requirement that "the local period changes with the curve coordinates" can be transformed into the requirement that the incident angle of the recording beam changes with the recording coordinates. That is, an interference pattern with a gradually changing spatial frequency is formed on the recording plane, so that after being attached to the curved surface, it still corresponds to the gradually changing period in the curve coordinate direction, thereby achieving a stable out-pointing direction and a small directional drift on the curved surface.

[0050] The process of determining the periodic distribution curve includes: representing the periodic distribution curve as a parameterized function containing multiple parameters to be optimized, and using the analytical initial periodic distribution obtained based on the surface geometric model as the initial solution of the parameterized function; constructing a set of combinations describing the position distribution of the exit pupil plane of the near-infrared optomechanical system and the angle distribution of the illumination light, and for each combination in the set, calculating the waveguide propagation angle coupled into the curved waveguide lens by the incident coupler diffractive optical element, thereby obtaining the coupling angle error relative to a predetermined reference propagation angle; constructing a set of combinations describing the possible position distribution of the user's pupil and the direction distribution of the field of view, and for each combination in the set, calculating the exit angle and incident position of the coupler diffractive optical element, thereby obtaining the exit angle error or angular dispersion relative to a predetermined reference exit angle; and iteratively adjusting the parameters to be optimized until the comprehensive evaluation index meets the convergence condition, with the goal of minimizing the coupling angle error and the exit angle error.

[0051] Furthermore, the starting point of this optimization process is that, within the limited eye-tracking range, it is necessary to ensure that all multiple angular components are emitted into the limited area of ​​the eye-tracking box, while suppressing astigmatic errors introduced by surface propagation. Therefore, it is not possible to guarantee no error within the extended range solely based on the analytical periodic distribution. During optimization, the possible positions of the pupil within the eye-tracking box can be enumerated, and for each position, the light entering the pupil from different visual directions can be calculated. The "angular error under all combinations" is used as an indicator to evaluate the degree of distortion, and the minimization of this angular error is used as the objective to correct the periodic variation pattern.

[0052] Furthermore, during computation, the premise that "light has been coupled into the waveguide and propagates without additional distortion within the waveguide" can be used as an approximation, allowing the optimization focus to be on the angular deviation introduced during the coupling process. Optimization can also be performed only in the meridional direction to reduce computational complexity. The parameterized periodic distribution obtained after iterative convergence not only satisfies the coaxial constraint under the central reference condition but also maintains a smaller angular dispersion when the pupil position and field of view change, thus making the formed corneal reflective spot more stable in spatial position and shape, and more conducive to continuous detection and matching.

[0053] In one embodiment, establishing an eye geometry model and calculating the gaze direction based on corneal reflection includes: detecting the pupil boundary in an eye image and obtaining the pupil center through boundary fitting; detecting multiple corneal reflective spots in an eye image and establishing a correspondence between the corneal reflective spots and multiple near-infrared emitting points; based on the spatial positions of the multiple near-infrared emitting points on the near-infrared waveguide lens, the emission directions of the multiple near-infrared emitting points, and the law of light reflection on the corneal surface, inversely calculating the spatial position of the corneal center, and combining the corneal refraction effect to perform three-dimensional position correction of the pupil center to obtain the eye optical axis; fusing the eye optical axis with the calibrated visual axis offset to obtain the gaze direction; wherein, during calibration, a mapping relationship between the gaze direction and external reference coordinates is established under multiple fixation target conditions, and when the eye tracking device experiences changes in assembly stress, the near-infrared waveguide lens undergoes slight displacement, or the actual emission directions of the multiple near-infrared emitting points deviate, the mapping relationship is updated and compensated based on the relative geometric relationship of the corneal reflective spots.

[0054] Furthermore, pupil boundary detection can first perform grayscale normalization and highlight suppression on the eye image, and then obtain a set of candidate boundary points through threshold segmentation, edge detection or gradient-based contour search. Ellipse fitting or circle / ellipse model fitting is then performed on the boundary point set to obtain a stable pupil center even when there is local occlusion, eyelash interference or non-ideal pupil projection. Outlier removal can be introduced during the fitting process to avoid corneal highlight reflection, eyelid edges, etc. being mistakenly included in the boundary point set, causing center drift.

[0055] Furthermore, corneal reflective spot detection can leverage the characteristics of high brightness and localized intensity surges under near-infrared illumination. Multiple corneal reflective spot centers can be obtained through candidate bright spot extraction and morphological screening. Features such as area, peak intensity, and compactness can be calculated for each bright spot to eliminate non-reflective spot targets. When establishing correspondences, a predetermined pattern topology of the emitting points on the lens can be used to construct a graph structure or distance matrix from multiple corneal reflective spots in the image, and then matched with the predetermined pattern. This associates each corneal reflective spot with its source emitting point, avoiding identity swapping under multiple spot conditions.

[0056] Furthermore, the inverse calculation of the corneal center can rely on the geometric constraints of the law of reflection, which are satisfied by the "incident ray - corneal surface - reflected ray": the outgoing direction of the light source can be regarded as the incident direction, and the imaging light path of the infrared eye-tracking camera can be regarded as the direction of the reflected ray. Finding the center position of the surface that satisfies multiple sets of reflection constraints in three-dimensional space is equivalent to solving for the most consistent intersection point or the minimum residual point among multiple constraint rays. Multiple corneal reflected light spots provide redundant constraints, so that the solution can still be maintained by the remaining light spots when the quality of some light spots deteriorates or is lost for a short time, and incorrect matches can be screened out through residual consistency.

[0057] Furthermore, the three-dimensional position correction of the pupil center can take into account the refractive effect of the corneal medium: the pupil center observed by the infrared eye-tracking camera is the apparent position after refraction through the cornea. By combining the obtained corneal center position with the corneal curvature approximation model to compensate for the refraction of the line of sight, the apparent pupil center can be converted into a pupil center that is closer to the real spatial position, thereby obtaining a more stable optical axis of the eyeball; there is a fixed offset between the optical axis and the visual axis, which can be obtained through multi-target fixation calibration and then fused in the subsequent process.

[0058] Furthermore, the established mapping relationship can take the form of regression or geometric mapping, mapping the "directional quantity obtained based on the optical axis" to the line-of-sight direction of the external reference coordinate system. The mapping can simultaneously absorb individual differences, device wearing differences, and camera installation deviations. When changes in assembly stress or micro-displacement of the lens cause changes in the equivalent position and emission direction of the light-emitting point, the equivalent exit pupil replication and emission direction of the waveguide system will shift accordingly. This is directly manifested as an overall drift or local stretching of the relative geometric configuration of the corneal reflective spot in the image. At this time, the "relative distance, angle, and topological relationship between corneal reflective spots" can be used as self-consistency quantities to detect mapping mismatch and update the mapping parameters without changing the user's gaze task or requiring only a small amount of correction, thereby compensating for slow-changing drifts. This update method based on relative geometric relationships does not depend on the absolute position of a single spot and can maintain the consistency of the line-of-sight output even with slippage and slight deviations in the optical path.

[0059] As can be seen from the above embodiments, compared with the solution of directly arranging multiple near-infrared light sources around the frame, the present invention couples near-infrared light into the optical waveguide lens and emits it towards the eye area at a predetermined position on the lens in the form of an equivalent light-emitting point. This allows the pupil area to be illuminated for forming corneal reflective spots without significantly increasing the size of the frame, reducing visual field obstruction and wearing burden caused by widening the frame or raising the nose pads.

[0060] Furthermore, this disclosure addresses the challenges of information redundancy and sequence alignment in continuous sign language recognition by introducing a recognition framework based on spatial temporal modeling and attention. This enables accurate end-to-end continuous sign language recognition, and the recognition results are further used to drive animation displays, thereby improving the stability and comprehensibility of recognition and expression, and facilitating a better user experience in real-world scenarios.

[0061] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0062] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An eye-tracking method based on near-infrared emitting points of optical waveguides, characterized in that, The method includes: The near-infrared optical engine is controlled to generate illumination light in the near-infrared band, and the illumination light is guided to the ingress coupling region of the near-infrared optical waveguide lens; The illumination light is coupled into the near-infrared waveguide lens by an ingress coupling diffraction optical element disposed in the ingress coupling region, so that the illumination light is transmitted along a predetermined propagation path in the near-infrared waveguide lens and reaches the outgress coupling region. In the out-coupling region, the illumination light is emitted from the near-infrared waveguide lens using an out-coupling diffraction optical element, forming multiple near-infrared emitting points distributed on the near-infrared waveguide lens, and the emission direction of the multiple near-infrared emitting points is directed towards the user's pupil area, so as to form a corresponding corneal reflective spot on the user's corneal surface. An infrared eye-tracking camera is used to capture an image of the eye, including the pupil area and the corneal reflective spot. Feature extraction is performed on the eye image to obtain the image coordinates of the pupil center and the image coordinates of the corneal reflective spot. Based on the corneal reflection method, an eyeball geometric model is established to calculate the user's gaze direction.

2. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 1, characterized in that, The near-infrared waveguide lens consists of two lenses, corresponding to the user's eyes. Each lens has an input coupling area near the side of the eyeglass frame and an output coupling area near the bridge of the nose. The near-infrared optical engine is positioned in the side of the eyeglass frame and aligned with the corresponding input coupling area, so that the near-infrared optical engine is located outside the user's main visual channel but can still supply light to the near-infrared waveguide lens. The infrared eye-tracking camera is positioned in the bridge of the eyeglass frame and faces the user's eyes.

3. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 1, characterized in that, The near-infrared waveguide lens includes a waveguide substrate forming a total internal reflection transmission channel, and a diffraction structure disposed on the surface, inner surface, or interlayer of the waveguide substrate; the out-coupled diffraction optical element includes multiple out-coupled sub-regions distributed along the surface of the near-infrared waveguide lens, each out-coupled sub-region being used to emit the illumination light and form multiple near-infrared emitting points, such that the multiple near-infrared emitting points are distributed in a predetermined pattern on the near-infrared waveguide lens and form multiple mutually distinguishable corneal reflective spots on the user's cornea; wherein, the predetermined pattern distribution matches the imaging field of view of the infrared eye-tracking camera and the reachable range of the pupil region, so that when the user's gaze rotates or the range of eye movement changes, the infrared eye-tracking camera can still simultaneously observe the center of the pupil and at least part of the corneal reflective spots.

4. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 1, characterized in that, The near-infrared waveguide lens is a curved waveguide lens adapted to the user's facial contours. The curved waveguide lens is a concentric curved waveguide structure formed by inner and outer curved surfaces with a common center of curvature, or a bent waveguide structure formed by an equivalent curvature distribution. Furthermore, the ingress coupling diffraction optical element and the egress coupling diffraction optical element have varying grating periods along the curvilinear coordinate direction of the curved waveguide lens to compensate for astigmatic propagation errors introduced by the curved waveguide. This ensures that the incident angle of the illumination light relative to the local surface normal remains consistent or approximately consistent during multiple total internal reflections when propagating within the curved waveguide lens, thereby reducing the outgoing direction drift of the multiple near-infrared emitting points and reducing the deformation expansion of the corneal reflected light spot.

5. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 4, characterized in that, Obtaining the variable periodicity distribution of the incident-coupled diffractive optical element includes: A coordinate system is established with the curvature center or equivalent optical axis of the curved waveguide lens as a reference, and curve coordinates are defined in the coupling region along the surface unfolding direction. Multiple sampling positions are selected on the curve coordinates. For each sampling position, the local surface normal direction of the curved waveguide lens and the incident direction of the incident light emitted by the near-infrared optomechanism when it reaches the sampling position are determined. The target coupling condition is determined so that the incident light enters the curved waveguide lens after diffraction and propagates relative to the local surface normal at a predetermined propagation angle within the waveguide, thereby achieving an angle range that satisfies total internal reflection propagation within the waveguide and ensuring consistency of propagation characteristics in the meridional and sagittal directions. Under the conditions of predetermined diffraction order and predetermined waveguide refractive index, based on the phase matching relationship between the incident direction, the local surface normal and the target coupling condition, the local grating period corresponding to each sampling position is calculated and a periodic distribution curve of the grating period as a function of the curve coordinates is formed. The processing data of the ingress coupling diffraction optical element is generated based on the periodic distribution curve, so that after the illumination light is coupled into the curved waveguide lens through the ingress coupling diffraction optical element, the propagation angle of the light entering the waveguide at different sampling positions remains consistent, thereby reducing the focusing effect and coupling aberration introduced by the curved surface.

6. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 4, characterized in that, Obtaining the variable periodicity distribution of the out-coupled diffraction optical element includes: Based on the geometric model of the curved waveguide lens, curve coordinates are defined in the out-coupling region and multiple sampling positions are selected; For each sampling position, determine the angle of incidence of the light ray incident on that sampling position within the waveguide relative to the local surface normal; The target emission conditions are determined based on the predetermined spatial relationship between the user's eye and the curved waveguide lens. The target emission conditions are used to direct the diffracted light rays toward the pupil area and cover a predetermined eye movement range so that an observable corneal reflective spot can still be formed on the cornea when there is a change in the position of the user's eye. Under predetermined diffraction order conditions, based on the phase matching relationship between the target emission conditions and the incident angle, the local grating period corresponding to each sampling position is obtained, and a periodic distribution curve of the grating period changing with the curve coordinate is formed; and the local grating period at the reference sampling position located near the equivalent optical axis is matched with the reference grating period of the input coupling diffraction optical element to form an approximately coaxial input coupling, waveguide propagation, and output coupling optical path.

7. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 6, characterized in that, Determining the periodic distribution curve also includes: The periodic distribution curve is represented as a parameterized function containing multiple parameters to be optimized, and the analytical initial periodic distribution obtained based on the surface geometric model is used as the initial solution of the parameterized function. Construct a set of combinations describing the position distribution of the exit pupil of the near-infrared optomechanical system and the angle distribution of the illumination light, and for each combination in the set, calculate the waveguide in-waveguide propagation angle coupled into the curved waveguide lens by the input coupling diffraction optical element, thereby obtaining the coupling angle error relative to a predetermined reference propagation angle; Construct a set of combinations describing the possible positional distribution of the user's pupil and the directional distribution of the field of view, and for each combination in the set, calculate the exit angle and incident position of the out-coupled diffractive optical element, thereby obtaining the exit angle error or angular dispersion relative to a predetermined reference exit angle. With the goal of minimizing the combined evaluation index of the coupling angle error and the emission angle error, the parameters to be optimized are iteratively adjusted until the combined evaluation index reaches the convergence condition.

8. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 7, characterized in that, The input coupling diffraction optical element and the output coupling diffraction optical element are implemented in the form of a holographic optical element, and the fabrication of the holographic optical element includes: A recording layer is formed by laying photosensitive holographic material on a planar recording substrate; Based on the periodic distribution curve, the target surface period of the recording layer at multiple curve coordinate positions is converted into the corresponding recording beam incident angle condition, and an interference recording optical path composed of a reference beam and an object beam is set accordingly. The reference beam is an approximately planar wavefront to characterize the equivalent wavefront of light propagating in the waveguide, and the object beam is an approximately cylindrical wavefront or a wavefront with curvature to characterize the equivalent wavefront of light propagating in external space. The object beam is converged or diverged by a cylindrical lens or an equivalent wavefront shaping device so that the interference fringes form the variable periodic distribution on the recording layer. After recording is completed, the formed holographic optical element is attached to the corresponding surface or interlayer of the curved waveguide lens to form the ingress coupling diffraction optical element and the egress coupling diffraction optical element; wherein, the curved waveguide lens is formed by bending or equivalent forming process of planar transparent waveguide sheet through mold, so that the curved waveguide lens maintains a predetermined curvature and reduces the period deviation caused by deformation after being assembled with the eyeglass frame in the wearing state; and, the egress coupling diffraction optical element has a varying diffraction efficiency distribution at different positions along the propagation direction of the illumination light in the curved waveguide lens.

9. The eye-tracking method based on near-infrared emitting points of optical waveguides according to claim 1, characterized in that, The process of establishing an eyeball geometric model based on the corneal reflection method and calculating the line of sight direction includes: The pupil boundary is detected in the eye image, and the pupil center is obtained by boundary fitting; Detect multiple corneal reflective spots in the eye image and establish a correspondence between the corneal reflective spots and multiple near-infrared emitting points; Based on the spatial positions of the multiple near-infrared emitting points on the near-infrared waveguide lens, the emission directions of the multiple near-infrared emitting points, and the law of light reflection on the corneal surface, the spatial position of the corneal center is obtained by inverse calculation, and the three-dimensional position correction of the pupil center is performed in combination with the corneal refraction effect to obtain the optical axis of the eyeball. The eye's optical axis is fused with the calibrated visual axis offset to obtain the gaze direction. During calibration, a mapping relationship between the gaze direction and external reference coordinates is established under multiple fixation target conditions. When the eye-tracking device experiences changes in assembly stress, the near-infrared waveguide lens undergoes slight displacement, or the actual emission direction of multiple near-infrared emitting points deviates, the mapping relationship is updated and compensated based on the relative geometric relationship of the corneal reflected light spot.

10. An eye-tracking device based on near-infrared light-emitting points of optical waveguides, characterized in that, The eye-tracking device includes an eyeglass frame, a near-infrared waveguide lens, a near-infrared optical engine optically coupled to the near-infrared waveguide lens, an infrared eye-tracking camera facing the wearer's eyes, and a data processing unit electrically connected to the near-infrared optical engine and the infrared eye-tracking camera; the data processing unit is used to execute the eye-tracking method based on near-infrared light-emitting points of the waveguide as described in any one of claims 1-9.

Images