Eye movement tracking method, interaction method and wearable eye movement tracking device

Abstract

The embodiment of the invention provides an eye movement tracking method, an interaction method and a wearable eye movement tracking device.The eye movement tracking method is applied to the wearable eye movement tracking device.The eye movement tracking method comprises the steps that in response to an optical signal projected to the eye, the incidence direction of eyeball surface reflected light is obtained; and determining eye movement parameters according to the incident direction. According to some embodiments of the invention, the incident angle of the reflected light reflected by the cornea is acquired through the angle sensor, and the eye movement parameter is determined according to the value of the incident angle, so that the high precision of 0.1 degree can be realized, and the incident light angle can be determined and calculated only based on the readings of a few photodiodes (PD) to obtain the eye movement parameter; therefore, the function of integrating eye movement tracking on related equipment is realized.

Description

Technical Field

[0001] This application relates to the field of eye tracking. Specifically, embodiments of this application relate to an eye tracking method, an interaction method, and an eye tracking device. Background Art

[0002] The core objective of eye tracking technology is to infer the user's gaze direction (or line of sight direction) or fixation point, etc. by capturing and analyzing eye movement characteristics (such as pupil position, corneal reflection, electrooculogram signals of the eye muscles, etc.).

[0003] Eye tracking is a crucial human-machine interface in extended reality (XR) and artificial intelligence (AI) glasses, undertaking a human-machine input function similar to that of a mouse or multi-touch. At the same time, eye tracking also plays a crucial role in providing an immersive user experience. For example, it optimizes the graphics processing efficiency through foveated rendering, reduces the probability of motion sickness, and enables dynamic color correction.

[0004] However, the current integration of eye tracking technology in the fields of XR and AI glasses is still limited. The main technical challenges it faces in popularization include: First, existing solutions are difficult to effectively eliminate the optical path interference generated by corrective lenses and display optical systems. Second, the power consumption of eye tracking devices is large. Therefore, it is challenging to maintain the tracking performance under design constraints such as device positioning, volume, power consumption, weight, and cost, which brings challenges to the large-scale application of eye tracking technology in devices such as XR. Summary of the Invention

[0005] The objective of embodiments of this application is to provide an eye tracking method, an interaction method, and a wearable eye tracking device. Some embodiments of this application provide a method for collecting the incident angle of the reflected light via corneal reflection on a light signal detection unit by an angle sensor and determining eye movement parameters based on the value of the incident angle. For example, the difference between the gaze direction determined by embodiments of this application and the ideal gaze direction is only at the 0.1° level, improving the high-precision detection of the obtained eye movement parameters and enabling the determination of the incident light angle of the reflected light and thus the eye movement parameters based only on the readings of a few light signal detection units (such as photodiodes PD), thereby realizing the integration of the eye tracking function in related devices (such as extended reality (XR) and artificial intelligence (AI) glasses).

[0006] In a first aspect, an embodiment of the present application provides an eye movement tracking method, which is applied to a wearable eye movement tracking device. The eye movement tracking method includes: in response to a light signal projected onto the eye, obtaining the incident direction of the reflected light on the eye surface; and determining an eye movement parameter according to the incident direction.

[0007] By obtaining the incident direction of the reflected light and then determining the eye movement tracking parameter according to the incident direction, compared with the method of using a camera to capture an image and analyze the image for eye movement tracking, or using a scanning light source for eye movement tracking, the eye movement tracking method of the embodiment of the present application significantly reduces the power consumption of the device integrating the eye movement tracking method.

[0008] In some embodiments, the wearable eye movement tracking device includes an angle sensor. The angle sensor includes multiple channels, and one channel corresponds to one optical signal detection unit. Each optical signal detection unit corresponds to a target field of view within a set range; wherein, obtaining the incident direction of the reflected light on the eye surface includes: determining an incident angle corresponding to the incident direction according to the electrical signal output value of at least one optical signal detection unit.

[0009] In the embodiment of the present application, multiple optical signal detection units are integrated in the angle sensor, and the incident direction of the reflected light (whether the incident light can be) is determined according to the output values of these optical signal detection units. Furthermore, the incident direction can be calculated by a few optical signal detection units, effectively reducing the power consumption of the device integrating the eye movement tracking algorithm.

[0010] In some embodiments, the angular response spectra of the optical signal detection units in adjacent channels overlap; determining the incident angle corresponding to the incident direction according to the electrical signal output value of at least one optical signal detection unit includes: determining the incident angle according to the output values of multiple optical signal detection units with overlapping angular response spectra.

[0011] The angular response spectra between different channels of the angle sensor in some embodiments of the present application overlap, and the final incident direction is determined according to the overlap value. Therefore, it is not difficult to understand that a small angular change in the overlapping region will cause a coordinated change in the signals of multiple channels, breaking through the resolution limit of a single channel. If each channel covers a part of the FOV and the responses of adjacent channels overlap, then even if the eye moves to the junction of two channels, the data of the two channels can be interpolated to improve the accuracy of the eye movement tracking result.

[0012] In some embodiments, before obtaining the incident direction of the reflected light on the eye surface, the method further includes: pre-calibrating the angular response curve of each optical signal detection unit in the angle sensor, where the angular response curve is used to characterize the mapping relationship between the output electrical signal of the corresponding optical signal detection unit and the incident angle of the reflected light; wherein, obtaining the incident direction of the reflected light on the eye surface includes: determining the incident direction according to the angular response curve.

[0013] Some embodiments of the present application determine the angular response curve of each optical signal detection unit in the angle sensor by calibrating the mapping table. For example, the angular response curve includes the correspondence between the incident angle and the output current, so that the incident direction can be determined according to the angular response curve subsequently, improving the speed of data processing and reducing the power consumption of the device.

[0014] In some embodiments, before obtaining the incident direction of the reflected light on the eye surface, the method further includes: using a controllable light source to irradiate the angle sensor based on multiple groups of predefined two-dimensional incident angles respectively, and recording the electrical signals output by each optical signal detection unit in the angle sensor under the irradiation of each group of two-dimensional incident angles to obtain calibration data; using a multi-variable interpolation algorithm to construct a continuous function model based on the calibration data; wherein, obtaining the incident direction of the reflected light on the eye surface includes: reading the electrical signal output value of at least one optical signal detection unit in the angle sensor; inputting the electrical signal output value into the continuous function model, and obtaining the incident direction of the reflected light through inverse lookup or interpolation inversion.

[0015] Some embodiments of the present application rely on pre-calibrated data, have a fast calculation speed, and are suitable for real-time applications; the interpolation model can compensate for the non-linear response of the optical signal detection unit (for example, photodiode PD).

[0016] In some embodiments, before obtaining the incident direction of the reflected light on the eye surface, the method further includes: using a controllable light source to irradiate the angle sensor based on multiple groups of predefined two-dimensional incident angles respectively, and recording the electrical signals output by each optical signal detection unit in the angle sensor under the irradiation of each group of two-dimensional incident angles to obtain calibration data; wherein, obtaining the incident direction of the reflected light on the eye surface includes: taking the angle of the incident direction of the reflected light as an optimization variable; using a numerical optimization algorithm to search for the target solution of the objective function in the angular parameter space, and taking the target solution as the value of the optimization variable to obtain the incident direction.

[0017] Some embodiments of the present application can improve the high-precision detection of eye movement tracking results with only a small number of PDs through an optimization algorithm, avoiding relying on a complex sensor array and reducing power consumption.

[0018] In some embodiments, before determining the eye movement parameters according to the incident direction, the method further includes: collecting the known position and the gaze direction of the user's eyeball, and synchronously recording the incident angle readings output by the angle sensor to obtain calibration data; based on the calibration data, obtaining a function or mapping relationship between the eyeball position or the gaze direction and the incident angle to obtain a mapping relationship model; the determining the eye movement parameters according to the incident direction includes: inputting the incident angle reading corresponding to the incident direction into the mapping relationship model, and obtaining the real-time eyeball position and the gaze direction corresponding to the incident direction according to the mapping relationship model.

[0019] The calibration process of the embodiments of the present application supports personalized adjustment to adapt to the physiological differences of the eyeballs of different users.

[0020] In some embodiments, the determining the eye movement parameters according to the incident direction includes: obtaining the three-dimensional coordinates of the center of the sphere formed by the cornea; obtaining the three-dimensional coordinates of the pupil center; obtaining a vector to be recognized according to the three-dimensional coordinates of the center of the sphere formed by the cornea and the three-dimensional coordinates of the pupil center; obtaining the gaze direction corresponding to the vector to be recognized according to the vector visual direction mapping relationship, where the vector visual direction mapping relationship is a one-to-one correspondence relationship between the known gaze direction obtained by calibration and the vector formed by the coordinates of the center of the cornea sphere and the pupil coordinates.

[0021] In a second aspect, some embodiments of the present application provide an interaction method, where the interaction method includes: obtaining eye movement parameters according to the eye movement tracking method provided in any one of the embodiments described in the first aspect, where the eye movement parameters are the gaze direction; dynamically adjusting the focus area or the display content of the control in the interaction interface according to the gaze direction.

[0022] In a third aspect, some embodiments of the present application provide an interaction method for a virtual-real fusion scenario, where the interaction method includes: obtaining eye movement parameters according to the eye movement tracking method provided in any one of the embodiments described in the first aspect, where the eye movement parameters are the gaze direction; operating a virtual object according to the gaze direction.

[0023] In a fourth aspect, some embodiments of the present application provide a wearable eye movement tracking device, where the wearable eye movement tracking device includes: at least one group of light sources configured to project light signals onto the eyes so that the light signals can cover the surface of the eyeball; at least one angle sensor, one angle sensor being matched with one group of light sources, and each angle sensor being at least configured to collect the reflection signals of the light signals emitted by the matched group of light sources and output the incident angles of the reflection signals; a data processing unit configured to calculate eye movement parameters based on the output of the angle sensor.

[0024] In some embodiments, the wearable eye tracking device includes: a fixing component adapted to the user's head to maintain the relative distribution of each group of light sources and the matching angle sensors around the eyes.

[0025] In some embodiments, the fixing component includes a frame portion, a first angle sensor is embedded in the first side of the frame portion, and a group of light sources matching the first angle sensor is embedded in the second side of the frame portion, and the first side and the second side are opposite sides.

[0026] In some embodiments, the angle sensor further includes: a timing control circuit configured to make the matching light source group and the angle sensor work together through timing control.

[0027] In some embodiments, the angle sensor includes: an optical signal detection array composed of a plurality of optical signal detection units; a mask layer disposed on the array; wherein, the mask layer controls the angular range of the incident signals entering each optical signal detection unit in the optical signal detection array through a compound aperture.

[0028] In some embodiments, the compound aperture includes: a main aperture configured as a shared incident channel for all optical signal detection units in the angle sensor to provide the ability to receive wide-angle incident signals in a first angular range; secondary apertures distributed around the main aperture, each secondary aperture is correspondingly arranged with one optical signal detection unit in the angle sensor, and each secondary aperture forms a directional light channel with a set field of view angle through a metal mask.

[0029] In some embodiments, the optical signal detection array includes five photodiodes, wherein four of the five photodiodes are symmetrically distributed up, down, left, and right along the outer periphery of the remaining one photodiode, the main aperture is an annular hole disposed in the center above the five photodiodes, and the secondary apertures are 4 micro-holes distributed around the main aperture.

[0030] In some embodiments, the angle sensor includes: an optical signal detection array composed of a plurality of optical signal detection units; a microlens array disposed above the optical signal detection array; wherein, one optical signal detection unit is correspondingly arranged with one microlens in the microlens array, and there is a preset position offset between the microlens array and the optical signal detection array, and the preset position offset is used to ensure the difference and overlap of the angular response spectra of different optical signal detection units.

[0031] In some embodiments, the angle sensor includes: a plurality of light detection units; a lens correspondingly arranged with the plurality of light detection units for covering the plurality of light detection units; wherein, none of the plurality of light detection units is located on the focal plane of the lens.

[0032] In some embodiments, the angle sensor further includes: a polarizer located in the incident light path and configured to filter diffusely reflected light.

[0033] In some embodiments, the wearable eye tracking device further includes: a polarizer; wherein the angle sensor has a first channel and a second channel, and the polarizer is disposed in the first channel.

[0034] In some embodiments, the wearable eye tracking device further includes: a collimated light source module configured to convert the divergent light emitted by the light source into parallel light to ensure that the light covers the target area; or a dynamic angle control unit configured to monitor the pupil position in real time and adjust the angle of the light source so that the light source irradiates the target area.

[0035] In some embodiments, the wearable tracking device further includes: a light source signal modulation circuit configured to modulate a target group of light sources to project an optical signal toward the eye at a target frequency; and an optical signal detection modulation circuit configured to perform frequency modulation on each optical signal detection unit in a target angle sensor that matches the target group of light sources, so that each optical signal detection unit in the target angle sensor reads the detected optical signal at the target frequency.

[0036] In some embodiments of the present application, by performing a high-frequency modulation (such as from 10 kHz to 100 kHz) on the light source, both the light source LED and the optical signal detection unit (e.g., a photodiode PD) operate according to this frequency (i.e., the light source LED emits at this frequency and the photodiode reads the signal at this frequency), so that the optical signal detection unit is only sensitive to signals at this frequency, effectively filtering direct current signals DC or ambient light with slow variations.

[0037] In some embodiments, each angle sensor is further configured to detect a blink event through all output signals of all optical signal detection units in the corresponding angle sensor, so as to trigger an interaction operation or end the eye tracking operation.

[0038] In some embodiments of the present application, an angle sensor is used to detect a blink. Specifically, when it is confirmed that the signals of all optical signal detection units (e.g., photodiodes PD) in the angle sensor change instantaneously together (mainly skin reflection), a blink event can be determined. After that, the blink can be used as a signal for interaction or as a signal indicating that the eye movement no longer outputs data.

[0039] In some embodiments, the angle sensor is further configured to confirm the occurrence of a saccade through the attributes of the output signals of at least one optical signal detection unit.

[0040] Some embodiments of the present application detect discontinuous signal changes through an angle sensor to confirm the occurrence of saccades, and then can improve the interaction performance based on the saccade event.

[0041] In a fifth aspect, some embodiments of the present application provide an eye movement tracking device, where the eye movement tracking device includes: a frame assembly including at least one frame part; at least one group of optoelectronic components disposed on the frame part, and each group of optoelectronic components includes: a plurality of light sources configured to project a light signal with a preset wavelength onto an eye region, disposed in a first region of the frame part; an angle sensor disposed in a second region of the frame part, configured to receive the reflected light after the light signal is reflected by the eye and determine the incident direction of the reflected light; a processing unit configured to calculate eye movement parameters based on the incident direction; wherein, the first region and the second region are opposite sides of the frame part, and the angle sensor in each group of optoelectronic components is used to receive the reflected signal of the light signals emitted by the plurality of light sources in the corresponding optoelectronic component.

[0042] In some embodiments, the at least one group of optoelectronic components includes a first optoelectronic component, and the first optoelectronic component is any one group of optoelectronic components. Wherein, the plurality of light emitting diodes included in the first optoelectronic component are located in the left region of the frame part and the angle sensor included in the first optoelectronic component is located in the right region of the frame part, or the plurality of light emitting diodes included in the first optoelectronic component are located in the right region of the frame part and the angle sensor included in the first optoelectronic component is located in the left region of the frame part.

[0043] In some embodiments, the angle sensor includes: an optical signal detection array composed of a plurality of optical signal detection units; a mask layer located above the optical signal detection array, wherein the mask layer is provided with a composite aperture, and the composite aperture includes a main aperture and a plurality of secondary apertures, the main aperture is correspondingly arranged with all the optical signal detection units in the optical signal detection array, and one secondary aperture is correspondingly arranged with one optical signal detection unit in the optical signal detection array.

[0044] In some embodiments, the angle sensor includes: an optical signal detection array composed of a plurality of optical signal detection units; a microlens array disposed above the optical signal detection array; wherein, the optical signal detection units are correspondingly arranged with the microlenses in the microlens array one by one, and there is a preset position offset between the microlens array and the optical signal detection array, and the preset position offset is used to ensure that the angular response spectra of different optical signal detection units have differences and overlaps.

[0045] Sixth aspect, some embodiments of the present application provide an interaction method based on a gaze direction. The interaction method includes: in response to a change in the incident angle of the reflected light reflected from the eye surface caused by eye rotation, at least one optical signal detection unit located in different regions within an angle sensor is respectively configured to receive reflected light signals at different angles; output respective determined incident angle signals of the reflected light according to the angle response curves of the at least one optical signal detection unit; convert the incident angle signals into infrared light incident angles through a calibration mapping table or a machine learning model; map the incident angles to a gaze direction in combination with an eye geometry model; and perform an interaction operation according to the gaze direction.

[0046] In some embodiments, the interaction method further includes: in response to a detected blink event, triggering an interaction operation or ending an eye movement tracking operation, where the blink event is determined by the output values of all optical signal detection units in the angle sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 It is a block diagram of the module composition of the wearable eye movement tracking device provided by the embodiments of the present application;

[0049] Figure 2 It is one of the architecture diagrams of the angle sensor provided by the embodiments of the present application;

[0050] Figure 3 It is another architecture diagram of the angle sensor provided by the embodiments of the present application;

[0051] Figure 4 It is a third architecture diagram of the angle sensor provided by the embodiments of the present application;

[0052] Figure 5 It is a fourth architecture diagram of the angle sensor provided by the embodiments of the present application;

[0053] Figure 6 It is a schematic diagram of the wearable eye movement tracking device provided by the embodiments of the present application;

[0054] Figure 7 It is provided by the embodiments of the present application corresponding to Figure 8 One of the schematic diagrams of the working principle corresponding to glasses;

[0055] Figure 8Schematic diagram II of the working principle corresponding to the glasses provided by the embodiments of the present application Figure 8 ;

[0056] Figure 9 Flowchart of the eye movement tracking method provided by the embodiments of the present application

[0057] Figure 10 Schematic diagram of the forehead ratio isoline map provided by the embodiments of the present application

[0058] Figure 11 Flowchart of the interaction method based on the gaze direction provided by the embodiments of the present application Detailed implementation manners

[0059] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0060] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, terms such as "first" and "second" are only used for descriptive distinction and cannot be construed as indicating or implying relative importance.

[0061] In the research and development of eye movement tracking devices (such as wearable AI and AR glasses), how to reduce the device power consumption has become a technical problem to be solved urgently. This is because such devices are often limited by the compact battery design (usually with a capacity <500 mAh) and strict power budget (typical value <1 W). The current technical status shows that the power consumption of a single eye of the eye movement tracking module in the VR system is about 100 mW level, while the AI / AR glasses need to break through to achieve the ultra-low power consumption index of <10 mW for a single eye.

[0062] The related art uses a traditional camera to obtain eye-tracking parameters. The camera can be regarded as being composed of one million discrete photodiodes (i.e., 1000×1000 pixels, each pixel corresponding to a photodiode). Therefore, it is not difficult to understand that reading all pixels (regardless of the signal sparsity) consumes a large amount of power. To reduce power consumption, the related art provides a technical solution that uses a sparse photodiode array (on the order of a dozen) distributed around the eyes instead of a camera for eye tracking. These technical solutions choose to use light spots as the signal source (however, it is difficult to distinguish light spots from non-light spot signals such as diffuse reflection in the sclera region) to detect eye movement parameters. Some other related examples use the reflection difference between the white sclera and the gray / dark iris for eye tracking, but these methods usually have limited accuracy and cannot meet the requirements of low power consumption and high accuracy for eye tracking. Another eye-tracking solution that replaces the camera provided by the related art is to use a scanning light source. The specific implementation process includes: the light source (similar to a projector) sequentially scans the eye area at an angle, and at the same time, four photodiodes synchronously receive signals. By combining the light source direction (from the scanning projector), the diode signal (mainly specular reflection), and the position information of the light source and the sensor, the relative position and posture of the eyeball / cornea can be calculated. However, the power consumption of this device is also 10 milliwatts, so the power consumption is large and cannot meet the requirements of low power consumption for eye-tracking devices.

[0063] In at least one embodiment of the present application, in order to reduce the power consumption of an eye-tracking device or apparatus, a camera used in traditional eye-tracking technology is replaced with an angle sensor. The angle sensor in the embodiment of the present application can be used to accurately detect the incident direction of the reflected light on the surface of the eyeball, and then eye movement parameters such as the gaze direction can be obtained based on this incident direction. In the embodiment of the present application, there is a matching relationship between one angle sensor and multiple light sources, that is, the signals emitted by some light sources can only be collected and sensed by the angle sensor that matches them, and the incident direction of the reflected signal is obtained through this angle sensor. In this way, the embodiment of the present application can significantly reduce interference and reduce the power consumption of the eye-tracking device.

[0064] First, an exemplary wearable eye-tracking device that can be used to execute the eye-tracking method in the embodiment of the present application will be described below.

[0065] Some embodiments of the present application provide a wearable eye-tracking device, which includes: at least one group of light sources, configured to project light signals onto the eyes so that the light signals can cover the surface of the eyeball; at least one angle sensor, one angle sensor is matched with one group of light sources, and each angle sensor is configured to collect the reflected signal of the light signal emitted by the matching group of light sources and output the incident angle of the reflected signal; a data processing unit, configured to calculate eye movement parameters based on the output of the angle sensor

[0066] Please refer to Figure 1, Figure 1 A wearable eye movement tracking device 100 provided for some embodiments of the present application. The wearable eye movement tracking device 100 includes: a first group of light sources 110 and a first angle sensor 130 that matches the first group of light sources (matching means Figure 1 the first angle sensor 130 is configured to collect the reflected signal of the optical signal emitted by the first group of light sources 110 and output the incident angle of the reflected signal), a second group of light sources 120 and a second angle sensor 140 that matches the second group of light sources (matching means Figure 1 the second angle sensor 140 is configured to collect the reflected signal of the optical signal emitted by the second group of light sources 120 and output the incident angle of the reflected signal). It can be understood that Figure 1 is only used to exemplarily illustrate the exemplary architecture of the wearable eye movement tracking device 100 provided for some embodiments of the present application. In some other embodiments of the present application, the wearable eye movement tracking device 100 may also exemplarily include a third group of light sources, a fourth group of light sources, etc. and angle sensors respectively matching each group of light sources.

[0067] It should be noted that, in some embodiments of the present application, Figure 1 the data processing unit 160 is exemplarily provided in each angle sensor. For example, Figure 1 a data processing unit 160 is provided in the first angle sensor or Figure 1 a data processing unit 160 is provided in the second angle sensor. Correspondingly, the data processing unit 160 is configured to calculate eye movement parameters based on the output of the corresponding angle sensor; in some other embodiments of the present application, a common data processing unit may be provided for multiple angle sensors. Correspondingly, the data processing unit 160 is configured to calculate eye movement parameters based on the outputs of each angle sensor.

[0068] Next, in combination with Figures 2 - 5 exemplarily illustrate the angle sensors of some embodiments of the present application.

[0069] In some embodiments of the present application, each angle sensor of the wearable eye movement tracking device includes: an optical signal detection array and a mask layer disposed on the array. Among them, the optical signal detection array is composed of a plurality of optical signal detection units, and the mask layer controls the angle range of the incident signal entering each optical signal detection unit in the optical signal detection array through a composite aperture. It should be noted that the mask layer described in the embodiments of the present application includes but is not limited to metal materials, and other mask materials suitable for semiconductor processes may also be used. In some embodiments of the present application, the patterned mask layer contains materials that can achieve a predetermined shielding function, and its formation timing covers an optional process window from wafer-level processing to package-level processing.

[0070] For example, in some embodiments of the present application, the compound aperture includes: a main aperture configured as a shared incident channel for all optical signal detection units in the angle sensor (i.e., the sensor to which the main aperture belongs) to provide a wide-angle incident signal reception capability for a first angular range; a secondary aperture distributed around the main aperture, each secondary aperture corresponding to one optical signal detection unit in the angle sensor to which it belongs, and each secondary aperture forming a directional optical channel with a set field of view (FOV) through a metal mask.

[0071] The following will exemplarily illustrate an angle sensor with a compound aperture structure according to some embodiments of the present application in conjunction with Figure 2 and Figure 3 an angle sensor with a compound aperture structure according to some embodiments of the present application.

[0072] In some embodiments of the present application, there is provided an angle sensor as shown in Figure 2 The angle sensor includes an optical signal detection array 132 and a metal mask layer 131 located above the optical signal detection array. Among them, the compound aperture of the metal mask layer 131 exemplarily includes a main aperture 131-1 and three secondary apertures (i.e., a first secondary aperture 131-2, a second secondary aperture 131-3, and a third secondary aperture 131-4). The optical signal detection array 132 exemplarily includes a first photodiode 132-1, a second photodiode 132-2, and a third photodiode 132-3 (these photodiodes are taken as an example of optical signal detection units); in conjunction with Figure 2 it can be seen that Figure 2 all the photodiodes share a main aperture 131-1, and each photodiode is correspondingly arranged with a secondary aperture, that is, the first secondary aperture 131-2 is correspondingly arranged with the first photodiode 132-1, the second secondary aperture 131-3 is correspondingly arranged with the second photodiode 132-2, and the third secondary aperture 131-4 is correspondingly arranged with the third photodiode 132-3.

[0073] As Figure 3 shown, in some embodiments of the present application, the optical signal detection array of the angle sensor exemplarily includes five photodiodes (respectively located below Figure 3 the four oval orange patterns and a central circular orange pattern), Figure 3 only one photodiode 322 is shown (specifically referring to Figure 3 the top view 320, and this top view also provides the aperture structure formed by the mask layer above the photodiode). Among them, four of the five photodiodes (respectively located below Figure 3 the four oval orange patterns) are symmetrically distributed up, down, left, and right along the outer periphery of the remaining one photodiode (i.e., the photodiode located below the central orange circular pattern) (specifically referring to Figure 3Top view 310), the main aperture is an annular hole in the center above the five photodiodes, and the secondary aperture is 4 micro-holes distributed around the main aperture. It should be noted that Figure 3 The angular response spectral curves of the photodiode located at the central position and the photodiodes located on the left and right sides of this photodiode are also provided exemplarily. From this angular response spectrum, it can be seen that the angular response spectra of adjacent photodiodes have both differences and overlaps. That is to say, in the embodiments of the present application, the angular response spectra of the photodiodes PD in the angle sensor have differentiated main sensitive regions and the response spectra of adjacent photodiodes PD partially overlap. Since in the embodiments of the present application, each photodiode PD can form different field-of-view coverage ranges due to the mask layer with a compound aperture and has different light intensity responses to the incident angle, the incident direction can be solved by comparing the multi-PD signals. In some embodiments of the present application, the response spectra of adjacent photodiodes PD partially overlap, which can ensure that the angles that the angle sensor can detect are continuously covered without blind spots, improving the angle measurement accuracy and anti-noise ability. Figure 3 The abscissa of the angular response spectral image is used to provide the one-dimensional field-of-view angles of the set ranges corresponding to the respective photodiodes. For example, the field-of-view angle of the set range of the photodiode located at the central position is: -25 degrees to +25 degrees, the field-of-view angle of the set range of the photodiode located on the left side of this central photodiode is: -50 degrees to 0 degrees, and the field-of-view angle of the set range of the photodiode located on the right side of this central photodiode is: 0 degrees to 50 degrees. These field-of-view angle ranges belong to the horizontal field-of-view angle or the vertical field-of-view angle.

[0074] Such as Figure 3 As shown, the compound aperture of the angle sensor in some embodiments of the present application belongs to an optical microstructure, which realizes fine control of light through the design and arrangement of micron-level optical elements. The following is the annotation description of each component unit in Figure 3 It should be noted that the following description is only used to provide a specific example, and those skilled in the art can adopt technical solutions different from the following description according to specific needs:

[0075] 1. Shared main aperture ( Figure 3 The top annular area in

[0076] Function: As a common incident channel for all PDs (an example of an optical signal detection unit), it provides a wide-angle reception ability of ±45°.

[0077] Parameter: The diameter of this main aperture Vignetting design optimizes stray light suppression.

[0078] 2. Independent secondary aperture ( Figure 3 The oval or circular micro-holes above each PD in

[0079] Function: Form a directional receiving channel with a 15° field of view (FOV) through the mask layer

[0080] Parameter: Secondary aperture diameter It is distributed in a 45° equiangular pitch ring

[0081] 3. Mask layer structure

[0082] · Function: Define the secondary aperture boundary to prevent light crosstalk

[0083] · Process: Adopt photolithography technology to achieve μm-level precision control

[0084] Some embodiments of the present application ensure wide field of view tracking through the main aperture of the compound aperture, and high-precision angle resolution is achieved through the secondary aperture, which is particularly suitable for eye movement tracking scenarios that require simultaneous satisfaction of large-range detection and precise measurement. Its optical microstructure design enables the sensor to achieve multiple-fold improvement in angle resolution while maintaining a compact volume compared to traditional solutions.

[0085] Such as Figure 2 and Figure 3 In some embodiments of the present application, the angle sensor arranges multiple photodiodes (as a specific example of the optical signal detection unit) behind the mask layer. This structure can be monolithically integrated through chip design and manufacturing processes, and the overall package is of chip-level size (extremely small).

[0086] Taking the photodiode as the optical signal detection unit as an example below, the angle sensor with a compound aperture at the chip level as shown in Figure 2 or Figure 3 is analyzed from the aspects of the structure and manufacturing process, working principle, and technical advantages of the angle sensor. These angle sensors belong to the sensors of the mask layer stacked photodiode (PD) array scheme.

[0087] (1) Structure and manufacturing process

[0088] Mask layer design:

[0089] Cover a mask layer (such as a metal mask layer or a non-metal mask layer made of aluminum or copper, etc.) above the PD array, and an array of holes with a specific shape (such as a rectangle, a circle, or a slit) is opened on the mask. Each hole corresponds to a PD unit to limit the path of the incident light (as Figure 3 schematically shown). Example: The diameter of the mask hole is 50 μm, the pitch is 100 μm, and each hole only allows light within a specific angle range to pass through.

[0090] Monolithic integrated manufacturing:

[0091] 1. During the CMOS chip manufacturing process, a mask layer is directly formed above the PD array through photolithography and etching processes to achieve chip-level integration. 2. The overall size after packaging is extremely small (such as 3mm × 3mm), suitable for miniature devices (such as AR glasses).

[0092] (2) Working principle

[0093] Angle selectivity:

[0094] The holes in the mask layer, through geometric constraints (such as aperture size and position), only allow light rays with specific incident angles to reach the underlying PD (similar to the principle of a pinhole camera). Example: When light rays are incident at an angle θ, only the secondary aperture A allows the light to reach PD1, while the light rays of the secondary aperture B are blocked.

[0095] Multi-channel response overlap:

[0096] Those skilled in the art can design the coverage range of adjacent holes to make the sensitive angle parts of different PDs partially overlap, thereby achieving multi-channel signal redundancy.

[0097] (3) Technical advantages

[0098] Ultra-miniaturization: Chip-scale packaging (CSP) saves space and is suitable for wearable devices.

[0099] High consistency: The CMOS process ensures the precise alignment of the mask and the PD, reducing the calibration difficulty.

[0100] Low cost: Utilizing mature semiconductor processes is suitable for large-scale mass production.

[0101] In some embodiments of the present application, each angle sensor of the wearable eye movement tracking device includes: an optical signal detection array composed of a plurality of optical signal detection units (such as Figure 4 the optical signal detection array 132 shown) and a microlens array (such as Figure 4 the microlens array 133 shown), and the microlens array is disposed above the optical signal detection array; wherein, one optical signal detection unit is correspondingly disposed with one microlens in the microlens array, and there is a preset position offset between the microlens array and the optical signal detection array, and the preset position offset is used to ensure that the angle response spectra of different optical signal detection units have differences and overlaps. As Figure 4As shown, the optical signal detection array 132 exemplarily includes a first photodiode 132-1, a second photodiode 132-2, and a third photodiode 132-3, and the corresponding microlens array 133 exemplarily includes a first microlens 133-1, a second microlens 132-2, and a third microlens 132-3. The first microlens 133-1 is located above the first photodiode 132-1 and the centers of the two are offset. The second microlens 133-2 is located above the second photodiode 132-2 and the centers of the two are offset. The third microlens 133-3 is located above the third photodiode 132-3 and the two are offset. Those skilled in the art can adjust the size of the offset according to specific needs. The embodiment of the present application does not limit the value of the size of the offset.

[0102] It should be noted that the angle sensor of some embodiments of the present application adopts Figure 4 The architecture can also realize that the angular response spectrum of each photodiode has a differentiated main sensitive area and the response spectra of adjacent photodiodes PD partially overlap. In the embodiment of the present application Figure 4 Each photodiode PD can form a different field of view coverage range due to the center offset of its corresponding microlens, and responds differently to the light intensity of the incident angle. The incident direction can be solved by comparing multiple PD signals. In some embodiments of the present application, the response spectra of adjacent photodiodes PD partially overlap, which can ensure that the angle detectable by the angle sensor continuously covers no blind spots, thereby improving the angle measurement accuracy and noise resistance.

[0103] In some embodiments of the present application, Figure 5 As shown, the angle sensor includes: a plurality of light detection units (eg, Figure 5 The optical signal detection array 132 is composed of three optical signal detection units and a lens ( Figure 5 The lens 134 is shown, and the lens is arranged correspondingly to the optical signal detection array composed of multiple optical detection units (in Figure 5 The middle lens 134 corresponds to the light signal detection array 132 composed of three photodiodes, and the lens is used to cover the multiple light detection units, wherein the multiple light detection units are not located on the focal plane of the lens (such as Figure 5 The three photodiodes shown are not on the focal plane of the lens 134). Those skilled in the art can set the distance between the optical signal detection array 132 and the focal plane of the lens 134 according to actual needs, and this application does not limit the specific value of the distance.

[0104] It should be noted that the angle sensor of some embodiments of the present application adopts Figure 5The architecture can also realize that the angle response spectrum of each photodiode PD in the angle sensor has a differentiated main sensitive area and the response spectra of adjacent photodiodes PD partially overlap. Figure 5 Each photodiode PD has a different field of view coverage due to the lens, and responds differently to the light intensity of the incident angle. The incident direction can be solved by comparing multiple PD signals. In some embodiments of the present application, the response spectra of adjacent photodiodes PD partially overlap, which can ensure that the angle detectable by the angle sensor continuously covers no blind spots and improves the angle measurement accuracy and noise resistance.

[0105] In some embodiments of the present application, Figures 2 - 5 The angle sensor can be integrated with the bare-die photodiode at the module level using mask layers or microstructures.

[0106] Taking the photodiode as the light signal detection unit as an example, the following is an analysis of the structure and manufacturing process, working principle and technical advantages of the angle sensor obtained from the module level. Figures 2 - 5 Angle sensor shown.

[0107] 1) Structure and manufacturing process

[0108] Modular design:

[0109] Assemble the bare PD (unpackaged chip) with external microstructures (mask layer or microlens) into an independent module. Example: Bonding a microlens array on top of the PD bare die and coating the lens surface with a mask layer.

[0110] Microstructure implementation:

[0111] 1. Mask layer: Holes are formed on the glass / polymer substrate by photolithography or laser cutting.

[0112] 2. Microlens: Use photoresist thermal reflow or nanoimprint technology to make microlenses and adjust the focal length or tilt angle.

[0113] (2) Working principle

[0114] Light path control: 1. Incident light first passes through the mask aperture, filtering out light at non-target angles. 2. Microlenses focus light at specific angles onto corresponding PDs. Example: A tilted microlens focuses light at +20° horizontally onto PD1 and light at +25° horizontally onto PD2.

[0115] Angular response expansion: The focusing characteristics of the microlens can expand the effective detection angle range of a single PD and reduce the number of PDs required.

[0116] (3)Technological advantages

[0117] Design flexibility: The mask, lens, and PD parameters (such as material, aperture, focal length) can be independently optimized. Repairability: The modular design allows for the replacement of damaged microstructures, reducing maintenance costs. Compatibility: It is compatible with different models of PD dies and supports multi-vendor supply chains.

[0118] That is to say, the angle sensors of the embodiments of the present application are specially designed and manufactured (such as Figures 2 - 5 the angle sensor structure). These angle sensors can achieve overlapping angle response spectra (that is, the reading characteristics of each optical signal detection unit in the same angle sensor for light rays with different incident angles) for multiple channels (one channel is correspondingly set for one optical signal detection unit) within the target field of view (FOV). Figures 2 - 5 Each channel of the angle sensor has a specific response curve, that is, an angle response spectrum (for example, a function of voltage / current changing with the incident angle) for light rays with different incident angles. The response curves of multiple channels partially overlap within the target field of view (FOV), ensuring that light rays at certain incident angles can be detected by at least two channels, thereby improving the angle calculation accuracy and robustness. The angle sensors of some embodiments of the present application are composed of multiple independent detection units (channels), and each unit is regulated by an optical microstructure (such as a nanograting, a metasurface lens, or a microlens array), an aperture, or optoelectronic material properties to be sensitive to incident light within a specific angle range.

[0119] As disclosed above, in some embodiments of the present application, multiple photodiodes are arranged behind the mask layer (such as Figure 2 or as shown in 3). This structure can be monolithically integrated through chip design and manufacturing processes and is integrally packaged to a chip-level size (extremely small). In other embodiments of the present application, the angle sensor uses the microstructure of a mask and a microlens at the module level and is integrated with a die photodiode.

[0120] It should be noted that in some embodiments of the present application, signal intensity can be maximized and interference noise can be reduced through polarization control (for example, polarization control technology can be used to maximize signal intensity and suppress interference noise. The parallel polarization configuration between the LED and the detector can maximize the signal intensity of specular reflection).

[0121] For example, in some embodiments of the present application, the angle sensor further includes: a polarizer, which is located in the incident light path corresponding to the angle sensor and functions to filter the diffusely reflected light. This is because when the light source irradiates the eye, the reflected signal is divided into specular reflection and diffuse reflection. Among them, the reflection of Glint for determining eye movement parameters is specular reflection, and other parts of the eye (such as the iris, sclera or skin around the eye, etc.) mostly perform diffuse reflection on the signal incident by the light source. And most of the light in specular reflection is p-polarized light. Therefore, in some embodiments of the present application, a linear polarizer with p-polarization can be set on the angle sensor to reduce the influence result of diffuse reflection and reduce the interference signal for obtaining eye movement parameters.

[0122] For example, in some embodiments of the present application, the wearable eye movement tracking device further includes: a polarizer (for filtering the diffusely reflected light on the eye surface), wherein the angle sensor has a first channel and a second channel, and the polarizer is disposed in the first channel. It can be understood that when the light source projects a light signal onto the eye, in addition to specular reflection, the diffusely reflected light from other positions cannot be completely removed. Therefore, in some embodiments of the present application, two channels are set for each angle sensor. The first channel has no polarizer, and the second channel has an s-polarizer. The signal of the second channel is a non-Glint signal (the diffuse reflection signal from the iris, skin or sclera, and the intensity is half). Thus, a purer spot signal, that is, the incident signal of the reflected light, can be obtained through calculation (for example, the reflected signal received by the first channel - the reflected signal received by the second channel * 2).

[0123] For example, in some embodiments of the present application, the wearable eye movement tracking device further includes: a collimated light source module, which is configured to convert the divergent light emitted by the light source into parallel light to ensure that the light covers the target area (i.e., the area where the cornea is located); or, a dynamic angle control unit, which is configured to monitor the pupil position in real time and adjust the angle of the light source so that the light source irradiates the target area (i.e., the area where the cornea is located). That is to say, in some embodiments of the present application, the angles of the emitted light and the received light can be strictly controlled, so that less light shines on other positions (iris, skin or sclera), and less light reflected from these positions is received, making the incident signal for eye movement parameter estimation a specular reflection signal.

[0124] It should be noted that, in some embodiments of the present application, the angle sensor further includes: a timing control circuit, which is configured to make the matching light source group and the angle sensor work together through timing control, that is, to enable a certain angle sensor to only receive the signal after the reflection of the light signal projected onto the eye by the matching light source group.

[0125] In some embodiments of the present application, the wearable tracking device further includes: a light source signal modulation circuit configured to modulate a target group of light sources (belonging to any group of light sources) to project an optical signal towards the eyes at a target frequency (e.g., a relatively high set frequency); and an optical signal detection modulation circuit configured to perform frequency modulation on each optical signal detection unit in a target angle sensor (this description does not specifically refer to a certain angle sensor but an angle sensor that matches the target group of light sources), so that each optical signal detection unit in the target angle sensor reads the detected optical signal at the target frequency.

[0126] It should be noted that in some embodiments of the present application, the light source signal modulation circuit and the modulated LED light source are placed on the same circuit board or separately. It can be understood that if the modulation circuit is integrated with the LED light source, signal interference may be reduced and the response speed may be improved, but the heat generation will increase. Placing them separately may be more flexible, but there may be signal delay or noise problems. Similarly, in some embodiments of the present application, the optical signal detection modulation circuit and the modulated optical signal detection unit can be placed on the same circuit board or separately. It can be understood that if the modulation circuit is integrated with the modulated unit, signal interference may be reduced and the response speed may be improved, but the heat generation will increase. Placing them separately may be more flexible, but there may be signal delay or noise problems.

[0127] In some embodiments of the present application, each angle sensor is further configured to detect a blink event through the output signals of all optical signal detection units in the corresponding angle sensor, so as to trigger an interaction operation or end the operation of eye movement tracking.

[0128] Some embodiments of the present application can use a blink as a signal for interaction or as a signal indicating that eye movement no longer outputs data. The angle sensors in some embodiments of the present application can detect a blink through a common sudden change (mainly skin reflection) of all PD signals included therein.

[0129] In some embodiments of the present application, the angle sensor is further configured to confirm the occurrence of a saccade through the attributes of the output signals of at least one optical signal detection unit.

[0130] Since saccades occur frequently, accurately detecting target jumps can improve the effect of eye movement tracking. In some embodiments of the present application, a saccade event is determined by detecting discontinuous signal changes through an angle sensor.

[0131] In some embodiments of the present application, the wearable eye movement tracking device includes: a fixing component adapted to the user's head to maintain a symmetric distribution of each group of light sources and the matching angle sensors around the eyes.

[0132] For example, in some embodiments of the present application, the fixing component includes a frame part, a first angle sensor is embedded in the first side of the frame part, and a set of light sources matching the first angle sensor is embedded in the second side of the frame part, and the first side and the second side are opposite sides.

[0133] For example, some embodiments of the present application provide an eye movement tracking glasses 200 (as Figure 6 shown), the eye movement tracking glasses 200 include: a frame component and at least one set of optoelectronic components (not shown in the figure) arranged on the frame part included in the frame component, wherein the frame component includes at least one frame part (as Figure 6 shown a frame part 230).

[0134] In some embodiments of the present application, each set of optoelectronic components includes: a plurality of light sources, an angle sensor, and a processing unit. For example, as Figure 6 shown, the blue first light source 211 and the blue second light source 212 and the green first angle sensor 210 are used as components in a set of optoelectronic components, and the purple third light source 221 and the purple fourth light source 222 and the red second angle sensor 220 are used as components in another set of optoelectronic components. In some embodiments of the present application, a common processing unit is provided for a plurality of angle sensors, and in some embodiments of the present application, a separate processing unit is provided for each angle sensor, wherein the processing unit is configured to calculate eye movement parameters according to the incident direction.

[0135] In some embodiments of the present application, the plurality of light sources in the first optoelectronic component (as any one of the multiple sets of optoelectronic components) are configured to project a light signal with a preset wavelength to the eye area, and if the plurality of light sources in the first optoelectronic component are arranged in the first area of the frame part, then the angle sensor in the first set of optoelectronic components is arranged in the second area of the frame part, and the angle sensor is configured to receive the reflected light after the light signal projected by the plurality of light sources in the first set of optoelectronic components to the eye is reflected by the eye and determine the incident direction of the reflected light, wherein, in each set of optoelectronic components, the angle sensor is used to receive the reflected signal of the light signal emitted by the plurality of light sources in the corresponding optoelectronic component, and the first area and the second area are opposite sides of the frame part. For example, in some embodiments of the present application, the plurality of light-emitting diodes (as an example of a light source) included in the first optoelectronic component are located in the left area of the frame part and the angle sensor included in the first optoelectronic component is located in the right area of the frame part, or, the plurality of light-emitting diodes (as an example of a light source) included in the first optoelectronic component are located in the right area of the frame part and the angle sensor included in the first optoelectronic component is located in the left area of the frame part.

[0136] For example, as Figure 6As shown, the blue first light source 211 and the blue second light source 212 are located on the left side of the frame part, and correspondingly, the first angle sensor 210 matched with these two light sources is located on the right side of the frame part. This set of optoelectronic components includes: the blue first light source 211, the blue second light source 212, and the first angle sensor 210. The purple third light source 221 and the purple fourth light source 222 are located on the right side of the frame part, and correspondingly, the second angle sensor 220 matched with these two light sources is located on the right side of the frame part. This set of optoelectronic components includes: the purple third light source 221, the purple fourth light source 222, and the second angle sensor 220.

[0137] Some embodiments of the present application can determine the placement positions of the matching light sources and angle sensors in the frame part by using the following two examples: For glint-based tracking: The gaze direction is determined by measuring the reflection point of the corneal surface (i.e., the glint of specular reflection), and generally the accuracy is relatively high (about 1 degree). In some embodiments of the present application, an LED (as an example of a light source) is placed opposite the matching angle sensor across the eye, such as Figure 6 the two left light-emitting diodes LED (blue) are matched with the one right angle sensor sensor (green), and the two right light-emitting diodes LED (purple) are matched with the one left angle sensor sensor (i.e., red).

[0138] It should be noted that in some embodiments of the present application, Figure 6 the eye movement tracking glasses may only include a set of optoelectronic components. In other embodiments of the present application, Figure 6 the eye movement tracking glasses may include more than one set of optoelectronic components. The embodiments of the present application do not limit the total number of sets of optoelectronic components, nor do they limit the number of light sources included in each set of optoelectronic components.

[0139] For the specific structures of the angle sensors and other units included in the eye movement tracking glasses, reference can be made to the relevant descriptions in the above for wearable eye movement tracking devices. To avoid repetition, not too much elaboration will be made on the eye movement tracking glasses here.

[0140] Next, in combination with Figures 7 - 8 exemplarily elaborate on the schematic diagram of the working principle of the eye movement tracking glasses provided by the embodiments of the present application as Figure 6 shown.

[0141] Figure 7 and Figure 8 are the schematic diagrams of the working principle of the eye movement tracking glasses of the embodiments of the present application. Figure 7 and Figure 8They are respectively the angular changes of the eye movement process, the projected light rays, and the incident light rays of the reflected light observed from different perspectives, and Figure 7 is a three-dimensional image Figure 8 is a two-dimensional planar image. Figure 7 and Figure 8 The blue and purple origin points (different colors are used to represent different groups of optoelectronic components rather than the emission colors of LEDs) in represent LED light sources (corresponding to the light sources with the corresponding color identifiers respectively Figure 6 ). Figure 7 and Figure 8 The red and green squares (different colors are used to represent two angle sensors belonging to different groups of optoelectronic components) in represent two angle sensors. In the embodiments of the present application, Figure 7 and Figure 8 The light signals projected by the two purple LEDs in onto the eyes will only be received by the red angle sensor (for example, it can be achieved through timing control that at a certain point in time, only the purple LEDs are on and the red angle sensor receives the signals), that is, the two purple LEDs are matched with the red angle sensor or the two belong to a group of optoelectronic components. Figure 7 and Figure 8 The hemispherical shape of the grid shown in represents the cornea of the eye. It can be seen that as the Figure 7 or Figure 8 the eyeball moves in, the incident angle of the light rays incident on the red angle sensor changes. The angle sensor in the embodiments of the present application can accurately detect and output this incident angle. With the help of the user calibration process in the embodiments of the present application, the position and direction of the eyeball can form a functional or mapping relationship with the output incident angle, so that the eye movement parameters such as the direction of eye fixation (i.e., the fixation direction) and the position of the eyeball can be inferred based on the readings of the angle sensor. It can be understood that Figure 7 and Figure 8 The coordinate values on the coordinate axes of represent the coverage range of the field of view angle in the corresponding directions.

[0142] Next, some embodiments of the present application will be exemplarily described in combination with Figure 9 The eye movement tracking method provided by some embodiments of the present application can be applied to the wearable eye movement tracking device as described above or to the eye movement tracking glasses shown in Figure 1 or applied to Figure 6 .

[0143] As Figure 9 shown, some embodiments of the present application provide an eye movement tracking method, and the eye movement tracking method includes:

[0144] S110, in response to the light signal projected onto the eye, obtain the incident direction of the light reflected from the surface of the eyeball (for example, the cornea or the retina).

[0145] For example, in some embodiments of the present application, this step exemplarily includes: a light source arranged around the eye projects a light signal toward the eye, the light signal is reflected by the surface of the eyeball to form a reflection signal, and then an angle sensor matched with the light source collects and senses the reflection signal and determines the incident angle of the reflected light corresponding to the reflection signal, that is, obtains the incident direction of the reflected light.

[0146] Combined with the above Figure 7 and Figure 8 It can be seen that the working principle of obtaining the incident angle in this application is:

[0147] Incident light detection: Eye movement causes the incident angle of reflected light to change, entering a sub-area within the angle sensor FOV and being received by at least one optical signal detection unit (e.g., photodiode PD).

[0148] Multi-channel response: Adjacent channels (such as PD1 and PD2) output their own determined incident angle signals based on overlapping angular response curves.

[0149] Incident angle calculation: Through a calibration mapping table or machine learning model, the multi-channel signal (that is, the two or more incident angle signals obtained in the previous step) is combined and converted into the precise infrared light incident angle (the angle incident to the optical signal detection unit).

[0150] S120: Determine eye movement parameters according to the incident direction.

[0151] For example, in some embodiments of the present application, the line of sight mapping exemplarily includes: combining the geometric model of the eyeball (such as corneal curvature) to map the incident angle obtained by the angle sensor into a gaze direction (obtaining screen coordinates) or mapping it into other eye movement parameters. In the embodiments of the present application, eye movement parameters include but are not limited to the direction of eye gaze, the position of the eyeball, the pupil diameter, etc. For example, eye movement parameters exemplarily include: gaze direction, eyeball position, etc. For example, the eye movement model calculates the gaze direction by the relative position of the corneal reflection point and the pupil center.

[0152] It can be understood that the application scenarios of the technical solution of the present application can cover all applications based on eye movement data (medical, AI, VR, etc.), that is, the eye movement parameters obtained by using the wearable eye tracking device or eye tracking method of the embodiment of the present application can be used in medical, AI or AR and other fields.

[0153] The angle sensor of this application can detect the incident angle of infrared light reflected by the eyeball, and then determine eye movement parameters based on the incident angle, for example, calculating the gaze direction. This eye tracking method can cover the field of view (FOV) of the human eye with an accuracy of 0.5° to 1°. Compared with the eye tracking solutions provided by related technologies, the eye tracking process of the embodiment of this application has lower power consumption and the obtained eye tracking parameters are more accurate.

[0154] The following is an example of Figure 9 The implementation process of each step.

[0155] It should be noted that the wearable eye tracking device of the embodiment of the present application includes an angle sensor, which includes multiple channels, one channel corresponds to one optical signal detection unit, and each optical signal detection unit is set corresponding to a target field of view within a set range; the corresponding S110 exemplarily includes: determining the incident angle corresponding to the incident direction based on the electrical signal output value of at least one optical signal detection unit.

[0156] The optical signal detection unit may be a photodiode or a CMOS photosensitive unit, etc.

[0157] The target field of view is FOV. The target field of view (FOV) usually covers the range of human eye rotation (horizontally ±30°, vertically ±20°). The angle sensor of the embodiment of the present application needs to achieve high-precision overlapping response within this range. FOV refers to the angular range that the angle sensor can detect. For the eye tracking of this application, it must cover the possible rotation angles of the human eye, otherwise the gaze direction cannot be accurately tracked. For example, the human eye can usually rotate horizontally up to ±30 degrees and vertically ±20 degrees, so the FOV of the angle sensor needs to cover at least this range. The target field of view (FOV) refers to the range of light incidence angles that the sensor can detect. In eye tracking, the FOV setting directly determines the range of eye movements that the system can capture. For example, the multi-channel angle sensor of the embodiment of the present application covers the target FOV through multiple independent detection units (such as photodiodes PD). Example: If the target FOV is horizontally ±30° and vertically ±20°, the angle sensor can be designed as a 5×3 channel matrix, with each channel responsible for covering a sub-area of 6°×7°.

[0158] In some embodiments of the present application, S110 obtains the incident angle corresponding to the incident direction according to the output value of the electrical signal through a table lookup or a machine learning model.

[0159] In order to improve the resolution limit of a single channel and to enable the angle sensor to achieve smooth measurement within a continuous angle range and avoid blind spots, in some embodiments of the present application, the angular response spectra of the optical signal detection units of adjacent channels overlap. Then, the process of determining (for example, by looking up a table or determining by a machine learning model) the incident angle corresponding to the incident direction based on the electrical signal output value of at least one optical signal detection unit in S110 exemplarily includes: determining the incident angle based on the output values of multiple optical signal detection units with overlapping angular response spectra.

[0160] For example, in some embodiments of the present application, the incident angle is determined according to the proportional relationship. Each optical signal detection unit has a limited FOV, and there is an overlap of FOV between some optical signal detection units and some other optical signal detection units (not necessarily all optical signal detections). The angle sensor in some embodiments of the present application can infer the incident direction according to Figure 10 the ratio between different channels shown. As Figure 10 shown, the horizontal axis and the vertical axis of this figure represent the incident angles X and Y of the light rays. The red line and the green line are contour lines. In this implementation, each angle sensor has four optical signal detection units (photodiodes, PDs), one PD in each of the four directions of east, south, west, and north. For example, on the "0.55" line of the red line, it represents that S / (N + S) is equal to 0.55. In this way, through these two ratios (the red line and the green line), the incident angle of the light ray can be known.

[0161] The embodiments of the present application require that the angle response spectra of each PD in the angle sensor have both differences and overlaps. The differences ensure that each PD has different sensitivities to optical signals at different angles, so as to determine the incident angle by comparing the signals of each PD. The overlaps are for achieving smooth measurement within a continuous angular range and avoiding blind spots. It is not difficult to understand that a small angular change in the overlapping area will cause a coordinated change in the signals of multiple channels, breaking through the resolution limit of a single channel (similar to the principle of super-resolution). If each channel covers a part of the FOV and the responses of adjacent channels overlap, then even if the eyeball moves to the junction of two channels, the system can perform interpolation through the data of the two channels to improve the accuracy.

[0162] As an example, in some embodiments of the present application, before performing S110 to obtain the incident direction of the reflected light on the eyeball surface, the eye movement tracking method further includes: pre-calibrating the angle response curves of each optical signal detection unit in the angle sensor, where the angle response curve is used to characterize the mapping relationship between the output electrical signal of the corresponding optical signal detection unit and the incident angle of the reflected light; and obtaining the incident direction of the reflected light on the eyeball surface includes: determining the incident direction according to the angle response curve.

[0163] That is to say, some embodiments of the present application determine the angle response curve of each PD in the angle sensor by calibrating the mapping table. For example, the angle response curve includes the one-to-one correspondence between the incident angle and the output current, or the one-to-one correspondence between the incident angle and the output voltage, etc.

[0164] As an example, in some embodiments of the present application, before executing S110 to obtain the incident direction of the light reflected from the surface of the eyeball, the eye tracking method further includes: using a controllable light source to obtain the incident direction of the light reflected from the surface of the eyeball based on multiple sets of predefined two-dimensional incident angles (for example, the horizontal angle θ and the vertical angle θ). combinations of) respectively illuminate the angle sensor, and record the electrical signals (for example, current or voltage values) output by each optical signal detection unit in the angle sensor under each group of two-dimensional incident angles to obtain calibration data; use a multivariate interpolation algorithm (such as bilinear interpolation, spline interpolation, radial basis function interpolation, etc.) to construct a continuous function model based on the calibration data (the continuous function model can determine the incident angle according to the electrical signals output by N optical signal detection units in the angle sensor, where N is an integer greater than or equal to 2); wherein, the process of obtaining the incident direction of the light reflected from the surface of the eyeball in S110 exemplarily includes: reading the electrical signal output value of each of at least one optical signal detection units in the angle sensor; inputting the electrical signal output value into the continuous function model, and obtaining the incident direction of the reflected light by reverse search or interpolation inversion.

[0165] That is, S110 of some embodiments of the present application provides a method for determining a continuous function model based on a calibration and interpolation method and determining an incident direction according to the continuous function model, the method comprising:

[0166] Step 1: Sensor Calibration (Establishing a Known Angle-Signal Mapping)

[0167] Step 1.1, generate a 2D Angle Mesh with known incident angles

[0168] o Using a controllable light source, the light is emitted from a series of predefined two-dimensional incident angles (e.g. horizontal angle θ and vertical angle θ). combination) illumination angle sensor.

[0169] o Example grid: θ = [0°, 5°, 10°, ... 90°], A two-dimensional angle matrix is formed.

[0170] Step 1.2: Record the response signal of each photodiode (PD) in the angle sensor

[0171] For each known angle The output signal (such as current or voltage value) of each PD is recorded to form a calibration database.

[0172] oData format:

[0173]

[0174] Step 2: Build an interpolation model (2×3D or 2×nD interpolation)

[0175] Step 2.1, Interpolation Dimension Definition

[0176] o Input Dimension: 2D Angle

[0177] o Output Dimension:

[0178] ■ 2×3D Interpolation: Each angle corresponds to 3 PD readings (e.g., PD1, PD2, PD3).

[0179] ■ 2×nD Interpolation: Each angle corresponds to n PD readings (n is the number of PDs).

[0180] Step 2.2, Interpolation Method Selection

[0181] o Use a multivariate interpolation algorithm (such as bilinear interpolation, spline interpolation, radial basis function interpolation, etc.) to construct a continuous function model based on calibration data.

[0182] o Model Objective:

[0183] Given any combination of PD readings [PD1, PD2,..., PDn], inverse map to the corresponding incident angle through interpolation

[0184] Step 3: Calculate the Incident Angle Direction in Real Time

[0185] Step 3.1, Obtain Real-Time PD Readings

[0186] o The angle sensor receives the optical signal of the unknown incident angle and records the real-time output values of each PD.

[0187] Step 3.2, Inverse Interpolate the Incident Angle

[0188] o Input the real-time PD readings into the interpolation model, and through reverse lookup or interpolation inversion, obtain the corresponding That is, the incident direction described in S110 is obtained.

[0190] Some embodiments of the present application are methods for determining a continuous function model based on calibration and interpolation methods and determining the incident direction according to the continuous function model. The technical advantages include: fast calculation based on pre-calibrated data, suitable for real-time applications, and the interpolation model can compensate for the non-linear response of PDs.

[0191] As an example, in some embodiments of the present application, before executing S110 to obtain the incident direction of the light reflected from the surface of the eyeball, the eye tracking method further includes: using a controllable light source to illuminate the angle sensor based on multiple groups of predefined two-dimensional incident angles, and recording the electrical signals output by each light signal detection unit in the angle sensor under each group of two-dimensional incident angles to obtain calibration data; (i.e., pre-establishing a mapping relationship between a known angle and the response data of each light signal detection unit included in the angle sensor; i ... The illumination sensor is illuminated, the response signal of each photodiode (PD) is recorded, and an angle-signal mapping database is established), wherein the process of obtaining the incident direction of the light reflected from the surface of the eyeball in S110 exemplarily includes: constructing an optimization objective function based on pre-calibrated angle-signal mapping data, solving the incident angle parameter by a numerical optimization algorithm, specifically including: taking the angle of the incident direction of the reflected light to be solved as an optimization variable; using a numerical optimization algorithm (such as gradient descent, Levenberg-Marquardt algorithm) to search for a target solution of the objective function (for example, the objective function is to minimize the mean square error between the measured PD reading and the theoretical value) in the angle parameter space (that is, outputting the solution that minimizes the objective function) As the incident angle estimation value), the target solution is taken as the value of the optimization variable to obtain the incident direction.

[0192] That is, some embodiments S110 of the present application provide an embodiment of determining an incident method based on a solution method for an optimization problem, specifically including:

[0193] 1. Calibration data preparation

[0194] oPre-processed with a known 2D angle grid The angle sensor is illuminated, the response signals of each photodiode (PD) in the angle sensor are recorded, and an angle-signal mapping database is established.

[0195] 2. Optimization Problem Modeling

[0196] oDefine variables: The incident angle to be determined Set as optimization variable.

[0197] oObjective function (below): Minimize the mean square error between the measured PD reading and the theoretical value:

[0198]

[0199] o Prior knowledge integration: Use calibration data as initial constraints or reference models to narrow the solution space.

[0200] 3. Optimization solution

[0201] o Algorithm selection: Use numerical optimization algorithms (such as gradient descent, Levenberg-Marquardt algorithm)

[0202] Search for the optimal solution in the angular parameter space.

[0203] oInitial value setting: Provides initial angle guesses based on calibration data or historical results to accelerate convergence.

[0204] 4. Result Output

[0205] oOutput the minimum objective function as an estimate of the angle of incidence.

[0206] The advantages of the embodiment of the incident method determined based on the optimization problem-solving method provided in this application include: strong robustness: PD reading noise or nonlinear response deviation is tolerated through the optimization algorithm; flexible model fusion: physical optical models (such as light intensity attenuation formula) can be combined to improve accuracy; low hardware cost: only a small number of PDs are required to achieve high accuracy (<0.1 degrees), avoiding dependence on complex sensor arrays.

[0207] As an example, in some embodiments of the present application, before executing S110 to obtain the incident direction of the light reflected from the surface of the eyeball, the eye tracking method further includes: collecting the known position and gaze direction of the user's eyeball, and synchronously recording the incident angle reading output by the angle sensor to obtain calibration data; based on the calibration data, obtaining a function or mapping relationship between the eyeball position or gaze direction and the incident angle to obtain a mapping relationship model; the corresponding S120 exemplarily includes: inputting the incident angle reading corresponding to the incident direction into the mapping relationship model, and obtaining the real-time eyeball position (x', y', z') and gaze direction corresponding to the incident direction according to the mapping relationship model.

[0208] That is, in some embodiments of the present application, in order to obtain eye movement parameters according to the incident direction, S120 exemplarily includes:

[0209] Step 1: Perform the user calibration process

[0210] Under preset conditions, the user's eyeball's known position and direction data are collected, and the incident angle readings output by the angle sensor are simultaneously recorded. Specifically, it includes:

[0211] Data collection: The user looks at multiple preset markers on the screen, and the following data is synchronously recorded: the reference eye position and gaze direction output by the high-precision eye tracker, and the incident angle data detected by the angle sensor (such as the offset angle of the infrared reflection spot).

[0212] Step 2: Establish a mapping relationship model

[0213] Based on calibration data, a function or mapping relationship between the eye position / gaze direction and the incident angle is generated through an algorithm, including the following sub-steps:

[0214] a. Data fitting: Using regression analysis, neural networks or interpolation algorithms, associate the eye position (x, y, z) and direction with the incident angles (α, β).

[0215] b. Model storage: Save the generated mapping relationship as a look-up table (LUT) or a parametric equation.

[0216] Step 3, Real-time data acquisition

[0217] o Detect and output the current incident angle readings (α', β') in real time through an angle sensor.

[0218] Step 4, Infer the eye state

[0219] Input the current incident angle readings into the mapping relationship model to calculate the corresponding real-time eye position (x', y', z') and gaze direction

[0220] It should be noted that in some embodiments of the present application, the calibration trigger conditions include: automatically starting when first used or when the ambient light changes. The mapping relationship: can cover linear models (such as affine transformation), non-linear models (such as polynomial fitting) or machine learning models (such as support vector machines). The embodiments of the present application replace complex optical systems through algorithmic mapping, reducing the dependence on high-density sensors. The calibration process supports personalized adjustment to adapt to the physiological differences of the eyes of different users.

[0221] That is to say, in some embodiments of the present application, through the user calibration process, a mapping relationship between the eye position, direction and the incident angle readings of the angle sensor is established; the real-time incident angle readings of the angle sensor are obtained; according to the mapping relationship, the user's gaze direction and eye position are inferred from the incident angle readings. Replace complex optical systems through algorithmic mapping, reducing the dependence on high-density sensors; the calibration process supports personalized adjustment to adapt to the physiological differences of the eyes of different users.

[0222] In some embodiments of the present application, determining the eye movement parameters according to the incident direction in S120 includes: obtaining the three-dimensional coordinates of the center of the sphere formed by the cornea; obtaining the three-dimensional coordinates of the center of the pupil; obtaining a vector to be identified based on the three-dimensional coordinates of the center of the sphere formed by the cornea and the three-dimensional coordinates of the center of the pupil; comparing the vector to be identified with the vector visual direction mapping relationship to obtain the gaze direction corresponding to the vector to be identified, wherein the vector visual direction mapping relationship is a one-to-one correspondence between a known gaze direction obtained by calibration and a vector formed by the coordinates of the center of the corneal sphere and the coordinates of the pupil.

[0223] That is, in some embodiments of the present application, S120 exemplarily includes:

[0224] First, the three-dimensional position of the center of the sphere formed by the cornea is estimated by an optimization algorithm, that is, the three-dimensional coordinates of the center of the sphere formed by the cornea are estimated.

[0225] Specifically, based on the position of the angle sensor, the position of the LED matching the angle sensor, and the internal parameters of the angle sensor, and based on the principle that the distance from the point formed by each light spot on the corneal surface to the center of the sphere formed by the cornea is the same, the spatial position of the center of the sphere formed by the cornea is determined by multiple light spots.

[0226] Next, the absolute position of the pupil center (after corneal refraction) in the physical space is calculated, that is, the three-dimensional coordinates of the pupil center in the physical space are calculated.

[0227] Afterwards, a spatial vector vector to be analyzed is obtained based on the three-dimensional coordinates of the cornea center and the pupil center, that is, a vector is obtained based on the spatial position of the center of the sphere formed by the cornea and the three-dimensional coordinates of the pupil center in physical space.

[0228] Finally, based on the mapping relationship obtained during the user calibration process, the gaze direction that matches the spatial vector to be analyzed in the previous step is found.

[0229] It is understood that in some embodiments of the present application, user calibration can be performed in advance before executing S120. Through this user calibration, a correspondence between multiple known points (known gaze directions) and vectors can be established. Then, based on the correspondence, a gaze direction that matches the spatial vector to be analyzed is found as the final gaze direction for eye tracking. In other words, some embodiments of the present application can correspond the spatial vector to be analyzed with the gaze direction using multiple points with known directions, so that after determining the three-dimensional coordinates of the center of the sphere formed by the cornea and the three-dimensional coordinates of the pupil, the gaze direction of the eye can be obtained.

[0230] The working principle of the eye movement tracking method according to some embodiments of the present application is as follows: When the eyeball rotates, the direction of the light entering the angle sensor changes. The embodiments of the present application can detect these angle changes and absolute angle values through the angle sensor, and combine the position information of the LED light source and the angle sensor to calculate the attitude position of the cornea and the eyeball, and even the cornea size. The process can refer to the processing flow of the computer vision (CV) method in eye movement tracking (ET).

[0231] The gaze direction obtained by the wearable eye movement tracking device and the eye movement tracking method provided by the embodiments of the present application achieves high precision at the sub-0.1° level. The embodiments of the present application can calculate the incident light angle based only on the readings of a few photodiodes (PDs). Some embodiments of the present application can perform optimization, interpolation, and inverse problem solving. One implementation method is (specifically, refer to the method of determining a continuous function model based on calibration and interpolation and determining the incident direction according to this continuous function model): Calibrate the angle sensor with incident light of a known two-dimensional angle grid, and then use the calibration data as the interpolation basis (2×3D or 2×nD interpolation) to solve the incident light direction from the PD readings; Another method is to use the calibration data as prior knowledge and construct the angle solution as an optimization problem. Specifically, refer to the relevant steps in the embodiments of the method for determining the incident method based on the solution of the optimization problem above.

[0232] Through the integrated design of the mask layer or microlens and the PD in the embodiments of the present application, high-precision multi-channel angle detection can be achieved in an extremely small size, perfectly meeting the requirements of eye movement tracking for miniaturization, low power consumption, and high real-time performance. The core of this technology lies in the combination of physical optical constraints and signal redundancy solution, and can be used in next-generation human-computer interaction devices.

[0233] Some embodiments of the present application provide an interaction method, and the interaction method includes: obtaining eye movement parameters according to the eye movement tracking method described in any of the above embodiments, where the eye movement parameters are the gaze direction; dynamically adjusting the focus area or display content of the controls in the interaction interface according to the gaze direction.

[0234] Some embodiments of the present application provide an interaction method for a virtual-real fusion scenario, and the interaction method includes: obtaining eye movement parameters according to the eye movement tracking method described in any of the above embodiments, where the eye movement parameters are the gaze direction; operating a virtual object according to the gaze direction. For example, the operations include selection, movement, etc.

[0235] As Figure 11As shown in the figure, some embodiments of the present application provide an interaction method based on the gaze direction. The interaction method includes: S801. In response to the change in the incident angle of the reflected light reflected from the eye surface caused by the rotation of the eyeball, at least one optical signal detection unit in different regions of the angle sensor is respectively used to receive the reflected light signals at different angles; S802. Output the incident angle signals of the reflected light determined by each according to the angle response curve of the at least one optical signal detection unit; S803. Convert the incident angle signal into the incident angle of the infrared light through a calibration mapping table or a machine learning model; S804. Combine the eyeball geometric model (such as the model of corneal curvature) to map the incident angle to the gaze direction; S805. Perform an interaction operation according to the gaze direction. For example, obtain the screen coordinates according to the gaze direction and complete the interaction operation according to the screen coordinates.

[0236] In some embodiments of the present application, the interaction method further includes: in response to the detected blink event, triggering an interaction operation or ending the eye movement tracking operation, where the blink event is determined by the output values of all optical signal detection units in the angle sensor.

[0237] To avoid repetition, the method for obtaining the gaze direction will not be described in too much detail. For the structure of the angle sensor and the related processing process, specific reference can be made to the relevant content recorded above.

[0238] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0239] In addition, each functional module in various embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0240] If the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0241] The above are only the embodiments of this application and are not used to limit the protection scope of this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application. It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0242] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0243] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

Claims

1. An eye movement tracking method, applied to a wearable eye movement tracking device, the eye movement tracking method comprising: In response to a light signal projected onto the eye, obtaining the incident direction of the reflected light on the surface of the eyeball; Determining eye movement parameters according to the incident direction.

2. The eye movement tracking method according to claim 1, wherein The wearable eye movement tracking device includes an angle sensor, the angle sensor includes a plurality of channels, one channel corresponds to one light signal detection unit, and each light signal detection unit is correspondingly arranged with a target field of view within a set range; The obtaining of the incident direction of the reflected light on the surface of the eyeball includes: Determining an incident angle corresponding to the incident direction according to the electrical signal output value of at least one light signal detection unit.

3. The method for eye movement tracking according to claim 2, wherein The angle response spectra of the light signal detection units of adjacent channels overlap; The determining of the incident angle corresponding to the incident direction according to the electrical signal output value of at least one light signal detection unit includes: Determining the incident angle according to the output values of a plurality of light signal detection units having overlapping angle response spectra.

4. The eye movement tracking method according to any one of claims 2-3, characterized in that, Before the obtaining of the incident direction of the reflected light on the surface of the eyeball, the eye movement tracking method further includes: Pre-calibrating the angle response curves of the respective light signal detection units in the angle sensor, wherein the angle response curve is used to characterize the mapping relationship between the output electrical signal of the corresponding light signal detection unit and the incident angle of the reflected light; Wherein, The obtaining of the incident direction of the reflected light on the surface of the eyeball includes: Determining the incident direction according to the angle response curve.

5. The eye movement tracking method according to any one of claims 2-3, characterized in that, Before the obtaining of the incident direction of the reflected light on the surface of the eyeball, the eye movement tracking method further includes: Using a controllable light source, irradiating the angle sensor respectively based on multiple groups of predefined two-dimensional incident angles, and recording the electrical signals output by each light signal detection unit in the angle sensor under the irradiation of each group of two-dimensional incident angles to obtain calibration data; Using a multi-variable interpolation algorithm to construct a continuous function model based on the calibration data; Wherein, The obtaining of the incident direction of the reflected light on the surface of the eyeball includes: Reading the electrical signal output value of at least one light signal detection unit in the angle sensor; Inputting the electrical signal output value into the continuous function model, and obtaining the incident direction of the reflected light through reverse lookup or interpolation inversion.

6. The eye movement tracking method according to any one of claims 2-3, characterized in that, Before the obtaining of the incident direction of the reflected light on the surface of the eyeball, the eye movement tracking method further includes: Using a controllable light source, irradiating the angle sensor respectively based on multiple groups of predefined two-dimensional incident angles, and recording the electrical signals output by each light signal detection unit in the angle sensor under the irradiation of each group of two-dimensional incident angles to obtain calibration data; Wherein, The obtaining of the incident direction of the reflected light on the surface of the eyeball includes: Taking the angle of the incident direction of the reflected light as an optimization variable; Adopting a numerical optimization algorithm to search for the target solution of the objective function in the angle parameter space, and taking the target solution as the value of the optimization variable to obtain the incident direction.

7. The eye movement tracking method according to claim 1, wherein Before the determining of the eye movement parameters according to the incident direction, the eye movement tracking method further includes: Collecting the known positions and fixation directions of the user's eyeballs, and synchronously recording the incident angle readings output by the angle sensor to obtain calibration data; Based on the calibration data, obtain the functional or mapping relationship between the eye position or the gaze direction and the incident angle, and obtain a mapping relationship model; The determining the eye movement parameters according to the incident direction includes: Input the incident angle reading corresponding to the incident direction into the mapping relationship model, and obtain the real-time eye position and gaze direction corresponding to the incident direction according to the mapping relationship model.

8. The eye movement tracking method according to claim 1, wherein The determining the eye movement parameters according to the incident direction includes: Obtain the three-dimensional coordinates of the center of the sphere formed by the cornea; Obtain the three-dimensional coordinates of the pupil center; Obtain a vector to be recognized according to the three-dimensional coordinates of the center of the sphere formed by the cornea and the three-dimensional coordinates of the pupil center; According to the vector to be recognized and the vector visual direction mapping relationship, obtain the gaze direction corresponding to the vector to be recognized, where the vector visual direction mapping relationship is the corresponding relationship between the vector formed by the corneal sphere center coordinates and the pupil coordinates obtained through calibration and the gaze direction.

9. An interaction method, characterized in that, The interactive method includes: Obtain eye movement parameters according to any one of claims 1-8, where the eye movement parameters are gaze directions; Dynamically adjust the focus area or display content of the control in the interactive interface according to the gaze direction.

10. An interaction method for a virtual-real fusion scenario, characterized in that, The interactive method includes: ​ ​ 11. A wearable eye movement tracking device, characterized in that, ​ ​ ​ ​ 12. The wearable eye movement tracking device according to claim 11, wherein, ​ ​ 13. The wearable eye movement tracking device according to claim 12, wherein, ​ 14. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, ​ ​ 15. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, ​ ​ ​ ​ ​ ​ ​ Secondary apertures are distributed around the main aperture. Each secondary aperture is correspondingly arranged with one optical signal detection unit in the corresponding angle sensor. Each secondary aperture forms a directional optical channel with a set field of view angle through a metal mask.

17. The wearable eye movement tracking device according to claim 16, wherein the optical signal detection array includes five photodiodes. Among them, four of the five photodiodes are symmetrically distributed up, down, left, and right along the outer periphery of the remaining one photodiode. The main aperture is an annular hole arranged in the center above the five photodiodes, and the secondary apertures are 4 micro-holes distributed around the main aperture.

18. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, The angle sensor includes: an optical signal detection array composed of multiple optical signal detection units; a microlens array arranged above the optical signal detection array; wherein, one optical signal detection unit is correspondingly arranged with one microlens in the microlens array. There is a preset position offset between the microlens array and the optical signal detection array, and the preset position offset is used to ensure that the angle response spectra of different optical signal detection units have differences and overlaps.

19. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, The angle sensor includes: multiple light detection units; a lens correspondingly arranged with the multiple light detection units for covering the multiple light detection units; wherein, the multiple light detection units are not located on the focal plane of the lens.

20. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, The angle sensor further includes: a polarizer located in the incident light path and configured to filter the diffusely reflected light.

21. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, The wearable eye movement tracking device further includes: a polarizer; wherein, the angle sensor has a first channel and a second channel, and the polarizer is arranged in the first channel.

22. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, The wearable eye movement tracking device further includes: a collimated light source module configured to convert the divergent light emitted by the light source into parallel light to ensure that the light covers the target area; Or, a dynamic angle control unit configured to monitor the pupil position in real time and adjust the angle of the light source so that the light source irradiates the target area.

23. The wearable eye movement tracking device according to any one of claims 11-13, characterized in that, The wearable tracking device further includes: a light source signal modulation circuit configured to control the target group of light sources to project optical signals to the eyes at a target frequency; and a light signal detection modulation circuit configured to perform frequency modulation on each optical signal detection unit in the target angle sensor that matches the target group of light sources, so that each optical signal detection unit in the target angle sensor reads the detected optical signals at the target frequency.

24. The wearable eye movement tracking device according to claim 11, wherein, Each angle sensor is further configured to detect a blinking event through the output signals of all optical signal detection units in the corresponding angle sensor to trigger an interaction operation or end the eye movement tracking operation.

25. The wearable eye movement tracking device according to claim 11, characterized in that The angle sensor is further configured to confirm the occurrence of a saccade through the attributes of the output signals of at least one optical signal detection unit.

26. An interaction method based on the gaze direction, characterized in that, The interaction method includes: In response to the change in the incident angle of the reflected light reflected from the eye surface caused by the rotation of the eyeball, at least one optical signal detection unit located in different regions within the angle sensor is respectively used to receive reflected light signals at different angles; Output the determined incident angle signals of the reflected light respectively according to the angle response curves of the at least one optical signal detection unit; Convert the incident angle signal into an infrared light incident angle through a calibration mapping table or a machine learning model; Combine the eye geometry model to map the incident angle to the gaze direction; Perform an interaction operation according to the gaze direction.

27. The interactive method according to claim 26, wherein The interaction method further includes: In response to a detected blink event, trigger an interaction operation or end the eye movement tracking operation, where the blink event is determined by the output values of all optical signal detection units in the angle sensor.

Images