Eye movement tracking method based on MEMS micromirror and glasses device

Abstract

The invention discloses an eye movement tracking method based on an MEMS micromirror and a glasses device. The eye movement tracking method based on the MEMS micromirror comprises the following steps: S1, setting an initial eyeball position, and adjusting an initial angle of the micromirror to enable a reflected laser vector to be incident to the center of an eyeball cornea; s2, controlling a micro-mirror resonance scanning laser beam to continuously deflect on the surface of the cornea; s3, the laser spot generates a periodic scanning track on the surface of the cornea of the eyeball, a photoelectric detector receives a signal and generates a characteristic pulse peak at the central position of the cornea, and the time difference of the pulse peak values in a single scanning period is analyzed to obtain the rotation condition of the eyeball; the glasses device comprises a glasses frame, a photoelectric detector, an MEMS micromirror and a laser component thereof, and is used for realizing the above eye movement tracking steps. According to the accurate eye position judgment method based on the geometrical relationship, the computing power requirement is low, positioning is more accurate, the performance of an eye movement tracking system is improved, the MEMS micromirror has the advantages of being small in size, high in response speed, low in power consumption and the like, and the problems that an existing eye movement tracking technology is low in accuracy and low in portability are effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of eye tracking technology, and specifically relates to an eye tracking method based on MEMS micromirrors. In particular, it relates to an eye tracking glasses device based on MEMS micromirrors. Background Art

[0002] Eye tracking is a technique that studies visual attention and cognitive processes by recording and analyzing eye movements. The ability to monitor and analyze eye movements in real time provides a powerful tool for studying the underlying mechanisms of human visual processing, attention dynamics, and cognitive function. Over the past few decades, eye tracking research has evolved from laboratory studies into a versatile tool with applications across fields such as psychology, neuroscience, human-computer interaction, marketing, and medicine.

[0003] Initially, direct observation and mechanical recording methods were used to simply record eye movements. In the early 20th century, invasive eye tracking methods such as mirror reflection, electrooculography, and scleral search coils were developed to improve recording accuracy. However, these techniques required complex surgery or instrumentation. Since the mid-20th century, non-invasive methods based on photoelectric sensing, such as edge tracking, retinal image tracking, and dual-Purkinje eye tracking, have enabled high-precision eye movement data recording without contacting the eyeball. However, technical limitations have hindered portability. Currently, infrared camera-based eye tracking is the most widely used eye tracking technology. These methods primarily use infrared cameras to capture eye movements and then, through a series of algorithms, determine the real-time gaze position. Existing technologies use the pupil-corneal reflection vector to establish a geometric model, achieving 3D gaze point estimation using a single camera. Alternatively, a method using only a low-resolution infrared stereo camera and a set of infrared illuminators, using evidential reasoning and geometric rules to detect candidate eye regions, can reduce processing time to less than 24 milliseconds and is suitable for real-time systems. However, these methods only achieve eye localization, failing to achieve high-precision eye tracking. In existing technologies, the interplay between camera performance, image processing capabilities, and device size results in poor system portability and limited accuracy. Furthermore, image recognition-based eye tracking systems require real-time processing of high-resolution images, placing higher demands on algorithm efficiency and computing resources. Particularly in mobile devices or embedded systems, these factors often become bottlenecks to system performance due to limitations in computing power and power consumption. Summary of the Invention

[0004] The object of the present invention is to provide an eye tracking method and eyewear device based on MEMS micromirrors to solve the problems of low accuracy and low portability of existing eye tracking mentioned in the above background technology.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] An eye tracking method based on MEMS micromirrors includes the following steps:

[0007] S1. Set the initial eyeball position and adjust the initial angle of the micromirror so that the reflected laser vector is incident on the center of the cornea of the eyeball;

[0008] S2, controlling the micromirror to resonate and scan the laser beam, causing it to continuously deflect on the corneal surface, wherein the driving frequency of the micromirror is 1-10 kHz and the maximum deflection angle is ±20°;

[0009] S3. The laser spot generates a periodic scanning track on the surface of the cornea of the eye. The photodetector receives the signal and generates a characteristic pulse peak at the center of the cornea. The time difference of the pulse peak within a single scanning cycle is analyzed to obtain the rotation of the eye. The characteristic pulse peak is extracted by a threshold comparison circuit, and the threshold range is 70%-90% of the reflected signal intensity.

[0010] Preferably, the micromirror scans the laser beam in the following situations: the eyeball rotates on the polarization side, causing the laser beam to rotate toward the direction of the incident light; the eyeball rotates on the far beam side, causing the laser beam to rotate away from the direction of the incident light.

[0011] Preferably, in the initial state of the eyeball, a three-dimensional coordinate system is established with the eyeball rotation center as the origin, the y-axis is perpendicular to the horizontal plane, the visual axis coincides with the positive direction of the y-axis, the x-axis represents the horizontal gaze direction, the incident laser beam must be strictly kept coplanar with the y-axis, and the angle between the incident laser and the reflected laser has a quantitative geometric relationship:

[0012]

[0013] Where v0 represents the reference angle of the incident / reflected laser beam, a and b are the horizontal / vertical offsets from the eyeball rotation center to the micromirror reflection surface, respectively.

[0014] Preferably, when the micromirror resonates and deflects, the real-time deflection angle is:

[0015] θ=θ max sin(2πft)

[0016] Where f is the driving frequency of the micromirror, and θmax is the maximum deflection angle of the micromirror.

[0017] Preferably, the time-varying characteristics of the real-time deflection angle of the micromirror cause the laser spot to produce a periodic scanning trajectory on the surface of the cornea of the eyeball. Based on the difference in corneal-scleral reflectivity and the characteristic that the reflected signal intensity is the largest at the center of the cornea, the photodetector receives the signal and generates a characteristic pulse peak at the center of the cornea. The relationship between the incident-emission laser angle corresponding to the peak time and the time difference between adjacent pulse peaks is:

[0018]

[0019] Preferably, the pulse peak-to-peak interval has the following quantitative relationship:

[0020] Polarized side:

[0021] High beam side:

[0022] Where Δt is the time difference between adjacent pulse peaks, T is the galvanometer scanning period, and tp is the peak arrival time.

[0023] Preferably, the functional relationship between the time interval of the photodiode receiving peak value and the eye rotation angle is:

[0024]

[0025] η is the eye rotation angle, λ represents the real-time angle between the incident laser beam and the reflected laser beam, and a and b are the horizontal / vertical offsets from the eye rotation center to the micromirror reflection surface, respectively.

[0026] A MEMS micromirror-based eye-tracking eyewear device comprises an eyeglass frame, two APD photodetectors, two MEMS micromirrors and their lasers, wherein the MEMS micromirrors and their lasers are arranged on the eyeglass frame, and the APD photodetectors are arranged on the inner side of the eyeglass frame. A charging interface is provided on the temple portion of the eyeglass frame, and a wire assembly is also provided inside the eyeglass frame to connect the charging interface to the APD photodetectors and the MEMS micromirrors and their lasers.

[0027] Technical effects and advantages of the present invention: Compared with the existing technology, the eye tracking method based on MEMS micromirrors proposed in the present invention has the following advantages:

[0028] First, the high-frequency rotation of the micromirror reflects low-power infrared laser light onto the eyeball, completing an entire scan of the eyeball. The geometric model of the micromirror scan directly calculates the eye movement angle, avoiding complex image processing, thereby reducing computing power requirements and improving the performance of the eye tracking system.

[0029] Second, the present invention provides a method and device for eye tracking based on a MEMS micromirror. The MEMS micromirror has advantages such as small size, fast response speed, and low power consumption. Therefore, this eye tracking solution also has the advantages of miniaturization, low power consumption, and low cost.

[0030] Third, the eye position is calculated based on the time peak of the diode signal without relying on a complex imaging system. High-speed acquisition and galvanometer scanning technology are used to achieve high-resolution eye tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the eye tracking method based on MEMS micromirrors of the present invention;

[0032] Figure 2 Schematic diagram of the principle of the eye tracking method based on MEMS micromirrors of the present invention;

[0033] Figure 3 This is a schematic diagram of the structure of the eye-tracking glasses device based on MEMS micromirrors of the present invention.

[0034] In the figure: 1. Eyeglass frame; 2. APD photodetector; 3. MEMS micromirror and its laser. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0036] The present invention provides Figure 1-2 An eye tracking method based on a MEMS micromirror is shown, comprising the following steps:

[0037] S1. Set the initial eye position, with the 0° eyeball direct viewing direction as the y-axis, the horizontal position as the x-axis, and the origin as the eyeball center position (0,0). Adjust the initial angle of the micromirror so that the reflected laser vector is incident on the center of the eyeball cornea. Due to the structure of the eyeglass frame, set the laser light horizontal to the y-axis.

[0038] Depend on Figure 1 The geometric relationship shows that the angle v0 between the incident laser and the outgoing laser is

[0039]

[0040] Where v0 represents the reference angle of the incident / reflected laser beam, and a and b are the horizontal and vertical offsets from the eye's rotation center to the micromirror's reflective surface, respectively. During the experiment, based on the geometric relationship between the preset eye model and the laser reflection path, closed-loop feedback control was used to adjust the micromirror's initial angle so that the reflected laser vector was incident on the center of the cornea. A PID controller was used to adjust the micromirror's drive voltage to achieve a deflection accuracy of θmax = 10° ± 0.5°.

[0041] S2, when the micromirror deflects the incident laser beam to rotate toward the incident laser side, as Figure 2As shown in (b), the relationship between the micromirror rotation angle and the incident laser beam angle is as follows:

[0042] θ=(v-v0) / 2

[0043] v=v0+2θ

[0044] Where v represents the real-time angle between the incident laser beam and the reflected laser beam, and v is a function of the real-time angle θ of the micromirror rotation. When the micromirror deflects and causes the incident laser beam to rotate away from the incident laser side, as shown in Figure 2 As shown in (c), there is the following relationship between θ and the incident laser beam angle:

[0045] θ=(v-v0) / 2

[0046] v=v0+2θ

[0047] The real-time angle θ of the micromirror is related to the vibration of the micromirror and is a function of time t.

[0048] θ=θ max sin(2πft)

[0049] Where f is the driving frequency of the micromirror and θmax is the maximum deflection angle of the micromirror. Therefore, the real-time change of v can be obtained as:

[0050] Polarized side: υ=υ0-2θ max sin(2πft)

[0051] High beam side: υ=υ0+2θ max sin(2πft)

[0052] S3. This time-varying characteristic causes the laser spot to produce a periodic scanning trajectory on the surface of the cornea. Due to the difference in corneal-scleral reflectivity and the fact that the reflected signal intensity is the highest at the center of the cornea, the photodetector will receive a characteristic pulse peak at the center of the cornea. The pulse peak-to-peak interval has the following quantitative relationship:

[0053] Polarized side:

[0054] High beam side:

[0055] Where Δt is the time difference between adjacent pulse peaks, T is the galvanometer scanning period, and tp is the peak arrival time.

[0056] The mathematical model constructed by combining the above equations shows that Δt and tp have a strict hyperbolic tangent function relationship:

[0057]

[0058] Further solving the functional relationship between the time interval of the photodiode receiving peak and the eye rotation angle, according to the relationship between the eye constraint conditions and the eye position and rotation angle, we can know that:

[0059] Relationship between angle and eye position:

[0060] Eyeball constraints: x 2 +y 2 =R 2

[0061] Where x and y are the coordinates of the eyeball center. Assuming that the eyeball is spherical, the relationship between the eye position and the rotation angle can be obtained:

[0062]

[0063] The mathematical model constructed by combining the above equations can obtain the functional relationship between the peak time interval Δt and the eye rotation angle η:

[0064]

[0065] After the photodetector signal is converted by the transimpedance amplifier, it is collected and calculated by the FPGA in real time;

[0066] After testing on an embedded system, the time required for calculating a single eye movement angle is less than 0.5ms, and the power consumption is less than 50mW.

[0067] Suitable for foveated rendering of AR / VR glasses, supporting 120Hz refresh rate.

[0068] The present invention also provides Figure 3 The device, which uses MEMS micromirrors for eye tracking, includes a glasses frame 1, two APD photodetectors 2, and two MEMS micromirrors and their associated lasers 3. The MEMS micromirrors and their associated lasers 3 are mounted on the frame 1, while the APD photodetectors 2 are located inside the frame 1. The temples of the frame 1 are equipped with charging ports, and the frame 1 also includes wire assemblies connecting the charging ports to the APD photodetectors 2 and the MEMS micromirrors and their associated lasers 3. The APDs are InGaAs, with a response time of <10ns.

[0069] Working principle: The user wears the eye tracking device on the head, and the MEMS micromirror deflects to dynamically scan the laser beam onto the eyeball. The photodetector receives the signal and generates a characteristic pulse peak at the center of the cornea. The time difference of the pulse peak within a single scanning cycle is analyzed to obtain the rotation of the eyeball. It has the advantages of miniaturization, low power consumption and low cost. The laser beam scanning range is controlled by micromirror vibration to achieve high-precision measurement of eye movement. The eye position is calculated based on the time peak of the diode signal without relying on a complex imaging system. High-speed acquisition and galvanometer scanning technology are used to achieve high-resolution eye tracking. After testing, the system angular resolution is 0.1° and the sampling frequency reaches 1kHz.

[0070] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An eye tracking method based on MEMS micromirrors, characterized in that: The steps include: S1. Set the initial eyeball position and adjust the initial angle of the micromirror so that the reflected laser vector is incident on the center of the cornea of the eyeball; S2. Control the micromirror to resonate and scan the laser beam, causing it to continuously deflect on the corneal surface. The laser wavelength is 905 nm to 1550 nm, the power is 0.02 mW to 0.1 mW, the driving frequency of the micromirror is 1 to 10 kHz, and the maximum deflection angle is ±20°. S3. The laser spot generates a periodic scanning track on the surface of the cornea of the eye. The photodetector receives the signal and generates a characteristic pulse peak at the center of the cornea. The time difference of the pulse peak within a single scanning cycle is analyzed to obtain the rotation of the eye. The characteristic pulse peak is extracted by a threshold comparison circuit, and the threshold range is 70%-90% of the reflected signal intensity.

2. The eye tracking method based on MEMS micromirrors according to claim 1, characterized in that: The situations in which the micromirror scans the laser beam include: the polarization side of the eye rotates, and the laser beam rotates toward the direction of the incident light; the far beam side of the eye rotates, and the laser beam rotates away from the direction of the incident light.

3. The eye tracking method based on MEMS micromirrors according to claim 2, characterized in that: In the initial state of the eyeball, a three-dimensional coordinate system is established with the eyeball rotation center as the origin. The y-axis is perpendicular to the horizontal plane, the visual axis coincides with the positive direction of the y-axis, and the x-axis represents the horizontal gaze direction. The incident laser beam must be strictly kept coplanar with the y-axis. The angle between the incident laser and the reflected laser has a quantitative geometric relationship: Where v0 represents the reference angle of the incident / reflected laser beam, a and b are the horizontal / vertical offsets from the eyeball rotation center to the micromirror reflection surface, respectively.

4. The eye tracking method based on MEMS micromirrors according to claim 2, characterized in that: When the scanning micromirror resonates and deflects, the real-time deflection angle is: θ=θ max sin(2πft) Where f is the driving frequency of the micromirror, and θmax is the maximum deflection angle of the micromirror.

5. The eye tracking method based on MEMS micromirrors according to claim 4, characterized in that: The time-varying characteristics of the real-time deflection angle of the micromirror cause the laser spot to produce a periodic scanning trajectory on the surface of the cornea of the eye. Based on the difference in corneal-scleral reflectivity and the characteristic that the reflected signal intensity is the largest at the center of the cornea, the photodetector receives the signal and generates a characteristic pulse peak at the center of the cornea. The relationship between the incident-emission laser angle corresponding to the peak time and the time difference between adjacent pulse peaks is as follows:

6. The eye tracking method based on MEMS micromirrors according to claim 4, characterized in that: The pulse peak-to-peak interval has the following quantitative relationship: Polarized side: High beam side: Where Δt is the time difference between adjacent pulse peaks, T is the galvanometer scanning period, and tp is the peak arrival time.

7. The eye tracking method based on MEMS micromirrors according to claim 4, characterized in that: The functional relationship between the time interval of the photodiode receiving peak and the eye movement angle is: η is the eye rotation angle, λ represents the real-time angle between the incident laser beam and the reflected laser beam, a and b are the horizontal / vertical offsets from the eye rotation center to the micromirror reflection surface, respectively, and c is the distance from the eye rotation center to the corneal vertex, with a value range of 10-12 mm.

8. An eye-tracking eyewear device based on a MEMS micromirror according to any one of claims 1 to 7, comprising an eyeglass frame (1), two APD photodetectors (2), two MEMS micromirrors and their lasers (3), and characterized in that: The MEMS micromirror and its laser (2) are arranged on a glasses frame (1), the APD photodetector (3) is arranged on the inner side of the glasses frame (1), the temple portion of the glasses frame (1) is provided with a charging interface, and the glasses frame (1) is also provided with a wire assembly for connecting the charging interface to the APD photodetector (2) and the MEMS micromirror and its laser (3).

Images