Eye feature acquisition system, method and eye tracking device

Abstract

The application provides an eye feature acquisition system, method and eye movement tracking device. The system comprises a light source module, a control module, a sensor module and a data processing module. The control module is used to control the light source module to realize a first timing or a second timing, and the irradiance of the light source module is substantially constant in the first timing, and the irradiance of the light source module periodically changes in the second timing. The sensor module is used to acquire event stream information corresponding to the first timing and the second timing respectively, and send the event stream information to the data processing module. The data processing module is used to determine a light spot image frame and a pupil image frame according to the event stream information. The two timings are realized by controlling the light source module, so that the dynamic sensor can acquire event stream information corresponding to the two timings respectively, thereby accurately obtaining the light spot image frame and the pupil image frame, accurately acquiring eye features based on the dynamic sensor, and reducing system power consumption. In addition, the eye features are acquired based on the dynamic sensor, and the accuracy of the eye movement tracking result can be improved.

Description

Technical Field

[0001] This invention relates to the field of eye-tracking technology, and more particularly to an eye feature acquisition system, method, and eye-tracking device. Background Technology

[0002] Eye tracking is a technology that acquires information by tracking and recording human eye movements. In extended reality devices, eye tracking is key to human-computer interaction.

[0003] Typically, eye-tracking technology determines the direction of gaze and eye movement based on pupil-corneal reflection. Specifically, a light source illuminates the eye, and a camera captures an image of the eye. Image recognition identifies the pupil, and the corneal reflection formed by the light source is identified. The eye movement vector is derived by calculating the relative positional change between the pupil center and the corneal reflection. This eye movement vector is used to infer the direction of eye movement and gaze. Some technologies use frame sensors to acquire eye images; however, frame sensors have several drawbacks. For example, when the eye is stationary, the frame sensor still acquires an image, but this image contains invalid information. The system needs to process this invalid information, increasing power consumption.

[0004] Therefore, reducing system power consumption is a technical problem that needs to be solved when acquiring eye features. Summary of the Invention

[0005] This invention provides an eye feature acquisition system, method, and eye-tracking device, which uses a control method of the light source module to accurately acquire eye features based on dynamic sensors, thereby solving the problem of high system power consumption.

[0006] In a first aspect, the present invention provides an eye feature acquisition system, comprising:

[0007] Light source module;

[0008] The control module is used to control the light source module to achieve a first timing sequence or a second timing sequence, wherein the irradiance of the light source module remains basically unchanged in the first timing sequence, and the irradiance of the light source module changes periodically in the second timing sequence.

[0009] The sensor module includes a dynamic sensor for acquiring event stream information and sending the event stream information to the data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence.

[0010] The data processing module is used to determine a light spot image frame based on the first event stream information, and to determine a pupil image frame based on the second event stream information; the light spot image frame and the pupil image frame are used to determine eye movement information.

[0011] Optionally, the control module is used to generate an encoded signal and send the encoded signal to the light source module. The encoded signal includes a differential encoded signal and a synchronization encoded signal. The light source module includes at least two light source groups, each light source group including multiple light source units. The at least two light source groups of the light source module are alternately turned on based on the differential encoded signal, and the at least two light source groups of the light source module are simultaneously turned on or simultaneously turned off based on the synchronization encoded signal.

[0012] Optionally, the light source group includes a first light source group and a second light source group; each light source group includes multiple light source units; the output signals from the first and second output terminals of the control module are in phase, and the output signals from the third and first output terminals of the control module are in opposite phase; the fourth output terminal of the control module is used to output the encoding mode; the first output terminal is connected to the first light source group through a first driver; the second, third, and fourth output terminals are respectively connected to a switching unit, and the switching unit is also connected to the second light source group through a second driver; each light source unit in the first and second light source groups is connected in series.

[0013] The encoding modes include differential encoding mode and synchronous encoding mode; the switching unit is used to connect the third output terminal to the second light source group in the differential encoding mode; the switching unit is also used to connect the second output terminal to the second light source group in the synchronous encoding mode.

[0014] Optionally, the control module is connected to a multi-channel driver, and the control module is used to output switching information for each light source unit; the multi-channel driver includes multiple output terminals, each output terminal is connected to a light source unit, so as to control the corresponding light source unit to turn on or off based on the switching information.

[0015] Optionally, the control module is further configured to generate a trigger signal while generating the encoded signal, and send the trigger signal to the dynamic sensor module; the trigger signal is the same as the encoded signal;

[0016] The dynamic sensor is used to add a mark to the event stream information according to the trigger signal after acquiring the event stream information, so that the data processing module determines the spot image frame based on the mark and the first event stream, and determines the pupil image frame based on the mark and the second event stream.

[0017] Optionally, the light source module blinks at a first blink frequency based on the differential coded signal; the light source module blinks at a second blink frequency based on the synchronization coded signal; the first blink frequency is less than the second blink frequency.

[0018] Optionally, the dynamic sensor includes a master dynamic sensor and a slave dynamic sensor, which correspond to the two eyes respectively. The master dynamic sensor is used to send a clock signal to the slave dynamic sensor; the slave dynamic sensor is used to generate a timestamp in the event stream information based on the received clock signal.

[0019] Optionally, the sensor module further includes N frame sensors, and the control module is further configured to send a first pulse signal to the main dynamic sensor, and, for any frame sensor, send a second pulse signal to the frame sensor according to the frame rate of the frame sensor; wherein the rising edges of the first pulse signal and the second pulse signal are aligned.

[0020] In a second aspect, the present invention provides a method for obtaining eye features, the method comprising:

[0021] The control module controls the light source module to achieve a first timing sequence or a second timing sequence, and the irradiance of the light source module remains basically unchanged in the first timing sequence, while the irradiance of the light source module changes periodically in the second timing sequence.

[0022] Event stream information is acquired through a sensor module and sent to a data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence; the sensor module includes a dynamic sensor.

[0023] The data processing module determines a light spot image frame based on the first event stream information and a pupil image frame based on the second event stream information; the light spot image frame and the pupil image frame are used to determine eye movement information.

[0024] Thirdly, the present invention provides an eye-tracking device, comprising: an eye feature acquisition system and a processor as described in any of the first aspects, wherein the processor is configured to acquire a light spot image frame and a pupil image frame sent by the eye feature acquisition system, and determine eye movement information based on the light spot image frame and the pupil image frame.

[0025] This invention provides an eye feature acquisition system, method, and eye-tracking device. The system includes: a light source module; a control module for controlling the light source module to achieve a first time sequence or a second time sequence, wherein the irradiance of the light source module remains essentially constant in the first time sequence and the irradiance of the light source module changes periodically in the second time sequence; a sensor module, including a dynamic sensor, for acquiring event stream information and sending the event stream information to a data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence; and a data processing module for determining a spot image frame based on the first event stream information and determining a pupil image frame based on the second event stream information. The spot image frame and the pupil image frame are used to determine eye movement information. By controlling the light source module to achieve two time sequences, the dynamic sensor can acquire the event stream information corresponding to the two time sequences respectively, thereby accurately obtaining the spot image frame and the pupil image frame, so as to achieve accurate acquisition of eye features based on the dynamic sensor, thereby reducing system power consumption. In addition, acquiring eye features based on the dynamic sensor can also improve the accuracy of eye-tracking results. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0027] Figure 1 An application scenario diagram provided by an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the structure of an eye feature acquisition system provided in an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of an eye-tracking method provided in an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of a spot image frame provided in an embodiment of the present invention;

[0031] Figure 5 A schematic diagram of a pupil image frame provided in an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of a differentially coded signal provided in an embodiment of the present invention;

[0033] Figure 7 A schematic diagram of a synchronization coded signal provided in an embodiment of the present invention;

[0034] Figure 8 A schematic diagram of a light source module provided in an embodiment of the present invention;

[0035] Figure 9 This is a schematic diagram of a series-driven light source unit provided in an embodiment of the present invention;

[0036] Figure 10 This is a schematic diagram illustrating an independent driving method for each light source unit, provided as an embodiment of the present invention.

[0037] Figure 11 A schematic diagram of an event flow information provided in an embodiment of the present invention;

[0038] Figure 12 A schematic diagram of a trigger signal provided in an embodiment of the present invention;

[0039] Figure 13 A schematic diagram of another trigger signal provided in an embodiment of the present invention;

[0040] Figure 14 This is a schematic diagram illustrating multi-sensor synchronization as provided in an embodiment of the present invention;

[0041] Figure 15 A schematic diagram of a first pulse signal and a second pulse signal provided for an embodiment of the present invention;

[0042] Figure 16 This is a flowchart illustrating a method for obtaining eye features according to an embodiment of the present invention.

[0043] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0044] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention.

[0045] Figure 1 An application scenario diagram provided by an embodiment of the present invention, such as... Figure 1As shown, in a device containing an eye-tracking system, there is a controller. The controller can send control information to the eye feature acquisition system within the eye-tracking system to initiate eye tracking and acquire eye movement information. The eye feature acquisition system can acquire light spot image frames and pupil image frames. The light spot image frames are obtained due to the reflection of light by the cornea. The processor can determine the user's eye movement information based on the light spot image frames and pupil image frames, and send the determined eye movement information to the controller. A light source and a camera can be set in the eye feature acquisition system to illuminate the user's eyes and capture eye images, respectively.

[0046] In existing technologies, the camera used in eye feature acquisition systems is an infrared camera, also known as a frame sensor. In the images acquired by the frame sensor, usually only the information corresponding to the pixels of the pupil is valid, while the information recorded by most pixels is redundant, thereby increasing the power consumption of the system.

[0047] To address the aforementioned issues, a dynamic sensor is considered to acquire event information and determine eye images based on this information. The dynamic sensor only generates data when pixel brightness changes, and it doesn't capture information from all pixels at once, thus reducing data processing and transmission requirements and lowering system power consumption. When using a dynamic sensor, the light source module needs corresponding improvements. The light source module can implement either a first timing sequence or a second timing sequence. In the first timing sequence, the irradiance of the light source module remains essentially constant; in the second timing sequence, the irradiance of the light source module changes periodically. This allows the dynamic sensor to capture event information related to brightness changes and simultaneously obtain spot image frames and pupil image frames based on the event information, thereby determining eye movement information.

[0048] Figure 2 This is a schematic diagram of the structure of an eye feature acquisition system provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the system includes:

[0049] Light source module;

[0050] The control module is used to control the light source module to achieve a first timing sequence or a second timing sequence, wherein the irradiance of the light source module remains basically unchanged in the first timing sequence, and the irradiance of the light source module changes periodically in the second timing sequence.

[0051] The sensor module includes a dynamic sensor for acquiring event stream information and sending the event stream information to the data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence.

[0052] The data processing module is used to determine a light spot image frame based on the first event stream information, and to determine a pupil image frame based on the second event stream information; the light spot image frame and the pupil image frame are used to determine eye movement information.

[0053] The control module is the main control unit of the entire eye feature acquisition system, and it can control the light source module. Optionally, when the eye feature acquisition system is set up in a VR (Virtual Reality) device, the controller in the VR device sends the eye tracking command to the control module of the eye feature acquisition system, thereby enabling the control module to control the light source module.

[0054] Optionally, the control module can send a control signal to the light source module, so that the light source module can be turned on or off based on the control signal.

[0055] Optionally, the control module can control the light source module to implement either a first timing sequence or a second timing sequence. The light source module can first illuminate the eyeball according to the first timing sequence, and then illuminate the eyeball according to the second timing sequence; or, it can first illuminate the eyeball according to the second timing sequence, and then illuminate the eyeball according to the first timing sequence. The order in which the control module sends the two signals to the light source module is not limited. Since the time difference between the control module controlling the light source module to implement the first and second timing sequences is small, it can be assumed that the user's line of sight has not changed.

[0056] Optionally, the light source module includes two light source groups. When the light source module implements the first timing sequence, the two light source groups are in opposite open or closed states; when the light source module implements the second timing sequence, the two light source groups are in the same open or closed state.

[0057] The phrase "the irradiance of the light source module remains basically unchanged in the first time sequence" means that the dynamic sensor does not generate redundant events caused by changes in irradiance in the first time sequence. "Redundant events" usually refer to extra or unnecessary signals, data or events generated by the sensor during normal operation.

[0058] The eye feature acquisition system also includes a sensor module, which is a dynamic sensor. The dynamic sensor operates in an asynchronous pixel mode, only outputting the address and information of pixels where the light intensity changes, thereby eliminating redundant data at the source.

[0059] The dynamic sensor independently records the brightness change events of each pixel, thereby acquiring event stream information. This event stream information includes: a timestamp (the precise time of the brightness change event); pixel position (the pixel coordinates where the event occurred); and polarity information (whether the brightness increased or decreased). After acquiring the event stream information, the dynamic sensor sends it to the data processing module. The data processing module accumulates the event stream information to obtain an image. Optionally, since the light source module implements both a first timing sequence and a second timing sequence, the event stream information corresponding to the first timing sequence is called the first event stream information, and the event stream information corresponding to the second timing sequence is called the second event stream information. The data processing module then accumulates and processes the two types of event stream information separately to obtain a spot image frame and a pupil image frame.

[0060] After acquiring the light spot image frame and the pupil image frame, eye movement information can be determined based on pupil-corneal reflection technology. This allows us to determine the pupil position and the position of the corneal reflected light spot, and thus determine eye movement information based on the relative positional changes of these two elements. After obtaining the light spot image frame and the pupil image frame, the data processing module can send this information to the control module.

[0061] Figure 3 This is a schematic diagram of an eye-tracking method provided by an embodiment of the present invention. The control module and the light source module cooperate to implement LED (Light Emitting Diode) encoding. The sensor module acquires event stream information, and the data processing module obtains the corresponding image. The image can then be processed, specifically, light spot information processing and pupil information processing, to determine the user's gaze direction based on the eye-tracking algorithm. Specifically, the light spot information processing, pupil information processing, and the execution of the eye-tracking algorithm can be performed in the control module. Alternatively, the control module can send the light spot image frame and pupil image frame to a PC (Personal Computer) or other electronic device for execution of the light spot information processing, pupil information processing, and eye-tracking algorithm on the PC or other electronic device to determine the gaze direction.

[0062] The process of determining the light spot image frame based on the first time sequence is briefly explained below. The cornea of ​​the eye is a near-hemispherical mirror surface that can produce specular reflection of the light source; the skin, iris, and sclera near the eyeball, which are near-Lambertian surfaces, can produce diffuse reflection of the light source. When the light source illuminates the surface of the eyeball in the first time sequence, since the overall irradiance remains roughly constant at different times, the diffuse reflection of the light source on the eyeball is relatively uniform and does not change significantly. Therefore, the dynamic sensor cannot collect pixel information of the diffuse reflection areas. The dynamic sensor only collects the corneal reflection light spot information caused by the light intensity change due to specular reflection. Therefore, the light spot image frame can be obtained based on the first event stream information corresponding to the first time sequence.

[0063] Figure 4 This is a schematic diagram of a light spot image frame provided in an embodiment of the present invention. As can be seen, the acquired light spot image frame contains only the light spot.

[0064] The process of determining pupil image frames based on the second time sequence is briefly explained below. When the light source illuminates in the second time sequence, the overall irradiance changes at different times. The dynamic sensor can record the changes in light intensity in the scene and output an event stream to the data processing module based on these changes. The pupil has low light reflection characteristics, making it difficult to trigger events, while the surrounding iris, cornea, and sclera will exhibit diffuse or specular reflection, thus recording a large number of event points. Therefore, the pupil edge can be determined by fitting the distribution of event points.

[0065] Figure 5 This is a schematic diagram of a pupil image frame provided in an embodiment of the present invention. It can be seen that the acquired pupil image frame contains information about the pupil and other parts of the eye. The light spot in the pupil image frame is not complete; therefore, the direction of gaze can be determined by combining the pupil image frame and the light spot image frame.

[0066] Optional, such as Figure 2 As shown, since a user has two eyes, and both eyes typically focus on the same object simultaneously, a first light source module, a first sensor module, and a first data processing module can be configured for the left eye, and a second light source module, a second sensor module, and a second data processing module can be configured for the right eye. The control module can simultaneously send control signals to both the first and second light source modules, enabling them to simultaneously implement either a first timing sequence or a second timing sequence. This allows the two sensor modules to simultaneously acquire event stream information, ultimately determining the user's gaze direction based on the corresponding light spot image frames and pupil image frames for the left and right eyes, thereby improving the accuracy of eye tracking.

[0067] In continuous eye-tracking data acquired by frame sensors, typically only the data showing changes in pupil information is useful; most pixels record redundant information with limited actual usable data. This leads to repetitive work during image processing. Furthermore, in near-eye fixation tracking, the frame sensor continues operating even when the eye remains stationary, increasing system power consumption. In contrast, motion sensors only generate data when pixel brightness changes, and they don't capture information from all pixels at once. Therefore, motion sensors don't continuously capture complete image frames. This means that in static scenes or scenes with minimal brightness changes, motion sensors generate almost no data, reducing the need for data processing and transmission, thus lowering system power consumption.

[0068] Furthermore, existing frame sensors are limited by bandwidth and power, typically restricting frame rates to below 300Hz. Achieving higher frame rates requires expensive high-speed cameras. Motion sensors, on the other hand, generally offer higher frame rates and can effectively capture eye-movement information, improving the accuracy of eye tracking. Moreover, because motion sensors only transmit changing information, rather than the entire frame, they significantly reduce data bandwidth requirements and system power consumption.

[0069] Furthermore, when using frame sensors, the accuracy of eye tracking is also affected by ambient lighting. When the ambient light is insufficient or too strong, images captured by traditional frame sensors may be underexposed or overexposed, resulting in missing information about the eye and its surroundings. This affects subsequent calculations and reduces the accuracy of eye tracking results. However, dynamic sensors work by detecting changes in brightness to acquire event information, and to a certain extent, they can better adapt to different lighting conditions. Compared with traditional frame sensors, they can reduce the impact of ambient lighting on the accuracy of eye tracking results.

[0070] This invention provides an eye feature acquisition system, comprising: a light source module; a control module for controlling the light source module to implement a first time sequence or a second time sequence, wherein the irradiance of the light source module remains essentially constant in the first time sequence and the irradiance of the light source module varies periodically in the second time sequence; a sensor module, including a dynamic sensor, for acquiring event stream information and sending the event stream information to a data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence; and a data processing module for determining a spot image frame based on the first event stream information and determining a pupil image frame based on the second event stream information. The spot image frame and the pupil image frame are used to determine eye movement information. By controlling the light source module to implement two time sequences, the dynamic sensor can acquire the event stream information corresponding to the two time sequences respectively, thereby accurately obtaining the spot image frame and the pupil image frame, so as to achieve accurate acquisition of eye features based on the dynamic sensor, thereby reducing system power consumption. In addition, acquiring eye features based on the dynamic sensor can also improve the accuracy of eye tracking results.

[0071] Optionally, the control module is used to generate an encoded signal and send the encoded signal to the light source module. The encoded signal includes a differential encoded signal and a synchronization encoded signal. The light source module includes at least two light source groups, each light source group including multiple light source units. The at least two light source groups of the light source module are alternately turned on based on the differential encoded signal, and the at least two light source groups of the light source module are simultaneously turned on or simultaneously turned off based on the synchronization encoded signal.

[0072] Figure 6 This is a schematic diagram of a differentially coded signal provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the two light source groups are the first light source group and the second light source group, respectively. The differential coding signal can instruct the first light source group and the second light source group to be turned on alternately. That is, when each light source unit in the first light source group is turned on, each light source unit in the second light source group is turned off; when each light source unit in the second light source group is turned on, each light source unit in the first light source group is turned off.

[0073] The above encoding method allows the two light source groups to be turned on alternately, so that the total irradiance received by the eye at different times remains unchanged, thereby allowing the spot image frame to be determined based on the differential coded signal.

[0074] By controlling the two light source groups using differential coding, the overall irradiance can remain constant at different times, thus accurately obtaining the light spot image frame.

[0075] Figure 7This is a schematic diagram of a synchronous encoding signal provided in an embodiment of the present invention. The synchronous encoding signal refers to the simultaneous activation or deactivation of two light source groups, so that the total irradiance received by the eyeball varies at different times.

[0076] Optionally, when the light source group is turned on, each light source unit in the light source group is turned on.

[0077] By controlling the two light source groups through synchronous encoding, the overall irradiance can be varied at different times, thereby accurately obtaining pupil image frames.

[0078] Figure 8 This is a schematic diagram of a light source module provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the light source module includes a first light source group and a second light source group, and the light source in the light source group can be an infrared LED light source.

[0079] The light source units in the first light source group and the light source units in the second light source group appear alternately on the lamp board to ensure that the overall irradiance remains largely constant at different times in differential encoding mode. Furthermore, the distribution of each light source unit on the lamp board and the duty cycle of each light source unit can be adjusted to maintain a relatively constant overall irradiance in differential encoding mode.

[0080] In synchronous coding mode, each light source unit in the first light source group and each light source unit in the second light source group are turned on and off at the same frequency, duty cycle and timing.

[0081] Optionally, the above-mentioned light source module is only an example. The light source units in the light source module can not only be circular, but also other shapes. This application does not limit this.

[0082] The control module can drive the light source module in two ways: series driving and independent driving, which will be explained below.

[0083] Optionally, the light source group includes a first light source group and a second light source group; each light source group includes multiple light source units; the output signals from the first and second output terminals of the control module are in phase, and the output signals from the third and first output terminals of the control module are in opposite phase; the fourth output terminal of the control module is used to output the encoding mode; the first output terminal is connected to the first light source group through a first driver; the second, third, and fourth output terminals are respectively connected to a switching unit, and the switching unit is also connected to the second light source group through a second driver; each light source unit in the first and second light source groups is connected in series.

[0084] The encoding modes include differential encoding mode and synchronous encoding mode; the switching unit is used to connect the third output terminal to the second light source group in the differential encoding mode; the switching unit is also used to connect the second output terminal to the second light source group in the synchronous encoding mode.

[0085] Figure 9 This is a schematic diagram of a series-driven light source unit provided in an embodiment of the present invention, as shown below. Figure 9 As shown, the control module has four output terminals: IO0, IO1, IO2, and IO3. The first output terminal is directly connected to the first driver, which is connected to the first light source group. The light source units in the first light source group are connected in series. The second, third, and fourth output terminals are respectively connected to a switching unit. The switching unit can also be directly connected to the second driver, which is connected to the second light source group. The light source units in the second light source group are connected in series.

[0086] The fourth output of the control module outputs an encoding mode, such as differential encoding mode or synchronous encoding mode, so that the switching unit can connect the second or third output to the second driver. Since the output signals of the first and second outputs are in phase, and the output signals of the third and first outputs are in phase, when the encoding mode output by the fourth output is differential encoding mode, the switching module can connect the third output to the second driver; when the encoding mode output by the fourth output is synchronous encoding mode, the switching module can connect the second output to the second driver.

[0087] Optionally, the switching unit can be a single-pole double-throw switch.

[0088] Through the connection method between the control module and the light source module described above, it is possible to control the first light source group or the second light source group simultaneously. This method has the advantages of simple connection, fewer pins, and low cost.

[0089] Optionally, the control module is connected to a multi-channel driver, and the control module is used to output switching information for each light source unit; the multi-channel driver includes multiple output terminals, each output terminal is connected to a light source unit, so as to control the corresponding light source unit to turn on or off based on the switching information.

[0090] Figure 10 This is a schematic diagram of an embodiment of the present invention providing an independently driven light source unit, as shown below. Figure 10As shown, the control module can output switching information for each light source unit to the multi-channel driver, such as turning on each light source unit in the first light source group and turning off each light source unit in the second light source group. The multi-channel driver can control each light source unit to turn on or off based on the acquired switching information.

[0091] By independently controlling each light source unit, the duty cycle and timing of each light source unit (LED) can be individually adjusted, thereby improving the flexibility of light source module control.

[0092] Optionally, the control module is further configured to generate a trigger signal while generating the encoded signal, and send the trigger signal to the dynamic sensor module; the trigger signal is the same as the encoded signal;

[0093] The dynamic sensor is used to add a mark to the event stream information according to the trigger signal after acquiring the event stream information, so that the data processing module determines the spot image frame based on the mark and the first event stream, and determines the pupil image frame based on the mark and the second event stream.

[0094] The dynamic sensor also provides a Trigger In interface, which can receive trigger signals sent by the control module. The control module can then send the trigger signals to the data processing module, which in turn sends the trigger signals to the sensor module.

[0095] The control module generates a trigger signal simultaneously with the encoded signal, and the trigger signal and the encoded signal are identical. Upon receiving the trigger signal, the dynamic sensor adds a marker to the acquired event stream information. Specifically, the marker can be inserted into the event stream information at the switching times of the light source unit. Thus, after acquiring the event stream information sent by the sensor, the data processing module can focus on the information before and after the marker position to quickly determine the light spot image frame based on the marker and the first event stream, and to quickly determine the pupil image frame based on the marker and the second event stream. Figure 11 This is a schematic diagram of an event flow information provided in an embodiment of the present invention, such as... Figure 11 As shown, markers can be inserted into the event stream when the trigger signal is a rising edge or a falling edge, that is, when the light source is turned on and off.

[0096] Figure 12 This is a schematic diagram of a trigger signal provided in an embodiment of the present invention. Figure 13 This is a schematic diagram of another trigger signal provided in an embodiment of the present invention. As can be seen from the figure, the trigger signal is consistent with the encoding signal of the first light source group or the second light source group.

[0097] By setting the same trigger signal as the encoding signal and marking the event stream information based on the trigger signal, the data processing module can quickly obtain the spot image frame and the pupil image frame.

[0098] Optionally, the light source module blinks at a first blink frequency based on the differential coded signal; the light source module blinks at a second blink frequency based on the synchronization coded signal; the first blink frequency is less than the second blink frequency.

[0099] In synchronous coding mode, the skin, iris, and sclera, which are based on the surface of an approximate Lambertian body, will diffusely reflect the light source. The characteristic of diffuse reflection is that less light is reflected and less information about the brightness change of the light. This can increase the second flash frequency corresponding to the flashing of the synchronous coding signal, making the second flash frequency greater than the first flash frequency corresponding to the flashing of the differential coding signal. This allows for the collection of more event information, thereby improving the accuracy of the acquired pupil image frames.

[0100] To improve the accuracy of eye tracking, binocular data acquisition needs to be synchronized. Furthermore, in addition to acquiring spot image frames and pupil image frames based on a motion sensor, image frames, such as complete reference images, can also be acquired based on a frame sensor to improve eye tracking accuracy. Therefore, synchronization between the motion sensor and the frame sensor is also necessary.

[0101] Optionally, the dynamic sensor includes a master dynamic sensor and a slave dynamic sensor, which correspond to the two eyes respectively. The master dynamic sensor is used to send a clock signal to the slave dynamic sensor; the slave dynamic sensor is used to generate a timestamp in the event stream information based on the received clock signal.

[0102] The motion sensor has an internal timestamp clock, such as 1MHz, meaning it records event information at 1µs intervals. The motion sensor has Sync Out and Sync In interfaces. When synchronizing multiple motion sensors, one sensor can be configured as the master sensor, and the others as slave sensors. The master sensor outputs a 1MHz clock signal through its Sync Out interface, while the slave sensors receive this external clock signal through their Sync In interfaces and generate timestamps based on it.

[0103] Optionally, in the above configuration, the slave dynamic sensor can be started first, followed by the master dynamic sensor. This will cause the dynamic sensors to generate synchronized timestamp records.

[0104] Figure 14This is a schematic diagram of multi-sensor synchronization provided in an embodiment of the present invention. The master dynamic sensor sends a clock signal to the slave dynamic sensor to effectively realize synchronization between dynamic sensors and improve the accuracy of eye-tracking results.

[0105] Optionally, the sensor module further includes N frame sensors, and the control module is further configured to send a first pulse signal to the main dynamic sensor, and, for any frame sensor, send a second pulse signal to the frame sensor according to the frame rate of the frame sensor; wherein the rising edges of the first pulse signal and the second pulse signal are aligned.

[0106] The control module outputs a first pulse signal and a second pulse signal, with the rising edges of both signals being output simultaneously. The frame sensor captures an image based on the second pulse signal. Therefore, the markers in the event stream information of the motion sensor are synchronized with the image information of the frame sensor.

[0107] Figure 15 This is a schematic diagram of a first pulse signal and a second pulse signal provided in an embodiment of the present invention. As can be seen, the rising edges of the first pulse signal and the second pulse signal are output simultaneously.

[0108] By synchronizing the data stream information of the dynamic sensor and the frame sensor, the analysis efficiency of the data processing module can be improved.

[0109] Figure 16 This is a flowchart illustrating an eye feature acquisition method according to an embodiment of the present invention. The method includes steps S1601 to S1603:

[0110] Step S1601: Control the light source module through the control module to achieve the first timing sequence or the second timing sequence, and the irradiance of the light source module remains basically unchanged in the first timing sequence, while the irradiance of the light source module changes periodically in the second timing sequence.

[0111] Step S1602: Obtain event stream information through the sensor module and send the event stream information to the data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence; the sensor module includes a dynamic sensor;

[0112] Step S1603: The data processing module determines the light spot image frame based on the first event stream information and the pupil image frame based on the second event stream information; the light spot image frame and the pupil image frame are used to determine eye movement information.

[0113] In the above-described method for acquiring eye features, the control module first controls the light source module to achieve either a first timing sequence or a second timing sequence. The light source module can emit light sequentially according to the first and second timing sequences. After the light source module emits light, the sensor module can acquire event stream information and transmit it to the data processing module. The data processing module can then determine the spot image frame and the pupil image frame based on the two event stream information, respectively.

[0114] The detailed implementation process of the above method can be found in the foregoing embodiments, and will not be repeated here.

[0115] This invention provides a method for acquiring eye features. The method includes: controlling a light source module via a control module to achieve a first time sequence or a second time sequence, wherein the irradiance of the light source module remains essentially constant in the first time sequence, and the irradiance of the light source module changes periodically in the second time sequence; acquiring event stream information via a sensor module and sending the event stream information to a data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence; the sensor module includes a dynamic sensor; the data processing module determines a spot image frame based on the first event stream information and a pupil image frame based on the second event stream information; the spot image frame and the pupil image frame are used to determine eye movement information. By controlling the light source module to achieve two time sequences, the dynamic sensor can acquire the event stream information corresponding to the two time sequences respectively, thereby accurately obtaining the spot image frame and the pupil image frame, achieving accurate acquisition of eye features based on the dynamic sensor, thereby reducing system power consumption; furthermore, acquiring eye features based on the dynamic sensor can also improve the accuracy of eye tracking results.

[0116] This application also provides an eye-tracking device, including: an eye feature acquisition system and a processor, wherein the processor is used to acquire a spot image frame and a pupil image frame sent by the eye feature acquisition system, and to determine eye movement information based on the spot image frame and the pupil image frame.

[0117] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0118] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0120] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An eye feature acquisition system, characterized in that, include: Light source module; The control module is used to control the light source module to achieve a first timing sequence or a second timing sequence, wherein the irradiance of the light source module remains basically unchanged in the first timing sequence, and the irradiance of the light source module changes periodically in the second timing sequence. The sensor module includes a dynamic sensor for acquiring event stream information and sending the event stream information to the data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence. The data processing module is used to determine a light spot image frame based on the first event stream information, and to determine a pupil image frame based on the second event stream information; the light spot image frame and the pupil image frame are used to determine eye movement information.

2. The system according to claim 1, characterized in that, The control module is used to generate an encoded signal and send the encoded signal to the light source module. The encoded signal includes a differential encoded signal and a synchronization encoded signal. The light source module includes at least two light source groups, each light source group including multiple light source units. The at least two light source groups of the light source module are turned on alternately based on the differential encoded signal, and the at least two light source groups of the light source module are turned on or off simultaneously based on the synchronization encoded signal.

3. The system according to claim 2, characterized in that, The light source group includes a first light source group and a second light source group; each light source group includes multiple light source units; the first and second output terminals of the control module output signals with the same phase, and the third output terminal of the control module output signals with opposite phases to the first output terminal; the fourth output terminal of the control module is used to output the encoding mode; the first output terminal is connected to the first light source group through a first driver; the second, third, and fourth output terminals are respectively connected to a switching unit, and the switching unit is also connected to the second light source group through a second driver; each light source unit in the first and second light source groups is connected in series. The encoding modes include differential encoding mode and synchronous encoding mode; the switching unit is used to connect the third output terminal to the second light source group in the differential encoding mode; the switching unit is also used to connect the second output terminal to the second light source group in the synchronous encoding mode.

4. The system according to claim 3, characterized in that, The control module is connected to a multi-channel driver and is used to output switching information for each light source unit. The multi-channel driver includes multiple output terminals, each of which is connected to a light source unit to control the corresponding light source unit to turn on or off based on the switching information.

5. The system according to claim 2, characterized in that, The control module is further configured to generate a trigger signal while generating the encoded signal, and send the trigger signal to the dynamic sensor module; the trigger signal is the same as the encoded signal; The dynamic sensor is used to add a mark to the event stream information according to the trigger signal after acquiring the event stream information, so that the data processing module determines the spot image frame based on the mark and the first event stream, and determines the pupil image frame based on the mark and the second event stream.

6. The system according to claim 2, characterized in that, The light source module blinks at a first blinking frequency based on the differential coded signal; the light source module blinks at a second blinking frequency based on the synchronous coded signal; the first blinking frequency is less than the second blinking frequency.

7. The system according to any one of claims 1-6, characterized in that, The dynamic sensor includes a master dynamic sensor and a slave dynamic sensor, which are respectively associated with the two eyes. The master dynamic sensor is used to send a clock signal to the slave dynamic sensor. The dynamic sensor is used to generate a timestamp in the event stream information based on the received clock signal.

8. The system according to claim 7, characterized in that, The sensor module further includes N frame sensors, and the control module is also used to send a first pulse signal to the main dynamic sensor, and to send a second pulse signal to any frame sensor according to the frame rate of the frame sensor; wherein the rising edges of the first pulse signal and the second pulse signal are aligned.

9. A method for obtaining eye features, characterized in that, The method includes: The control module controls the light source module to achieve a first timing sequence or a second timing sequence, and the irradiance of the light source module remains basically unchanged in the first timing sequence, while the irradiance of the light source module changes periodically in the second timing sequence. Event stream information is acquired through a sensor module and sent to a data processing module; the event stream information includes first event stream information corresponding to the first time sequence and second event stream information corresponding to the second time sequence; the sensor module includes a dynamic sensor. The data processing module determines a light spot image frame based on the first event stream information and a pupil image frame based on the second event stream information; the light spot image frame and the pupil image frame are used to determine eye movement information.

10. An eye-tracking device, characterized in that, include: The eye feature acquisition system and processor according to any one of claims 1-9, wherein the processor is configured to acquire a spot image frame and a pupil image frame sent by the eye feature acquisition system, and determine eye movement information based on the spot image frame and the pupil image frame.

Images