Eyeball tracking system, method, device and equipment and storage medium

Abstract

The invention discloses an eyeball tracking system, method, device and equipment and a storage medium. Acquiring a first signal, wherein the first signal comprises eye features; wherein the first signal is collected when the light source scans and irradiates the set area at the first power, and the light source comprises a plurality of sub light sources; determining an eye position according to the first signal; adjusting the reflection direction of the reflector according to the eye position so as to reflect the light beam of the light source to the eye position; a second signal is obtained, the second signal comprises eye features, and the power of the sub-light source irradiating the eye position is adjusted from the first power to the second power; the second power is greater than the first power; and performing eyeball tracking according to the second signal. According to the eyeball tracking method provided by the embodiment of the invention, the signal scanning is performed by using the relatively small first power at the beginning, and the power of the sub light source irradiated to the eye position is adjusted from the first power to the relatively large second power after the eye position is determined, so that the light source utilization rate in the eyeball tracking process can be improved, resources are saved, and power consumption is reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of eye tracking technology, and in particular to an eye tracking system, method, apparatus, device, and storage medium. Background Art

[0002] Currently, there are two main methods for fill lighting in eye tracking devices: one uses multiple light sources for fill lighting, and the other uses a polarized light source. Both methods have the following drawbacks: most of the light power emitted by the light source is wasted in areas unrelated to the eyes, resulting in low light source utilization. Summary of the Invention

[0003] Embodiments of the present invention provide an eye tracking system, method, apparatus, device, and storage medium, which can improve light source utilization and save resources during eye tracking.

[0004] In a first aspect, an embodiment of the present invention provides an eye tracking system, comprising: a light source, at least one reflector, at least one micro-electromechanical controller, a signal sensor, and a processor;

[0005] The light source is used to emit a light beam; the at least one reflector is arranged on the light path of the light beam emitted by the light source, and is used to reflect the light beam to the position of the eyes or face;

[0006] The at least one micro-electromechanical controller is used to control the reflection direction of the at least one reflector;

[0007] The signal sensor is used to collect signals; the signals include eye features;

[0008] The processor is connected to the signal sensor and is used to receive the signal and determine the eye position based on the first signal; or receive the signal and determine the gaze point based on the signal; the processor is also used to adjust the power of the light source.

[0009] In a second aspect, an embodiment of the present invention further provides an eye tracking method, which is performed by the eye tracking system described in this embodiment. The method includes:

[0010] Acquire a first signal, the first signal including an eye feature; wherein the first signal is acquired when a light source scans and illuminates a set area at a first power, and the light source includes a plurality of sub-light sources;

[0011] determining an eye position based on the first signal;

[0012] adjusting the reflection direction of the reflector according to the eye position to reflect the light beam from the light source to the eye position;

[0013] Acquire a second signal, the second signal including an eye feature, and adjust the power of the sub-light source irradiated at the eye position from the first power to a second power; wherein the second power is greater than the first power;

[0014] Eye tracking is performed according to the second signal.

[0015] In a third aspect, an embodiment of the present invention further provides an eye tracking device, which is provided in the eye tracking system described in this embodiment, and includes:

[0016] a signal acquisition module, configured to acquire a first signal, the first signal including eye features; wherein the first signal is acquired when a light source scans and illuminates a set area at a first power, and the light source includes a plurality of sub-light sources;

[0017] an eye position determining module, configured to determine an eye position according to the first signal;

[0018] a reflection direction adjustment module, configured to adjust the reflection direction of the reflector according to the eye position, so as to reflect the light beam from the light source to the eye position;

[0019] a power adjustment module, configured to adjust the power of the sub-light source irradiating the eye position from the first power to a second power; wherein the second power is greater than the first power;

[0020] The signal acquisition module is further configured to acquire a second signal, wherein the second signal includes eye features;

[0021] An eye tracking module is configured to perform eye tracking according to the second signal.

[0022] In a fourth aspect, an embodiment of the present invention further provides an electronic device, comprising:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the eye tracking method described in the embodiment of the present invention.

[0026] In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the eye tracking method described in the embodiment of the present invention when executed.

[0027] The embodiments of the present invention disclose an eye tracking system, method, apparatus, device and storage medium. A first signal is obtained, the first signal including eye features; wherein the first signal is collected when a light source scans and illuminates a set area with a first power, and the light source includes multiple sub-light sources; the eye position is determined according to the first signal; the reflection direction of the reflector is adjusted according to the eye position to reflect the light beam of the light source to the eye position; a second signal is obtained, the second signal including eye features, and the power of the sub-light source irradiated to the eye position is adjusted from the first power to the second power; wherein the second power is greater than the first power; and eye tracking is performed according to the second signal. The eye tracking method provided in this embodiment uses a smaller first power to perform signal scanning at the beginning, and after the eye position is determined, the power of the sub-light source irradiated to the eye position is adjusted from the first power to the larger second power, which can improve the utilization rate of the light source during eye tracking, save resources and reduce power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a structural diagram of an eye tracking system in embodiment 1 of the present invention;

[0029] Figure 2 is a flow chart of an eye tracking method in embodiment 2 of the present invention;

[0030] Figure 3 is a schematic structural diagram of an eye tracking device in Embodiment 3 of the present invention;

[0031] Figure 4 It is a structural diagram of an electronic device in embodiment 4 of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0033] Eye tracking, also known as gaze tracking, is a technique for estimating eye gaze and / or fixation by measuring eye movements.

[0034] The line of sight can be understood as a three-dimensional vector, and the gaze point can be understood as the two-dimensional coordinate of the three-dimensional vector projected on a certain plane.

[0035] The most widely used method is optical recording: using a camera or video camera to record the subject's eye movements, obtaining eye images reflecting the eye movements. Eye features are then extracted from these images to build a model for line of sight / gaze point estimation. Eye features can include pupil position, pupil shape, iris position, iris shape, eyelid position, canthus position, and light spot (also known as Purkinje's spot).

[0036] Among optical recording methods, the most mainstream eye tracking method is called pupil-corneal reflection (PCCR), which is as follows:

[0037] Part 1. Eye Image Acquisition:

[0038] The light source is directed toward the eye, and the image acquisition device photographs the eye, and the corresponding reflection point of the light source on the cornea is photographed, namely the light spot (also called Purkinje spot), thereby obtaining an eye image with the light spot.

[0039] Part 2. Line of sight / gaze estimation:

[0040] As the eyeball rotates, the relative position relationship between the pupil center and the light spot changes accordingly, and the corresponding collected eye images with the light spot reflect this position change relationship;

[0041] Line of sight / gaze point estimation is performed based on the position change relationship.

[0042] It can be seen from this that when using the pupil-corneal reflection method to estimate the line of sight / gaze point, it is very important to determine the matching relationship between the light spot and the light source.

[0043] In addition to optical recording methods, eye tracking devices can also be MEMS micro-electromechanical systems, such as MEMS infrared scanning reflectors, infrared light sources, and infrared receivers; or capacitive sensors, which detect eye movements through the capacitance value between the eyeball and the capacitor plate; or they can be myoelectric detectors, which detect eye movements by placing electrodes on the bridge of the nose, forehead, ears, or earlobes and detecting myoelectric signal patterns.

[0044] Example 1

[0045] Figure 1 This is a structural diagram of an eye tracking system provided in Example 1 of the present invention, as shown in FIG. Figure 1 As shown, the system includes: a light source, at least one reflector, at least one micro-electro-mechanical system (MEMS) controller, a signal sensor and a processor.

[0046] The light source is used to emit a light beam and can be a laser, LED, or other light source. At least one reflector is disposed in the optical path of the light beam emitted by the light source and is used to reflect the light beam toward the eyes or face. At least one microelectromechanical controller is used to control the reflection direction of the at least one reflector. A signal sensor is used to collect signals, and the signals include eye features, which may include pupil position, pupil shape, iris position, iris shape, eyelid position, canthus position, and light spot (also known as Purkinje spot) position. The signal sensor can be an image sensor, an optical sensor, or the like. The image sensor can be included in an infrared camera, a DVS camera, or a visible light camera, while the optical sensor can be included in a scanner. A processor is connected to the signal sensor and is used to receive the signal and determine the eye position based on the signal; alternatively, it receives the signal and determines the gaze point based on the signal. The processor is also used to adjust the power of the light source.

[0047] The light source includes multiple sub-light sources, for example, two sub-light sources. In this embodiment, the eye tracking system operates in two stages. In the first stage, the light source emits a light beam at a relatively low power. The microelectromechanical controller controls the reflector to move within a certain range, causing the reflected light beam to scan and illuminate a predetermined area (e.g., a person's facial area). At this stage, the signal sensor collects a first signal. It is understood that the first signal can be an image captured by an image sensor or a light signal captured by an optical sensor. In this example, the first signal is an image containing the facial area of a person. The signal sensor sends the image containing the facial area to the processor. The processor extracts the grayscale value of the image and determines whether the grayscale value is less than a set threshold. If the grayscale value is less than the set threshold, the power of the light source is gradually increased according to an iterative approximation method. During each adjustment, the microelectromechanical controller controls the reflector to move within a certain range, causing the reflected light beam to scan the predetermined area. The signal sensor then controls the signal sensor to collect an image and sends the image to the processor. This continues until the grayscale value of the image is greater than or equal to the set threshold, indicating that the power of the light source has met the required level. The processor determines the eye position based on the current first signal. The method for determining the eye position can be to identify human facial features such as the inner and outer corners of the eyes, upper and lower eyelids, eyelashes, eyebrows, and pupils based on feature data in the image or light signal, that is, to determine the eye position through image recognition technology. In the second stage, the micro-electromechanical controller adjusts the reflection direction of the reflector according to the eye position determined in the first stage to reflect the light beam of the light source to the eye position, controls the signal sensor to collect the second signal (such as the eye image), and increases the power of the sub-light source in the light source that illuminates the eye position. Finally, eye tracking is performed based on the collected second signal. Here, the second signal can be an image collected by the image sensor or a light signal collected by the optical sensor. When performing eye tracking, the signal that needs to be collected needs to be clearer in order to obtain accurate eye features, so it is necessary to increase the power of the sub-light source at the corresponding position.

[0048] The eye tracking system provided in this embodiment includes: a light source, at least one reflector, at least one micro-electromechanical controller, a signal sensor, and a processor. The light source is configured to emit a light beam; at least one reflector is disposed in the optical path of the light beam emitted by the light source and is configured to reflect the light beam toward the eye or face; at least one micro-electromechanical controller is configured to control the reflection direction of the at least one reflector; a signal sensor is configured to collect signals; a processor is connected to the signal sensor and is configured to receive the signals and determine the eye position or gaze point based on the signals; the processor is further configured to adjust the power of the light source. This can improve light source utilization during eye tracking, conserve resources, and reduce power consumption.

[0049] Example 2

[0050] Figure 2 : is a flow chart of an eye tracking method provided in the second embodiment of the present invention. The method is applicable to the case of tracking the eye. The method is executed by the eye tracking system provided in the above embodiment. Figure 2 As shown, the method includes the following steps:

[0051] S210: Acquire a first signal.

[0052] Among them, the first signal includes eye features, and is collected when the light source scans and illuminates the set area at a first power, and the light source includes multiple sub-light sources. Eye features may include: pupil position, pupil shape, iris position, iris shape, eyelid position, eye corner position, light spot (also known as Purkinje spot) position, etc. The characteristic value of the first signal is greater than the set threshold, that is, the brightness of the first signal meets the signal processing requirements. In this embodiment, the set area can be a facial area. The first power can be understood as the minimum light source power that can determine the position of the human eye through signal recognition technology. This can ensure that when the device starts running, it can operate at a lower power and can identify the position of the human eye.

[0053] Specifically, the process of obtaining the first signal can be: controlling the light source to emit a light beam at a first power; determining the movement range of the reflector, and controlling the reflector to move within the movement range so that the reflected light beam scans and illuminates the set area; and collecting the first signal of the light beam scanning and illumination.

[0054] The range of motion includes a translation range and / or a rotation range. According to the principle of reflection, the position and rotation angle of the reflector affect the direction of light reflection. In this embodiment, in order to enable the reflected light to scan and illuminate a set area, a micro-electromechanical controller is required to control the reflector to move within a certain translation range and / or rotation range. The range of motion of the reflector is determined by the position information of the light source, the position information of the reflector, the position information of the set area, and the size of the set area.

[0055] The size of the set area can be understood as the size or area of the set area. Specifically, the motion range of the reflector can be determined by: obtaining the position information of the light source, the position information of the reflector, the position information of the set area, and the size of the set area; and determining the motion range of the reflector based on the position information of the light source, the position information of the reflector, the position information of the set area, and the size of the set area.

[0056] Among them, the posture information of the light source includes the position and / or posture of the light source, and the position and posture of the light source determine the direction of the light beam emitted by the light source. The posture information of the reflector includes the position and / or posture of the reflector, which determines the reflection direction of the incident light source by the reflector, wherein the posture of the reflector can be characterized by the rotation angle of the reflector. The posture information of the set area includes the position and / or posture of the set area, wherein the posture of the set area can be characterized by the orientation of the set area. The posture information of the set area and the size or area of the set area determine the direction in which the reflector needs to reflect the light. In this embodiment, the posture information of the light source, the posture information of the reflector, the posture information of the set area and the size or area of the set area are calculated according to the principle of light propagation and reflection, and the motion range of the reflector can be obtained, so that the reflected light beam scans and illuminates the set area (such as the facial area).

[0057] Optionally, when the signal is an image, the method of controlling the light source to emit a light beam at a first power can be: controlling the light source to emit a light beam at a third power, and controlling the reflector to move within a motion range so that the reflected light beam scans and illuminates a set area; obtaining a first image scanned and illuminated by the light beam, and extracting a grayscale value of the first image; judging whether the grayscale value is less than a set threshold; if the grayscale value is less than the set threshold, adjusting the power of the light source in an iterative approximation manner until the grayscale value of the first image corresponding to the adjusted power is greater than or equal to the set threshold, determining the final adjusted power as the first power, and controlling the light source to emit a light beam at the first power.

[0058] Wherein, the third power is less than the first power. The method for determining the range of motion of the reflector can refer to the above embodiment and will not be repeated here. The set threshold value can be understood as the minimum grayscale value that can meet the requirements of image processing. The process of adjusting the power of the light source by the iterative approximation method can be: first increase the power of the light source by a set value, and obtain the first image irradiated with the increased power, and determine the grayscale value of the first image; if the grayscale value is less than the set threshold value, continue to increase the power of the light source by a set value, and obtain the first image irradiated with the increased power, and determine the grayscale value of the first image. And so on, until the grayscale value of the first image is greater than or equal to the set threshold value, stop adjusting the power of the light source, and use the last adjusted power as the first power. That is, after each adjustment of the power of the light source, the grayscale value of the first image corresponding to the adjusted power must be obtained.

[0059] Specifically, the process of adjusting the power of the light source in an iterative approximation manner can be: increasing the power of the light source by a set value, and controlling the reflector to move within the motion range so that the reflected light beam scans and illuminates the set area; obtaining a first image of the light beam scanning and illuminating, and extracting the grayscale value of the first image; judging whether the grayscale value is less than a set threshold; if the grayscale value is less than the set threshold, returning to execute the operation of increasing the power of the light source by a set value until the grayscale value is greater than or equal to the set threshold; if the grayscale value is greater than or equal to the set threshold, determining the increased power as the first power.

[0060] In this embodiment, the power of the light source is adjusted by iterative approximation, which can prevent the light source power from being adjusted to a large value at one time, thereby wasting resources.

[0061] S220: Determine the eye position according to the first signal.

[0062] The eye position can be represented by the position of a detection frame within which the eye is located. The detection frame can be a rectangular frame, a circular frame, an elliptical frame, or the like. As described above, the first signal can be an image captured by an image sensor, or a light signal captured by an optical sensor. The optical sensor is included in a scanner, which ultimately converts the light signal into an image. In this embodiment, determining the eye position based on the first signal can be performed by performing eye detection on the first image and obtaining the position of the detection frame within which the eye is located as the eye position. Eye detection in the first image can be performed using existing target detection algorithms, which are not limited herein.

[0063] S230: Adjust the reflection direction of the reflector according to the eye position to reflect the light beam from the light source to the eye position.

[0064] In this embodiment, the position of the light source remains unchanged. In order to focus the light source on the eye, the reflection direction of the reflector needs to be adjusted according to the position of the eye so that the reflected light propagates to the eye. The method for adjusting the reflection direction of the reflector according to the eye position can be: first, determine the position adjustment amount of the reflector based on the eye position, the position information of the reflector, and the position information of the light source; then adjust the position information of the reflector based on the position adjustment amount so that the light beam reflected by the reflector propagates to the eye position. The position adjustment amount of the reflector can be obtained by calculating the eye position, the position information of the reflector, and the position information of the light source based on the principle of light propagation and reflection.

[0065] S240: Acquire a second signal, and adjust the power of the sub-light source irradiated to the eye position from the first power to the second power.

[0066] The second signal includes eye features, which may include pupil position, pupil shape, iris position, iris shape, eyelid position, eye corner position, light spot (also known as Purkinje spot) position, etc.

[0067] The second power is greater than the first power. In this embodiment, the light source is composed of multiple sub-light sources, for example, two sub-light sources, and light from some of the sub-light sources is irradiated to the eye position, while light from some of the sub-light sources is not irradiated to the eye position. In this case, in order to obtain a clearer eye image, the power of the sub-light source irradiating to the eye position is adjusted from the first power to the second power.

[0068] Specifically, the power of the sub-light source irradiating the eye position can be adjusted from the first power to the second power by: obtaining the distance between the eye and the light source; determining the second power based on the distance; and adjusting the power of the sub-light source irradiating the eye position from the first power to the second power. Alternatively, the power can be directly set to a value for the second power, for example, by setting the second power to the rated power of the light source; and adjusting the power of the sub-light source irradiating the eye position from the first power to the second power.

[0069] Among them, the method of determining the second power based on distance can be: obtaining the initial power of the light source and the minimum distance between the eye and the light source. The initial power can be understood as the minimum power of the light source that meets the eye tracking requirements when the eye is closest to the light source, and needs to be set before the product leaves the factory according to actual conditions. The second power is determined based on the initial power of the light source, the minimum distance and the actual distance between the eye and the light source. Among them, the minimum distance can be understood as the minimum distance that can be set between the light source and the eye, for example, the minimum distance is 30cm. The formula can be expressed as: Where α is the setting coefficient, P0 is the initial power, S0 is the minimum distance between the eye and the light source, and S1 is the actual distance between the light source and the eye. It can be understood that the closer the eye is to the light source, the smaller the second power, and the farther the eye is from the light source, the larger the second power.

[0070] S250: Perform eye tracking according to the second signal.

[0071] In this embodiment, the method of performing eye tracking based on the second signal can adopt any existing eye tracking algorithm, which is not limited here. The second signal here can be understood as a second image or a second light signal, and the light signal is also converted into an image in the eye tracking calculation.

[0072] The technical solution of this embodiment is to obtain a first signal, the first signal including eye features; wherein the first signal is collected when a light source scans and illuminates a set area at a first power, and the light source includes multiple sub-light sources; determine the eye position based on the first signal; adjust the reflection direction of the reflector based on the eye position to reflect the light beam of the light source to the eye position; obtain a second signal, the second signal including eye features, and adjust the power of the sub-light source irradiated to the eye position from the first power to the second power; wherein the second power is greater than the first power; and perform eye tracking based on the second signal. The eye tracking method provided in this embodiment initially uses a smaller first power for signal scanning, and after determining the eye position, adjusts the power of the sub-light source irradiated to the eye position from the first power to the larger second power, which can improve the utilization rate of the light source during the eye tracking process, save resources and reduce power consumption.

[0073] Example 3

[0074] Figure 3 FIG. 1 is a schematic diagram of the structure of an eye tracking device provided in a third embodiment of the present invention. The device is provided in the eye tracking system described in this embodiment, and includes:

[0075] The signal acquisition module 310 is configured to acquire a first signal, wherein the first signal includes eye features; wherein the first signal is acquired when a light source scans and illuminates a set area at a first power, and the light source includes a plurality of sub-light sources;

[0076] an eye position determination module 320, configured to determine an eye position according to the first signal;

[0077] A reflection direction adjustment module 330 is used to adjust the reflection direction of the reflector according to the position of the eye, so as to reflect the light beam from the light source to the position of the eye;

[0078] A power adjustment module 340 is configured to adjust the power of the sub-light source irradiated at the eye position from a first power to a second power; wherein the second power is greater than the first power;

[0079] The signal acquisition module 310 is further configured to acquire a second signal, the second signal including eye features;

[0080] The eye tracking module 350 is configured to perform eye tracking according to the second signal.

[0081] Optionally, the signal acquisition module 310 is further configured to:

[0082] controlling the light source to emit a light beam at a first power;

[0083] Determining a motion range of the reflector and controlling the reflector to move within the motion range so that the reflected light beam scans and illuminates a set area; wherein the motion range includes a movement range and / or a rotation range;

[0084] The first signal of the light beam scanning irradiation is collected.

[0085] Optionally, the signal acquisition module 310 is further configured to:

[0086] Obtaining the position information of the light source, the position information of the reflector, the position information of the set area, and the size of the set area;

[0087] The motion range of the reflector is determined according to the position information of the light source, the position information of the reflector, the position information of the set area, and the size of the set area.

[0088] Optionally, the signal acquisition module 310 includes an image acquisition module 3101, further configured to:

[0089] Controlling the light source to emit a light beam at a third power, and controlling the reflector to move within a range of motion so that the reflected light beam scans and illuminates a set area; wherein the third power is less than the first power;

[0090] Acquire a first image irradiated by the light beam scanning, and extract a grayscale value of the first image;

[0091] Determine whether the grayscale value is less than the set threshold;

[0092] If the grayscale value is less than the set threshold, the power of the light source is adjusted in an iterative approximation manner until the grayscale value of the first image corresponding to the adjusted power is greater than or equal to the set threshold. The final adjusted power is determined as the first power, and the light source is controlled to emit a light beam at the first power.

[0093] Optionally, the image acquisition module 3101 is further configured to:

[0094] Increase the power of the light source to a set value and control the reflector to move within the range of motion so that the reflected light beam scans and illuminates the set area;

[0095] Acquire a first image irradiated by the light beam scanning, and extract a grayscale value of the first image;

[0096] Determine whether the grayscale value is less than the set threshold;

[0097] If the grayscale value is less than the set threshold, the process returns to the operation of increasing the power of the light source by the set value until the grayscale value is greater than or equal to the set threshold;

[0098] If the grayscale value is greater than or equal to the set threshold, the increased power is determined as the first power.

[0099] Optionally, the power adjustment module 340 is further configured to:

[0100] Get the distance between the eye and the light source;

[0101] determining a second power based on the distance;

[0102] The power of the sub-light source irradiated to the eye position is adjusted from the first power to the second power.

[0103] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not fully described in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0104] Example 4

[0105] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0106] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0107] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0108] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the eye tracking method.

[0109] In some embodiments, the eye tracking method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the eye tracking method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the eye tracking method in any other suitable manner (e.g., via firmware).

[0110] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0115] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0116] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0117] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An eye tracking system, characterized in that: include: a light source, at least one reflector, at least one micro-electromechanical controller, a signal sensor and a processor; The light source is used to emit a light beam; The at least one reflector is arranged on the optical path of the light beam emitted by the light source, and is used to reflect the light beam to the position of the eyes or face; The at least one micro-electromechanical controller is used to control the reflection direction of the at least one reflector; The signal sensor is used to collect signals, and the signals include eye features; The processor is connected to the signal sensor, and is used to receive the signal and determine the eye position according to the signal; or, determine the gaze point according to the signal; the processor is also used to adjust the power of the light source.

2. An eye tracking method, characterized in that: The method is performed by the eye tracking system according to claim 1, and the method comprises: Acquire a first signal, the first signal including an eye feature; wherein the first signal is acquired when a light source scans and illuminates a set area at a first power, and the light source includes a plurality of sub-light sources; determining an eye position based on the first signal; adjusting the reflection direction of the reflector according to the eye position to reflect the light beam from the light source to the eye position; Acquire a second signal, the second signal including an eye feature, and adjust the power of the sub-light source irradiated at the eye position from the first power to a second power; wherein the second power is greater than the first power; Eye tracking is performed according to the second signal.

3. The method according to claim 2, characterized in that Obtaining a first signal, including: controlling the light source to emit a light beam at a first power; Determining a motion range of a reflector, and controlling the reflector to move within the motion range so that the reflected light beam scans and illuminates a set area; wherein the motion range includes a movement range and / or a rotation range; A first signal of the light beam scanning irradiation is collected.

4. The method according to claim 3, characterized in that Determine the range of motion of the reflector, including: Obtaining the position information of the light source, the position information of the reflector, the position information of the set area, and the size of the set area; The motion range of the reflector is determined according to the posture information of the light source, the posture information of the reflector, the posture information of the set area, and the size of the set area.

5. The method according to claim 3, characterized in that The signal includes an image, and controlling the light source to emit a light beam at a first power includes: Controlling the light source to emit a light beam at a third power, and controlling the reflector to move within the range of motion so that the reflected light beam scans and illuminates a set area; wherein the third power is less than the first power; Acquire a first image irradiated by a scanning light beam, and extract a grayscale value of the first image; Determine whether the grayscale value is less than a set threshold; If the grayscale value is less than the set threshold, the power of the light source is adjusted in an iterative approximation manner until the grayscale value of the first image corresponding to the adjusted power is greater than or equal to the set threshold. The final adjusted power is determined as the first power, and the light source is controlled to emit a light beam at the first power.

6. The method according to claim 5, characterized in that Adjusting the power of the light source in an iterative approximation manner, comprising: Adjusting the power of the light source to a set value, and controlling the reflector to move within the range of motion, so that the reflected light beam scans and illuminates the set area; Acquire a first image irradiated by a scanning light beam, and extract a grayscale value of the first image; Determine whether the grayscale value is less than a set threshold; If the grayscale value is less than the set threshold, returning to the operation of increasing the power of the light source by a set value until the grayscale value is greater than or equal to the set threshold; If the grayscale value is greater than or equal to the set threshold, the increased power is determined as the first power.

7. The method according to claim 2, characterized in that Adjusting the power of the sub-light source irradiating the eye position from the first power to the second power includes: Get the distance between the eye and the light source; determining a second power based on the distance; The power of the sub-light source irradiating the eye position is adjusted from the first power to a second power.

8. An eye tracking device, characterized in that: The device is provided in the eye tracking system according to claim 1, and the device comprises: a signal acquisition module, configured to acquire a first signal, the first signal including eye features; wherein the first signal is acquired when a light source scans and illuminates a set area at a first power, and the light source includes a plurality of sub-light sources; an eye position determining module, configured to determine an eye position according to the first signal; a reflection direction adjustment module, configured to adjust the reflection direction of the reflector according to the eye position, so as to reflect the light beam from the light source to the eye position; a power adjustment module, configured to adjust the power of the sub-light source irradiating the eye position from the first power to a second power; wherein the second power is greater than the first power; The signal acquisition module is further configured to acquire a second signal, wherein the second signal includes eye features; An eye tracking module is configured to perform eye tracking according to the second signal.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the eye tracking method according to any one of claims 2 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the eye tracking method according to any one of claims 2 to 7 when executed.

Images