Eyeball tracking apparatus and eyeball tracking method

Abstract

The present application belongs to the technical field of electronic devices. Disclosed are an eyeball tracking apparatus and an eyeball tracking method. The eyeball tracking apparatus comprises: a housing; a lens, which is arranged on the housing; a first light source, which is used for emitting first infrared light towards the eyes of a user; a second light source, which is used for emitting second infrared light towards the eyes of the user, wherein the wavelength of the first infrared light is different from the wavelength of the second infrared light; a first infrared camera, which receives the first infrared light reflected by the eyes of the user and filters out the second infrared light to form a first eyeball image; and a second infrared camera, which receives the second infrared light reflected by the eyes of the user and filters out the first infrared light to form a second eyeball image.

Description

Eye tracking device and eye tracking method

[0001] Cross-references

[0002] This application claims priority to a Chinese patent application filed with the Patent Office of China on February 8, 2024, with application number 202410177228.1 and application name “Eye Tracking Device and Eye Tracking Method”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application belongs to the technical field of electronic equipment, and specifically relates to an eye tracking device and an eye tracking method. Background Art

[0004] With the continuous development of technology and the increasing demand for convenience and safety, various mixed reality (MR) and virtual reality (AR) devices now feature both eye tracking and iris recognition. Eye tracking primarily uses an infrared camera to capture infrared reflections (i.e., the reflections of infrared light on the eye) contained in the eye image. This information is then used to locate the pupil and, using a specific algorithm, determine the user's gaze point.

[0005] During eye tracking, infrared light reflection points can easily appear at the edge of the pupil, the iris area, or both. This makes it difficult to identify eye features, reducing the effectiveness of eye tracking. Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide an eye tracking device, an eye tracking method and an apparatus, which can at least solve the problem that the position of the infrared light reflection point affects the recognition of eye features during the eye tracking process.

[0007] In a first aspect, an embodiment of the present application provides an eye tracking device, comprising:

[0008] case;

[0009] a lens, disposed on the housing;

[0010] a first light source, configured to emit a first infrared light toward the user's eyes;

[0011] a second light source, configured to emit a second infrared light toward the user's eyes, wherein the wavelength of the first infrared light is different from the wavelength of the second infrared light;

[0012] a first infrared camera, receiving the first infrared light reflected by the user's eye and filtering out the second infrared light to form a first eye image;

[0013] The second infrared camera receives the second infrared light reflected by the user's eyes and filters out the first infrared light to form a second eye image.

[0014] In a second aspect, an embodiment of the present application provides an eye tracking method, which is applied to the eye tracking device described in the first aspect, comprising:

[0015] Acquire a first eye image captured by the first infrared camera and a second eye image captured by the second infrared camera;

[0016] Determining position information of a highlight reflective point in the first eye image and positioning information of a pupil in the second eye image;

[0017] The user's gaze point and / or sight direction is determined based on the position information of the highlighted reflective point, the position information of the first light source, and the positioning information of the pupil.

[0018] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the second aspect are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the second aspect are implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the second aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the second aspect.

[0022] In an embodiment of the present application, an eye tracking device includes: a housing; a lens disposed in the housing; a first light source for emitting a first infrared light toward a user's eye; a second light source for emitting a second infrared light toward the user's eye, wherein the wavelength of the first infrared light is different from the wavelength of the second infrared light; a first infrared camera for receiving the first infrared light reflected by the user's eye and filtering out the second infrared light to form a first eye image; and a second infrared camera for receiving the second infrared light reflected by the user's eye and filtering out the first infrared light to form a second eye image. In an embodiment of the present application, since the first light source and the second light source emit infrared light of different wavelengths toward the user's eye, the first infrared camera and the second infrared camera can respectively receive infrared light of corresponding wavelengths reflected by the user's eye. This can reduce the influence of the position of the infrared light reflection point on the recognition of eye features in each eye image, thereby improving the effect of eye tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figures 1a-1c are schematic diagrams of possible locations of bright reflective spots formed by infrared light during eye tracking;

[0024] FIG2 is a schematic diagram of the structure of an eye tracking device provided in an embodiment of the present application;

[0025] FIG3 is a schematic diagram of the layout of the cameras provided in an embodiment of the present application;

[0026] FIG4 is a schematic structural diagram of the front side of the optical machine lens barrel provided in an embodiment of the present application;

[0027] FIG5 is a flow chart of an eye tracking method according to an embodiment of the present application;

[0028] FIG6 is a flow chart of a method for determining a gaze point position according to an embodiment of the present application;

[0029] FIG7 is a schematic diagram of a method for determining a gaze point position according to an embodiment of the present application;

[0030] FIG8 is a flow chart of the iris recognition method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0032] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0033] The electronic device provided in the embodiments of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0034] Eye tracking uses image processing equipment to locate the pupil, obtain pupil coordinates, and calculate the point of gaze or gaze through a specific algorithm. Iris recognition uses iris texture information to identify the person. Currently, various MR or AR devices have both eye tracking and iris recognition functions. However, during the eye tracking and iris recognition process, the following situations often occur:

[0035] Scenario 1: As shown in Figure 1a, a bright reflective spot 1 formed by infrared light appears at the edge 2 of the pupil, affecting the recognition of pupil edge information. This makes it impossible to accurately locate the position of the pupil center, directly affecting the effectiveness of eye tracking and iris recognition.

[0036] Scenario 2: As shown in Figure 1b, a bright reflective point 1 formed by infrared light appears in iris region 3. Since iris region 3 contains a large number of texture and color features, the bright reflective point 1 in iris region 3 affects the feature extraction and recognition of the iris, resulting in inaccurate iris recognition results.

[0037] Scenario 3: As shown in Figure 1c, the bright reflective spot 1 formed by infrared light appears both at the pupil edge 2 and in the iris area 3, affecting the effects of eye tracking and iris recognition.

[0038] To address the problems encountered by the aforementioned MR or AR devices in the eye tracking and iris recognition process, an embodiment of the present application provides an eye tracking device, as shown in FIG2 , which includes:

[0039] case;

[0040] A lens 210 is provided in the housing;

[0041] A first light source 220 is configured to emit a first infrared light toward the user's eyes 5;

[0042] A second light source 230 is configured to emit a second infrared light toward the user's eye 5 , wherein the wavelength of the first infrared light is different from the wavelength of the second infrared light;

[0043] The first infrared camera 240 receives the first infrared light reflected by the user's eye 5 and filters out the second infrared light to form a first eye image;

[0044] The second infrared camera 250 receives the second infrared light reflected by the user's eye 5 and filters out the first infrared light to form a second eyeball image.

[0045] In an embodiment of the present application, the eye tracking device includes a shell, a lens 210 is arranged in the shell, and a first light source 220 and a second light source 230 are arranged at the edge of the lens. Since the first light source 220 and the second light source 230 emit infrared light of different wavelengths to the user's eyes 5, the first infrared camera 240 and the second infrared camera 250 can respectively receive the infrared light of corresponding wavelengths reflected by the user's eyes 5. In this way, the influence of the position of the highlight reflection point 1 formed by the infrared light on the recognition of eye features in each eye image can be reduced. For example, the influence of the position of the highlight reflection point 1 formed by the second infrared light on the pupil edge 2 in the first eye image can be reduced, or the influence of the position of the highlight reflection point 1 formed by the first infrared light on the iris area 3 in the second eye image can be reduced. Therefore, clearer eye features can be identified through the first eye image and the second eye image, thereby improving the effect of eye tracking.

[0046] In a possible implementation, the wavelength of the first infrared light is adjustable within a first wavelength band, and the wavelength of the second infrared light is adjustable within a second wavelength band; the first wavelength band does not overlap with the second wavelength band.

[0047] In the embodiment of the present application, the first band corresponding to the first infrared light emitted by the first light source 220 is recorded as band A, and the second band corresponding to the second infrared light emitted by the second light source 230 is recorded as band B. The wavelength of the first infrared light is adjustable within the range of band A, and the wavelength of the second infrared light is adjustable within the range of band B. The AB band range is generally 780nm to 2500nm, and the A band and the B band are within the range of the AB band and there is no overlap between the A band and the B band.

[0048] In one possible implementation, the first infrared camera 240 includes a first infrared filter, which is configured to transmit the first infrared light and filter out the second infrared light;

[0049] The second infrared camera 250 includes a second infrared filter, which is used to transmit the second infrared light and filter out the first infrared light.

[0050] In the embodiment of the present application, the first infrared camera 240 and the second infrared camera 250 operate in different wavelength bands. That is, the first infrared camera 240 receives the first infrared light reflected by the user's eye 5 and filters out the second infrared light to form a first eye image; the second infrared camera 250 receives the second infrared light reflected by the user's eye 5 and filters out the first infrared light to form a second eye image. A first infrared filter can be provided on the first infrared camera 240 to transmit the first infrared light and filter out the second infrared light; and a second infrared filter can be provided on the second infrared camera 250 to transmit the second infrared light and filter out the first infrared light. This prevents the first infrared camera 240 from receiving the second infrared light reflected by the user's eye 5, so that the formed first eye image does not include the highlighted reflective point 1 of the second infrared light. Similarly, the second infrared camera 250 can be prevented from receiving the first infrared light reflected by the user's eye 5, so that the formed second eye image does not include the highlighted reflective point 1 of the first infrared light.

[0051] In a possible implementation, multiple first light sources 220 may be included, and the multiple first light sources 220 are arranged around the optical axis 211 of the lens 210 ; the second light source 230 is arranged on a side of the first light source 220 away from the optical axis 211 of the lens 210 .

[0052] In actual applications, an MR or AR device typically includes six first light sources 220, which are arranged in a ring along the edge of the lens and facing the user's eye 5. The second light source 230 is arranged on the side of the first light source 220 away from the optical axis 211 of the lens 210 to prevent the reflection of the second infrared light from the user's eye 5 from being received by the second infrared camera 250 and forming a highlight reflection point 1 on the second eye image.

[0053] A plurality of second light sources 230 may be included, and the plurality of second light sources 230 are disposed on opposite sides of the lens 210 .

[0054] In actual applications, MR or AR devices usually set a second light source 230 on the upper and lower sides of the lens edge, so that the second infrared camera 250 can capture a clear second eye image without the highlight reflective point 1 based on the second infrared light.

[0055] In one possible implementation, the second light source 230 is disposed on the first side and / or the second side of the lens 210, and the second infrared camera 250 is disposed on the third side and / or the fourth side of the lens 210; the first side and the second side are two opposite sides of the lens 210, and the third side and the fourth side are the other opposite sides of the lens 210; the first infrared camera 240 is disposed on the first side or the second side of the lens 210.

[0056] In the embodiment of the present application, the second light source 230 is disposed on the upper and lower sides of the lens 210, and the second infrared camera 250 can be disposed on the left and right sides of the lens 210. Optionally, as shown in FIG3 , the second infrared camera 250 can be disposed so that its projection on the plane where the lens 210 is located is located on the target horizontal line l, for example, deployed at positions 6 and 7. The first infrared camera 240 is disposed on the upper or lower side of the lens 210. The user may blink, in which case the upper eyelid moves downward while the lower eyelid remains in its original position. Optionally, the projection of the first infrared camera 240 on the plane where the lens 210 is located is generally located below the target horizontal line l, for example, deployed at positions 8, 9, and 10. The target horizontal line l can be a horizontal line passing through the optical center of the lens 210.

[0057] In one possible implementation, the first infrared light causes the eye image in the first eye image to have multiple highlight reflective points 1; the second infrared light causes the pupil image and the iris image in the second eye image to have no highlight reflective points 1.

[0058] In the embodiment of the present application, the first infrared camera 240 receives the first infrared light reflected by the user's eye 5 and filters out the second infrared light to form a first eye image. A plurality of first light sources 220 are arranged around the optical axis 211 of the lens 210. The light reflected by the first infrared light from the user's eye 5 enters the first infrared camera 240, forming a light spot (i.e., a highlighted reflective point) on the first eye image captured by the first infrared camera 240. The second infrared camera 250 receives the second infrared light reflected by the user's eye and filters out the first infrared light to form a second eye image. The second light source 230 is arranged on a side of the first light source 220 away from the optical axis 211 of the lens 210. The light reflected by the second infrared light from the user's eye 5 rarely enters the second infrared camera 250, and the probability of forming a light spot on the second eye image captured by the second infrared camera 250 is very low. That is, the pupil image and iris image in the second eye image do not have a highlighted reflective point 1.

[0059] In the process of acquiring the first eyeball image and the second eyeball image, the first light source 220 and the second light source 230 are synchronously turned on and off;

[0060] The first infrared camera 240 and the second infrared camera 250 are exposed synchronously, the output frame rate of the first infrared camera 240 is the same as the output frame rate of the second infrared camera 250 , and the exposure time of the first infrared camera 240 is the same as the exposure time of the second infrared camera 250 .

[0061] In this way, the first eye image and the second eye image can be acquired simultaneously, ensuring the timeliness of eye tracking.

[0062] In a possible implementation, the emission direction of the first infrared light forms a first preset angle r1 with the optical axis 211 ; the emission direction of the second infrared light forms a second preset angle r2 with the optical axis 211 , and the second preset angle r2 is smaller than the first preset angle r1 .

[0063] In the embodiment of the present application, the first preset angle r1 can be set according to actual needs. When the user is wearing the device, the point where the first infrared light intersects the optical axis 211 can be the center of the eyeball, so that the first eyeball image formed by the first infrared light by the first infrared camera 240 includes a clear highlight reflection point 1. The second preset angle r2 is smaller than the first preset angle r1 to reduce the probability of the second infrared light forming a light spot on the second eyeball image captured by the second infrared camera 250, so that the pupil image and iris image in the second eyeball image do not include the highlight reflection point 1. In practical applications, as shown in FIG4 , the eye tracking device includes a lens 210 embedded in an optical machine housing 260 . A light ring frame 250 is provided at the edge of the lens 210 . A first light source 220 and a second light source 230 are provided on the light ring frame 250 . The second light source 230 is positioned away from the edge of the lens 210 relative to the first light source 220 . The first light source 220 is perpendicularly oriented toward the center of the eye, while the second light source 230 is perpendicularly oriented toward the eyelid. The second light source 230 is positioned at a greater horizontal inclination angle than the first light source 220 . This allows the primary ray of the second infrared light to pass through the user's eye 5 and not be reflected by the second infrared camera 250 . Only a small amount of light from the edge of the eye is reflected by the second infrared camera 250 . This significantly reduces the probability of light spots forming on the second eye image captured by the second infrared camera 250 . In this manner, eye tracking can be performed using the first and second eye images, enabling both the acquisition of reflective points on the eye image and a clear eye image. This reduces the impact of the infrared reflective point position on eye feature recognition and improves the effectiveness of eye tracking.

[0064] In a possible implementation, eye tracking may be performed based on the first eye image and the second eye image, including:

[0065] Determine the position information of the highlight reflective point 1 in the first eye image and the positioning information of the pupil 4 in the second eye image; determine the user's gaze point and / or line of sight direction based on the position information of the highlight reflective point 1, the position information of the first light source 220, and the positioning information of the pupil 4.

[0066] In a possible implementation, the method further includes:

[0067] Determine an iris feature image in the second eye image; extract a target iris feature from the iris feature image, and match the target iris feature with a pre-stored iris feature to obtain an iris recognition result of the user.

[0068] That is, the first eye image and the second eye image taken using the above-mentioned eye tracking device can realize eye tracking and iris recognition, reduce the influence of the position of the highlighted reflective point 1 of infrared light on the recognition of eye features in each eye image, and improve the effect of eye tracking and iris recognition.

[0069] Please refer to Figure 5, which is a flowchart of an eye tracking method provided by an embodiment of the present application. The eye tracking method 500 can be applied to the above-mentioned eye tracking device, and specifically may include the following steps:

[0070] S501: Acquire a first eyeball image taken by the first infrared camera 240 and a second eyeball image taken by the second infrared camera 250 .

[0071] In an embodiment of the present application, during the eye tracking process, a first eye image taken by the first infrared camera 240 and a second eye image taken by the second infrared camera 250 are obtained, wherein the first infrared light causes the eye image in the first eye image to have multiple highlight reflection points; and the second infrared light causes the pupil image and the iris image in the second eye image to have no highlight reflection points.

[0072] S502: Determine the position information of the highlight reflection point in the first eye image and the positioning information of the pupil in the second eye image.

[0073] In an embodiment of the present application, the first eye image can be preprocessed, where the preprocessing includes stretching, compression, grayscale conversion, and binarization. Specifically, an adaptive threshold can be set to binarize the first eye image to obtain a binary image; then, the position information of the highlight reflective points in the binary image can be extracted. Simultaneously, an edge detection algorithm (such as the Canny edge detection method) can be used to obtain the characteristic edges of the pupil region in the second eye image, and the Hough circle detection method can be used to detect the pupil circle or ellipse boundary to obtain the pupil location information in the second eye image.

[0074] S503: Determine the user's gaze point and / or sight direction based on the position information of the highlight reflective point, the position information of the first light source 220, and the positioning information of the pupil.

[0075] In an embodiment of the present application, as shown in FIG6 , the second eye image is preprocessed and pupil detection is performed to obtain pupil positioning information; and the first eye image is subjected to reflection point detection to obtain position information of highlighted reflection points; and then the user's gaze point and / or line of sight direction are determined based on the spatial geometric relationship.

[0076] In one possible implementation, in step S503, determining the user's gaze point and / or sight direction based on the position information of the highlighted reflective point, the position information of the first light source 220, and the pupil positioning information includes:

[0077] The coordinates of the center point of the eyeball are determined based on the position information of the highlighted reflective point and the position information of the first light source 220 corresponding to the highlighted reflective point; the optical axis of the user's eyeball is determined based on the coordinates of the center point of the eyeball and the positioning information of the pupil; the line of sight axis is determined based on the optical axis of the user's eyeball and preset line of sight axis conversion information; wherein the line of sight axis conversion information is the conversion information between the pre-calibrated pupil optical axis and the line of sight axis; the user's gaze point and / or line of sight direction are determined based on the line of sight axis.

[0078] Here, the pupil positioning information may be the position coordinates of the pupil center point.

[0079] In an embodiment of the present application, as shown in FIG7 , the coordinates of the center of the eyeball can be determined based on the position coordinates of the highlighted reflective point in the first eyeball image and the pre-calibrated internal and external parameters of the first infrared camera, using the pinhole imaging principle and the position information of the first light source 220 corresponding to the reflective point. The spatial position coordinates of the center of the pupil in the image are determined by fitting the pupil edge information in the second eyeball image, and the pre-calibrated internal and external parameters of the second infrared camera, using the pinhole imaging principle and eyeball refraction (light incident from air to the center of the pupil of the eyeball); the optical axis of the user's eyeball is determined based on the coordinates of the center of the eyeball and the coordinates of the center of the pupil. The line of sight axis is determined based on the optical axis of the user's eyeball and the preset line of sight axis conversion information; and the user's gaze point and / or line of sight direction are determined based on the line of sight axis.

[0080] In this way, performing eye tracking based on the first eye image and the second eye image can reduce the influence of the position of the infrared light reflection point on the recognition of eye features and improve the effect of eye tracking.

[0081] In a possible implementation, the eye tracking method 500 further includes:

[0082] S504: Determine an iris feature image in the second eyeball image.

[0083] In an embodiment of the present application, the iris area has rich biometric features, which can be used for identity authentication by identifying these biometric features. Therefore, the iris feature image in the second eye image can be obtained to perform subsequent feature extraction processing based on the iris feature image to verify the identity of the user.

[0084] In a possible implementation, in step S504, determining the iris feature image in the second eye image includes:

[0085] Acquire a feature extraction region between a first edge curve of a pupil region and a second edge curve of an iris region in the second eyeball image; and perform polar coordinate transformation on the feature extraction region to obtain the iris feature image.

[0086] In the embodiment of the present application, polar coordinate transformation is to transform the rectangular coordinate system of the image into the polar coordinate system, and a circular image can be transformed into a rectangular image. After the polar coordinate transformation of the image, some specific image processing operations can be implemented, such as filtering, feature recognition, etc.

[0087] S505: Extract target iris features from the iris feature image, match the target iris features with pre-stored iris features, and obtain an iris recognition result of the user.

[0088] In the embodiment of the present application, target iris features (such as Gabor wavelet features, etc.) are extracted from the iris feature image, and the target iris features are matched with pre-stored iris features to obtain the user's iris recognition results.

[0089] In a possible implementation, in step S505, matching the target iris features with pre-stored iris features to obtain the user's iris recognition result includes:

[0090] quantizing and encoding the target iris feature to obtain a target iris feature code;

[0091] The iris recognition result of the user is determined according to the Hamming distance between the target iris feature code and the iris feature code corresponding to the pre-stored iris feature.

[0092] In an embodiment of the present application, as shown in FIG8 , the second eye image is first preprocessed and positioned and segmented to obtain an iris feature image; the iris feature image is normalized to extract the target iris feature, and the target iris feature is quantized and encoded to obtain a target iris feature code; and then the user's iris recognition result is determined by feature matching. Specifically, the user's iris recognition result can be determined based on the Hamming distance between the target iris feature code and the iris feature code corresponding to the pre-stored iris feature.

[0093] An embodiment of the present application provides an eye tracking method, which obtains a first eye image captured by a first infrared camera and a second eye image captured by a second infrared camera; determines the position information of a highlight reflective point in the first eye image and the positioning information of the pupil in the second eye image; and determines the user's gaze point and / or line of sight direction based on the position information of the highlight reflective point, the position information of the first light source, and the positioning information of the pupil. Because the pupil area in the second eye image has less light interference, the positioning of the pupil area is more accurate. Therefore, based on the position information of the highlight reflective point in the first eye image and the positioning information of the pupil in the second eye image, the user's gaze point and / or line of sight direction can be accurately identified, reducing the impact of infrared light reflection on the eye tracking results, thereby improving the accuracy and robustness of eye tracking.

[0094] Furthermore, the system determines the iris feature image in the second eye image, extracts the target iris features from the iris feature image, and matches the target iris features with pre-stored iris features to obtain the user's iris recognition results. Because the pupil and iris regions in the second eye image have less light interference, their positioning is more accurate, and more effective texture information from the iris region can be extracted, thereby improving the accuracy and robustness of iris recognition.

[0095] An embodiment of the present application also provides an eye tracking device, including the above-mentioned eye tracking device.

[0096] In the embodiments of the present application, the eye tracking device can be wearable, such as AR glasses, or non-wearable, such as a mobile phone or tablet with a lens.

[0097] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0098] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0099] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0100] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0101] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0103] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An eye tracking device, comprising: case; A lens (210) is provided on the housing; a first light source (220), configured to emit a first infrared light toward the user's eyes; a second light source (230) for emitting a second infrared light toward the user's eyes, wherein the wavelength of the first infrared light is different from the wavelength of the second infrared light; a first infrared camera (240) receiving the first infrared light reflected by the user's eyes and filtering out the second infrared light to form a first eyeball image; The second infrared camera (250) receives the second infrared light reflected by the user's eyes and filters out the first infrared light to form a second eyeball image.

2. The eye tracking device according to claim 1, wherein: The wavelength of the first infrared light is adjustable within a first wavelength band, and the wavelength of the second infrared light is adjustable within a second wavelength band; the first wavelength band and the second wavelength band do not overlap.

3. The eye tracking device according to claim 1, wherein: The first infrared camera (240) comprises a first infrared filter, the first infrared filter being used to transmit the first infrared light and filter out the second infrared light; The second infrared camera (250) comprises a second infrared filter, and the second infrared filter is used for transmitting the second infrared light and filtering out the first infrared light.

4. The eye tracking device according to claim 1, wherein: comprising a plurality of the first light sources (220), wherein the plurality of the first light sources (220) are arranged around the optical axis (211) of the lens (210); The second light source (230) is arranged on a side of the first light source (220) away from the optical axis (211) of the lens (210).

5. The eye tracking device according to claim 4, wherein: It comprises a plurality of second light sources (230), and the plurality of second light sources (230) are arranged on two opposite sides of the lens (210).

6. The eye tracking device according to claim 4, wherein: The second light source (230) is arranged on a first side and / or a second side of the lens (210), and the second infrared camera (250) is arranged on a third side and / or a fourth side of the lens (210); the first side and the second side are two opposite sides of the lens (210), and the third side and the fourth side are the other two opposite sides of the lens (210); The first infrared camera (240) is arranged on the first side or the second side of the lens (210).

7. The eye tracking device according to claim 1, wherein: The first infrared light causes the eye image in the first eye image to have multiple highlight reflection points; the second infrared light causes the pupil image and the iris image in the second eye image to have no highlight reflection points.

8. The eye tracking device according to claim 1, wherein: During the process of acquiring the first eyeball image and the second eyeball image, the first light source (220) and the second light source (230) are synchronously turned on, and the first light source (220) and the second light source (230) are synchronously turned off; The first infrared camera (240) and the second infrared camera (250) are exposed synchronously, the output frame rate of the first infrared camera (240) is the same as the output frame rate of the second infrared camera (250), and the exposure time of the first infrared camera (240) is the same as the exposure time of the second infrared camera (250).

9. The eye tracking device according to claim 1, wherein: The emission direction of the first infrared light forms a first preset angle (r1) with the optical axis (211); The emission direction of the second infrared light forms a second preset angle (r2) with the optical axis (211), and the second preset angle (r2) is smaller than the first preset angle (r1).

10. The eye tracking device according to claim 7, wherein: Performing eye tracking according to the first eye image and the second eye image includes: Determining position information of the highlight reflective point in the first eye image and positioning information of the pupil in the second eye image; The user's gaze point and / or sight direction is determined based on the position information of the highlighted reflective point, the position information of the first light source (220), and the positioning information of the pupil.

11. The eye tracking device according to claim 10, wherein: Determining the user's gaze point and / or sight direction based on the position information of the highlight reflective point, the position information of the first light source (220), and the pupil positioning information includes: Determining the coordinates of the center point of the eyeball according to the position information of the highlighted reflective point and the position information of the first light source (220) corresponding to the highlighted reflective point; Determining the optical axis of the user's eyeball based on the eyeball center coordinates and the pupil positioning information; Determining the sight axis according to the optical axis of the user's eyeball and preset sight axis conversion information; wherein the sight axis conversion information is pre-calibrated conversion information between the pupil optical axis and the sight axis; The gaze point and / or gaze direction of the user is determined according to the gaze axis.

12. The eye tracking device according to claim 7, wherein: Also includes: determining an iris feature image in the second eyeball image; The target iris feature is extracted from the iris feature image, and the target iris feature is matched with the pre-stored iris feature to obtain the user's iris recognition result.

13. The eye tracking device according to claim 12, wherein: The determining of the iris feature image in the second eyeball image includes: Acquire a feature extraction region between a first edge curve of a pupil region and a second edge curve of an iris region in the second eye image; Perform polar coordinate transformation on the feature extraction area to obtain the iris feature image.

14. The eye tracking device according to claim 12, wherein: The step of matching the target iris features with pre-stored iris features to obtain the user's iris recognition result includes: quantizing and encoding the target iris feature to obtain a target iris feature code; The iris recognition result of the user is determined according to the Hamming distance between the target iris feature code and the iris feature code corresponding to the pre-stored iris feature.

15. An eye tracking method, applied to the eye tracking device according to any one of claims 1 to 14, comprising: Acquiring a first eyeball image captured by the first infrared camera (240) and a second eyeball image captured by the second infrared camera (250); Determining position information of a highlight reflective point in the first eye image and positioning information of a pupil in the second eye image; The user's gaze point and / or sight direction is determined based on the position information of the highlighted reflective point, the position information of the first light source (220), and the positioning information of the pupil.

16. The method according to claim 15, wherein Determining the user's gaze point and / or sight direction based on the position information of the highlight reflective point, the position information of the first light source (220), and the pupil positioning information includes: Determining the coordinates of the center point of the eyeball according to the position information of the highlighted reflective point and the position information of the first light source (220) corresponding to the highlighted reflective point; Determining the pupil optical axis according to the eyeball center coordinates and the pupil positioning information; Determining the sight axis according to the pupil optical axis and preset sight axis conversion information; wherein the sight axis conversion information is pre-calibrated conversion information between the pupil optical axis and the sight axis; The gaze point and / or gaze direction of the user is determined according to the gaze axis.

17. The method according to claim 15, wherein: After obtaining the first eyeball image captured by the first infrared camera (240) and the second eyeball image captured by the second infrared camera (250), the method further includes: determining an iris feature image in the second eyeball image; The target iris feature is extracted from the iris feature image, and the target iris feature is matched with the pre-stored iris feature to obtain the user's iris recognition result.

18. The method according to claim 17, wherein: The determining of the iris feature image in the second eyeball image includes: Acquire a feature extraction region between a first edge curve of a pupil region and a second edge curve of an iris region in the second eye image; Perform polar coordinate transformation on the feature extraction area to obtain the iris feature image.

19. The method according to claim 17, wherein The step of matching the target iris features with pre-stored iris features to obtain the user's iris recognition result includes: quantizing and encoding the target iris feature to obtain a target iris feature code; The iris recognition result of the user is determined according to the Hamming distance between the target iris feature code and the iris feature code corresponding to the pre-stored iris feature.

Images