Eye movement tracking device and method

By calculating and combining the line of sight directions at multiple time points in the eye tracking device, the problem of insufficient line of sight direction calculation speed in the prior art is solved, the display update rate and user experience are improved, and the processor load is controlled by adjusting the calculation cycle. , adapt to different application needs.

CN119002687APending Publication Date: 2024-11-22HTC CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410592393.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-24
Filing Date
2024-05-14
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing eye tracking technology is difficult to improve the computing speed of the line of sight under hardware limitations, resulting in insufficient update rate of the display screen and affecting the user experience.

Method used

By using a camera in an eye tracking device to collect multiple eye images, and using a processor to calculate a single eye image at each time point to determine the line of sight direction, merge the line of sight direction at different time points to produce a combined gaze direction, adjust the line of sight direction Track frequency to control processor load and improve update rate.

Benefits of technology

It achieves a higher update rate and better user experience, and at the same time, by dynamically adjusting the calculation cycle of the line of sight direction, the processor load is controlled to meet different application needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002687A_ABST
    Figure CN119002687A_ABST
Patent Text Reader

Abstract

An eye tracking device is used for executing the following operations: based on a plurality of eye images of a user, obtaining a first single-eye image of a first eye at a first time point and a first single-eye image of a second eye at a second time point; calculating a first line-of-sight direction at the first time point based on the first single-eye image of the first eye, and calculating a second line-of-sight direction at the second time point based on the first single-eye image of the second eye; and generating a combined gazing direction at the second time point based on the first line-of-sight direction and the second line-of-sight direction. According to the eye movement tracking technology provided by the invention, after the sight line direction of the single-side eyes is calculated, the combined staring direction is generated, so that the updating speed is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an eye tracking device and method, and more particularly to an eye tracking device and method capable of improving an update rate. Background Art

[0002] In the application of virtual reality, mixed reality and other technologies, eye tracking technology is often needed to confirm the position of the user's eyes, which is then reflected on the display screen (for example, moving a cursor).

[0003] However, existing eye tracking technology needs to calculate the gaze direction of the user based on the image of the user's eyes. Therefore, if the calculation speed of the gaze direction cannot be improved, the refresh rate of the display screen (e.g., frame rate) is difficult to increase due to hardware limitations, resulting in a poor user experience.

[0004] In view of this, how to improve the computing efficiency of the user's gaze position is a goal that the industry urgently needs to work hard on. Summary of the invention

[0005] In order to solve the above problems, the present disclosure proposes an eye tracking device, comprising a camera and a processor. The camera is used to capture multiple eye images of a user in a continuous period of time. The processor is coupled to the camera, and is used to: obtain a first monocular image of a first eye at a first time point and a first monocular image of a second eye at a second time point based on the multiple eye images; calculate a first sight direction at the first time point based on the first monocular image of the first eye, and calculate a second sight direction at the second time point based on the first monocular image of the second eye; and generate a combined gaze direction at the second time point based on the first sight direction and the second sight direction.

[0006] In one embodiment of the present invention, the second time point is later than the first time point.

[0007] In one embodiment of the present invention, the processor is further used to: obtain a second monocular image of a first eye at a third time point based on the multiple eye images, wherein the third time point is later than the first time point and the second time point; calculate a third gaze direction at the third time point based on the second monocular image of the first eye; and generate the merged gaze direction at the third time point based on the second gaze direction and the third gaze direction.

[0008] In one embodiment of the present invention, the processor is further used to: obtain a second monocular image of a second eye at a fourth time point based on the multiple eye images, wherein the fourth time point is later than the third time point; calculate a fourth line of sight direction at the fourth time point based on the second monocular image of the second eye; and generate the merged gaze direction at the fourth time point based on the third line of sight direction and the fourth line of sight direction.

[0009] In one embodiment of the present invention, the processor is further configured to: select one of a plurality of update cycles; and determine a time interval between the second time point and the third time point based on the one of the plurality of update cycles.

[0010] In one embodiment of the present invention, the processor is further configured to: before the third time point, generate the combined gaze direction again based on the first sight line direction and the second sight line direction.

[0011] In one embodiment of the present invention, the operation of calculating the first sight line direction further includes: calculating a position of the first eye in a first monocular image of the first eye to calculate the first sight line direction.

[0012] In one embodiment of the present invention, the processor is further used to: based on a flag value, select a first operation or a second operation to generate the merged gaze direction; wherein the first operation includes calculating the gaze direction of the first eye and generating the merged gaze direction based on the gaze direction of the first eye; and the second operation includes calculating the gaze direction of the second eye and generating the merged gaze direction based on the gaze direction of the second eye.

[0013] In one embodiment of the present invention, the flag value corresponds to a first flag value and a second flag value, the first flag value corresponds to the first operation, and the second flag value corresponds to the second operation.

[0014] In one embodiment of the present invention, the processor is further configured to: in response to completing the first operation, set the flag value to the second flag value; and in response to completing the second operation, set the flag value to the first flag value.

[0015] The present disclosure also provides an eye tracking method applicable to an electronic device, the steps of which include: based on multiple eye images of a user, obtaining a first monocular image of a first eye at a first time point and a first monocular image of a second eye at a second time point; based on the first monocular image of the first eye, calculating a first sight direction at the first time point, and based on the first monocular image of the second eye, calculating a second sight direction at the second time point; and based on the first sight direction and the second sight direction, generating a combined gaze direction at the second time point.

[0016] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are intended to provide further explanation of the disclosure as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to make the above and other purposes, features, advantages and embodiments of the present disclosure more clearly understood, the accompanying drawings are described as follows:

[0018] Figure 1 is a schematic diagram of an eye tracking device in a first embodiment of the present disclosure;

[0019] Figure 2 A schematic diagram of generating a merged gaze direction operation in some embodiments of the present disclosure;

[0020] Figure 3 A schematic diagram of another operation of generating a merged gaze direction in some embodiments of the present disclosure; and

[0021] Figure 4 Flow chart of the eye tracking method in the second embodiment of the present disclosure.

[0022] Explanation of symbols:

[0023] 1: Eye tracking device

[0024] 12: Processor

[0025] 14: Camera

[0026] t10~t14,t20~t24: time interval

[0027] R10, R11, R13, R20, R21, R24: Right eye sight direction

[0028] L10, L12, L14, L20, L23: Left eye sight direction

[0029] E10~E14, E20~E24: Merge gaze direction

[0030] 200:Eye Tracking Methods

[0031] S201~S203: Steps DETAILED DESCRIPTION

[0032] In order to make the description of the present disclosure more detailed and complete, reference may be made to the attached drawings and various embodiments described below, in which the same numbers in the drawings represent the same or similar elements.

[0033] Please refer to Figure 1 , which is a schematic diagram of an eye tracking device 1 in the first embodiment of the present disclosure. The eye tracking device 1 includes a processor 12 and a camera 14, wherein the processor 12 is coupled to the camera 14. The eye tracking device 1 determines the position of the user's eye gaze based on the image of the user's eye, wherein the eye tracking device 1 can improve the efficiency of the operation by generating a merged gaze direction with the image of a single eye. In some embodiments, the eye tracking device 1 can be integrated into a head-mounted display, so that the eye tracking device 1 can improve the picture update rate of the head-mounted display.

[0034] In some embodiments, the processor 12 includes a central processing unit (CPU), a graphics processing unit (GPU), a multi-processor, a distributed processing system, an application specific integrated circuit (ASIC), and / or a suitable computing unit.

[0035] In some embodiments, the camera 14 may include one or more image capturing elements for capturing the user's eyes, and is used to capture multiple eye images of the user in a continuous period of time. In some embodiments, the camera 14 includes one image capturing element for capturing both eyes of the user. In other embodiments, the camera 14 includes multiple image capturing elements for capturing both eyes of the user, for example, each of the user's eyes is captured by one image capturing element, or each of the user's eyes is captured by multiple image capturing elements.

[0036] First, after obtaining the eye images of the user, the processor 12 processes a first monocular image of a first eye of the user at a first time point based on the plurality of eye images.

[0037] For example, if the camera 14 is composed of a plurality of image capturing elements to respectively photograph the two eyes of the user, the processor 12 directly selects a single eye image of one eye (eg, the left eye or the right eye) of the user from the eye image.

[0038] In some embodiments, in order to speed up the efficiency of image processing, after the camera 14 captures multiple images from different perspectives through multiple image capture elements, the firmware stitches the multiple images into a larger eye image. Specifically, each of the multiple eye images is generated by combining a left eye image and a right eye image. For example, the eye tracking device 1 obtains the left eye image through the image capture element that captures the left eye and obtains the right eye image through the image capture element that captures the right eye, and then generates an eye image by combining the images on both sides through firmware design.

[0039] Correspondingly, in order to save computing time, the processor 12 may crop the eye image into a smaller single-eye image, and use the image recognition model to determine the position of one eye of the user in the eye image.

[0040] Specifically, the processor 12 uses an image recognition model to identify a position of the first eye in one of the multiple eye images; and the processor 12 captures the first monocular image from one of the multiple eye images based on the position of the first eye.

[0041] For example, the processor 12 may utilize an image recognition model based on You Only Look Once (YOLO) or other image recognition algorithms to identify the position of the eyes in the eye image, and crop the eye image accordingly to capture a monocular image.

[0042] After obtaining the monocular image, the eye tracking device 1 calculates the sight direction of the eye of the user. Specifically, the processor 12 calculates a first sight direction based on the first monocular image of the first eye.

[0043] In some embodiments, the operation of calculating the first sight line direction further includes the processor 12 calculating a position of the first eye in a first monocular image of the first eye to calculate the first sight line direction.

[0044] For example, the processor 12 may calculate the relative position of the pupil of the eye in the eye socket based on the monocular image, and determine the sight direction of the eye accordingly.

[0045] Similarly, after calculating the sight direction of a single eye, the eye tracking device 1 processes a first monocular image of the user's other eye at a second time point based on the multiple eye images, and calculates the sight direction of the other eye accordingly.

[0046] Specifically, the processor 12 obtains a first monocular image of a second eye at a second time point based on the plurality of eye images, and calculates a second sight line direction at the second time point based on the first monocular image of the second eye.

[0047] It should be noted that the calculation of the second eye's sight direction is based on a monocular image corresponding to another time point (i.e., the second time point), and the time point is different from the first time point. In other words, the eye tracking device 1 uses the monocular images at different time points (i.e., the first time point and the second time point) to calculate the sight directions of the two eyes respectively. In addition, the eye tracking device 1 also calculates the sight directions of the first eye and the second eye at different time points (i.e., the first time point and the second time point).

[0048] For example, after calculating the sight line direction of the first eye, the eye tracking device 1 obtains the latest monocular image of the second eye and correspondingly calculates the sight line direction of the second eye to obtain the latest sight line direction of the second eye.

[0049] In some embodiments, the second time point is later than the first time point.

[0050] After calculating the gaze direction, the processor 12 generates the user's combined gaze direction based on the gaze directions of the two eyes. In other words, the eye tracking device 1 combines the gaze directions corresponding to different times to generate the gaze positions of the user's two eyes.

[0051] Specifically, the processor 12 generates a combined gaze direction at the second time point based on the first sight line direction and the second sight line direction.

[0052] Furthermore, in some embodiments, in subsequent operations, the eye tracking device 1 generates a combined gaze direction each time after tracking the sight direction of a single eye.

[0053] Specifically, the processor 12 obtains a second monocular image of a first eye at a third time point based on the multiple eye images, wherein the third time point is later than the first time point and the second time point; the processor 12 calculates a third gaze direction at the third time point based on the second monocular image of the first eye; and the processor 12 generates the merged gaze direction at the third time point based on the second gaze direction and the third gaze direction.

[0054] Similarly, in some embodiments, the eye tracking device 1 tracks the gaze directions of the two eyes in turn to ensure that the combined gaze direction is not distorted.

[0055] Specifically, the processor 12 obtains a second monocular image of a second eye at a fourth time point based on the multiple eye images, wherein the fourth time point is later than the third time point; the processor 12 calculates a fourth line of sight direction at the fourth time point based on the second monocular image of the second eye; and the processor 12 generates the merged gaze direction at the fourth time point based on the third line of sight direction and the fourth line of sight direction.

[0056] For convenience, please refer to Figure 2 , which is a schematic diagram of the operation of the eye tracking device 1 generating merged gaze directions E10 to E14 in some embodiments of the present disclosure.

[0057] As shown in the figure, first, in the time interval t10, the eye tracking device 1 calculates the combined gaze direction E10 based on the right eye gaze direction R10 and the left eye gaze direction L10.

[0058] Next, in the time interval t11, the eye tracking device 1 tracks the gaze direction of the user's right eye through the aforementioned operation, and calculates the combined gaze direction E11 based on the newly generated right eye gaze direction R11 and the existing left eye gaze direction L10.

[0059] Next, in the time interval t12, the eye tracking device 1 switches to tracking the gaze direction of the user's left eye, and calculates the combined gaze direction E12 based on the existing right eye gaze direction R11 and the newly generated left eye gaze direction L12.

[0060] Similarly, in time intervals t13 and t14, the eye tracking device 1 tracks the sight direction of one of the right eye and the left eye of the user respectively, and calculates merged gaze directions E13 and E14 by combining the newly generated sight direction with the existing sight direction of the other eye.

[0061] It should be noted that the right eye gaze direction R11 is calculated and generated in the time interval t11, the left eye gaze direction L12 is calculated and generated in the time interval t12, the right eye gaze direction R13 is calculated and generated in the time interval t13, and the left eye gaze direction L14 is calculated and generated in the time interval t14. In other words, the eye tracking device 1 generates corresponding merged gaze directions E11-E14 based on the eye images obtained in real time in each time interval t11-t14.

[0062] It can be seen that in the above example, the eye tracking device 1 only needs to track the sight direction of one eye each time to generate a merged gaze direction (i.e., generate a new merged gaze direction). In this way, compared to generating a merged gaze direction after tracking the sight directions of both eyes each time, the eye tracking device 1 of the present disclosure can shorten the update cycle (i.e., time interval t10-t14) to achieve a higher update rate.

[0063] In another example, the eye tracking device 1 of the present disclosure also tracks the sight lines in a different order to generate a combined gaze direction. Specifically, before the third time point, the processor 12 generates the combined gaze direction based on the first sight line direction and the second sight line direction again.

[0064] For convenience, please refer to Figure 3 , which is a schematic diagram of the operation of the eye tracking device 1 generating merged gaze directions E20 to E24 in some embodiments of the present disclosure.

[0065] and Figure 2 Similarly to the operation of , in time interval t20, similar to time interval t10, the eye tracking device 1 first calculates the merged gaze direction E20 based on the right eye gaze direction R20 and the left eye gaze direction L20. Further, in time interval t21, similar to time interval t11, the eye tracking device 1 tracks the gaze direction of the user's right eye, and calculates the merged gaze direction E21 based on the newly generated right eye gaze direction R21 and the existing left eye gaze direction L20.

[0066] However Figure 2 and Figure 3 The difference is that, compared Figure 2 In the time interval t12, Figure 3 In the time interval t22, the eye tracking device 1 calculates the merged gaze direction E22 based on the existing right eye sight direction R21 and the existing left eye sight direction L20. Next, in the time interval t23, the eye tracking device 1 calculates the merged gaze direction E23 based on the existing right eye sight direction R21 and the newly generated left eye sight direction L23. Further, in the time interval t24, the eye tracking device 1 calculates the merged gaze direction E24 based on the newly generated right eye sight direction R24 and the existing left eye sight direction L23.

[0067] Similarly, the right eye gaze direction R21 is calculated and generated in the time interval t21, the left eye gaze direction L23 is calculated and generated in the time interval t23, and the right eye gaze direction R24 is calculated and generated in the time interval t24. In other words, the eye tracking device 1 generates the combined gaze directions E21, E23, and E24 based on the eye images obtained in real time in each of the time intervals t21, t23, and t24.

[0068] It should be noted that, in some embodiments, although the merged gaze direction E22 is generated based on the right eye line of sight direction R21 and the left eye line of sight direction L20 like the merged gaze direction E21, in fact the merged gaze direction may also be calculated with reference to other parameters (for example: predicting future trajectories based on past movement trajectories). Therefore, the merged gaze directions E21 and E22 generated in different time intervals may be different.

[0069] It can be seen that in the above example, the eye tracking device 1 tracks the gaze direction of both eyes once every three time intervals, and tracks the gaze direction of one eye at most each time, so that a combined gaze direction can be generated (i.e., a new combined gaze direction is generated). Figure 2 An embodiment of Figure 3 The load of the embodiment is lower under the same update rate condition (ie, the same time interval length).

[0070] From the above two examples, it can be seen that the eye tracking device 1 controls the load of the processor 12 by adjusting the frequency of tracking the gaze direction (i.e., the number of times the gaze direction is calculated in the same time interval). In the same time interval, the higher the frequency of tracking the gaze direction, the higher the load of the processor 12; the lower the frequency of tracking the gaze direction, the lower the load of the processor 12.

[0071] Accordingly, the eye tracking device 1 adjusts the frequency of tracking the line of sight direction according to the current operating situation to control the load of the processor 12 .

[0072] Specifically, the processor 12 adjusts the frequency of tracking the gaze direction with reference to the application executed by the eye tracking device 1. For example, when the eye tracking device 1 executes an application with a high gaze tracking update rate requirement (e.g., action games), the processor 12 can increase the frequency to obtain a more immediate tracking effect; conversely, when the eye tracking device 1 executes an application with a low gaze tracking update rate requirement (e.g., playing a video), the processor 12 can reduce the frequency to save energy consumption.

[0073] On the other hand, the eye tracking device 1 adjusts the frequency according to the load of the processor 12. For example, when the load of the processor 12 is low, the frequency can be increased to obtain a more immediate tracking effect; conversely, when the load of the processor 12 is high, the frequency can be reduced to save computing resources.

[0074] In some embodiments, the eye tracking device 1 controls the update rate (eg, the frame rate of the display screen) by adjusting the period of tracking the line of sight, and can also control the load of the processor 12 .

[0075] Specifically, the processor 12 selects one of a plurality of update cycles; and the processor 12 determines a time interval between the second time point and the third time point based on the one of the plurality of update cycles.

[0076] by Figure 2 or Figure 3 For example, the eye tracking device 1 controls the update rate by adjusting the lengths of the time intervals t10-t14 and t20-t24 (ie, the update period).

[0077] It should be noted that, similar to the operation of adjusting the frequency of tracking the gaze direction in the aforementioned embodiment, the eye tracking device 1 can also adjust the update period based on factors such as the application it executes and the load of the processor 12, which will not be elaborated in this disclosure.

[0078] In some embodiments, in order to switch between the two eyes, the eye tracking device 1 sets a flag value to instruct the processor 12 which eye's gaze direction to track.

[0079] Specifically, the processor 12 selects to generate the merged gaze direction by a first operation or a second operation based on a flag value; wherein the first operation includes calculating a line of sight direction of the first eye and generating the merged gaze direction based on the line of sight direction of the first eye; and the second operation includes calculating the line of sight direction of the second eye and generating the merged gaze direction based on the line of sight direction of the second eye.

[0080] Furthermore, the eye tracking device 1 sets the flag value to two different values ​​(e.g., left or right, 0 or 1) to correspond to the two eyes respectively. Specifically, the flag value corresponds to a first flag value and a second flag value, the first flag value corresponds to the first operation, and the second flag value corresponds to the second operation.

[0081] Furthermore, in order to switch between the two eyes, after completing the eye tracking of one eye, the eye tracking device 1 sets the flag value to the flag value corresponding to the other eye, so as to instruct the processor 12 to perform eye tracking of the other eye the next time the eye tracking is performed.

[0082] Specifically, in response to completing the first operation, the processor 12 sets the flag value to the second flag value; and in response to completing the second operation, the processor 12 sets the flag value to the first flag value.

[0083] In this way, the eye tracking device 1 uses the flag value to switch between the two eyes, so as to perform eye tracking on the two eyes in turn.

[0084] In summary, the eye tracking device 1 proposed in the present disclosure generates a combined gaze direction by calculating the line of sight direction of one eye and then combining it with the existing line of sight direction of the other eye. Compared with calculating the line of sight directions of both sides to generate a combined gaze direction, the eye tracking device 1 of the present disclosure has a higher update speed and provides a better user experience. In addition, the eye tracking device 1 can also control the load of the processor 12 by adjusting the update cycle of calculating the line of sight direction. Accordingly, the eye tracking device 1 can dynamically control the load of the processor 12 according to the currently executed application or the load of the processor 12.

[0085] Please refer to Figure 4 , which is a flow chart of the eye tracking method 200 in the second embodiment of the present disclosure. The eye tracking method 200 comprises steps S201 to S203. The eye tracking method 200 is used to determine the position of the user's eye gaze according to the image of the user's eye, wherein the eye tracking method 200 can improve the updating efficiency by generating a merged gaze direction with the image of a single eye. The eye tracking method 200 can be executed by an electronic device (e.g.: Figure 1 The eye tracking device 1 is shown.

[0086] First, in step S201 , the electronic device obtains a first monocular image of a first eye at a first time point and a first monocular image of a second eye at a second time point based on a plurality of eye images of a user.

[0087] Next, in step S202 , the electronic device calculates a first sight line direction at the first time point based on the first monocular image of the first eye, and calculates a second sight line direction at the second time point based on the first monocular image of the second eye.

[0088] Finally, in step S203 , the electronic device generates a combined gaze direction at the second time point based on the first sight line direction and the second sight line direction.

[0089] In some embodiments, the second time point is later than the first time point.

[0090] In some embodiments, the eye tracking method 200 further includes the electronic device obtaining a second monocular image of a first eye at a third time point based on the multiple eye images, wherein the third time point is later than the first time point and the second time point; the electronic device calculating a third gaze direction at the third time point based on the second monocular image of the first eye; and the electronic device generating the merged gaze direction at the third time point based on the second gaze direction and the third gaze direction.

[0091] In some embodiments, the eye tracking method 200 further includes the electronic device obtaining a second monocular image of a second eye at a fourth time point based on the multiple eye images, wherein the fourth time point is later than the third time point; the electronic device calculating a fourth gaze direction at the fourth time point based on the second monocular image of the second eye; and the electronic device generating the merged gaze direction at the fourth time point based on the third gaze direction and the fourth gaze direction.

[0092] In some embodiments, the eye tracking method 200 further includes the electronic device selecting one of a plurality of update cycles; and the electronic device determining a time interval between the second time point and the third time point based on one of the plurality of update cycles.

[0093] In some embodiments, the eye tracking method 200 further includes the electronic device generating the combined gaze direction again based on the first sight line direction and the second sight line direction before the third time point.

[0094] In some embodiments, step S201 further includes the electronic device using an image recognition model to identify a position of the first eye in one of the multiple eye images; and the electronic device capturing the first monocular image from one of the multiple eye images based on the position of the first eye.

[0095] In some embodiments, step S202 further includes the electronic device calculating a position of the first eye in a first monocular image of the first eye to calculate the first sight direction.

[0096] In some embodiments, the eye tracking method 200 further includes the electronic device selecting, based on a flag value, to generate the merged gaze direction in a first step or a second step; wherein the first step includes calculating the gaze direction of the first eye and generating the merged gaze direction based on the gaze direction of the first eye; and the second step includes calculating the gaze direction of the second eye and generating the merged gaze direction based on the gaze direction of the second eye.

[0097] In some embodiments, the flag value corresponds to a first flag value and a second flag value, the first flag value corresponds to the first step, and the second flag value corresponds to the second step.

[0098] In some embodiments, the eye tracking method 200 further includes: in response to completing the first step, the electronic device sets the flag value to the second flag value; and in response to completing the second step, the electronic device sets the flag value to the first flag value.

[0099] In summary, the eye tracking method 200 proposed in the present disclosure generates a combined gaze direction by calculating the line of sight direction of one eye and then combining it with the existing line of sight direction of the other eye. Compared with updating after calculating the line of sight directions of both sides, the eye tracking method 200 of the present disclosure has a higher update speed and provides users with a better user experience. In addition, the eye tracking method 200 can also control the load of the electronic device by adjusting the update cycle of the calculated line of sight direction. Accordingly, the eye tracking method 200 can flexibly control the load of the electronic device according to the currently executed application or the load of the electronic device.

[0100] Although several embodiments are described above as examples, the eye tracking device and method proposed in the present disclosure may also be implemented by other systems, hardware, software, storage media or a combination thereof. Therefore, the protection scope of the present disclosure should not be limited to the specific implementation methods described in the embodiments of the present disclosure, but should be determined by the appended claims.

[0101] It is obvious to those with ordinary knowledge in the technical field to which the present disclosure belongs that various modifications and changes can be made to the structure of the present disclosure without departing from the scope or spirit of the present disclosure. In view of the foregoing, the scope of protection of the present disclosure also covers the modifications and changes made within the appended claims.

Claims

1. An eye tracking device, characterized in that: Include: a camera for capturing multiple eye images of a user over a continuous period of time; and a processor, coupled to the camera, for: Based on the multiple eye images, obtaining a first monocular image of a first eye at a first time point and a first monocular image of a second eye at a second time point; Calculating a first sight direction at the first time point based on the first monocular image of the first eye, and calculating a second sight direction at the second time point based on the first monocular image of the second eye; and A combined gaze direction is generated at the second time point based on the first gaze direction and the second gaze direction.

2. The eye tracking device according to claim 1, wherein: The second time point is later than the first time point.

3. The eye tracking device according to claim 1, wherein: The processor is further configured to: Based on the multiple eye images, obtaining a second monocular image of the first eye at a third time point, wherein the third time point is later than the first time point and the second time point; Calculating a third sight line direction at the third time point based on the second monocular image of the first eye; and The combined gaze direction is generated at the third time point based on the second gaze direction and the third gaze direction.

4. The eye tracking device according to claim 3, wherein: The processor is further configured to: Based on the plurality of eye images, obtaining a second monocular image of a second eye at a fourth time point, wherein the fourth time point is later than the third time point; Calculating a fourth sight line direction at the fourth time point based on the second monocular image of the second eye; and The combined gaze direction is generated at the fourth time point based on the third gaze direction and the fourth gaze direction.

5. The eye tracking device according to claim 3, wherein: The processor is further configured to: Select one of several update cycles; and A time interval between the second time point and the third time point is determined based on one of the plurality of update cycles.

6. The eye tracking device according to claim 3, wherein: The processor is further configured to: Before the third time point, the combined gaze direction is generated again based on the first sight line direction and the second sight line direction.

7. The eye tracking device according to claim 1, wherein: The operation of calculating the first sight direction further comprises: A position of the first eye in a first monocular image of the first eye is calculated to calculate the first sight line direction.

8. The eye tracking device according to claim 1, wherein: The processor is further configured to: Based on a flag value, selecting a first operation or a second operation to generate the merged gaze direction; The first operation includes calculating the sight direction of the first eye and generating the combined gaze direction based on the sight direction of the first eye; as well as The second operation includes calculating a sight line direction of the second eye and generating the combined gaze direction based on the sight line direction of the second eye.

9. The eye tracking device according to claim 8, wherein: The flag value corresponds to a first flag value and a second flag value, the first flag value corresponds to the first operation, and the second flag value corresponds to the second operation.

10. The eye tracking device according to claim 9, wherein: The processor is further configured to: In response to completing the first operation, setting the flag value to the second flag value; and In response to completing the second operation, the flag value is set to the first flag value.

11. An eye tracking method, characterized in that: Applicable to an electronic device, the eye tracking method comprises the following steps: Based on a plurality of eye images of a user, obtaining a first monocular image of a first eye at a first time point and a first monocular image of a second eye at a second time point; Calculating a first sight direction at the first time point based on the first monocular image of the first eye, and calculating a second sight direction at the second time point based on the first monocular image of the second eye; and A combined gaze direction is generated at the second time point based on the first gaze direction and the second gaze direction.