Eyewear device, system, and method and for performing eye tracking

Abstract

A personal eyewear device 100 and method are provided for obtaining eye tracking data of a wearer of the personal eyewear device 100. The personal eyewear device 100 comprises a frame 104, an electronics module 106 including a processor 110, a memory, a power source 112 for powering the electronics module 106, and an eye tracking subsystem comprising a first pair of imaging sensors forming a first stereo imaging sensor configured to capture images of a left eye of the wearer, and a second pair of imaging sensors forming a second stereo imaging sensor configured to capture images of a right eye of the wearer. An accelerometer and gyroscope may provide positional data.

Claims

1. A personal eyewear device for obtaining eye tracking data of a wearer of the personal eyewear device, the personal eyewear device comprising:a frame for supporting one or more lenses in front of a left eye and a right eye of a wearer when the personal eyewear device is worn by the wearer;an electronics module including:a processor;a memory;a power source for powering the electronics module; andan eye tracking subsystem, wherein the processor and the memory are communicatively coupled with the eye tracking subsystem and wherein the eye tracking subsystem comprises:a first pair of imaging sensors forming a first stereo imaging sensor configured to capture images of a left eye of the wearer; anda second pair of imaging sensors forming a second stereo imaging sensor configured to capture images of a right eye of the wearer.

2. The personal eyewear device of claim 1, wherein:the first pair of imaging sensors include a first imaging sensor arranged on the frame at a first position and a second imaging sensor arranged on the frame at a second position, wherein the first position and the second position are substantially opposed to each other such that the first imaging sensor and second imaging sensor are configured to face the left eye of the wearer from substantially opposite angles; andthe second pair of imaging sensors include a third imaging sensor arranged on the frame at a third position and a fourth imaging sensor arranged on the frame at a fourth position, wherein the third position and the fourth position are substantially opposed to each other such that the third imaging sensor and fourth imaging sensor are configured to face the right eye of the wearer from substantially opposite angles.

3. The personal eyewear device of claim 2, wherein:the first position and the second position are in the vicinity of a first rim of the frame, wherein the first rim of the frame is adjacent to the left eye of the wearer when the personal eyewear device is worn; andthe third position and the fourth position are in the vicinity of a second rim of the frame, wherein the second rim of the frame is adjacent to the right eye of the wearer when the personal eyewear device is worn.

4. The personal eyewear device of any preceding claim, wherein the first and second pairs of imaging sensors are configured to operate synchronously according to a global shutter such that the first and second pairs of imaging sensors are configured to capture simultaneous images.

5. The personal eyewear device of any preceding claim, wherein the first and second pairs of imaging sensors are infrared imaging sensors, each configured to be sensitive to light in the infrared spectral band.

6. The personal eyewear device of claim 5, wherein the eye tracking subsystem further comprises:a first illumination source for illuminating the left eye of the wearer with light in the infrared spectral band; anda second illumination source for illuminating the right eye of the wearer with light in the infrared spectral band, wherein the first and second illumination sources are communicatively coupled with the processor.

7. The personal eyewear device of any preceding claim, wherein the eye tracking subsystemfurther comprises:an accelerometer; anda gyroscope, wherein the accelerometer and the gyroscope are configured to detect an orientation of the personal eyewear device, wherein the accelerometer and gyroscope are communicatively coupled to the processor.

8. The personal eyewear device of any preceding claim, further comprising a transceiver communicatively coupled to the processor, for transmitting eye tracking data to a user terminal device.

9. The personal eyewear device of any preceding claim, further comprising a photovoltaic cell disposed on the frame for charging the power source.

10. A system for obtaining eye tracking data of a wearer of a personal eyewear device, the system comprising:the personal eyewear device of any preceding claim;a user device configured to communicate with the personal eyewear device and to receive data including eye tracking data of a wearer from the personal eyewear device; anda data processing module, configured to further process the eye tracking data of the wearer to determine a status of the wearer.

11. The system of claim 10, wherein the data processing module is external to the user device, wherein the user device is configured to communicate with the data processing module, and wherein the data processing module is configured to receive the eye tracking data of the wearer from the user device.

12. A method of obtaining eye tracking data of a wearer of the personal eyewear device of any of claims 1 to 9, wherein the method comprises:simultaneously capturing, with the first pair of imaging sensors, a first pair of images of a left eye of the wearer, and with the second pair of imaging sensors, a second pair of images of a right eye of the wearer.

13. The method of claim 12, further comprising a first processing step including:processing, using a stereo computer vision process, the first pair of images to obtain first three-dimensional information data of the left eye of the wearer;processing, using the stereo computer vision process, the second pair of images to obtain second three-dimensional information data of the right eye of the wearer; and62recording the first and second three-dimensional information data as eye tracking data.

14. The method of claim 13, further comprising a second processing step including: identifying a first location of a first biological feature of the left eye of the wearer in the first three-dimensional information data; andidentifying a second location of a second biological feature of the right eye of the wearer in the second three-dimensional information data.

15. The method of claim 14, further comprising a third processing step including: determining a left eye gaze vector from based on the first location;determining a right eye gaze vector based on the second location; and recording the left and right eye gaze vectors as eye tracking data.

16. The method of claim 15, further comprising a fourth processing step including: determining a gaze point, by determining a point of minimum distance or convergence between the left eye gaze vector and the right eye gaze vector; andrecording the gaze point as eye tracking data.

17. The method of claim 16, further comprising a fifth processing step including: determining a distance from the wearer to the gaze point; and recording the distance from the wearer to the gaze point as eye tracking data.

18. The method of any of claims 12 to 17, further comprising: capturing a plurality of consecutive first and second pairs of images;recording a plurality of eye tracking data corresponding to the plurality of consecutive first and second pairs of images; andassociating each of the plurality of eye tracking data with timing data indicative of respective times of capture of the plurality of consecutive first and second pairs of images.

19. The method of claim 18 when dependent on claim 17, further comprising a sixth processing step including:retrieving gaze points and distances from the wearer to the gaze points from the plurality of eye tracking data;joining the gaze points in a chronological manner, according to their associated timing data, to form a gaze path, wherein the gaze path is associated with the distances from the wearer to the gaze points forming the gaze path; andrecording the gaze path as eye tracking data.

20. The method of any of claims 15 to 19, further comprising calibrating the personal eyewear device, wherein calibrating comprises:capturing, in a controlled environment, a first calibration pair of images of a left eye of the wearer, and a second calibration pair of images of a right eye of the wearer, wherein the controlled environment63includes a viewable target point of known approximate first and second angles from the left and the right eyes of the wearer respectively, with wearer looking at the target point;processing the first calibration pair of images to identify a first calibration location of a first calibration biological feature of the left eye of the wearer;processing the second calibration pair of images to identify a second calibration location of a second calibration biological feature of the right eye of the wearer;assigning a first calibration eye gaze vector to the first calibration location based on the known approximate first angle between the target point of the wearer and the left eye of the wearer; andassigning a second calibration eye gaze vector to the second calibration location based on the known approximate angle between the target point of the wearer and the left eye of the wearer,wherein:determining the left eye gaze vector based on the first location comprises determining the left eye gaze vector from the first calibration eye gaze vector and a first difference between the first location and the first calibration location; anddetermining the right eye gaze vector based on the second location comprises determining the right eye gaze vector from the second calibration eye gaze vector and a second difference between the second location and the second calibration location.

21. The method of any of claims 13 to 20, wherein the first, second, third, fourth, fifth and / or sixth processing steps are performed by the processor of the electronics module of the personal eyewear device.

22. The method of claim 21, further comprising, using a scheduling module that is coupled to, or forms part of the processor of the electronics module of the personal eyewear device:activating or deactivating one or more additional subsystems of the personal eyewear device by comparing the eye tracking data to respective activation conditions for the one or more additional subsystems.

23. The method of any of claims 12 to 22, further comprising:performing further processing of the eye tracking data to determine a status of the wearer.

24. The method of claim 23, wherein performing the further processing of the eye tracking data comprises:training a machine learning model with a training dataset, the training dataset including a plurality of data entries, the plurality of data entries including training eye tracking data correlated with training status data;inputting, to the trained machine learning model, the eye tracking data;receiving, from the trained machine learning model, an output including status data; and determining the status of the wearer from the output including status data.

25. A computer program, which, when executed by a processor, is configured to perform the method of any of claims 12 to 24.

Images