173 results about "Eye tracking system" patented technology

The invention belongs to the field of eye movement tracking, and particularly relates to an eye movement tracking system, method and equipment integrating electroencephalogram signals and a non-contact eye tracker. The objective of the invention is to solve the problems of limited acquisition range of a non-contact eye tracker and low spatial resolution of eye movement behavior deduction by EEG signals in the prior art. The method comprises the steps that eye movement data and electroencephalogram signals are synchronously collected, after calibration mapping and time sequence alignment preprocessing are conducted, an electroencephalogram modal behavior recognition model is supervised and trained through eye movement fixation point labels, and the incidence relation of the two modals is established. A dynamic fusion strategy is adopted in the online stage, when eye movement data are valid, a model prediction result and the eye movement data are fused to output a fixation point, when eye movement fails, the fixation point is calculated in combination with model prediction and historical data, enhanced compensation of eye movement tracking by electroencephalogram signals is achieved, the defect that the spatial resolution of a single EEG signal is low is effectively overcome, and the accuracy of eye movement tracking is improved. And the robustness and the tracking precision of the system in a complex scene are improved.

Particular embodiments are directed to passthrough image generation for a mixed-reality experience. A device may determine, using an eye-tracking system of a head-mounted device, eye-tracking data associated with a user of the head-mounted device. The device may determine, based on the eye-tracking data, a desired scene depth for the user. The device may instruct a first autofocus camera of the head-mounted device to adjust a first focus distance based on the desired scene depth and capture a first image of a real-world environment of the user. The device may generate a first passthrough image based on the first image. The device may display the first passthrough image to a first eye of the user via a first display of the head-mounted device.

Examples relate to systems and methods for enhancing generative AI outputs using eye tracking data. An eye tracking system accesses eye gaze information associated with a field of view of a head-wearable apparatus and generates contextual information associated with the field of view of the head-wearable apparatus based on the eye gaze information. The eye tracking system processes, by a generative machine learning model, the contextual information and at least one image of the field of view of the head-wearable apparatus to generate an output and presents on a display of the head-wearable apparatus the output generated by the generative machine learning model.

An optical device can include a first securement arm, a second securement arm, and a lens frame defining a lens aperture and including a nose bridge, the lens frame connected to the first and second securement arms. The optical device can also include an eye-tracking camera mounted to a platform extending from the lens frame, the platform disposed proximate the nose bridge. An electronic component is disposed in the lens frame, the electronic component electronically connected to the camera.

An eye tracking system can include a first camera configured to capture a first plurality of visual data of a right eye at a first sampling rate. The system can include a second camera configured to capture a second plurality of visual data of a left eye at a second sampling rate. The second plurality of visual data can be captured during different sampling times than the first plurality of visual data. The system can estimate, based on at least some visual data of the first and second plurality of visual data, visual data of at least one of the right or left eye at a sampling time during which visual data of an eye for which the visual data is being estimated are not being captured. Eye movements of the eye based on at least some of the estimated visual data and at least some visual data of the first or second plurality of visual data can be determined.

The invention provides a machine learning and deep learning combined calibration-free eye movement tracking system and method, and the system comprises the following modules: a pre-calibration module which is used for collecting a near-to-eye image of the eyes of a user through a near-to-eye camera and an infrared light source, extracting pupil center coordinates and cornea reflection point coordinates, and outputting the pupil center coordinates and the cornea reflection point coordinates; calculating a personalized kappa angle of the user based on the extracted pupil center coordinates and cornea reflection point coordinates; the decision maker module adopts an AdaBoost ensemble learning algorithm, classifies the users according to the kappa angle obtained through calculation, and outputs a corresponding weight combination; and the deep learning module is used for receiving the output weight combination and the real-time near-eye image, and predicting the coordinates of the fixation point of the user through a convolutional neural network and a residual connection structure. According to the invention, the application flexibility and user friendliness of the eye movement tracking technology are greatly enhanced while the precision is guaranteed.

The embodiment of the invention discloses an eye movement tracking system and wearable equipment. The eye movement tracking system comprises a first light source module, a second light source module, a waveguide, an image acquisition module and a control module. Wherein the control module is configured to control the first light source module and the second light source module to be turned on in a time-sharing manner so as to alternately emit light rays towards the first eye and the second eye, and the waveguide transmits reflected light of the first eye and the second eye to the image acquisition module so that the image acquisition module can acquire image information of the first eye and the second eye. And the control module determines the fixation point information according to the image information. Therefore, the eye movement tracking system is favorable for reducing the production complexity and cost, improving the yield, and improving the aesthetic property and the user experience.

Disclosed are gaming systems, methods and machines that include one or more eye gaze tracking systems. In particular, the eye gaze tracking system may be incorporated with a gaming system, or located with a gaming environment and otherwise connected to a gaming system. When the focus of the user's gaze is identified, the gaming system can activate a response in the feature associated with the user's gaze. This can include a virtual feature on a display screen or a feature, such as lighting or signage, on the gaming system or within the gaming environment.

A three-dimensional (3D) compressive sensing based eye tracking system using single photon avalanche diode (SPAD) sensors achieves high resolution depth measurement by using low resolution SPAD sensors and active encoded illumination such as two- or three-dimensional fringe patterns, random speckles, random patterns, and / or superimposed patterns projected onto a surface of an eye. The patterns may be projected by high speed illuminators such as a digital micromirror device (DMD) or micro-electromechanical system (MEMS) projector in series changing at the same rate as the SPAD sensor capture rate. A processor may employ compressive sensing techniques to obtain a high resolution image and depth information from the captured images of varying patterns.

The invention discloses an eye movement tracking stability enhancing method, system and device and a storage medium, and belongs to the field of human-computer interaction and computer vision. The method comprises the following steps: acquiring an eye image and an original sight fixation point coordinate; performing quantitative evaluation on the image from five dimensions of definition, contrast and the like to generate an eye image quality evaluation grade; calculating the moving speed and acceleration of the current point based on the historical sight fixation point sequence; fusing, and judging and correcting the original coordinates. Through multi-modal information fusion and hierarchical adaptive correction, noise jump caused by poor image quality can be accurately distinguished and filtered out, meanwhile, the real eye movement dynamic state is effectively reserved, and the output stability and reliability of an eye movement tracking system are remarkably improved.

A lens assembly for a head-mounted display (HMD) comprises a reflective polarizer and an eye-tracking waveguide. The reflective polarizer is in an optical element. The eye-tracking waveguide is also included in the optical element. The eye-tracking waveguide is configured to direct near-infrared received illumination light to an eye-tracking camera.

A system enables a user to query based on a user's gaze by receiving a query from the user and capturing, via an eye tracking system on a headset, the user's gaze location near an object in a local area. The system captures one or more images of the local area with the object and formats the images based in part on a region of interest in the one or more images that includes the object. The system generates a formatted query based in part on the query. The formatted query is provided to a search engine. Information describing the object determined from the one or more formatted images and information describing the query are used by the search engine to determine an answer to the query about the object. The system presents the answer to the query about the object.

An eye tracking system for a head-mounted device includes an imaging assembly and an illumination assembly. The imaging assembly is configured to optically focus light from an object plane of an eye onto an image plane of an image sensor, and the illumination assembly is configured to optically project light from an illumination source onto the object plane of the eye. The imaging assembly is configured with an imaging angle coding mapping for optical relay coupling imaging. The illumination assembly is configured with an illumination angle coding mapping for optically coupling a projection.

The invention relates to the technical field of man-machine interaction, and discloses an eye movement tracking method, intelligent glasses, a mobile terminal and an eye movement tracking system.The method comprises the steps that current head movement data of a wearer is collected and sent to the mobile terminal; receiving the current gazing direction of the wearing user fed back by the mobile terminal, and tracking the eye movement of the wearing user according to the current gazing direction; wherein the current gazing direction is obtained through the mobile terminal according to the current head movement data and a current eyeball image of the wearing user, and the current eyeball image is obtained by shooting eyeballs of the wearing user through a camera. Since the mobile terminal with the camera arranged outside is utilized to complete eyeball collection and fixation direction determination of the wearing user, an independent camera and an independent computing power unit do not need to be arranged in the intelligent glasses, and the size of the intelligent glasses is reduced.

The present disclosure relates to an optical system comprising: a lightguide (110) having opposing first (112) and second (114) non-parallel optical surfaces, wherein at least one of the first or second optical surfaces is curved; an in-coupling diffractive optical element (120) arranged to receive reflected light from an object and couple the reflected light into the lightguide at a first location, wherein the reflected light propagates along the lightguide and exits the light guide at a second location; and a light detector (140) arranged proximal to the second location to receive the reflected light from the light guide. A head mounted eye tracking system or a world facing camera system may comprise the optical system.

The invention provides an eye movement tracking system for aphasia rehabilitation training, a vocabulary difficulty adaptive adjustment method, equipment and a medium. The method comprises the following steps: determining semantic diffusion characteristics of a current vocabulary for representing a target user; determining a semantic recognition gradient of a target user on the initial rehabilitation training voice; performing response feedback analysis on speech recognition of the target user in the initial rehabilitation training process based on the semantic diffusion features and the semantic recognition gradient to obtain a semantic response deviation of the target user in the current stage; determining a gaze association relationship of the target user under different semantic structure features; and adaptively adjusting the vocabulary difficulty of the rehabilitation training voice in the next stage according to the cognitive performance index of the target user in the current stage determined by the semantic response deviation and all the gaze association relationships. By adopting the scheme of the invention, the eye movement fixation characteristics of the patient can be deeply associated with the word order structure of the training voice content, and then the training voice in the rehabilitation training process can be adaptively adjusted.

A three-dimensional (3D) compressive sensing based eye movement tracking system using a single photon avalanche diode (SPAD) sensor achieves high resolution depth measurements by using a low resolution SPAD sensor and actively coded illumination (e.g., two or three dimensional fringe patterns, random spots, random patterns, and / or overlay patterns) projected onto the surface of the eye. These patterns may be projected by a high speed illuminator, such as a digital micromirror device (DMD) or a micro-electro-mechanical system (MEMS) projector, varying continuously at the same rate as the SPAD sensor acquisition rate. The processor may employ compressed sensing techniques to obtain high resolution images and depth information from the acquired images of the varying patterns.

An eye tracking system for detecting position and movements of a user's eyes in a head-mounted display (HMD). The eye tracking system includes at least one eye tracking camera, an illumination source that emits infrared light towards the user's eyes, and diffraction gratings located at the eyepieces. The diffraction gratings redirect or reflect at least a portion of infrared light reflected off the user's eyes, while allowing visible light to pass. The cameras capture images of the user's eyes from the infrared light that is redirected or reflected by the diffraction gratings.

The invention is related a method and a remote eye tracking system for determining a position of at least one eye of a subject. The method comprises the steps of: illuminating a face of a subject using at least one infrared (IR) illuminator off-axis from at least one image sensor; capturing at least one image of the face using the at least one image sensor at a time instant; and using a processing circuitry of the remote eye tracking system, performing processing steps of: determining at least one pupil candidate associated with the at least one image; determining at least one glint candidate associated with the at least one image; determining at least one pupil-glint candidate group, comprising at least one pupil candidate and at least one corresponding glint candidate; generating a score value for each of the at least one pupil-glint candidate groups; and determining an eye position of at least one eye of the subject in the at least one image based on the pupil-glint candidate group with the highest score value.

An apparatus, system, and method for an eye tracking system for a head-mounted device include a light source, an image sensor, and a frame tracking system. The light source and image sensor may be configured to determine absolute and relative eye orientations. The frame tracking system may be configured to determine the displacement of a head-mounted device frame relative to the face of the user. The frame tracking system may provide displacement data for the displacement of the head-mounted device frame to the eye tracking system to enable the eye tracking system to compensate for frame slippage, frame jostling, or other frame displacement (relative to the face of the user). The frame tracking system may include one or more position sensors disposed on the head-mounted device frame and logic configured to determine displacement data using information from the one or more position sensors.

Systems, devices, and methods for eye tracking and other sensing using self-mixing interferometry (SMI) through a waveguide in a near-eye device, i.e., in-field SMI eye tracking. The near-eye display device may include a waveguide having SMI coupler(s) and eye-side coupler(s) where one or more SMI(s): (i) project light into the SMI coupler(s) through the waveguide and out the eye-side coupler(s) to the user's eye; and (ii) receive light reflected back from the user's eye into the eye-side coupler(s) through the waveguide and out the SMI coupler(s). The one or more SMIs mix / modulate the received reflected light with presently projected light to provide an electrical signal for eye tracking. The SMI and / or eye-side couplers may be reflective, refractive, and / or diffractive; interior / exterior gratings, lenses, and / or coatings may be utilized in / on the waveguide to redirect, modulate, and / or otherwise alter the projected / received light.

The present disclosure allows for accurate and computationally efficient gaze tracking without requiring a corneal reflection. Eye tracking systems and methods are provided. They include obtaining images of the user's face, detecting a head pose, and locating the centre eye rotation, the pupil centre, the optical axis, the cornea centre and finally the visual axis corresponding to the gaze direction using user-specific parameters. Calibration systems and methods are also provided for acquiring the user-specific parameters. They include at least twice displaying a target point for the user to glaze at, directing light to obtain a corneal reflection, obtaining an image of the user's face, detecting the head pose, inferring a transformation to convert coordinates between the world coordinate system and a head coordinate system, and locating the corneal reflection, the cornea centre, the pupil centres and the optical axes. The data acquired thereby is used to compute the parameters.

To provide a more efficient system / method for eye motion tracking.SOLUTION: A system and method for eye motion tracking. An anchor point and a plurality of auxiliary points are selected from the reference image. Individual live images in the sequence of images are then searched for a match between the anchor point and the ancillary point. First, an anchor point is found, and then the search for individual auxiliary points is limited to a search window defined by the known distance and / or orientation of the auxiliary point to be searched relative to the anchor point.SELECTED DRAWING: Figure 4

Augmented and / or virtual reality (AR / VR), near-eye display devices that implement eye tracking via dense point cloud scanning are disclosed. In examples, an eye tracking system for an augmented reality / virtual reality (AR / VR) display device comprises a light beam emission and sensor element located in a first location of the display device to emit a light beam. The eye tracking system may comprises a redirection element located in a second location of the display device to redirect the light beam to illuminate one of line-of-sight and a field-of-view (FOV) of the display device.

The invention discloses an eye movement tracking system and method based on a position sensitive detector. The eye movement tracking system comprises an infrared lighting module, an optical element, the position sensitive detector and an MCU. The infrared lighting module emits infrared light which can be borne by human eyes to a cornea area of a user, the optical element performs angle compression and beam shrinkage on reflected light obtained after the infrared light is reflected in the cornea area of the user, and then the reflected light enters the position sensitive detector; four paths of voltage signals output by the position sensitive detector are input into the MCU; a reflected light spot position calculation module and a nonlinear mapping relation model between a reflected light spot position and an eyeball rotation angle are integrated in the MCU; the reflection light spot position calculation module is used for carrying out normalization processing on the four paths of voltage signals and calculating a ratio to obtain a reflection light spot position; the nonlinear mapping relation model between the reflected light spot position and the eyeball rotation angle takes the reflected light spot position as input and the eyeball rotation angle as output to realize eye movement tracking.

To provide systems and methods for eye tracking for use in various applications such as virtual reality or augmented reality applications that include head-mounted display devices.SOLUTION: An eye tracking subsystem may be provided, comprising a plurality of assemblies, each including a light source such as an LED, a light detector such as a silicon photodiode, and a polarizer configured to prevent light reflected via specular reflection from reaching the light detectors so that only scattered light is detected by the light detectors. Machine learning or other techniques may be used to track or otherwise determine a gaze direction of a user, which may be used by one or more components of an HMD device to improve its functionality in various ways.SELECTED DRAWING: Figure 5

A system and method of projecting augmentation imagery in a head-mounted display are disclosed. The system includes an eye tracking system configured to generate eye tracking data at an eye-tracking rate in response to a position of a pupil of a user, a head tracking system configured to estimate a change in orientation of the user's head at a head tracking rate and generate jitter data at a jitter sensing rate, a controller configured generate a field mapping in response to render data at a projector update rate, a light generator configured to generate a color encoded light associated with the augmentation imagery, the light generator includes a 2D pixelated display and an optical system configured to image the color encoded light into a field of view of the user, wherein the optical system includes a foveated optical system.

Systems may include an eye-tracking illumination source, a waveguide optically coupled to the eye-tracking illumination source, and an optical sensor positioned to detect light from the eye-tracking illumination source. The waveguide may include at least one optical element for directing light from the eye-tracking illumination source toward an eye of a user. Various other related systems, devices, and methods are also disclosed.

A method, system, and non-transitory computer-readable storage medium for classifying flashes using an eye-tracking system of a head-mounted device are provided by the following steps: obtaining a current image of a first eye (any position of a flash present in the image t+1 ); obtaining a data set that indicates the respective positions of at least one flash detected in one or more previously captured images of the first eye; and for each flash present in the current image (image t+1 ), determining whether the position of the flash corresponds to the position of a flash in the data set; and if the positions correspond, classifying the flash as a static flash. Static flashes can be excluded from further processing in the eye-tracking system. If the eyes of the user of the head-mounted device have moved, the embodiment may further include updating the data set based on the current image (image t+1 ).

A head-mounted eye tracking system including an eye tracker, a signal processor, and a plurality of light-emitting optical guides is provided. The eye tracker is adaptable for sensing eyeballs of a wearer. The eye tracker includes a plurality of light-emitting devices and a plurality of sensing devices. The plurality of light-emitting devices are adaptable for emitting a tracking beam. The sensing devices are adaptable for receiving the tracking beam reflected by the eyeballs of the wearer. The signal processor is signally connected to the eye tracker. The plurality of light-emitting optical guides is disposed corresponding to the plurality of light-emitting devices.