568 results about "Eye tracking" patented technology

A virtual reality (VR)-based system for improving students' reading comprehension and writing skills. The system includes: a VR headset with an integrated eye-tracking system, high-resolution stereoscopic displays and spatial audio output, where the eye-tracking system detects reading patterns, fixation points and saccadic movements to assess reading engagement; a pair of motion tracking gloves with embedded pressure-sensitive capacitive sensors and haptic feedback actuators, allowing students to work with virtual reading material, perform handwriting exercises, and receive tactile reinforcement of text-based interactions; an adaptive learning module based on artificial intelligence (AI) that is configured to dynamically adjust reading complexity and writing exercises, responding to real-time metrics for interaction with students, with the AI ​​module using deep learning-based natural language processing (NLP) models to analyze reading fluency, written coherence, and syntactic complexity; a speech recognition and synthesis subsystem comprising a neural network-based acoustic model and a phoneme segmentation processor, wherein the subsystem assesses pronunciation accuracy, fluency, and articulation deviations by analyzing phonetic waveforms of spoken responses; a multimodal interactive interface that enables students to interact with virtual reading environments via a combination of hand tracking, voice commands and eye-controlled text selection, with the interface allowing students to highlight, comment on and orally summarize text content; A real-time dashboard for teacher monitoring, which is communicatively connected to the VR system. The dashboard offers educators detailed analyses of student comprehension patterns, writing progress, and AI-generated adaptive learning recommendations; and a cloud-integrated content management system (CMS) configured to store AI-curated reading materials, personalized writing assignments, and NLP-processed insights into student performance, with the CMS enabling adaptive curriculum adjustments based on students' learning trajectories.

A system includes a head-up display (HUD) system including a display; a combiner; and at least one processor communicatively coupled to at least one image sensor of at least one tracking system and to the display. The at least one processor configured to: receive image data from one or more of the at least one image sensor, wherein the image data is (i) for performing eye tracking and / or head tracking operations and (ii) for monitoring the displayed images; and detect the at least one malfunction of the HUD system.

A display device may receive eye tracking data. A display device may detect an eye gaze intent state based on the eye tracking data. A display device may receive an input device signal of an input device. A display device may determine to switch from a first input modality to a second input modality based on at least one of the eye gaze intent state or the input device signal. A display device may move a position indicator on a display of the display device using at least one of the first input modality or the second input modality.

In one implementation, a method of authenticating a user is performed by a first device including a camera, an eye tracker, one or more processors, and non-transitory memory. The method includes obtaining, via the camera, an image of a physical environment. The method includes detecting, in the image of the physical environment, a second device. The method includes determining, using the eye tracker, a gaze of a user. The method includes determining that the gaze of the user is directed to the second device. The method includes, in response to determining that the gaze of the user is directed to the second device, transmitting authentication credentials to the second device.

Techniques for gaze-based video conference prompts are described that leverage eye-tracking algorithms within a video conference setting to perform a variety of functionality. For instance, a computing device displays a user interface of a video conference session that includes representations of video conference attendees. The computing device receives video content that depicts a user of the computing device and uses eye tracking techniques to determine a gaze location of the user based on the video content. The gaze location corresponds to one of the representations of an attendee. The computing device then communicates a prompt to one or more of the attendees that indicates the gaze location. In another example, the computing device detects a behavior of the user, e.g., a scanning behavior. Responsive to detection of the behavior, the computing device performs an action within the user interface, such as to display a roster view of the attendees.

An information processing apparatus includes analysis circuitry to receive captured images comprising at least one eye of a user when viewing display images displayed by a head mounted display “HMD” and detect one or more eye properties for the user in dependence on at least some of the captured images and generate eye information indicative of one or more of the eye properties for the user, control circuitry to control, in dependence on the eye information, image brightness associated with one or more eye tracking calibration images to be displayed to the user, and processing circuitry to output one or more of the eye tracking calibration images for display by the HMD.

The invention discloses an ideological and political large-scale course teaching attention assessment data acquisition method based on sight tracking. The method comprises the following steps: S1, high-definition large-scene eye movement tracking photographing equipment captures eye images of students in a course teaching process in real time through an infrared camera; s2, performing pupil area detection and pupil center fitting on the eye image to obtain sight tracking data; s3, performing gaze hotspot analysis and gaze duration statistics according to the sight tracking data to quantify attention distribution and change trend of the students and visually display the attention distribution and change trend; and S4, generating a teaching quality evaluation report and teaching optimization suggestions according to the gazing hotspot analysis and gazing duration statistical results. The classroom teaching effect can be objectively evaluated in real time, subjective evaluation errors are reduced, the accuracy and efficiency of teaching feedback are improved, the method is suitable for various scenes such as traditional classrooms, online education and experiment teaching, and the method has the industrial application prospect of education digital transformation.

An XR system is disclosed having a night mode to optimize performance in low light conditions. The system estimates a user's dark adaptation level based on captured image data and selects a rendering configuration aligned with the user's vision. For example, spatial resolution, temporal resolution, color mode, and brightness are adjusted based on whether the adaptation level is scotopic, mesopic, or photopic. With eye tracking, foveated rendering is used for cones and periphery rendering for rods. Without eye tracking, modes with different wavelengths and colorization are employed. The night mode aims to maximize information visible on the display, conserve power by only showing what eyes can perceive, and preserve night vision by avoiding over-stimulating rods.

The invention discloses an immersive travel experience system based on the meta-universe technology, which relates to the field of the meta-universe technology and comprises a scene construction module, an interaction equipment module, a data analysis module and a multi-mode interaction module. According to the immersive text travel experience system, virtual reality, augmented reality and 3D rendering technologies are integrated to construct a virtual text travel scene, special equipment is configured to enable a user to feel the virtual text travel scene personally on the scene, meanwhile, the system can further analyze user behavior data in real time, and the user experience is improved. The virtual scene is adjusted according to the interactive behavior of the user in the virtual travel scene, personalized touring experience is provided, the naturalness and depth of interaction are further enhanced by fusing gestures, voice interaction, eye movement tracking and electroencephalogram signal recognition technologies, the user can obtain richer and more real travel experience, and the user experience is improved. The immersion and individuation degree of cultural tourism are improved, and interesting tourism experience is brought.

The invention provides an eyeball tracking optical system and a display device. The eyeball tracking optical system comprises an infrared light emitting device, a first lens group and an infrared imaging device. Wherein the infrared light emitting device is used for emitting infrared light to human eyes; the first lens group is used for being arranged on the light emitting side of the display panel, the side, away from the display panel, of the first lens group is used for converging infrared light reflected by human eyes, and the side, close to the display panel, of the first lens group is used for diverging the infrared light deflected by the first lens group; the infrared imaging device is arranged on the side, close to the display panel, of the first lens set, and the imaging face of the infrared imaging device faces the first lens set. Infrared light reflected by human eyes is deflected after passing through the first lens group, and is incident to the infrared imaging device.

The invention discloses a driving skill level classification method based on multi-dimensional data fusion. The method comprises the following steps: (1) acquiring behaviors, physiological data and environmental data of a driver in real time through a driving simulator, eye movement tracking glasses and a physiological sensor; and (2) processing and analyzing the multi-dimensional data by using a CNN-LSTM classifier (CNN-LSTM-attention) so as to evaluate the skill level of the driver. According to the method, control skills and cognitive skills are comprehensively evaluated, key parameters including steering, braking, eye movement frequency, concentration duration and the like are captured in real time by using a driving simulator, eye movement tracking glasses and a physiological sensor, and a CNN-LSTM-attention classifier is constructed. According to the method, the complexity of the scene is quantified, the classifier is trained according to different scenes, the accuracy is improved, and meanwhile, the reliability of driving ability evaluation is enhanced by using multi-dimensional data.

The invention provides a teaching interaction-oriented digital human three-dimensional reconstruction system, and relates to the technical field of human-computer interaction, and the system comprises a data collection module which is used for collecting a voice signal, gesture action data, eye tracking data and facial expression data of a user in real time through a multi-modal sensor array, and obtaining a teaching type identifier, an interaction stage state parameter and a participant role identifier of the current teaching scene through a preset teaching scene database. According to the invention, accurate adaptation of teaching scenes and intelligent generation and dynamic optimization of interaction strategies are realized, the accuracy and suitability of teaching interaction are improved, and personalized and high-quality teaching interaction experience is brought to users.

Provided are methods, systems, and computer program products for using an augmented or alternative communications (AAC) system including an eye tracking device. The method may include receiving at least one gaze input from a user and displaying at least one selected text portion in response to each gaze input. The at least one selected text portion may form a message that is displayed in a message window of the AAC system. A suggested text portion may be generated, the suggested text portion being adaptive to the content of the message and being displayed in the message window with the message. The method may further include receiving a confirmatory gaze input from the user, the confirmatory gaze input causing the suggested text portion to be added to the message.

A system for treatment and / or psychology intervention of attention deficit disorders in a user in need thereof. The system uses a head mounted display equipped with an eye tracking device. The system is configured to display a scene that includes a principal object and at least one secondary object, then use the eye tracking device to determine whether the user's gaze is directed to one of the objects. If the user's gaze is directed to one of the secondary objects beyond a threshold criterion, that secondary object is removed from the scene. If the user's gaze is directed to the principal object beyond a threshold criterion, a previously removed secondary object is reintroduced into the scene. The principal object is a desired object of the user's attention, while the secondary object(s) may be potentially or known distracting objects. Corresponding software and methods are also disclosed.

The invention relates to the technical field of deep learning, and discloses an eyeball tracking method and device and electronic equipment, and the method comprises the steps: obtaining an eye image data set of a user, inputting the eye image data set into a preset deep learning model, determining the eyeball position information and the eyeball movement mode of the user, based on the eyeball position information and the eyeball movement mode, the intelligent robot is controlled to perform eye interaction with the user, and eye recognition and dynamic position real-time tracking can be accurately performed through a preset deep learning model.

The invention relates to the technical field of brightness control, in particular to a welding machine screen display brightness control method and system. The method comprises the following steps: acquiring a welding machine screen image, environment perception data and a user face image; performing illumination intensity calculation on the environment perception data to generate environment illumination intensity data; performing glare analysis on the environment perception data through the environment illumination intensity data to generate environment glare influence data; performing environment perception brightness adjustment on the welding machine screen image based on the environment glare influence data to generate environment perception brightness adjustment data; performing visual perception feature analysis on the user face image based on the environmental perception brightness adjustment data to generate user visual perception feature data; and performing eyeball tracking on the user face image by using a computer vision technology to generate user eyeball tracking data. According to the invention, efficient and accurate brightness control is realized.

The invention relates to the technical field of visual health management, in particular to a ciliary muscle adjustment training system and method based on eye movement tracking, and the method comprises the steps: collecting eye movement data of a fixation point track, an adjustment amplitude change rate and pupil response time in real time, and combining personalized parameters such as the age of a user, diopter and a training target; inputting a ciliary muscle training dynamic planning algorithm module, dynamically generating a stage scheme containing a training action type, a single-group duration, a stimulation interval and a focal length adjustment strategy, and adaptively adjusting the strategy according to real-time adjustment stability and a historical ability curve; through multi-dimensional matching analysis of a pre-stored standard eye movement template, a training specification score is calculated from the track goodness of fit, speed uniformity, amplitude standard-reaching rate and binocular coordination, visual fatigue parameters are obtained based on continuous training duration, pupil fluctuation and retina reflex change, a scoring result is dynamically corrected, and a visual fatigue evaluation result is obtained. Personalized, self-adaptive and quantitative safety control of the training process is realized.

Calibration for eye tracking calibration in a head-mounted display can refined during use by tracking head rotation of a user during a time period while the head-mounted display is worn by the user, tracking eye rotation of the user during the time period, comparing the eye rotation of the user to the head rotation of the user during the time period, and refining a calibration state of eye tracking for the head-mounted display based on comparing the eye rotation of the user to the head rotation of the user during the time period.

In example implementations, an apparatus is provided. The apparatus includes a display, an eye tracking sensor, and a processor. The eye tracking sensor is to capture eye movements of a user. The processor is in communication with the display and the eye tracking sensor. The processor is to identify a type of eye movement from the eye movements captured by the eye tracking sensor and activate a control that is shown in the display.

A cyber-physical system for conducting a telehealth session. In embodiments, the system includes a hardware control box that enables patients (e.g., elderly or cognitively-impaired patients) to easily participate in the telehealth session. In some embodiments, the system analyzes sensor data (e.g., thermal images, eye tracking data, etc.) and calculates state variables that form a mathematical representation (a “digital twin”) indicative of the physical, emotive, cognitive, or social state of the patient. In some of those embodiments, the system calculates Myasthenia Gravis core examination metrics by performing computer vision analysis on patient video data and / or audio analysis on patient audio data. In some embodiments, the system controls a pan-tile-zoom camera to zoom in on each region of interest that is relevant to the examination. In some embodiments, the digital twin is used as an input of a heuristic computer reasoning system, which uses artificial intelligence to support clinical diagnosis and decision-making.

The invention relates to the technical field of early detection of neurological diseases, and discloses an application of a portable three-dimensional eye tracker in cognitive function assessment, comprising: a portable three-dimensional eye tracker used for collecting eye movement data, including fixation time, glancing trajectory, review frequency and eye jump speed; the electroencephalogram acquisition equipment is used for acquiring electroencephalogram data, and the electroencephalogram data comprises alpha wave energy E alpha, beta wave energy E beta and theta wave energy E theta; the heart rate variability monitoring equipment is used for collecting heart rate data, and the heart rate data comprises the ratio RLF / HF of low frequency (LF) to high frequency (HF). The accuracy of cognitive function evaluation is greatly improved through multi-modal data fusion, real-time monitoring and personalized feedback are provided, early discovery of cognitive impairment is facilitated, and the device is high in portability, supports long-term monitoring and testing anytime and anywhere, and is suitable for early detection and continuous evaluation of Alzheimer's diseases.

A system includes one or more processors and one or more memories having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations include receiving detected ocular landmarks in a subject, determining ocular parameters based on the detected ocular landmarks, and determining, based on the ocular parameters, a coarse calibration output. The operations include receiving, eyetracking data relating to an eye movement made by the subject, determining a landing position of the eye at the end of the eye movement, sending a signal to display a target at the landing position on a screen, and receiving, from the subject, an input including a response relating the landing position to a gaze parameter of the subject. The gaze parameter is associated with the eye movement. The operations include updating the coarse calibration output based on the input.

An eye tracking device including an event-based optical sensor, configured to receive radiation reflected off an eye of a user and produce a signal stream of events, each event corresponding to detection of a temporal change in the received radiation at one or more pixels of said optical sensor, and a controller, which is connected to said optical sensor and configured to: a) receive the signal stream of events from the optical sensor, b) generate an inference frame based on the stream of events, c) utilize the inference frame as input to a machine learning module and operate the machine learning module to obtain output data, and e) extract from the output data information related to the eye of the user, wherein the controller is configured to generate the inference frame utilizing a first artificial neural network.

The invention discloses an intelligent exhibition service system for an art display space, and belongs to the technical field of art display services, and the system comprises an audience information collection module which obtains the personal information of audiences and the data of art interests at the entrance of the art display space through a terminal registration or code scanning mode; according to the system, multiple modules of audience information collection, interaction intention recognition, emotional interest analysis, personalized service planning and display service control are integrated, so that comprehensive intelligent upgrading of audience experience in an art display space is realized; according to the method, the behavior habits and emotional states of audiences are analyzed in an all-around and multi-dimensional mode, and the highly personalized service planning greatly improves the visiting experience of the audiences, so that the audiences can freely select visiting routes and interaction modes according to own interests and preferences, and the interactivity and interestingness of exhibition are remarkably enhanced.

A multi-screen cockpit display system is suitable for being installed in a cockpit, and includes a first display device, a control unit and a second display device. The first display device is disposed in front of a driver's seat in the cockpit, and is suitable for displaying a first dynamic information. The control unit is electrically connected to the first display device. The control unit is suitable for providing the first dynamic information to the first display device according to an operating state of the cockpit. Another multi-screen cockpit display system with an eye tracking module is also provided.

To provide a highly reliable actuator.SOLUTION: An actuator according to the present invention includes a vibration plate, an electrode layer formed integrally with the vibration plate, and a substrate having a piezoelectric layer, the substrate includes a silicon material, the piezoelectric layer includes potassium sodium niobate, the electrode layer includes a Pt film, and the average particle diameter in the plane direction of the Pt particles forming the Pt film is larger than the thickness of the Pt film.SELECTED DRAWING: Figure 1

The disclosed method that may be used for eye tracking includes measuring, by a controller, a distance to an object (316) based on a combination of polarization information of detected light and time-of-flight information of the detected light, wherein the detected light results from at least one of scattering or reflection of a light beam from the object (316). The disclosed method additionally includes performing, by an artificial reality system that includes the controller, one or more user interaction procedures based on information that includes the measured distance to the object (316). A system includes a laser scanning polarization imager including and a detector system (302) including a plurality of photodetectors 304. Each photodetector (306) includes a polarizer (308) having a particular polarization direction oriented at zero, 45, 90 and 135 degrees. A light source (310) may include a vertical cavity surface emitting laser (VCSEL) to emit a light beam (312). A reflector (314), such as a MEMS scanner, directs the light beam (312) onto the object (316), such as an eye. Light (318), e.g., specular and / or diffusive, is reflected and / or scattered from the object (316) and detected by the detector system (302). A controller measures a distance to the object (316) based on this detected light (318). Additionally, the controller measures position and / or size of one or more regions (320) of the object, such as a pupil and / or cornea.

The invention discloses a resolution adjustment method, device and equipment based on an eye tracking technology. The bandwidth can be reduced while the definition is improved. The resolution adjustment method based on the eye tracking technology comprises the following steps: acquiring real-time eye tracking data based on the eye tracking technology; dividing a sight focus area and a non-focus area of the user based on a preset area range and the real-time eye tracking data; and rendering the sight focus area and the non-focus area according to different resolutions.

Techniques related to the computation of gaze vectors of users of wearable devices are disclosed. A neural network may be trained through first and second training steps. The neural network may include a set of feature encoding layers and a plurality of sets of task-specific layers that each operate on an output of the set of feature encoding layers. During the first training step, a first image of a first eye may be provided to the neural network, eye segmentation data may be generated using the neural network, and the set of feature encoding layers may be trained. During the second training step, a second image of a second eye may be provided to the neural network, network output data may be generated using the neural network, and the plurality of sets of task-specific layers may be trained.

Methods, systems, and storage media for performing multi-user gaze tracking in a vehicle space using multi-surface optical reflections are disclosed. Implementations may: acquire face and eye region image data of a plurality of occupants within a field of view of at least one camera associated with a vehicle; evaluate reflected image quality thresholds; locate and match occupants within the vehicle space; and perform eye tracking for multiple occupants independently via reflected multi-view images provided to a deep learning model.