34 results about "Saccade" patented technology

The invention provides an eye movement trajectory analysis method and system based on hybrid clustering and time constraint, and belongs to the technical field of computer vision. Calculating a time difference and a moving speed between continuous original eye movement data points; comparing the moving speed with a speed threshold value, and classifying the moving speed into candidate fixation points and glancing points; extracting spatial features and time features of the candidate fixation points, performing standardization processing, and performing weighted fusion to obtain spatial-temporal feature vectors; clustering the spatio-temporal feature vectors to obtain preliminary clustering labels of the candidate fixation points; performing time constraint processing on each cluster; calculating the duration of the clustering cluster after the time constraint processing, and generating a final clustering label of the candidate fixation point; and outputting a final classification label of each original eye movement data point in combination with the classification result of the glancing points.

The invention discloses an eye movement track analysis and classification method and system based on three-dimensional fixation estimation, and relates to the field of computer vision, and the method comprises the specific steps: obtaining a face video, estimating a head posture based on the face video, and extracting a plurality of face images; inputting the facial image into a multi-task learning model, extracting facial features and eye features in parallel, processing the fused features, and outputting an eye prediction state and a three-dimensional fixation vector estimation value; converting the three-dimensional fixation vector estimation value into a fixation point sequence under a screen coordinate system based on a relative pose relationship between the camera and the screen; clustering the fixation point sequence to obtain multi-dimensional eye movement features; and inputting the head posture and the multi-dimensional eye movement features into a fusion de-noising variational auto-encoder to obtain a classification result. According to the method, the text reading task is combined with the space-time distance function, and the extracted multi-dimensional eye movement features such as fixation, eye jump and review can accurately reflect the reading process of the subject, so that the classification accuracy is improved.

A method, system, and computer readable medium for switching between motion tracking mode. The method includes detecting a significant change in confidence level for motion tracking. A change from a first tracking mode to a second tracking mode is initiated when a significant change in confidence level is detected and the change is paused until a saccade or blink is detected. by eye tracking device. After a saccade or blink is detected the tracking mode switches from the first tracking mode to the second tracking mode.

The invention relates to the technical field of intelligent glasses, and particularly discloses an intelligent glasses system with a micro eye jump detection function, which comprises an eye movement image acquisition module, a data processing and transmission module, an eye movement feature extraction module, a micro eye jump detection and analysis module, an application response and interaction module and a hardware integration module. The eye movement image acquisition module synchronously captures a high-frame-rate eyeball image through a near-infrared light source and a camera, the data processing and transmission module carries out noise reduction and distortion correction on the image and transmits the image through a dual-mode link, and the eye movement feature extraction module locates a pupil center and compensates head movement to generate feature vectors including position, speed and the like. The micro-eye jump detection and analysis module identifies real micro-eye jumps and quantifies parameters through double-threshold screening and a machine learning model. Wearability and real-time performance of micro eye jump detection can be achieved, scene limitation of traditional equipment is broken through, portability and detection precision are both considered, and the method can be used for application scenes such as attention monitoring and disease detection.

A visual brain therapy device comprises a vertical panel, a horizontal panel, a plurality of lights and a controller. The vertical panel has an aperture, where the aperture has an upper edge and an opposite lower edge. The horizontal panel has a front end and an opposite back end, where the front end of the horizontal panel aligns with the lower edge of the aperture. Each of the plurality of lights is mounted on either the vertical panel or the horizontal panel. The controller is configured to activate each of the plurality of lights in a pattern, where the pattern comprises one of convergence, divergence, saccades and anti-saccades.

A gaze detection device and a setting processing method thereof, including an eye movement tracking module and a target detection module. The eye movement tracking module includes a sensing device that estimates and determines a gaze and a timing change of the gaze by sensing eye information, thereby distinguishing between a saccade state or a fixation state. The target detection module includes a plurality of target categories for setting at least one target category of interest. In the fixation state, the target detection module can detect a target. When the target detection module determines that a comparison image information matches the target category of interest, the image information is captured and stored for user review and reference.

The invention relates to the technical field of recruitment video analysis, and discloses a recruitment assisting method and system, and the method comprises the steps: obtaining continuous image frames and timestamps of an answering video through a fixed time interval, extracting the sight line direction of each frame through a convolutional neural network, and generating an azimuth angle sequence; then calculating azimuth angle changes of adjacent frames and positioning a peak value to obtain an occurrence moment and a main direction of a glancing event, and forming a glancing event time sequence; intercepting a final-section event in a final-section time window, and screening a target direction event set in combination with a direction range, an amplitude threshold and a time interval condition; mapping an event occurrence moment into a phase according to a preset period, and calculating a phase concentration ratio by adopting a circle statistical method to obtain a phase locking index; the phase locking index is corrected, and significance evaluation is carried out under the statistical condition; and finally, completing risk judgment according to the corrected phase locking index and the saliency result, and realizing quantitative detection and intelligent identification of the abnormal sight behavior in the online interview.

The invention relates to a visual field intensity modeling and parameter optimization method based on a directional glancing eye movement task, and the method comprises the steps: carrying out the parameterization design and data collection of the directional glancing task, constructing a visual field intensity model covering the three stages of central fixation, green spot glancing and red spot maintenance, carrying out the back-stepping of an individual visual field parameter based on maximum likelihood estimation, and carrying out the optimization of the visual field intensity. And performing distribution modeling and type division on multiple tested parameters, and optimizing task parameters through simulation and discrimination performance. According to the method, a unified space-time visual field intensity model is established, fine splitting of different cognitive mechanisms is realized, a task parameter optimization closed loop of simulation-backstepping-evaluation-screening is constructed, the distinguishing efficiency of different visual fields and execution control subtypes is remarkably improved, and the method has the advantages of being high in robustness and high in robustness. The method overcomes the defects of dependence on empirical parameters, insufficient cognitive interpretation and low crowd discrimination in the prior art, and can be used for quantitative evaluation and targeted task design of visual attention and eye movement control functions.

This invention provides a diagnostic method based on saccade analysis. The method includes: collecting eye movement displacement and time data of the subject to obtain a discrete eye movement velocity sequence; obtaining a denoised eye movement velocity sequence after denoising; selecting the velocity sequence to be processed using a moving window; determining the time length of the moving window based on the length of the velocity sequence and the calculation time requirements; determining an eye movement velocity threshold according to diagnostic needs; calculating local extrema of the velocity sequence within the moving window and determining the eye movement type and number of saccades based on the velocity threshold; calculating local extrema of the entire velocity sequence using this method to obtain the total number of saccades; diagnosing the subject's eye movement state based on the number of saccades and their time intervals, and determining whether detailed analysis of local data is needed to obtain a more accurate diagnostic result based on the diagnosis results. This invention can help doctors diagnose different types of saccades and their potential predictive conditions such as cognitive impairment and Parkinson's disease more quickly.

The present application relates to the technical field of computer vision and cognitive rehabilitation, and particularly relates to a self-adaptive reading efficiency detection and training system based on multi-modal fusion, comprising: a multi-modal data acquisition module, which is used for acquiring eye movement coordinates (x, y, t), pupil diameter and head posture data of a user; a space-time trajectory reconstruction engine, which comprises an improved I-DT algorithm module based on signal-to-noise ratio dynamic weighting and Kalman filter prediction compensation, and is used for extracting fixation points from low-frequency sampling data and retaining micro-saccades; and an elastic grid mapping module, which is used for generating a Gaussian ridge line model according to text topological features of a reading interface.The present application reduces dependence on special hardware, is conducive to cost control, and can realize fixation trajectory restoration precision close to that of a scientific research level device (> 120Hz) on a 30Hz-60Hz consumer-level camera or a tablet computer front lens through the improved I-DT algorithm, so that professional reading disorder screening and cognitive training are suitable for deployment and use in primary medical institutions or home environments.

The invention discloses a crew man-machine interaction cognitive performance determination method, and relates to the field of crew man-machine interaction cognitive behavior modeling, and the method comprises the steps: a judgment and decision model loads a target task, forms a generation set, searches a generation corresponding to a maximum utility value for execution through the judgment and decision model, and updates a work memory model. The visual attention model is driven to operate, the eye movement preparation time and the eye movement glancing speed are obtained, and the visual attention performance is determined; the operation control model is driven to operate, the hardware environment of the display control equipment and the Fresnel's law parameters of the sailors are obtained, and the operation control performance is determined; updating the working memory model according to the execution result; updating the generative memory model according to the updated working memory model, and determining judgment and decision performance; summing the visual attention performance, the manipulation control performance and the judgment and decision performance, and determining the total cognitive performance of man-machine interaction of the crew. According to the method, the man-machine interaction cognitive performance of the crew can be accurately calculated in a conventional or long-voyage state.

This invention discloses a visual data analysis system and method for work scenarios, relating to the field of augmented reality technology. The method includes: monitoring the peak value, duration, and post-rotation stillness duration of head rotation angular velocity; freezing a semantic snapshot of the visual field and recording the rotation amplitude when corresponding thresholds are met; defining the effective depth-of-field range using the depth of a reference object on the work surface; determining global motion, gaze point drift, and reference object overlap during visual field regression; matching new frames and snapshots after all parameters meet stability thresholds; marking disappearance candidates based on the number of consecutive frames confirmed by disappearance; assigning confidence and outputting visual cues after excluding operation occlusion and out-of-field offset; and correcting near-end offset, far-end offset, number of consecutive frames confirmed by disappearance, and stability thresholds based on user response. This invention effectively distinguishes between brief saccades and actual gaze departures through the combined triggering of multiple parameters, including head rotation angular velocity, duration, and post-rotation stillness duration.

The application discloses an eye movement track generation method, device and equipment and a readable storage medium. The method comprises the following steps: acquiring line-of-sight landing point data collected based on an eye tracker and an experiment duration; performing denoising processing on the line-of-sight landing point data through a sampling window, and determining saccade data and fixation data from the data points obtained after the denoising processing; determining a first time factor through a start time, an end time and the experiment duration of a fixation point, and determining a second time factor through a duration and the experiment duration; and generating eye movement track information based on the position of the fixation point, the first time factor and the second time factor. The application adopts a new eye movement track representation method, which not only contains duration and fixation position information, but also increases more specific time point information, can comprehensively express dynamic changes of visual attention with time, and constructs a new eye movement track model based on a conditional generative adversarial network, so that the model can accurately predict eye movement tracks.

The application discloses a driving distraction state recognition method based on eye movement sequence space-time semantic feature analysis, and relates to the technical field of visual analysis, and comprises the following steps: S1, acquiring eye movement data of a to-be-identified subject during driving; S2, constructing a driving distraction recognition model based on eye movement sequence space-time semantic features; and S3, inputting the eye movement data of the to-be-identified subject into the driving distraction recognition model based on eye movement sequence space-time semantic features which is constructed, and obtaining a distraction detection result of the to-be-identified subject. The application designs a driving distraction recognition model based on eye movement sequence space-time semantic features, extracts time, space and semantic features in visual saccade paths, and combines the features to represent the eye movement mode of a driver, so that whether the driver is in a distraction state can be identified.

Systems and methods are disclosed for modifying content items based on eye tracking data, which may be indicative of saccadic suppression of a user. A plurality of content items is displayed on a device, including a first content item displayed at a first region. In an example, based on eye tracking data of the user, the onset of a saccade of the user is detected. An end location and duration of the saccade may be predicted based at least in part on the eye tracking data. A saccadic suppression window is identified based on the predicted duration. Based at least in part on the predicted end location, the display of the first content item is modified at a time that is within the saccadic suppression window.

This invention relates to the field of human-computer interaction and interface optimization, and discloses a method for optimizing the visual hierarchy of an interface based on eye tracking. The method includes: acquiring a set of interface elements of the interface to be optimized, and constructing a target visual hierarchy map based on a preset interaction task flow, the functional type of each interface element, and its information priority; collecting eye tracking data during the user's execution of a preset interaction task, the eye tracking data including first fixation time, cumulative fixation duration, number of regressions, and saccade shift relationships; determining the perception priority parameters of each interface element based on the eye tracking data, and constructing an actual perception hierarchy map; comparing the actual perception hierarchy map with the target visual hierarchy map to identify hierarchy deviation element groups and their corresponding deviation types; and calling the corresponding visual expression parameter reconstruction rules according to the deviation type to locally reconstruct the hierarchy deviation element groups while maintaining the preset interface structure constraints.

The invention relates to the technical field of auxiliary dementia screening, in particular to an early dementia screening system and method based on an eye movement tracking technology. The method comprises the following steps: matching a material word sequence with a voice word sequence, and selecting a matched segmented word of each segmented word in the material word sequence from the voice word sequence; according to the distribution density and the track confusion degree of the fixation points of the to-be-tested patient in the pronunciation time period of the matched segmented word of each segmented word of the material word sequence, acquiring a voice glancing difference degree, and adjusting the attention degree of the fixation points of the to-be-tested patient in different positions of the test material in the pronunciation time period of the matched segmented word of the segmented word of the material word sequence by using the voice glancing difference degree. And obtaining a voice fusion thermodynamic diagram, and performing early dementia screening. According to the method, the voice data generated by reading behavior monitoring are considered, the thermodynamic diagram fused with the voice signals is obtained, and the accuracy of dementia screening is effectively improved.

The invention provides a gaze stability evaluation system and method, and the method comprises the steps: extracting an effective gaze point sequence of a testee from an eye movement signal, carrying out the spatial-temporal feature analysis of the effective gaze point sequence, and obtaining the spatial distribution features and time fluctuation features of gaze points of the testee in a gaze process; determining a static fixation deviation coefficient when the tested person executes the specified visual fixation task based on the spatial distribution characteristics and the fixation target reference position; according to the time fluctuation characteristics, micro-eye jump characteristics in the eye movement process of the testee are determined, and then the visual gazing dithering sensitivity of the testee in the dynamic interference environment is determined according to the micro-eye jump characteristics and the influence degree of illumination changes on eye movement signals; and obtaining a gaze stability evaluation result of the testee based on the static gaze deviation coefficient and the jitter sensitivity of the visual gaze. By adopting the scheme of the invention, the application range of gaze stability evaluation can be expanded in a dynamic interference environment based on the combined judgment of the static and dynamic dimensions in the gaze process.

The invention provides a composite frequency sound wave intervention method for ciliary muscle training, which comprises the following steps of: acquiring eye movement data through a portable eye tracker or AR / VR (Auto Regression / Virtual Reality) equipment, and extracting multi-dimensional characteristics such as fixation point transfer, glancing speed, residence time and pupil dynamics by utilizing a deep learning algorithm; after filtering and normalization processing, the light-weight LSTM model is input for future eye movement trend prediction, a prediction result is used for dynamically adjusting sound source distribution and an enhancement region in spatial audio rendering so as to match a user eye movement track in real time, and meanwhile, sound source diffusion parameter optimization is performed in combination with LSTM confidence, so that the user eye movement trend prediction accuracy is improved. And model parameters are adaptively adjusted according to real-time behavior feedback, collaborative optimization of guidance and behaviors is realized, and energy consumption management and personalized training mode switching under low electric quantity are realized, so that user experience and training efficiency are improved.

In various examples, a stitched image may be generated from overlapping image frames using a dynamic seam placement that depends on scene content and / or other factors. Since an optimized seam placement may jump from a previous location from time slice to time slice, one or more constraints may be applied to limit the movement of dynamically placed seams such that any given seam moves gradually over time, limiting potential discontinuities in a visualization of the stitched image on a display. Eye tracking may be used to detect a saccade of a monitored person and / or detect that the monitored person is not looking at the display, and one or more of the constraints used to limit the movement of dynamically placed seams may be relaxed or lifted when the monitored person is experiencing a saccade and / or is looking away from the display.

An electronic device may have projectors, waveguides, and a bridge sensor. The projectors may generate calibration patterns in image light provided to the waveguides. The sensor may generate sensor data from the calibration patterns that is used to detect and correct for optical misalignment. A gaze tracking sensor may generate gaze information using transmitted and reflected infrared light. The gaze information may include a gaze direction, a blind spot location, blink information, and / or saccade information. The projector may place the calibration pattern at a location in a field of view of the image light that is separated from the gaze direction, may place the calibration pattern at the blind spot location, may synchronize the calibration pattern with a blink, and / or may synchronize the calibration pattern with a saccade. This may help to hide the calibration pattern from being viewable or perceivable by a user.

A method for assessing cranial nerve function, the method includes presenting a stimulus to a patient, capturing position data from at least one eye of the subject using an eye tracker having one or more sensors, filtering the data to remove noise, detecting one or more saccades from the position data, computing at least one saccade summary metric, computing at least one saccade directional metric, and predicting a probability of a concussion based on the at least one saccade summary metric or the at least one saccade directional metric.

This invention relates to the field of visual monitoring, and more particularly to a method and system for real-time monitoring of student attention in a classroom setting. The method includes: performing high-frequency frame extraction and binocular region cropping on a student's facial video stream to obtain a binocular sub-image sequence; analyzing the visual-motor features of the binocular image sequence to obtain binocular pupil data; decoupling the binocular pupil data based on a preset saccade judgment threshold to obtain rapid saccade events and micro-saccade events, and statistically analyzing the frequencies of the two types of events to obtain rapid saccade frequency and micro-saccade frequency; evaluating the directional divergence of micro-saccade events to obtain micro-saccade directional entropy, and analyzing the temporal interval dispersion of rapid saccade events to obtain temporal interval variation coefficient; mapping the micro-saccade directional entropy and temporal interval variation coefficient in a state space to obtain an attention state label; comprehensively evaluating the attention state label, rapid saccade frequency, and micro-saccade frequency to obtain an attention level, and generating latent distraction intervention suggestions based on the attention level.

This technology makes it easy to evaluate the skill level of workers. [Solution] The evaluation device includes a control unit. The control unit acquires feature quantities indicating eye movements based on information detected by a sensor that acquires information on the eye movements of a person to be evaluated while performing a predetermined task, and evaluates the quality of the person's predetermined task based on the acquired feature quantities. The feature quantities include at least one of the following: drift, microsaccades, saccades, smooth tracking eye movements, convergence / divergence eye movements, and blinking.

The invention discloses a fatigue monitoring method based on eye movement and blinking behaviors. The fatigue monitoring method comprises the following steps: step 1, carrying out eye image acquisition by adopting glasses type equipment; step 2, based on the collected eye image, performing eye movement tracking by introducing eye image quality enhancement and helmet slip compensation; 3, detecting eye scanning features based on the fixation point position obtained through eye movement tracking in the step 2, and representing the eye scanning features by adopting the eye scanning speed; step 4, detecting blink features of the user based on the eyelid ROI image obtained through quality enhancement in the step 2; and step 5, judging the fatigue state of the user based on the detection results of the step 3 and the step 4. According to the method, the problem that the monitoring accuracy is insufficient due to the fact that various blink characteristic values calculated by an existing eye movement monitoring mode are easily interfered by factors such as individual differences and environment illumination is solved, and the problem that the fatigue state of a user with natural small eyes is easily misreported is solved.

The invention relates to the field of interface design, and particularly discloses a human factor verification method and device for a vehicle machine interface and a vehicle, and the method comprises the steps: determining a target interface region corresponding to a target button in a to-be-evaluated interface; obtaining eye movement fixation data of the user during driving in different scenes, wherein the eye movement fixation data comprises at least one of first fixation time, the number of fixation points in the target interface area, the maximum duration of single-time road deviation and the number of times of target interface area glancing; based on the eye movement fixation data, scene interface verification values corresponding to the to-be-evaluated interface in different scenes are determined; and determining a comprehensive interface verification value of the to-be-evaluated interface based on the scene interface verification values corresponding to the to-be-evaluated interface in different scenes. By obtaining the multi-dimensional eye movement fixation data of the user and combining the scoring results in different scenes, the user experience of the vehicle-mounted terminal interface can be effectively quantified, and a basis is provided for interface optimization.

This invention relates to the field of smart glasses technology, specifically disclosing a smart glasses system with micro-saccade detection functionality. The system includes an eye-tracking image acquisition module, a data processing and transmission module, an eye-tracking feature extraction module, a micro-saccade detection and analysis module, an application response and interaction module, and a hardware integration module. The eye-tracking image acquisition module simultaneously captures high-frame-rate eye images using a near-infrared light source and a camera. The data processing and transmission module performs image noise reduction and distortion correction, transmitting the images via a dual-mode link. The eye-tracking feature extraction module locates the pupil center and compensates for head movements, generating feature vectors containing position and velocity. The micro-saccade detection and analysis module identifies real micro-saccades and quantifies parameters using dual-threshold screening and a machine learning model. This invention enables wearable and real-time micro-saccade detection, breaking through the limitations of traditional devices and balancing portability and detection accuracy. It can be used in applications such as attention monitoring and disease detection.

This invention discloses a multi-dimensional perceptual gaze-point rendering method, system, and medium. The multi-dimensional perceptual gaze-point rendering method is characterized by the following steps: S1, acquiring the user's eye-tracking data in real-time within a virtual reality environment; S2, dividing the rendering sequence into saccade windows, recovery windows, and stable gaze windows; S3, constructing a multi-dimensional visual perception model that integrates saccade sequence features; S4, acquiring the perception critical frequency and mapping it to a resolution scaling factor to guide rasterization operations in the graphics pipeline; S5, invoking the resolution scaling factor and performing dynamic resolution rendering in the graphics rendering pipeline based on the resolution scaling factor. The system includes a data acquisition module, an event detection module, a model calculation module, a rendering scheduling module, and a graphics drawing module. This invention is rationally designed, effectively utilizing computing resources, improving the rendering system's frame rate, and enhancing the user experience.

This invention relates to an eye-tracking event classification method and apparatus based on time-domain-frequency domain fusion. The method includes: constructing an eye-tracking event classification model comprising a frequency domain learner, a stacked layer, and a classification layer; training the frequency domain learner on frequency domain features, training the stacked layer on time domain features, and training the classification layer on classification; determining whether the cross-entropy loss has reached its minimum; if not, adjusting the model parameters using the Adam optimizer and returning to training; if so, saving the model with the best classification performance. This invention, by introducing a frequency domain learner, can extract potential spectral features from eye-tracking signals, effectively suppressing noise interference and improving the model's stability and robustness in complex experimental scenarios. Simultaneously, by fusing time-domain multivariate dynamic features with frequency domain information and introducing a self-weighted attention mechanism to dynamically weight different types of features, the model can more accurately distinguish eye-tracking event types such as gaze, saccades, and smooth tracking.

Navigation of a virtual environment (VE) can mean navigating a VE that is spatially larger than the available Physical Tracked Space (PTS). Accordingly, the concept of redirected walking was introduced in order to provide a more natural way of navigating a VE, albeit with many restrictions on the shape and size of the physical and virtual spaces. However, prior art techniques have limitations such as requiring eye-tracking increasing costs and complexity of the user's head mounted display system or triggering major saccades in the user. Accordingly, the inventors have established a novel technique which overcomes these limitations. The technique is based on a machine learning / artificial intelligence system that predicts saccades such that the psychological phenomenon of inattentional blindness can be applied for re-directed walking without requiring the triggering major saccades in the users, complex expensive systems, etc.