59 results about "Saccade" patented technology

The invention relates to an eye movement tracking method and device based on adaptive filtering and eye movement equipment, and the method comprises the steps: obtaining an eye movement image of a to-be-tracked eyeball, and calculating a visual axis result of a current frame; performing feature comparison based on the eye movement image, and judging whether the eyeballs to be tracked move or not; when it is determined that the eyeballs to be tracked do not move, a visual axis result subjected to filtering processing is output; and when it is determined that the eyeballs to be tracked move, directly outputting a visual axis result. Through the eyeball tracking method and device, the state of the eyeball to be tracked can be determined through feature comparison, filtering processing is carried out according to the state of the eyeball to be tracked when fixation does not move, and the visual axis result is directly output when glancing moves, so that the problem that the requirements of sight calculation accuracy and quick response cannot be met is solved.

The invention relates to a novel reading disorder assessment and intelligent correction method based on EOG. The novel reading disorder assessment and intelligent correction method comprises the following steps: S1, respectively collecting eye movement signal data of a reading disorder child and a normal reading child in a reading process by adopting eye movement collection equipment; s2, filtering the original signal to generate an eye movement diagram, and analyzing transverse and longitudinal movement tracks and directions of eyes; s3, comparing the eye movement images of the reading disorder children and the normal children, and extracting key eye movement characteristic parameters of the reading disorder children and the normal children; s4, analyzing eye movement abnormal characteristics of frequent review of the individual; and S5, assessing the reading disorder through the reading behavior, and assessing the understanding degree of the text segment by using the text-related picture. The intelligent correction module is started to improve the reading effect. According to the method, the motion state of eyeballs, individual attention, visual processing speed, cognitive load and the like are reflected through the change of the electro-oculogram potential, a non-intrusive, real-time and quantitative evaluation and correction scheme is provided, and the evaluation and correction effect of the reading disorder can be remarkably improved.

The invention provides an eye movement trajectory analysis method and system based on hybrid clustering and time constraint, and belongs to the technical field of computer vision. Calculating a time difference and a moving speed between continuous original eye movement data points; comparing the moving speed with a speed threshold value, and classifying the moving speed into candidate fixation points and glancing points; extracting spatial features and time features of the candidate fixation points, performing standardization processing, and performing weighted fusion to obtain spatial-temporal feature vectors; clustering the spatio-temporal feature vectors to obtain preliminary clustering labels of the candidate fixation points; performing time constraint processing on each cluster; calculating the duration of the clustering cluster after the time constraint processing, and generating a final clustering label of the candidate fixation point; and outputting a final classification label of each original eye movement data point in combination with the classification result of the glancing points.

Disclosed herein are methods and systems for real-time detection of psychophysiological states from eye movement data. The methods involve capturing eye gaze vectors and eyelid openness levels over time using a user-facing camera. A sequence of discrete eye behaviors, including saccades, fixations, blinks, and long closures, is determined from the eye gaze vectors and eyelid openness levels. The sequence of discrete eye behaviors is transformed into a machine readable representation using a sliding time window. A mathematical or machine learning model then maps the machine readable representation of eye behaviors to one or more psychophysiological states. This approach provides a computationally efficient mechanism for predicting psychophysiological states by compressing gaze data into a continuous, numerical representation of eye behavioral events correlated with human psychophysiological states, including drowsiness, cognitive load, stress, and others.

The invention discloses an eye movement track analysis and classification method and system based on three-dimensional fixation estimation, and relates to the field of computer vision, and the method comprises the specific steps: obtaining a face video, estimating a head posture based on the face video, and extracting a plurality of face images; inputting the facial image into a multi-task learning model, extracting facial features and eye features in parallel, processing the fused features, and outputting an eye prediction state and a three-dimensional fixation vector estimation value; converting the three-dimensional fixation vector estimation value into a fixation point sequence under a screen coordinate system based on a relative pose relationship between the camera and the screen; clustering the fixation point sequence to obtain multi-dimensional eye movement features; and inputting the head posture and the multi-dimensional eye movement features into a fusion de-noising variational auto-encoder to obtain a classification result. According to the method, the text reading task is combined with the space-time distance function, and the extracted multi-dimensional eye movement features such as fixation, eye jump and review can accurately reflect the reading process of the subject, so that the classification accuracy is improved.

A method, system, and computer readable medium for switching between motion tracking mode. The method includes detecting a significant change in confidence level for motion tracking. A change from a first tracking mode to a second tracking mode is initiated when a significant change in confidence level is detected and the change is paused until a saccade or blink is detected. by eye tracking device. After a saccade or blink is detected the tracking mode switches from the first tracking mode to the second tracking mode.

A method, computer system, and a computer program product are provided for enabling a user to obtain additional information about a content. In one embodiment, the process starts by collecting, from at least one sensor, eye gazing data relating to eye movement of the user looking at a content. User's saccade rate is calculated based collected eye gazing data. It is determined when the saccade rate falls below a first value of the viewing content of a first area and the location is flagged as a first subset. It determined when the saccade rate shifts backwards over a second subset of the content one or more times and this area is flagged as a second area. The portion of the content located between the first and second flagged areas are emphasized. Explanatory tools are provided relating to content of emphasized portion.

The invention relates to the technical field of intelligent glasses, and particularly discloses an intelligent glasses system with a micro eye jump detection function, which comprises an eye movement image acquisition module, a data processing and transmission module, an eye movement feature extraction module, a micro eye jump detection and analysis module, an application response and interaction module and a hardware integration module. The eye movement image acquisition module synchronously captures a high-frame-rate eyeball image through a near-infrared light source and a camera, the data processing and transmission module carries out noise reduction and distortion correction on the image and transmits the image through a dual-mode link, and the eye movement feature extraction module locates a pupil center and compensates head movement to generate feature vectors including position, speed and the like. The micro-eye jump detection and analysis module identifies real micro-eye jumps and quantifies parameters through double-threshold screening and a machine learning model. Wearability and real-time performance of micro eye jump detection can be achieved, scene limitation of traditional equipment is broken through, portability and detection precision are both considered, and the method can be used for application scenes such as attention monitoring and disease detection.

The present invention discloses a method for detecting eye movement events based on Transformer, comprising the following steps: preprocessing the original gaze position time series; using a CNN network to extract features from the input sequence to obtain word vectors; performing positional encoding on the gaze position sequence to represent the positional information of the sequence; using Transformer to learn the global features of the eye movement sequence, and using a fully connected layer and Softmax to predict any moment of the input sequence as three basic event types: fixation, saccade, and post-saccadic oscillation; using event-level Cohen's Kappa to evaluate the performance. Compared with LSTM, Transformer can effectively establish long-term dependencies between observations at different moments on a long sequence and can achieve a high degree of parallelization in sequence processing. From the performance evaluation results, the present invention can achieve a good effect in detecting eye movement events.

A visual brain therapy device comprises a vertical panel, a horizontal panel, a plurality of lights and a controller. The vertical panel has an aperture, where the aperture has an upper edge and an opposite lower edge. The horizontal panel has a front end and an opposite back end, where the front end of the horizontal panel aligns with the lower edge of the aperture. Each of the plurality of lights is mounted on either the vertical panel or the horizontal panel. The controller is configured to activate each of the plurality of lights in a pattern, where the pattern comprises one of convergence, divergence, saccades and anti-saccades.

A gaze detection device and a setting processing method thereof, including an eye movement tracking module and a target detection module. The eye movement tracking module includes a sensing device that estimates and determines a gaze and a timing change of the gaze by sensing eye information, thereby distinguishing between a saccade state or a fixation state. The target detection module includes a plurality of target categories for setting at least one target category of interest. In the fixation state, the target detection module can detect a target. When the target detection module determines that a comparison image information matches the target category of interest, the image information is captured and stored for user review and reference.

The present invention discloses a method for detecting eye movement events based on multi-scale convolution, comprising the following steps: Step 1, preprocessing of the eye movement sequence; Step 2, using a UNet model to perform multi-scale feature extraction and feature fusion on the differential eye movement sequence; Step 3, using a recurrent neural network to simulate the eye movement event sequence; Step 4, using a linear fully connected layer and Softmax to classify the sample points at each moment in the eye movement sequence into fixation, saccade, and post-saccadic oscillation, so as to achieve eye movement event detection; Step 5, using event-level Cohen's Kappa to evaluate the performance of the three classified eye movement events. The method of the present invention solves the problem that the performance of the eye movement event detection method is limited by the fact that a convolutional neural network with a single-scale convolution kernel cannot effectively extract the features of small-sample events.

The present invention relates to a method and system for evaluating mental workload in a real environment based on eye movement data. The method comprises the following steps: respectively collecting eye movement data under different mental workload states, and preprocessing the eye movement data to construct an eye movement feature set; the eye movement feature set includes eye movement type statistical features, saccade speed statistical features, pupil diameter statistical features, and eye movement type transition features; establishing a mental workload evaluation model, training the mental workload evaluation model based on the eye movement feature set to obtain a trained mental workload evaluation model; collecting eye movement data of a user to be evaluated, preprocessing the eye movement data to construct an eye movement feature set of the user to be evaluated, and inputting the eye movement feature set into the trained mental workload evaluation model to obtain a mental workload evaluation result of the user to be evaluated.

The invention relates to the technical field of recruitment video analysis, and discloses a recruitment assisting method and system, and the method comprises the steps: obtaining continuous image frames and timestamps of an answering video through a fixed time interval, extracting the sight line direction of each frame through a convolutional neural network, and generating an azimuth angle sequence; then calculating azimuth angle changes of adjacent frames and positioning a peak value to obtain an occurrence moment and a main direction of a glancing event, and forming a glancing event time sequence; intercepting a final-section event in a final-section time window, and screening a target direction event set in combination with a direction range, an amplitude threshold and a time interval condition; mapping an event occurrence moment into a phase according to a preset period, and calculating a phase concentration ratio by adopting a circle statistical method to obtain a phase locking index; the phase locking index is corrected, and significance evaluation is carried out under the statistical condition; and finally, completing risk judgment according to the corrected phase locking index and the saliency result, and realizing quantitative detection and intelligent identification of the abnormal sight behavior in the online interview.

The invention relates to a visual field intensity modeling and parameter optimization method based on a directional glancing eye movement task, and the method comprises the steps: carrying out the parameterization design and data collection of the directional glancing task, constructing a visual field intensity model covering the three stages of central fixation, green spot glancing and red spot maintenance, carrying out the back-stepping of an individual visual field parameter based on maximum likelihood estimation, and carrying out the optimization of the visual field intensity. And performing distribution modeling and type division on multiple tested parameters, and optimizing task parameters through simulation and discrimination performance. According to the method, a unified space-time visual field intensity model is established, fine splitting of different cognitive mechanisms is realized, a task parameter optimization closed loop of simulation-backstepping-evaluation-screening is constructed, the distinguishing efficiency of different visual fields and execution control subtypes is remarkably improved, and the method has the advantages of being high in robustness and high in robustness. The method overcomes the defects of dependence on empirical parameters, insufficient cognitive interpretation and low crowd discrimination in the prior art, and can be used for quantitative evaluation and targeted task design of visual attention and eye movement control functions.

This invention provides a diagnostic method based on saccade analysis. The method includes: collecting eye movement displacement and time data of the subject to obtain a discrete eye movement velocity sequence; obtaining a denoised eye movement velocity sequence after denoising; selecting the velocity sequence to be processed using a moving window; determining the time length of the moving window based on the length of the velocity sequence and the calculation time requirements; determining an eye movement velocity threshold according to diagnostic needs; calculating local extrema of the velocity sequence within the moving window and determining the eye movement type and number of saccades based on the velocity threshold; calculating local extrema of the entire velocity sequence using this method to obtain the total number of saccades; diagnosing the subject's eye movement state based on the number of saccades and their time intervals, and determining whether detailed analysis of local data is needed to obtain a more accurate diagnostic result based on the diagnosis results. This invention can help doctors diagnose different types of saccades and their potential predictive conditions such as cognitive impairment and Parkinson's disease more quickly.

The invention provides an identity verification method and device, and relates to the technical field of data processing, and the method comprises the steps: obtaining the eye and head movement of a first user in each glancing time period to form a glancing feature set, and the eye and head movement in each staring time period to form a staring feature set in the process of inputting a password through the eye and head cooperative movement of the first user; the glancing feature set and the staring feature set are input into corresponding pre-trained random forest classifiers, output results of the random forest classifiers are summarized, a first authentication result is obtained through a voting mechanism, and the random forest classifiers are constructed according to the corresponding feature sets obtained when a target user registers; matching authentication is carried out according to a password indicated by the eye movement track and a password when the target user registers, and a second authentication result is obtained; and determining whether the identity verification of the first user is successful or not according to the first authentication result and the second authentication result. Therefore, the time cost can be reduced, and the user experience and safety can be improved.

The present application relates to the technical field of computer vision and cognitive rehabilitation, and particularly relates to a self-adaptive reading efficiency detection and training system based on multi-modal fusion, comprising: a multi-modal data acquisition module, which is used for acquiring eye movement coordinates (x, y, t), pupil diameter and head posture data of a user; a space-time trajectory reconstruction engine, which comprises an improved I-DT algorithm module based on signal-to-noise ratio dynamic weighting and Kalman filter prediction compensation, and is used for extracting fixation points from low-frequency sampling data and retaining micro-saccades; and an elastic grid mapping module, which is used for generating a Gaussian ridge line model according to text topological features of a reading interface.The present application reduces dependence on special hardware, is conducive to cost control, and can realize fixation trajectory restoration precision close to that of a scientific research level device (> 120Hz) on a 30Hz-60Hz consumer-level camera or a tablet computer front lens through the improved I-DT algorithm, so that professional reading disorder screening and cognitive training are suitable for deployment and use in primary medical institutions or home environments.

The invention discloses a crew man-machine interaction cognitive performance determination method, and relates to the field of crew man-machine interaction cognitive behavior modeling, and the method comprises the steps: a judgment and decision model loads a target task, forms a generation set, searches a generation corresponding to a maximum utility value for execution through the judgment and decision model, and updates a work memory model. The visual attention model is driven to operate, the eye movement preparation time and the eye movement glancing speed are obtained, and the visual attention performance is determined; the operation control model is driven to operate, the hardware environment of the display control equipment and the Fresnel's law parameters of the sailors are obtained, and the operation control performance is determined; updating the working memory model according to the execution result; updating the generative memory model according to the updated working memory model, and determining judgment and decision performance; summing the visual attention performance, the manipulation control performance and the judgment and decision performance, and determining the total cognitive performance of man-machine interaction of the crew. According to the method, the man-machine interaction cognitive performance of the crew can be accurately calculated in a conventional or long-voyage state.

This invention discloses a method and system for generating personalized text summaries that integrates eye movement data. The system collects eye movement data from users reading text on a screen and captures a text image interface. The system then analyzes the eye movement data and identifies the coordinates of individual characters in the text image. A gaze-to-word mapping method is then constructed. Word scores are then calculated based on eye movement fixations, saccades, and reflexes, and keywords are extracted based on the scores. After keyword extraction, the keywords are grouped into eye movement key information, encoded using a bidirectional long-short-term memory network, and an internal attention model is used to address duplicate words generated during the summary generation process. The system then inputs the keywords into a pointer generator network to generate a personalized text summary. The text summary generated by this method achieves significant personalization, scoring high in relevant evaluation metrics. It integrates user personalized information while reducing the number of manual steps required by the user.

This invention discloses a visual data analysis system and method for work scenarios, relating to the field of augmented reality technology. The method includes: monitoring the peak value, duration, and post-rotation stillness duration of head rotation angular velocity; freezing a semantic snapshot of the visual field and recording the rotation amplitude when corresponding thresholds are met; defining the effective depth-of-field range using the depth of a reference object on the work surface; determining global motion, gaze point drift, and reference object overlap during visual field regression; matching new frames and snapshots after all parameters meet stability thresholds; marking disappearance candidates based on the number of consecutive frames confirmed by disappearance; assigning confidence and outputting visual cues after excluding operation occlusion and out-of-field offset; and correcting near-end offset, far-end offset, number of consecutive frames confirmed by disappearance, and stability thresholds based on user response. This invention effectively distinguishes between brief saccades and actual gaze departures through the combined triggering of multiple parameters, including head rotation angular velocity, duration, and post-rotation stillness duration.

The present invention relates to the field of electronic photo frames and discloses a method and system for intelligent interactive control of electronic photo frames based on eye tracking. The method comprises: real-time tracking of the user's eyeball position based on eye image data, obtaining an eye movement trajectory sequence based on changes in pupil center coordinates and eyelid opening and closing; calculating gaze point coordinates, gaze duration, and saccade frequency based on the eye movement trajectory sequence; obtaining the user's current eye movement behavior type through a personalized recognition model; matching the user's current eye movement behavior type with a preset eye movement pattern library to obtain a set of candidate commands; and executing photo frame functions according to the control commands. This method has the following advantages: contactless intelligent control based on eye movements improves the convenience and accuracy of human-computer interaction.

The present invention discloses a cognitive risk assessment system, method, device and storage medium based on eye movement features, which relates to the field of cognitive risk assessment, including a learning phase of a visual paired comparison task, a reverse saccade task and a test phase of a visual paired comparison task presented in sequence on a mobile terminal. The above-mentioned task is a task paradigm specially designed to evaluate cognitive functions related to cognitive impairment, and the reverse saccade task is interspersed with the visual paired comparison task. The time to complete the reverse saccade task can be used as the memory time of the visual paired comparison task, thereby optimizing the task paradigm. Quantified eye movement features are determined based on the facial video of the user when completing the cognitive risk assessment task, and the quantitative eye movement features are analyzed using a machine learning model to determine the cognitive risk assessment results. The cognitive risk assessment results determined based on eye movement features are more objective, and cognitive risk assessment can be achieved using a mobile terminal, which is more efficient, convenient, low-cost and applicable to a large population.

The present invention relates to an eye movement behavior classification method based on a Gaussian mixture model and a hidden Markov model, and belongs to the field of human-computer interaction technology. The hardware system includes a desktop eye tracker, a host computer, and a display screen; the host computer has a Gaussian mixture model program and a hidden Markov model program; the eye movement behavior classification operation steps are as follows: (1) collecting the original eye movement data of the measured person; (2) preprocessing the original eye movement data, using a clustering method to group the original eye movement data into three or more groups of eye movement data; (3) establishing a Gaussian mixture model and a hidden Markov model; (4) based on the Gaussian mixture model and the hidden Markov model, classifying the eye movement behavior of the three or more preprocessed groups of eye movement data, and inferring whether the current eye movement behavior is fixation, saccade, or tracking eye movement behavior. The classification method of the present invention improves classification accuracy, reduces complexity, and has significant advantages in adapting to practical applications.

The application discloses an eye movement track generation method, device and equipment and a readable storage medium. The method comprises the following steps: acquiring line-of-sight landing point data collected based on an eye tracker and an experiment duration; performing denoising processing on the line-of-sight landing point data through a sampling window, and determining saccade data and fixation data from the data points obtained after the denoising processing; determining a first time factor through a start time, an end time and the experiment duration of a fixation point, and determining a second time factor through a duration and the experiment duration; and generating eye movement track information based on the position of the fixation point, the first time factor and the second time factor. The application adopts a new eye movement track representation method, which not only contains duration and fixation position information, but also increases more specific time point information, can comprehensively express dynamic changes of visual attention with time, and constructs a new eye movement track model based on a conditional generative adversarial network, so that the model can accurately predict eye movement tracks.

The application discloses a driving distraction state recognition method based on eye movement sequence space-time semantic feature analysis, and relates to the technical field of visual analysis, and comprises the following steps: S1, acquiring eye movement data of a to-be-identified subject during driving; S2, constructing a driving distraction recognition model based on eye movement sequence space-time semantic features; and S3, inputting the eye movement data of the to-be-identified subject into the driving distraction recognition model based on eye movement sequence space-time semantic features which is constructed, and obtaining a distraction detection result of the to-be-identified subject. The application designs a driving distraction recognition model based on eye movement sequence space-time semantic features, extracts time, space and semantic features in visual saccade paths, and combines the features to represent the eye movement mode of a driver, so that whether the driver is in a distraction state can be identified.

Systems and methods are disclosed for modifying content items based on eye tracking data, which may be indicative of saccadic suppression of a user. A plurality of content items is displayed on a device, including a first content item displayed at a first region. In an example, based on eye tracking data of the user, the onset of a saccade of the user is detected. An end location and duration of the saccade may be predicted based at least in part on the eye tracking data. A saccadic suppression window is identified based on the predicted duration. Based at least in part on the predicted end location, the display of the first content item is modified at a time that is within the saccadic suppression window.

A driver abnormality sign detection device includes a control circuit configured to detect an abnormality sign of a driver based on visual line information acquired from a visual line detector and on image data from a vehicle-outside camera. The control circuit is configured to acquire reference saccades of the driver, calculate dispersion of a section including a peak of saliency in a visual field of the driver based on the image data, calculate predicted saccades by correcting the reference saccades with correction values acquired based on the dispersion of the section including the peak of the saliency, and detect presence or absence of the abnormality sign of the driver based on a visual line abnormality degree representing an extent that measured saccades diverge from the predicted saccades.

The invention provides a dynamic three-dimensional point cloud model eye movement data acquisition method. The problems that in the eye movement data acquisition process of an existing three-dimensional model eye movement test scheme, the head position of a subject needs to be fixed, only a limited browsing view angle is provided, a 2D rendering graph is used for replacing a three-dimensional model, and vivid browsing experience is difficult to provide are solved. The method comprises the steps that Unity software is combined with VR equipment to construct a virtual test scene, and a dynamic three-dimensional point cloud model is displayed in the center of the virtual test scene; all subjects are divided into a plurality of teams, and each team starts from a preset starting point and stands towards a virtual test scene center so as to start to browse the dynamic three-dimensional point cloud model; allowing the subject to freely move in the browsing process; synchronously collecting eye movement data and head six-degree-of-freedom data of the subject through VR equipment, and calculating a sight intersection point of the sight of the subject and the dynamic three-dimensional point cloud model in real time; and distinguishing an eye movement fixation point and a glancing point from the sight line intersection point by adopting an I-VT method of a self-adaptive speed threshold.

Systems and techniques are described for extended reality (XR) operations. An XR system displays virtual content using a display according to display settings. The display settings can identify, for instance, a position, orientation, and / or size of the virtual content as displayed. The environment can be viewable using the display as the virtual content is displayed by the display, for example using a see-through display or a pass-through display. The imaging system can determine, based on one or more attributes of one or both eyes of the user of the imaging system, an extent of perception of the virtual content that is displayed using the display by the user. The attributes can identify, for instance, eye position, eye movement, pupil dilation, saccades, fixations, blinking, and / or squinting. The XR system can determine, based on the extent of perception of the virtual content by the user, a modification to the display settings.