256 results about "Fixation point" patented technology

According to an aspect of the present invention, there is provided an electrical audio signal processing system and device, comprising: a computer graphics processing and selective visual display system with a screen; an eye tracking device; a processor; one or more computer memory devices; wherein the processor is arranged for operations comprising: measuring the user's eye movements to ascertain the specific word on which the user is fixated, by the eye tracking device; modifying the display at the user's current fixation point; applying a delay between the presentation of successive graphic elements based on the user's calculated rate to accommodate the user's required time; and presenting elements to the user at a rate based upon the user's required time.

The invention relates to the technical field of data processing, and particularly provides an operation intention recognition method, system and device based on multi-mode fusion and a medium, and the method comprises the steps: synchronously collecting interaction data of at least two modes of a user, the modes comprising at least two of gestures, voice and eye gaze; carrying out alignment processing on the interaction data, wherein the alignment processing comprises time synchronization and space mapping to a unified coordinate system; recognizing structured semantic information from each piece of aligned modal data, wherein the structured semantic information comprises a gesture type, a voice text and a fixation point coordinate; and based on a preset semantic rule and context memory, performing semantic association and anaphora resolution on the structured semantic information to obtain an operation intention. The method effectively overcomes the inherent defects of unnatural single-mode interaction, easy ambiguity and poor fault tolerance.

The invention provides a liquid crystal display drive control method and system, and the method comprises the steps: obtaining a page loaded in a memory of a liquid crystal computer, calculating the maximum image motion degree of the page loaded in the memory of the liquid crystal computer through the difference between continuous frames, and adjusting the refresh rate of a liquid crystal display screen; acquiring a fixation point position and fixation time of the user through an eye movement tracking technology, and adjusting the brightness of the liquid crystal display screen according to the fixation point position and the fixation time of the user; predicting future display contents and user behaviors by using a machine learning algorithm in combination with the use habits of the user and historical data of the display, and adjusting the brightness and refresh rate of the display based on a prediction result; the energy consumption data of the display are recorded in real time, the actual energy consumption and the expected energy consumption are compared and analyzed, and abnormity is found in time and feedback adjustment is carried out. Through the method and the corresponding system, the use energy consumption of the liquid crystal screen can be saved under the condition that the user experience is not influenced.

A gonioscopic attachment can attach to a gonioscope and can include atraumatic retention elements that are configured to engage an eye to retain the gonioscope relative to the eye. The gonioscopic attachment can be configured to position the gonioscope so that at least a portion of the concave distal surface of the gonioscopic optical element is spaced apart from the eye, and / or so that the curvature of the distal surface is angularly offset from the curvature of the eye. The gonioscope can include a fixation point configured to be visible to the subject when the gonioscope is positioned on the eye. In some embodiments, the retention elements can be incorporated directly into the gonioscope. A coupling mechanism can couple the gonioscope to a lid speculum. The gonioscope can include a light pipe, which can be configured to directly light into the eye.

The invention relates to the field of electronic photo frames, and discloses an electronic photo frame intelligent interaction control method and system based on eye movement tracking, and the method comprises the steps: carrying out the real-time tracking of an eyeball position based on the eye image data of a user, and obtaining an eye movement track sequence according to the pupil center coordinate change and the eyelid opening and closing state change; calculating a fixation point coordinate, a fixation time length and an eye hopping frequency based on the eye movement track sequence; the current eye movement behavior type of the user is obtained through a personalized recognition model; matching with a preset eye movement pattern library based on the current eye movement behavior type of the user to obtain a candidate instruction set; and executing photo frame function operation according to the control instruction. The method has the following effect that the convenience and accuracy of man-machine interaction are improved based on non-contact intelligent control of eye movement.

The invention belongs to the field of eye movement tracking, and particularly relates to an eye movement tracking system, method and equipment integrating electroencephalogram signals and a non-contact eye tracker. The objective of the invention is to solve the problems of limited acquisition range of a non-contact eye tracker and low spatial resolution of eye movement behavior deduction by EEG signals in the prior art. The method comprises the steps that eye movement data and electroencephalogram signals are synchronously collected, after calibration mapping and time sequence alignment preprocessing are conducted, an electroencephalogram modal behavior recognition model is supervised and trained through eye movement fixation point labels, and the incidence relation of the two modals is established. A dynamic fusion strategy is adopted in the online stage, when eye movement data are valid, a model prediction result and the eye movement data are fused to output a fixation point, when eye movement fails, the fixation point is calculated in combination with model prediction and historical data, enhanced compensation of eye movement tracking by electroencephalogram signals is achieved, the defect that the spatial resolution of a single EEG signal is low is effectively overcome, and the accuracy of eye movement tracking is improved. And the robustness and the tracking precision of the system in a complex scene are improved.

The invention discloses an intelligent glasses image adjusting system based on eye movement tracking and gesture fusion, and relates to the technical field of intelligent equipment. A multi-modal sensing module is arranged to construct a multi-modal sensing layer to capture eyeball movement tracks and gesture actions; a fixation point prediction module is set to process a dynamic scene through a space-time attention mechanism to obtain a fixation point prediction area, a gesture semantic understanding module is set to process gesture actions based on a Transform architecture, and the gesture actions of a user are converted into image adjustment instructions. An image enhancement strategy setting module designs a multi-stage image enhancement strategy according to the fixation point prediction area and the image adjustment instruction, and sets a dynamic adjustment intensity control module to perform adaptive adjustment to obtain a dynamic adjustment intensity control result; an eye movement-gesture cooperative control module is arranged to provide an eye movement-gesture cooperative control mechanism to realize image area selection and parameter adjustment, and accurate image area selection and parameter adjustment are realized.

The invention discloses an AR glasses control system and method, and belongs to the field of artificial intelligence and AR glasses application, and the method comprises the steps: capturing a user face image, combining a voice assistant, collecting user voiceprint data, and carrying out the matching through a recurrent neural network; constructing a three-dimensional environment model, positioning and marking obstacles and interactive objects, and identifying objects in the environment; the method comprises the following steps of: tracking a gesture, acquiring a key point coordinate by adopting an OpenPose algorithm, identifying a predefined gesture by utilizing a support vector machine, and mapping the gesture into a specific operation instruction; analyzing the user intention based on natural language processing, and calling a corresponding function module according to an analysis result; acquiring a gazing direction of a user by using an eye movement tracking sensor, calculating a screen focus position through an algorithm, and dynamically highlighting options or displaying information according to a gazing point position; through multi-modal data fusion, gestures, voices and fixation point input are analyzed in a unified manner. By accurately understanding the intention of the user, the interaction precision of the AR glasses is improved, and the user experience is optimized.

The invention discloses a virtual reality interaction method and system applied to classical famous picture display, and particularly relates to the technical field of virtual reality interaction. Multi-dimensional high-definition image acquisition and composition element three-dimensional modeling are performed on a painting, a cultural context label system is constructed in combination with historical data, a fixation point, an action posture and a staying behavior of a user are perceived in real time, accurate matching of a user behavior and a semantic label is realized, and multi-modal interaction feedback content consistent with the style of the painting is generated, so that the user experience is improved. And typical paths and misunderstanding areas are identified through clustering analysis of multi-user behavior data, and context tags and response logic are dynamically optimized, so that personalized recommendation and semantic guidance are realized, the understanding depth and immersion experience of the users on classical art are effectively improved, and the cultural expression ability and intelligent adaptability of the system are enhanced.

The invention discloses a first view angle video generation method and device based on a mask diffusion model and fixation point constraint, and the method comprises the steps: constructing an end-to-end deep learning frame for the demands of video filling, prediction and visual attention region control in the generation of a first view angle video; the diversified first-view-angle video generation conforming to the visual logic is realized. The method comprises the following steps: firstly, dividing an input video into a condition frame set and an unknown frame set, and adding random noise to an unknown frame by using a dynamic mask module I to generate a noisized frame; performing reverse de-noising generation on the video through a full 3D convolutional neural network module II, and guiding the network to generate video content conforming to a space-time law by taking a diffusion step and a fixation point track as conditional constraints; and finally, saliency prediction is carried out by using a fixation point positioning module III. By jointly optimizing the loss of the reverse denoising generation process and the loss of the fixation point probability graph, the reasonability and diversity of the generated video are further improved. According to the method, through joint optimization of the mask diffusion strategy and fixation point constraint, the space-time continuity, the anti-noise capability and the adaptability to the fixation point trajectory of the generated video can be effectively improved, and the method is particularly suitable for a complex multi-face expression interaction scene.

The invention relates to the technical field of cognitive impairment image processing, in particular to an Alzheimer's disease early screening method which comprises the following steps: extracting time and space coordinates of a fixation point from a visual glancing task participated by an Alzheimer's disease patient, recording direction change of a jumping behavior in the time and space coordinates, and calculating the direction change of the jumping behavior in the time and space coordinates; and generating jump behavior distribution data. According to the method, by extracting the time and space coordinates of the fixation point and combining the direction change of the jump behavior, comprehensive description of the dynamic law of the fixation behavior is realized, and high-dimensional support is provided for fine analysis of the fixation behavior of a patient in the task execution process. By constructing jump event distribution, track dynamic change and complexity quantification of the gazing path, the offset characteristic, the jump rule and the path fractal dimension in the gazing behavior are accurately revealed, and quantitative description of the abnormal behavior is provided for screening.

The invention discloses a cognitive impairment screening method and device based on a VR eye tracker and near-infrared brain imaging. The device comprises a virtual reality scene generation module, an eye tracker data acquisition module, a near-infrared brain imaging module, a data evaluation processing module and a cognitive impairment screening module. The eye tracker data acquisition module is used for acquiring an eye movement data information set of a user in a virtual reality scene in real time; the eye movement data information set comprises a fixation point information sequence, a pupil area information sequence and an eye hopping frequency information sequence; the near-infrared brain imaging module is used for acquiring a blood oxygen information set of cerebral cortex when a user completes a cognitive task; the blood oxygen information set of the cerebral cortex comprises a brain blood oxygen information sequence; the data evaluation processing module is used for carrying out evaluation processing on the collected eye movement data information set and the blood oxygen information set of cerebral cortex to obtain cognitive disorder information of the user; the cognitive impairment information comprises information about whether cognitive impairment exists or not and the degree of the cognitive impairment.

The invention provides an on-line examination anti-cheating method and system based on AI fusion sight tracking, and relates to the technical field of on-line education safety, and the method comprises the steps: extracting a head motion feature sequence of an examinee, calculating the coordinates and duration of a sight fixation point, analyzing the fixation behavior of a functional region, and recognizing a periodic jump mode in a fixation migration feature map. The gazing mode mutation point is identified by combining autoregression spectrum analysis and wavelet analysis, so that the cheating behavior is accurately identified, the safety and fairness of the online examination are effectively improved, and the misjudgment rate is reduced.

The invention discloses a Parkinson's disease assessment method and system based on face video multi-modal physiological feature fusion, and relates to multiple technical fields of computer vision, physiological signal processing and the like, and the method comprises the following steps: S1, based on a continuous face video stream, extracting rPPG signals; then heart rate variability key parameters are calculated, and heart rate variability characteristics are obtained; s2, analyzing the dynamic change of an eye fixation point in the face video based on IPAST, extracting key eye movement behavior parameters, and obtaining eye movement behavior characteristics through a convolution gating loop unit; and S3, inputting the heart rate variability characteristics and the eye movement behavior characteristics into a multi-modal fusion network structure, and outputting a continuous risk score or an illness state label for assisting a doctor in early Parkinson risk assessment. According to the method, multiple physiological signals acquired through videos are deeply integrated, a multi-modal feature collaborative analysis framework is constructed, the subjective limitation of traditional scale evaluation is broken through, and the one-sidedness defect of single biomarker detection is overcome.

The invention relates to the technical field of large model cheating detection, in particular to an online interview large model cheating detection system and method based on eye movement tracking, and the detection method comprises the steps: S1, system initialization and environment adaptation; s2, eye movement tracking calibration and baseline establishment; s3, collecting and preprocessing real-time eye movement data; s4, executing an anomaly detection algorithm; and S5, carrying out risk assessment and outputting a result. According to the scheme, based on the WebGazer.js technology, the mapping relation between the fixation point and the screen coordinates is established through nine-point initialization calibration, and the personalized coefficient k is generated in combination with the pupil diameter and the facial features of the candidate, so that the eye movement tracking error is greatly reduced, and the large model cheating detection precision requirement can be met without special hardware. The design not only reduces the deployment cost, but also breaks through the hardware limitation, so that the technology can adapt to mainstream notebook equipment, is convenient for large-scale popularization and application, and solves the contradiction that high precision and low cost cannot be achieved at the same time in the traditional scheme.

The invention discloses an attention monitoring method, device and equipment based on multi-modal sensing and a medium, and relates to the technical field of attention monitoring. The method comprises the following steps: collecting eye movement image data of eyeballs of a user; extracting fixation point stability features, eye movement track features, blink mode features and pupil dynamic features based on the eye movement image data; determining a concentration score of the user based on the fixation point stability feature, the blink mode feature, the pupil dynamic feature and the eye movement track feature; and if the concentration score is lower than a preset threshold value, sending reminding information to a user through the wearable device. The attention distraction behavior of the user in a continuous high attention demand scene can be accurately identified and prompted in time.

The invention discloses an English classroom attention regulation and control method combined with brain wave detection, and relates to the technical field of English classroom attention regulation and control, and the method comprises the steps: obtaining a multi-channel brain wave signal of a student in English classroom learning through a brain wave collection device, collecting visual gazing track data through an eye tracker, and obtaining a brain wave signal; capturing a face image sequence through a camera; performing frequency band separation on the multi-channel electroencephalogram signal, extracting energy of a theta wave and a beta wave of a prefrontal lobe, and calculating a theta / beta power ratio; performing de-noising and clustering processing on the visual gaze track data, identifying an effective gaze interval, calculating a reciprocal of a gaze point distribution variance in unit time, generating a visual focusing stability parameter, performing action unit identification on a facial image sequence, extracting smile frequency, eyebrow intensity and blink period characteristics, and outputting a cognitive emotion state score; and constructing a multi-modal attention fusion evaluation model based on the theta / beta power ratio, the visual focusing stability parameter and the cognitive emotion state score.

The invention provides a display method, device and system for preventing and / or delaying myopia occurrence and / or development of a user in need, electronic equipment, a storage medium and a computer program product, and relates to the technical field of optics. The system provided by the invention combines an eye movement tracker, a display device, a controller and a dynamic fuzzy filter, and prevents or slows down the occurrence and development of myopia by accurately tracking the fixation point of a user, utilizing a camera module as an optical sensor and selectively blurring the peripheral visual content. The system is intended to eliminate optical signals that stimulate the increase in myopia degree, thereby promoting a healthier viewing habit.

The invention relates to the technical field of intelligent visual detection, in particular to visual field detection equipment, system and method based on extended reality (XR), and the method comprises the steps that an image feature extraction module collects image data in real time, extracts features and eliminates optical distortion; the fixation deviation correction module analyzes eye movement data in real time, quantifies fixation deviation and triggers visual prompt and correction; the adaptive stimulation generation module dynamically encrypts to generate a test point image and adaptively adjusts the test point image according to visual field analysis and test requirements; the dynamic adjustment image module predicts and optimizes a test point presentation sequence according to image features and fixation points, automatically encrypts a high-probability defect area and quickly jumps to a key screening area; and the visual field defect identification module extracts image features, calculates the defect probability of each region in real time by using a Bayesian network in combination with subject response data, and locates a high-risk region. The system provides an efficient and accurate visual field detection scheme through the synergistic effect of multiple modules.

The invention discloses an image display method for myopia prevention and control and a related product thereof. The image display method for myopia prevention and control comprises the following steps: determining a sight fixation point when a user watches an electronic display screen; determining an out-of-focus fuzzy display area on the electronic display screen according to the sight fixation point; and performing blurring processing for simulating a myopia defocusing effect on an image positioned on the defocusing blurred display area so as to obtain an image with the myopia defocusing effect on the retina of the user. By utilizing the method disclosed by the invention, the myopia prevention and control effect can be obtained under the condition that the myopia patient does not wear the frame glasses with the myopia defocus effect, so that the myopia prevention and control cost of the user is reduced.

The invention relates to the technical field of the Internet of Things, and discloses a safe eye protection method and system for realizing a free screen based on the Internet of Things, and the method comprises the steps: detecting a human face region of a target user under a screen display picture, calculating the region area and the head posture in the human face region, and calculating a fixation point in the human face region through the head posture; recognizing the blinking frequency of the face area, and calculating the face-screen display distance between the face area and the free screen; analyzing the visual fatigue of the target user under the screen display picture, and determining the eye state of the target user under the screen display picture; and when the eye state is abnormal, identifying an external environment and screen display characteristics of the screen display picture, extracting environment characteristics of the external environment, mining influence factors influencing the eye state based on the screen display characteristics and the environment characteristics, adjusting picture parameters of the free screen according to the influence factors and the eye state, and generating a safe eye protection report of the free screen. The safe eye protection effect of the free screen can be improved.

The invention relates to the technical field of image data processing, in particular to a method for improving the detail display quality of VR equipment by using a binocular difference mechanism. The method comprises the following steps: acquiring a binocular image of a current frame of VR equipment and a user watching behavior feature; the user gazing behavior characteristics comprise a gazing area and a gazing point; according to the disparity map of the binocular image, determining the disparity significance of the fixation area; determining an initial brightness interval and a binocular brightness difference degree according to a brightness order of a corresponding point of the fixation point in the binocular image; determining a brightness order adjustment weight according to the parallax saliency, the binocular brightness difference degree and the user watching behavior characteristics; adjusting the initial brightness interval according to the brightness order adjusting weight to obtain an adjusted brightness interval; and adjusting the brightness of the binocular image according to the adjusted brightness interval, and obtaining a binocular image currently output by the VR equipment based on the adjusted binocular image. The problem of display delay is avoided, and the quality of display details is ensured.

The invention provides a machine learning and deep learning combined calibration-free eye movement tracking system and method, and the system comprises the following modules: a pre-calibration module which is used for collecting a near-to-eye image of the eyes of a user through a near-to-eye camera and an infrared light source, extracting pupil center coordinates and cornea reflection point coordinates, and outputting the pupil center coordinates and the cornea reflection point coordinates; calculating a personalized kappa angle of the user based on the extracted pupil center coordinates and cornea reflection point coordinates; the decision maker module adopts an AdaBoost ensemble learning algorithm, classifies the users according to the kappa angle obtained through calculation, and outputs a corresponding weight combination; and the deep learning module is used for receiving the output weight combination and the real-time near-eye image, and predicting the coordinates of the fixation point of the user through a convolutional neural network and a residual connection structure. According to the invention, the application flexibility and user friendliness of the eye movement tracking technology are greatly enhanced while the precision is guaranteed.

The invention relates to the field of intelligent perception, and mainly relates to a user interaction intention perception method and system based on intelligent glasses, and the method comprises the steps: obtaining the abnormal degree of an eye jump drop point according to the main eye jump and the subsequent correction eye jump times after detecting that the current eye jump drop point of a user enters an interaction region of an interactive object; acquiring a pupil diameter change sequence after the main eye jump, and acquiring a pupil transient response score based on the pupil diameter change sequence; after a certain time from the falling point time of the main eye jump, acquiring a real-time double-variable contour ellipse area based on information of a plurality of latest fixation points, and acquiring a fixation stability score based on the real-time double-variable contour ellipse area; obtaining a real-time gaze residence time according to the falling point time of the main eye jump, and obtaining a residence time evaluation score based on the real-time gaze residence time; fusing the abnormal degree, the pupil instantaneous response score, the stability score and the evaluation score to obtain a dynamic interaction intention score of the user, and determining the interaction intention of the user based on the intention score. The perception accuracy of the user interaction intention is improved.

The invention discloses a focus depth determination method and a related device, and relates to the technical field of rendering, and the method comprises the steps: collecting eye movement data of a user according to a preset eye movement sampling rate in a process that the user watches a three-dimensional scene, so as to obtain multi-frame eye movement data collected in a target duration, and determining a focus depth of the user according to the multi-frame eye movement data, determining a fixation point position of the sight line of the user in the target three-dimensional model, generating a sphere with the radius as a target value by taking the fixation point position as a sphere center, obtaining a target vertex set in the sphere in the target three-dimensional model, and performing vertex density analysis on the target vertex set to obtain a focus depth for rendering the three-dimensional scene. According to the method, the focus depth dynamically adjusted along with the sight line of the user can be generated, the detail resolution capability of the user on the fixation point is improved, and the watching experience of the user on the three-dimensional scene is further improved.

The invention discloses a Rhatish culture VRAR virtual-real fusion interaction method based on eye movement tracking, and particularly relates to the technical field of virtual reality and augmented reality man-machine interaction. The method comprises the following steps: acquiring an eye image of a user through a near-infrared image acquisition unit, and inputting the eye image into a double-branch convolutional neural network model to estimate a two-dimensional fixation point of a screen; performing nonlinear correction on a fixation point based on the user personalized sight line offset correction model established in the offline calibration stage; in the online interaction stage, the corrected fixation point is combined with equipment pose data to be converted into three-dimensional space rays, the distance and angle relation between the three-dimensional space rays and an interaction target is calculated, and an interaction instruction is triggered when a threshold condition is continuously met. According to the method and the system, accurate, stable and natural eye movement interaction of the virtual contents of the Rhatish culture and the entity exhibits in the VR / AR environment is realized, and the user experience and the system robustness are improved.

The invention provides an eye muscle derived compensatory head position auxiliary identification device based on instantaneous eye movement capture, and relates to the technical field of medical instruments, the eye muscle derived compensatory head position auxiliary identification device comprises a rotary video playing module, a skeletal muscle key point capture module, a capture module and a photo output module; the detected person watches the video, the video rotates in the reverse direction from the compensatory head position to the horizontal position, if the head position at the horizontal position is not improved, the video rotates towards the compensatory head position, in the rotating process, skeletal muscle key points capture the opposite sides of face and neck key points of the detected person and the head position is digitized, a capture module is started, photos are automatically captured, and eye position abnormity is analyzed through the photos; when eye position abnormity exists in the snapshot image, the snapshot image can be determined as an eye muscle source compensatory head position, and when the head position improvement is not obvious, one eye of the examiner is covered, and the other eye continues to watch the video. And the detected person watches the video, and after the video stops rotating, the head position and the fixation point are kept stable until the photo is automatically captured. When the head position of the snapshot image is improved, the head position can be clearly determined as the eye muscle-derived compensatory head position.

The invention provides a track acquisition method, device and system. The method comprises the steps of obtaining a three-dimensional space model of a space environment where the wearable device is located; then, acquiring an eye sight line direction vector when the user wears the equipment; and then, based on the sight line direction vector and the three-dimensional space model, determining a fixation point position of the user in the space, and generating a motion track of the user in the space. The movement path, the speed change, the head orientation and rotation and the like of the user can be analyzed through the movement track, so that more personalized and interactive experience is provided for deep analysis of the watching behavior of the user, and a new angle is provided for exploring visual behaviors.

The present invention belongs to the technical field of eye movement tracking, and discloses a near-eye fixation point tracking method, device and medium based on an event sensor, including: an infrared light source provides infrared illumination for the eye and the surrounding area of the eye, and the event sensor collects information on the eye and the surrounding area of the eye to obtain a series of event stream information generated by the change in light caused by movement; the time window clustering event set T is sent to the ROI and pupil calculation module to output pupil ellipse information; the event stream information is sent to the vector calculation module, and in combination with the pupil ellipse information, the movement vector of the pupil center is calculated; in combination with the movement vector, the pupil ellipse information is updated; in combination with the dynamic filtering method, the pupil ellipse information is smoothed; the filtered pupil ellipse information is sent to the line-of-sight regression and compensation model, and in combination with the deep learning method, the fixation point coordinates are calculated. The present invention introduces an event sensor and can more accurately identify moving targets in a dynamic scene.

The invention provides a fitting method of a 3D eye model and electronic equipment, and aims to improve the accuracy of the eye model. Comprising the following steps: in response to an eye model fitting instruction, obtaining eye images obtained when eyes of a user watch fixation points at different positions; obtaining eyeball parameters of the user by using the image position coordinates of the pupil center of the user in each eye image and the standard visual axis vector corresponding to each eye image; obtaining a current eyeball rotation angle based on the eyeball parameters of the user and the image position coordinates of the pupil center in the current eye image every specified duration; according to the current eyeball rotation angle and the eyeball parameters, current eyeball model parameters are obtained, and the eyeball model parameters comprise the position vector of the eyeball center, the position vector of the pupil center, the position vector of the curvature center and the visual axis vector; and updating a preset initial eye model by using the current eyeball model parameters of the user to obtain a current 3D eye model of the user.