60 results about "Ocular motor" patented technology

The invention relates to the technical field of cognitive impairment detection, and discloses a cognitive impairment early warning method and device based on an electroencephalogram micro state and an eye movement trajectory, and the method comprises the following steps: S1, data acquisition, S2, electroencephalogram preprocessing, S3, electroencephalogram feature extraction, S4, eye movement feature extraction, S5, feature fusion, and S6, early warning judgment. According to the method, through independent convolution branch of electroencephalogram and eye movement features, a receptive field is expanded by utilizing cavity convolution to capture multi-scale features, and long-distance dependence is modeled by a self-attention layer; after tensor splicing, time sequence information is dynamically fused through a gating cycle unit (GRU), and cross-modal time correlation is captured. The method has the advantages that the multi-modal feature hierarchical extraction and self-adaptive modeling capability is enhanced, the complementarity fusion efficiency is optimized, meanwhile, by means of cavity convolution sparse connection, self-attention parameter sharing and GRU lightweight design, the model complexity and the calculation efficiency are balanced, and efficient feature representation is provided for cognitive impairment early warning.

The invention discloses a psychological health monitoring and self-adaptive nursing decision-making system and method based on electroencephalogram signals, and relates to the field, and the method comprises the following steps: S1, collecting electroencephalogram signals and eye movement characteristics, and generating a multi-modal original signal data set; s2, electroencephalogram alpha wave asymmetry and eye movement characteristics are extracted, and delta-alpha spectrum overlapping pathological interference is eliminated; s3, inputting an evaluation model and a rule base, matching a DLB pathological marker and a symptom association rule, and marking a mental health risk level; and S4, matching an intervention strategy, and dynamically updating the evaluation model and the symptom association rule. The multi-modal fusion strategy effectively covers an association path of a DLB patient from molecular pathology to clinical symptoms, further provides a high-reliability data basis for subsequent risk level marking, realizes extraction of pure alpha wave energy and correction of prefrontal lobe alpha wave asymmetry aiming at frequency spectrum overlapping interference of delta waves and alpha waves in electroencephalogram signals, and improves the accuracy of risk level marking. Emotion abnormal degree misjudgment caused by false increase of the FAA value is avoided.

The invention relates to the technical field of mechanical arm control, in particular to a mechanical arm control method and system based on motor imagery and eye movement tracking. (2) constructing an electroencephalogram signal classification model and performing training; (3) calibrating an eye movement tracking module and a mechanical arm control module; (4) inputting data collected in real time into an electroencephalogram signal classification model and an eye movement tracking module, and generating a user intention and a to-be-grabbed target label; and (5) an instruction decision is made according to the intention of the user, the to-be-grabbed target label is input into a mechanical arm control module, a mechanical arm control instruction is generated, the mechanical arm control instruction is driven to execute the grabbing action, and the control process of the mechanical arm is achieved. According to the scheme, more efficient man-machine interaction is achieved, through combination of motor imagery electroencephalogram decoding and eye movement tracking, mechanical arm selection target grabbing operation can be conducted through brain control and eye movement without manual operation, and the real-time performance and stability of a control system are improved through optimization of the decoding process.

The invention discloses a concentration training method and system based on multi-modal fusion, and the method comprises the steps: collecting an electroencephalogram signal, an eye movement signal and a head movement signal through a wearable device in a concentration training process, and calculating an electroencephalogram frequency characteristic parameter, an eye movement parameter and a head movement modal parameter in a one-to-one correspondence manner; calculating a concentration score according to the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement modal parameters so as to evaluate the concentration; and adjusting a concentration training strategy according to a concentration evaluation result. According to the method, the electroencephalogram, eye movement and head movement signals are collected at the same time through the wearable device, the corresponding characteristic parameters are calculated, the concentration score is comprehensively evaluated, and a comprehensive, accurate and stable evaluation result is provided. On the basis of real-time evaluation, task difficulty and types are dynamically adjusted, challenging and interestingness are ensured, the user participation degree and the training effect are improved, and concentration research and technical progress are promoted.

The invention provides a multi-modal data acquisition and intelligent analysis application method based on AI glasses. The method comprises the steps that the AI glasses collect text data through a camera and an OCR assembly, collect image video data through a binocular camera and an infrared sensor, collect audio data through an array microphone, and collect auxiliary data through an I MU and an eye movement tracking sensor; according to the AI glasses, a local lightweight AI model is used for basic analysis, and complex tasks are uploaded to a cloud large model for cooperative processing, including information summarization, deep mining and knowledge extraction; the AI glasses realize real-time pushing, scene assistance and content generation and output through an information application module according to an analysis result; according to the AI glasses, an end-cloud collaboration module and a federated learning framework are adopted, a local cloud processing strategy is dynamically adjusted, only feature parameters are uploaded, and it is ensured that original data are locally reserved.

The application discloses a rehabilitation robot control method based on multi-source information perception and electronic equipment, and the method comprises the following steps: acquiring upper limb modal signals of a patient without motor function, wherein the upper limb modal signals comprise electroencephalogram signals, and eye movement signals and / or electromyogram signals; for each kind of upper limb modal signal, using an intention recognition model to perform motor intention recognition on the upper limb modal signal to obtain a corresponding motor intention recognition result; acquiring a confidence degree corresponding to each kind of upper limb modal signal; determining the motor intention of the patient according to the motor intention recognition result corresponding to each kind of upper limb modal signal and the corresponding confidence degree; determining a motor control signal of an upper limb rehabilitation exoskeleton according to the motor intention; and controlling the upper limb rehabilitation exoskeleton to move according to the motor control signal. The application can improve the motor intention recognition accuracy of the patient without motor function in the upper limb rehabilitation training process.

The embodiment of the invention relates to the technical field of eye movement behavior detection, and discloses a high-precision eye movement behavior detection method driven by time sequence and time frequency fusion, and the method comprises the steps: carrying out the short-time Fourier transform of an original eye movement data sequence of a collected to-be-detected eye movement behavior, converting the original eye movement data sequence of the time domain into time-frequency domain information; performing feature extraction on time sequence information in the original eye movement data sequence by adopting a plurality of parallel Mamba networks to obtain time sequence features, and performing feature extraction on time-frequency domain information by using a convolutional neural network to obtain time-frequency features; splicing and fusing the time sequence features and the time frequency features to obtain fused features; inputting the fusion features into a bidirectional gating cycle unit network for sequence modeling; and performing classification prediction on the output characteristics of the bidirectional gating loop unit network by using a full connection layer to obtain a detection result of the to-be-detected eye movement behavior. The method effectively improves the accuracy and robustness of eye movement behavior detection.

An apparatus for assessing the fine motor skills of a target user includes: a pressure sensing module configured to capture pressure data associated with pressure exerted by small muscles during the performance of the fine motor skill by the target user; a motion tracking module configured to capture motion data associated with movement of the small muscles during the performance of the fine motor skill by the target user; an eye-tracking module configured to capture gaze data associated with the gaze direction of the target user during the performance of the fine motor skill; and a performance evaluation module configured to determine, in real time, one or more metrics associated with the accuracy of the target user's fine motor skills based on the captured motion data, the captured gaze data, and the captured pressure data.

A method for ocular simulated camera assisted robot training is provided. In some implementations, the method includes initializing, by a processor, a robotics assembly. The method further includes connecting, by the processor, to one or more computing devices. The method further includes operating, by the processor, the robotics assembly. The method further includes simulating, by the processor, an eye movement of a human or animal. The method further includes operating, by the processor, a laser to perform a determined exercise on an eye of the robotics assembly. Related systems, methods, and articles of manufacture are also described.

The application belongs to the field of intelligent transportation, and discloses a vehicle lane-changing recognition method and system based on multi-modal data fusion. In the application, eye movement tracking and multi-modal fusion technology of vehicle kinematics are combined to analyze vehicle motion data, and the statistical characteristics of the vehicle motion data are combined to extract a feature vector A representing the driving habit and stability of the driver. Meanwhile, the eye movement features and the feature vector A are spliced to extract a multi-modal feature vector B focusing on capturing lane-changing. A group classification mechanism based on the experience of the driver is further proposed, the feature vectors A and B are fused, and then the fused features are input into a meta-fusion gate network. The dynamic weight is output through the gate network to realize the classification of the driver based on the experience, and the most suitable expert recognition model is automatically weighted for each driver at each driving moment, so that more accurate and personalized recognition of lane-changing is realized.

The invention discloses a fusion intention prediction method, device and system, a storage medium and a program product. The method comprises the steps that electroencephalogram signals and eye movement signals of a human body are collected; acquiring an eye movement sampling rate of the eye movement signal and comparing the eye movement sampling rate with a preset sampling rate threshold; inputting the electroencephalogram signals and the eye movement signals into a large intention prediction model, wherein the large intention prediction model performs intention prediction based on the electroencephalogram signals and the eye movement signals; outputting an intention prediction result; when the eye movement sampling rate is greater than or equal to a preset sampling rate threshold value, executing fusion learning of the intention prediction large model; and when the eye movement sampling rate is smaller than a preset sampling rate threshold, closing fusion learning of the intention prediction large model. The problems that a preset static model is adopted in an existing scheme, a dynamic environment cannot be processed, the intention prediction accuracy is low, and long-term use cannot be achieved are solved.

This invention discloses an essential oil-based intervention system for the simultaneous treatment of heart and brain function based on dual feedback of eye movement and heart rate variability, belonging to the field of rehabilitation medicine technology. The system includes: a data synchronization acquisition module, which acquires heart rate variability data and eye movement characteristic data and generates a synchronized physiological data stream; a dual feedback joint assessment module, which assesses the patient's real-time physical and mental state level based on the synchronized physiological data stream; a synchronized intervention decision module, which generates essential oil atomization control instructions and eye movement training control instructions in parallel according to the physical and mental state level; and a synchronized intervention execution module, which drives the essential oil atomization unit and the eye movement training unit to execute corresponding intervention actions synchronously at the hardware level on the time axis. This invention achieves integrated and precise intervention of heart and brain function through dual-modal physiological feedback closed loop and synchronized control, effectively solving the problems of brain separation, delayed intervention, and poor coordination in traditional rehabilitation centers, and improving the synchronous rehabilitation effect of cognitive function and autonomic nervous function in stroke patients.

The invention discloses a cross-modal space-time alignment method and system for electroencephalogram signals and eye movement data, and the method comprises the steps: converting the electroencephalogram signals and eye movement data of each channel into a recurrence graph RP; performing state transition probability calculation on the discretized time sequence in the electroencephalogram signal and the eye movement data by using a Markov transition field (MTF) and obtaining an MTF transition distribution diagram of the time sequence among different states, and mapping the normalized time sequence in the electroencephalogram signal and the eye movement data to a polar coordinate space by using a GAF; according to the method, the function of carrying out cross-modal space-time alignment on electroencephalogram signals and eye movement data is achieved, the dynamic similarity of a recurrence graph RP, the state transition characteristics of an MTF transition distribution graph and time sequence angle association of a GAF two-dimensional matrix graph can be captured at the same time in an output RGB image, and encoding and association analysis can be carried out on multi-source time sequence information; hysteresis of man-machine collaboration is avoided, the man-machine collaboration effect is improved, and the system is suitable for being widely popularized and used.

The present disclosure is related to a peripheral containing an eye tracking device, a controller, and a battery. The peripheral may be able to receive a user device. When the user device is connected to the peripheral, the eye tracking device may transmit eye tracking data to the user device, and the controller may dynamically charge the user device, as well as power the eye tracking device.

The application discloses a kind of based on comfort perception's human living environment regulation method, comprising the following steps: collection environment parameter;User physiological data are collected;The user physiological data collected are handled, and attention concentration degree is obtained according to brain wave data, muscle tension is obtained according to myoelectric data, and cognitive load degree is obtained according to eye movement data;According to attention concentration degree, muscle tension and cognitive load degree, the comfort index of user under current environment is evaluated;If comfort index is less than threshold value, then with user interaction, obtain user feedback;According to user feedback and comfort index, adjust environment parameter.The application according to attention concentration degree, muscle tension and cognitive load degree, the comfort index of user under current environment is evaluated, and comprehensively and accurately reflects user experience;And increase user feedback, satisfy the individualized demand of user, provide more comfortable use environment for user.

The present invention relates to the field of neuropsychology and low-grade glioma treatment technology, and discloses a mobile spatial attention assessor and assessment method, comprising: a hardware device; and a self-developed spatial attention assessment module, a neuronavigation system, and an eye movement trajectory analysis module based on Matlab mounted on the hardware device. The hardware device comprises a touch screen, a bracket and an eye tracker adapted to a variety of application scenarios. The self-developed spatial attention assessment module comprises a line experiment and a C-type experiment. The spatial attention assessor and assessment method provided by the present invention can be applied to a variety of application scenarios, including: baseline neuropsychological tests, transcranial electrical stimulation tests, cortical and subcortical stimulation tests, etc., and has the advantages of one experiment and multiple sets of data, effectively avoiding individual differences and batch deviations in previous studies, and is simple to operate and time-saving, and can be used as an open system to add more functions.

The invention provides an autism diagnosis method based on multi-modal super-dimensional calculation and electronic equipment, and relates to the technical field of biomedical information. The method comprises the following steps: acquiring resting-state electroencephalogram data, eye movement tracking data and demographic data of a tested object; respectively carrying out typical feature extraction on the resting-state electroencephalogram data and the eye movement tracking data, respectively encoding electroencephalogram feature data and eye movement feature data into an electroencephalogram super-dimensional vector and an eye movement super-dimensional vector through super-dimensional calculation, and encoding the demographic data into a demographic super-dimensional vector; integrating the electroencephalogram super-dimensional vector and the eye movement super-dimensional vector through a weight fusion and residual enhancement mechanism to obtain a feature fusion super-dimensional vector, and integrating the feature fusion super-dimensional vector and a demographic super-dimensional vector through an adaptive fusion strategy to obtain a tested object super-dimensional vector of each tested object; and determining a diagnosis result of the target tested object according to a similarity calculation result with the plurality of class super-dimensional vectors. The autism diagnosis accuracy can be improved.

This invention discloses a method and system for identifying pilot EBT training target competencies driven by the fusion of eye-tracking data and flight data. The method includes: S0: acquiring eye-tracking data from the pilot's EBT training process, constructing a GDETC network to achieve spatiotemporal trajectory clustering of the eye-tracking data, and extracting the pilot's gaze spatiotemporal trajectory; S1: extracting competency assessment scores and error spatiotemporal point data from the EBT competency assessment worksheet and flight data; S2: constructing a VAE-STRF network, fusing the data extracted in S0 and S1, extracting the pilot's key competency assessment score matrix, establishing a pilot key competency assessment model, and outputting the target competencies for EBT training. This invention identifies the target competencies that pilots need to further improve through eye-tracking data during EBT training, replacing existing manual evaluation systems and providing effective decision support for improving the targeting of pilot training.

An embodiment of the present invention provides a cross-modal decision confidence estimation method and system based on generative adversarial learning. The method includes: collecting electroencephalogram (EEG) signals and eye movement signals of a subject during the decision-making process; extracting EEG features and eye movement features from the EEG signals and eye movement signals; performing a first training on an EEG generation model to extract decision confidence features through the EEG features and eye movement features, and performing a second training on the relationship between eye movement and EEG through generative adversarial learning from the decision confidence features; inputting the obtained real eye movement signals of the subject into the trained EEG generation model to obtain predicted EEG signals, and performing decision confidence estimation on the real eye movement signals and the predicted EEG signals based on a multi-modal classifier. The embodiment of the present invention utilizes generative adversarial learning to improve the decision confidence estimation ability based on eye movement signals, solves the problem of high complexity and cost in the process of collecting EEG signals, and ensures accurate decision confidence estimation.

The present disclosure is related to a peripheral containing an eye tracking device, a controller, and a battery. The peripheral may be able to receive a user device. When the user device is connected to the peripheral, the eye tracking device may transmit eye tracking data to the user device, and the controller may dynamically charge the user device, as well as power the eye tracking device.

A computer-implemented method which, when executed by data processing hardware, causes the data processing hardware to perform operations that include: receiving, in an eye-tracking application, eye-tracking data from an imaging system; executing, via the eye-tracking application, an eye-tracking protocol, wherein the eye-tracking protocol comprises a blink protocol and a direction protocol; comparing, via the eye-tracking protocol, key points of the eye-tracking data with a predefined pattern of the eye-tracking application; executing an action from the blink protocol and the direction protocol based on the comparison of the key points of the eye-tracking data with the predefined pattern; and generating, via the eye-tracking application, an ocular emergency alert based on the executed action from the blink protocol and the direction protocol.This includes receiving the ocular emergency alarm on a back-office server and generating a confirmation notification based on the ocular emergency alarm.

The invention discloses an eye-control interactive visual feedback evaluation method considering asthenopia, and belongs to the field of man-machine interaction and human factor engineering. The eye-control interaction visual feedback evaluation method is constructed for the key positioning stage and selection stage in the eye-control interaction process, the problem of visual fatigue caused by eye-control interaction is considered, positioning and selection tasks are played through mobile equipment, the positioning task is completed through a cross mark and a round mark, and the eye-control interaction visual feedback evaluation method is established. Completing a selection task by using a delay selection mechanism and a sub-selection reconfirmation mechanism; collecting eye movement data when the user finishes the positioning task and selects the task by using different visual feedbacks by adopting eye control interaction equipment, and calculating a positioning error and a selection accuracy rate; visual fatigue occurrence time is considered, and subjective load and evaluation of the user are combined to carry out eye control interaction visual feedback evaluation; when the eye-controlled interaction is used for man-machine interaction, the visual feedback is preferentially selected according to the eye-controlled interaction visual feedback evaluation result, and the visual fatigue is reduced and the accuracy of the eye-controlled interaction is ensured according to the preferentially selected visual feedback.

The application discloses a cooking processing method and device, electronic equipment and storage medium. The method comprises the following steps: in the cooking process, the eye movement data, gesture data and voice data of an operation subject to a cooking object are respectively subjected to feature extraction to obtain eye movement features and corresponding first confidence, gesture features and corresponding second confidence, and voice features and corresponding third confidence; based on the first confidence, the second confidence, the third confidence, and the correlation features among the eye movement features, the gesture features and the voice features, the weight information corresponding to each of the eye movement features, the gesture features and the voice features is determined, and then the fusion feature information is determined by combining the eye movement features, the gesture features and the voice features; the fusion feature information is input into an intention recognition network to determine the target intention of the operation subject to the cooking object, and the cooking object is controlled to interact. The application can improve the accuracy and intelligence of cooking collaborative interaction.

A driver awareness coverage (DAC) system includes a vehicle having notification devices and sensors capturing optical information about a vehicle driver. A DAC application defines areas of interest (AoIs) surrounding the vehicle; processes the optical information; estimates a driver gaze direction, field-of-view, and focal point, and maps the gaze direction to the AoIs. The DAC application determines the gaze direction is in a range of known passenger compartment locations; defines a lifespan for each AoI, and selectively updates each AoI lifespan depending on gaze direction. A stimulus subroutine is executed when a lifespan of an AoI has been exceeded. The lifespan is exceeded when the gaze direction has not passed through the AoIs for an extended period of time. Then, the DAC application generates a notification to the driver that prompts the driver to break a distracted, hypnotic or altered mental state and increase a driver attention level.

An auxiliary diagnostic apparatus for ocular myogenic compensatory head posture based on instantaneous eye movement capture, relating to the technical field of medical devices, and comprising a rotated video playback module, a skeletal muscle key point capture module, a capture module, and an image output module; a subject visually fixates on a video, wherein the video is rotated in a direction opposite to a compensatory head posture until reaching a horizontal position, and if a head posture at the horizontal position remains unimproved, then the video is rotated in a direction of the compensatory head posture; during rotation, the skeletal muscle key point capture module determines that captured facial and neck key points of the subject are symmetrical and that the head posture is properly aligned, and the capture module is started and automatically captures an image; the image is analyzed for ocular misalignment, and if ocular misalignment is present in the captured image, ocular myogenic compensatory head posture can be definitively determined; if the head posture does not improve significantly, one eye of the subject is covered while the other eye continues to fixate on the video. A subject visually fixates on a video, and after the video stops rotating, the subject maintains a stable head posture and fixation point until an automatic image capture process is completed. If the head posture in the captured image improves, ocular myogenic compensatory head posture can be definitively determined.

The invention discloses an emotion recognition method and system based on multi-modal biological characteristics, and relates to the technical field of emotion recognition, and the system comprises a multi-modal module, a calculation module, a dynamic module and a summary module. A coupling unit on the summarizing module also calculates a first coupling degree between the eye movement features and the micro-expression features and a second coupling degree between the eye movement features and the voice features through the eye movement feature fluctuation degree, the micro-expression feature fluctuation degree, the voice feature fluctuation degree and a coupling formula respectively, and when the value of the first coupling degree is closer to 1, the second coupling degree is closer to 1; the micro-expression features and the eye movement features are more synchronous, the micro-expression features and the eye movement features of the tested person are closer to natural physiological reactions, and the micro-expression features are not deliberately controlled; when the numerical value of the second coupling degree is closer to 1, the voice features and the eye movement features are more synchronous, further, the voice features and the eye movement features of the testee are closer to natural physiological reactions, and the voice features are not deliberately controlled.

The application discloses a kind of based on multi-modal fusion concentration training method and system, method includes, in the process of concentration training, brain electrical signal, eye movement signal and head movement signal are collected by wearable device, brain electrical frequency characteristic parameter, eye movement parameter and head movement mode parameter are calculated one by one correspondence;According to brain electrical frequency characteristic parameter, eye movement parameter and head movement mode parameter, concentration score is calculated to evaluate concentration;According to concentration evaluation result, adjust concentration training strategy.The application simultaneously collects brain electrical, eye movement and head movement signal by wearable device, calculates corresponding characteristic parameter and comprehensively evaluates concentration score, provides comprehensive, accurate and stable evaluation result.Based on real-time evaluation, dynamically adjust task difficulty and type, ensure challenge and interest, improve user participation and training effectiveness, promote concentration research and technological progress.

The invention belongs to the technical field of medical detection and medical instruments, and discloses a magnetic resonance compatible vestibular cold and hot water stimulation system, which comprises a cold and hot water module, a stimulation module and a control module, wherein the cold and hot water module is used for conveying water with a set temperature to the stimulation module and recycling the water output by the stimulation module for cyclic utilization; the stimulation module is made of a magnetic resonance compatible material and used for circularly and alternately conveying cold water and hot water to the two ears of the subject and conveying water flowing out of the ear canals to the cold and hot water module; the control module is used for controlling the stimulation module to circularly and alternately convey cold water and hot water to the ears of the subject according to a preset time sequence according to the synchronizing signal of the magnetic resonance scanner, collecting eye movement data of the subject in real time, and controlling and adjusting the water temperature of the cold and hot water module according to the eye movement data. According to the invention, accurate synchronization of the stimulation process and magnetic resonance scanning is realized, and personalized stimulation intensity customization and stable maintenance of the stimulation effect in long-time scanning are realized.

The present invention discloses a robot-assisted hand-eye coordination training system based on smooth eye tracking and guiding force field, comprising a virtual interactive scene module, a smooth eye tracking detection module, a robot-assisted interception module and an impact force rendering module. The virtual interactive scene module can generate a virtual interactive scene with a virtual moving object and a virtual handle agent; the smooth eye tracking detection module collects eye movement signals when the user tracks the virtual moving object to detect smooth eye tracking events. The robot-assisted interception module estimates the direction of movement of the virtual moving object and generates an interception guiding force field, thereby generating an auxiliary force to assist the user in interception. The impact force rendering module collects the kinematic information of the robot handle for collision detection and calculates the impact force according to the impact force calculation model after the collision is detected, and controls the motor to generate force feedback to the user's hand. This improves the user's enthusiasm and interest in participating in hand-eye coordination training and enhances the immersion of the system.