30 results about "Ocular motor" patented technology

The invention provides a multi-modal data acquisition and intelligent analysis application method based on AI glasses. The method comprises the steps that the AI glasses collect text data through a camera and an OCR assembly, collect image video data through a binocular camera and an infrared sensor, collect audio data through an array microphone, and collect auxiliary data through an I MU and an eye movement tracking sensor; according to the AI glasses, a local lightweight AI model is used for basic analysis, and complex tasks are uploaded to a cloud large model for cooperative processing, including information summarization, deep mining and knowledge extraction; the AI glasses realize real-time pushing, scene assistance and content generation and output through an information application module according to an analysis result; according to the AI glasses, an end-cloud collaboration module and a federated learning framework are adopted, a local cloud processing strategy is dynamically adjusted, only feature parameters are uploaded, and it is ensured that original data are locally reserved.

An apparatus for assessing the fine motor skills of a target user includes: a pressure sensing module configured to capture pressure data associated with pressure exerted by small muscles during the performance of the fine motor skill by the target user; a motion tracking module configured to capture motion data associated with movement of the small muscles during the performance of the fine motor skill by the target user; an eye-tracking module configured to capture gaze data associated with the gaze direction of the target user during the performance of the fine motor skill; and a performance evaluation module configured to determine, in real time, one or more metrics associated with the accuracy of the target user's fine motor skills based on the captured motion data, the captured gaze data, and the captured pressure data.

The application belongs to the field of intelligent transportation, and discloses a vehicle lane-changing recognition method and system based on multi-modal data fusion. In the application, eye movement tracking and multi-modal fusion technology of vehicle kinematics are combined to analyze vehicle motion data, and the statistical characteristics of the vehicle motion data are combined to extract a feature vector A representing the driving habit and stability of the driver. Meanwhile, the eye movement features and the feature vector A are spliced to extract a multi-modal feature vector B focusing on capturing lane-changing. A group classification mechanism based on the experience of the driver is further proposed, the feature vectors A and B are fused, and then the fused features are input into a meta-fusion gate network. The dynamic weight is output through the gate network to realize the classification of the driver based on the experience, and the most suitable expert recognition model is automatically weighted for each driver at each driving moment, so that more accurate and personalized recognition of lane-changing is realized.

This invention discloses an essential oil-based intervention system for the simultaneous treatment of heart and brain function based on dual feedback of eye movement and heart rate variability, belonging to the field of rehabilitation medicine technology. The system includes: a data synchronization acquisition module, which acquires heart rate variability data and eye movement characteristic data and generates a synchronized physiological data stream; a dual feedback joint assessment module, which assesses the patient's real-time physical and mental state level based on the synchronized physiological data stream; a synchronized intervention decision module, which generates essential oil atomization control instructions and eye movement training control instructions in parallel according to the physical and mental state level; and a synchronized intervention execution module, which drives the essential oil atomization unit and the eye movement training unit to execute corresponding intervention actions synchronously at the hardware level on the time axis. This invention achieves integrated and precise intervention of heart and brain function through dual-modal physiological feedback closed loop and synchronized control, effectively solving the problems of brain separation, delayed intervention, and poor coordination in traditional rehabilitation centers, and improving the synchronous rehabilitation effect of cognitive function and autonomic nervous function in stroke patients.

The present disclosure is related to a peripheral containing an eye tracking device, a controller, and a battery. The peripheral may be able to receive a user device. When the user device is connected to the peripheral, the eye tracking device may transmit eye tracking data to the user device, and the controller may dynamically charge the user device, as well as power the eye tracking device.

The application discloses a kind of based on comfort perception's human living environment regulation method, comprising the following steps: collection environment parameter;User physiological data are collected;The user physiological data collected are handled, and attention concentration degree is obtained according to brain wave data, muscle tension is obtained according to myoelectric data, and cognitive load degree is obtained according to eye movement data;According to attention concentration degree, muscle tension and cognitive load degree, the comfort index of user under current environment is evaluated;If comfort index is less than threshold value, then with user interaction, obtain user feedback;According to user feedback and comfort index, adjust environment parameter.The application according to attention concentration degree, muscle tension and cognitive load degree, the comfort index of user under current environment is evaluated, and comprehensively and accurately reflects user experience;And increase user feedback, satisfy the individualized demand of user, provide more comfortable use environment for user.

The invention provides an autism diagnosis method based on multi-modal super-dimensional calculation and electronic equipment, and relates to the technical field of biomedical information. The method comprises the following steps: acquiring resting-state electroencephalogram data, eye movement tracking data and demographic data of a tested object; respectively carrying out typical feature extraction on the resting-state electroencephalogram data and the eye movement tracking data, respectively encoding electroencephalogram feature data and eye movement feature data into an electroencephalogram super-dimensional vector and an eye movement super-dimensional vector through super-dimensional calculation, and encoding the demographic data into a demographic super-dimensional vector; integrating the electroencephalogram super-dimensional vector and the eye movement super-dimensional vector through a weight fusion and residual enhancement mechanism to obtain a feature fusion super-dimensional vector, and integrating the feature fusion super-dimensional vector and a demographic super-dimensional vector through an adaptive fusion strategy to obtain a tested object super-dimensional vector of each tested object; and determining a diagnosis result of the target tested object according to a similarity calculation result with the plurality of class super-dimensional vectors. The autism diagnosis accuracy can be improved.

The present disclosure is related to a peripheral containing an eye tracking device, a controller, and a battery. The peripheral may be able to receive a user device. When the user device is connected to the peripheral, the eye tracking device may transmit eye tracking data to the user device, and the controller may dynamically charge the user device, as well as power the eye tracking device.

A computer-implemented method which, when executed by data processing hardware, causes the data processing hardware to perform operations that include: receiving, in an eye-tracking application, eye-tracking data from an imaging system; executing, via the eye-tracking application, an eye-tracking protocol, wherein the eye-tracking protocol comprises a blink protocol and a direction protocol; comparing, via the eye-tracking protocol, key points of the eye-tracking data with a predefined pattern of the eye-tracking application; executing an action from the blink protocol and the direction protocol based on the comparison of the key points of the eye-tracking data with the predefined pattern; and generating, via the eye-tracking application, an ocular emergency alert based on the executed action from the blink protocol and the direction protocol.This includes receiving the ocular emergency alarm on a back-office server and generating a confirmation notification based on the ocular emergency alarm.

The application discloses a cooking processing method and device, electronic equipment and storage medium. The method comprises the following steps: in the cooking process, the eye movement data, gesture data and voice data of an operation subject to a cooking object are respectively subjected to feature extraction to obtain eye movement features and corresponding first confidence, gesture features and corresponding second confidence, and voice features and corresponding third confidence; based on the first confidence, the second confidence, the third confidence, and the correlation features among the eye movement features, the gesture features and the voice features, the weight information corresponding to each of the eye movement features, the gesture features and the voice features is determined, and then the fusion feature information is determined by combining the eye movement features, the gesture features and the voice features; the fusion feature information is input into an intention recognition network to determine the target intention of the operation subject to the cooking object, and the cooking object is controlled to interact. The application can improve the accuracy and intelligence of cooking collaborative interaction.

A driver awareness coverage (DAC) system includes a vehicle having notification devices and sensors capturing optical information about a vehicle driver. A DAC application defines areas of interest (AoIs) surrounding the vehicle; processes the optical information; estimates a driver gaze direction, field-of-view, and focal point, and maps the gaze direction to the AoIs. The DAC application determines the gaze direction is in a range of known passenger compartment locations; defines a lifespan for each AoI, and selectively updates each AoI lifespan depending on gaze direction. A stimulus subroutine is executed when a lifespan of an AoI has been exceeded. The lifespan is exceeded when the gaze direction has not passed through the AoIs for an extended period of time. Then, the DAC application generates a notification to the driver that prompts the driver to break a distracted, hypnotic or altered mental state and increase a driver attention level.

The invention discloses an emotion recognition method and system based on multi-modal biological characteristics, and relates to the technical field of emotion recognition, and the system comprises a multi-modal module, a calculation module, a dynamic module and a summary module. A coupling unit on the summarizing module also calculates a first coupling degree between the eye movement features and the micro-expression features and a second coupling degree between the eye movement features and the voice features through the eye movement feature fluctuation degree, the micro-expression feature fluctuation degree, the voice feature fluctuation degree and a coupling formula respectively, and when the value of the first coupling degree is closer to 1, the second coupling degree is closer to 1; the micro-expression features and the eye movement features are more synchronous, the micro-expression features and the eye movement features of the tested person are closer to natural physiological reactions, and the micro-expression features are not deliberately controlled; when the numerical value of the second coupling degree is closer to 1, the voice features and the eye movement features are more synchronous, further, the voice features and the eye movement features of the testee are closer to natural physiological reactions, and the voice features are not deliberately controlled.

The application belongs to the technical field of eye tracking, and particularly relates to a multi-eye tracker data processing method and device based on common port communication, aiming to solve the problems of large communication overhead, difficult time synchronization and low data fusion accuracy in the prior art when multiple eye trackers work cooperatively. The method comprises the following steps: via a common communication port, logical channels are respectively constructed according to the received communication data of multiple eye trackers; time synchronization is performed on the multiple eye trackers based on a global time reference, and calibration data of each eye tracker is sequentially fed back and dispatched; global calibration parameters are determined based on the calibration data to establish a global calibration model, and the subsequently collected eye movement data is corrected in coordinates based on the model. Through single-port multi-channel multiplexing, global synchronization scheduling and fusion modeling, the application significantly reduces the system resource consumption, realizes millisecond-level device cooperation, and greatly improves the overall analysis accuracy of eye movement data.

This invention provides a method for estimating neuropsychological activity that can accurately predict developmental disorders, mental illnesses, and other conditions. [Solution] Time-series data of the subject's pupil diameter and time-series data of the subject's eye movements are obtained. The time-averaged value of the pupil diameter is calculated from the time-series data of the pupil diameter, and the complexity of eye movements for each time scale is quantified from the time-series data of eye movements. Based on the time-averaged value of the pupil diameter and the complexity of eye movements, the subject's neuropsychiatric activity is estimated.

A driver attention monitoring (DAC) system comprises a vehicle equipped with alerting devices and sensors that capture visual information about the driver. A DAC application defines areas of interest (AoIs) surrounding the vehicle, processes the visual information, estimates the driver's gaze direction, field of view, and focal point, and maps the gaze direction onto the AoIs. The DAC application determines whether the gaze direction is within a range of known positions in the passenger compartment, defines a lifetime for each AoI, and selectively updates each AoI lifetime based on the gaze direction. A stimulus subroutine is executed when an AoI lifetime has been exceeded. The lifetime is exceeded if the gaze direction has not passed through the AoIs for an extended period.The DAC application then generates a notification to the driver, urging the driver to interrupt a distracted, hypnotic or altered mental state and increase the driver's level of attention.

The invention provides an adaptive SSMVEP-MI fusion brain-computer interface method and system based on eye movement tracking. The method comprises the following steps: determining a current effective fixation area of a user in real time through eye movement tracking; based on the region, only activating a corresponding local SSMVEP visual stimulation target, and generating associated specific motor imagery prompt information; electroencephalogram signals are synchronously collected, and SSMVEP features and MI features are extracted; self-adaptively selecting a decoding path according to whether the accumulated number of the MI effective samples with the labels reaches a threshold value or not: if not, independently decoding by using the SSMVEP features; if yes, fusion decoding based on signal quality evaluation is carried out on the bimodal features; and finally outputting a control signal to drive external equipment or update an interactive interface. According to the method, the visual interference is remarkably reduced, the recognition accuracy and the training efficiency at the initial stage of motor imagery are improved, and the robustness and the availability of the system are ensured through a dual adaptive mechanism of the sample number and the signal quality.

The invention relates to the technical field of intelligent driving, and discloses a multi-modal fusion-based driver state perception and cooperative driving regulation and control method, which comprises the following steps of S1, synchronously acquiring an electroencephalogram signal and an eye movement signal of a driver; s2, performing preprocessing and feature extraction to obtain an electroencephalogram feature sequence and an eye movement feature sequence; s3, performing feature-level fusion by using a multi-modal fusion model based on an attention mechanism to generate a fusion feature vector; s4, inputting the fusion feature vector into a state recognition model, and recognizing the cognitive state of the driver in real time; s5, based on the recognized cognitive state, the current vehicle state and the environment perception information, the matching degree of the driving task demand and the driver state is evaluated; and S6, according to the matching degree, dynamically selecting and executing corresponding regulation and control actions. According to the method, multi-modal physiological and behavior data can be deeply fused, a complex cognitive state is accurately decoded, and intelligent, personalized and prospective collaborative driving regulation and control are realized.

The invention relates to a monocular eye movement tracking and eye behavior recognition system and method of an embedded platform, and belongs to the technical field of face recognition. By adopting the system and the method provided by the invention, high-precision detection and behavior recognition of eyeball movement can be realized by analyzing the key points of the eye region. In the aspect of system design, aiming at the problem that computing resources of embedded equipment are limited, a neural network model structure and a reasoning process are optimized, and a multi-thread parallel processing mechanism is introduced to remarkably improve the real-time performance of the system; according to the monocular eye movement tracking and eye behavior recognition system, the processing speed of 29.85 ms / frame is achieved on a domestic RV1126B chip platform, complex setting and calibration processes are not needed, and use obstacles of a traditional eye movement tracking system are effectively eliminated. The system is simple in structure and convenient to deploy, has high real-time performance and good universality, and can be widely applied to application scenes such as driver state monitoring, virtual reality interaction and intelligent man-machine interfaces.

The invention discloses a vehicle-mounted oxygen supply regulation and control method and system in a plateau environment. The method comprises the steps that fatigue parameters of a driver and altitude information and bumping degree information of the place where the driver is located are obtained in real time; determining an emotional state of the driver according to the eye movement characteristic monitoring data and the electroencephalogram characteristic monitoring data of the driver; determining a first fatigue degree of the driver according to the oxyhemoglobin saturation data, the heart rate signal data and the electromyographic signal data of the driver; determining a second fatigue degree of the driver according to the altitude information and the bumping degree information of the place where the driver is located; determining a third fatigue degree of the driver according to the first fatigue degree and the second fatigue degree of the driver; according to the emotional state of the driver and the third fatigue degree, the single-time oxygen increasing amount is determined; oxygen is supplied to the driver according to the single-time oxygen increasing amount. According to the method, the single-time oxygen increasing amount can be determined in advance, so that oxygen is supplied to a driver according to the single-time oxygen increasing amount.

A fusion intention prediction method, device, system, storage medium and program product, comprising: collecting electroencephalogram signals and eye movement signals of a human body; obtaining an eye movement sampling rate of the eye movement signals and comparing the eye movement sampling rate with a preset sampling rate threshold; inputting the electroencephalogram signals and the eye movement signals into an intention prediction large model, the intention prediction large model performing intention prediction based on the electroencephalogram signals and the eye movement signals; outputting an intention prediction result; when the eye movement sampling rate is greater than or equal to the preset sampling rate threshold, performing fusion learning of the intention prediction large model; and when the eye movement sampling rate is less than the preset sampling rate threshold, closing the fusion learning of the intention prediction large model. The present disclosure solves the problems that the prior art uses a pre-set static model, cannot handle dynamic environments, has low intention prediction accuracy, and cannot be used for a long time.

The application relates to the technical field of automobile safety detection, and discloses a fatigue driving real-time detection method and system based on a monitoring camera. The method collects multi-source sensing data streams of a driver's face region, generates a synchronized physiological signal set through time sequence alignment processing; multi-modal feature extraction is performed on the physiological signal set, so that an eye movement trajectory feature vector, a head posture change sequence and a facial muscle activity intensity atlas are obtained; an eye movement fatigue index and a head behavior abnormality degree score are respectively generated, a comprehensive fatigue grade index is obtained by combining a weighted decision algorithm; a grading early warning trigger strategy is generated according to the comprehensive fatigue grade index and real-time driving environment parameters, a corresponding alarm instruction is called to generate a multi-level warning signal output stream; the physiological state change trend of the driver is monitored in real time, the dynamic priority of the warning signal output stream is adjusted, the alarm instruction set is updated, and fatigue driving intervention closed-loop control is completed.

The invention discloses a multi-mode interactive dynamic rehabilitation intelligent mattress for hemiplegic old people. The multi-mode interactive dynamic rehabilitation intelligent mattress is characterized by comprising a multi-mode intelligent interactive system, a dynamic adaptive mattress structure and adjusting system, a health-rehabilitation collaborative management system and an emergency intelligent response system, the method has the advantages that the multi-modal interaction reliability is high, voice, eye movement and muscle force multi-dimensional instructions are fused, the problem of single-modal failure is solved through a priority switching algorithm, and the instruction recognition accuracy is larger than 92%.

The present application belongs to the technical field of bionic machinery, and particularly relates to a bionic eye movement robot and a working method thereof. The present application comprises a base, a first linkage assembly, two second linkage assemblies, a first limiting piece, and two eyeball assemblies provided with bionic eyes. The first linkage assembly is driven to change a first height of a first parallelogram structure on a horizontal plane, and / or the second linkage assembly is driven to change a second height of a second parallelogram structure on a vertical plane, so as to respectively drive the two bionic eyes to move along a set spherical surface. The two bionic eyes of the bionic eye movement robot of the present application realize horizontal plane movement, vertical plane movement, and rotating movement of the bionic eyes on the set spherical surface through the first parallelogram structure and the second parallelogram structure which are perpendicular to each other, and through the limiting piece, thereby conforming to the natural movement law of human eyes.

The invention discloses a robot eye movement control method and system based on speed control, and the method comprises the steps: obtaining the eye movement data of an operator, and processing the eye movement data; the eye movement data is converted into a speed control instruction of the mechanical arm based on the speed eye movement control normal form; multi-view-angle environment information is obtained through an omni-directional camera matrix arranged at the working end of the mechanical arm, a multi-view-angle view is rendered on a user interface, and view angles are automatically switched according to the falling point of a fixation point; and the speed control instruction is received, kinematics transformation and environment state estimation are carried out, joint motion information of the mechanical arm is obtained, and the mechanical arm is controlled to execute a task. On the basis that direct position mapping is carried out on tail end movement of the robot through eye movement data, the movement speed of the robot serves as an interaction object, speed mapping of the robot is carried out through a gazing sight falling point distance control center displacement vector in the interaction process, and the performance is good in the aspects of continuous movement and dynamic adjustment of the robot.

The invention provides an upper limb rehabilitation training method based on eye movement, gesture recognition and resistance feedback, belongs to the technical field of upper limb rehabilitation training, and is used for solving the problems that in related technologies, training scenes are out of life, multi-mode cooperation is lacked, and parameter adaptability is poor. According to the method, a closed loop is formed by loading a living scene, collecting eye movement data to recognize a training intention, capturing unhealthy side gestures to achieve cooperation, outputting adaptive resistance feedback, combining multi-dimensional evaluation dynamic optimization parameters and fusing multiple algorithms. The training pertinence, the scene simulation degree and the collaborative naturalness are improved, and the upper limb function of the patient is helped to efficiently migrate to the daily life ability.

The invention relates to the technical field of virtual reality, and discloses a test method, device and equipment and a storage medium, and the method comprises the steps: controlling a human eye simulation part to rotate towards a first test point through a motor, and obtaining a first rotation angle of the motor and a first tracking angle obtained by tracking the human eye simulation part through an eye movement tracking system; controlling the human eye simulating part to rotate towards a second test point through the motor, and obtaining a second rotation angle of the motor and a second tracking angle obtained by tracking the human eye simulating part through the eye movement tracking system; obtaining an eye movement relative angle difference according to the first tracking angle and the second tracking angle, and obtaining a motor theoretical relative angle difference according to the first rotation angle and the second rotation angle; and testing the eye movement tracking system based on the eye movement relative angle difference and the motor theoretical relative angle difference to generate a test result. The accumulated influence of the absolute position error of the motor can be eliminated, so that the accuracy of a test result is improved.

The invention belongs to the technical field of wearable medical instruments and rehabilitation, and provides a flexible bionic eyelid rehabilitation system and a rehabilitation training method.The system comprises a flexible sensor array module which comprises a flexible stretchable substrate and a sensing material; the signal processing and control module is used for acquiring dynamic characteristic parameters of uninjured-side eyelid movement and generating an eyelid movement control instruction on the basis of the dynamic characteristic parameters; the electromagnetic driver module is used for generating a controllable magnetic field in response to the eyelid movement control instruction; and the flexible magnetic patch module can be pulled by non-contact magnetic force under the action of a controllable magnetic field to drive the affected-side eyelid of the user to execute follow-up operation synchronous with the movement of the uninjured-side eyelid. Functional closing or movement assistance can be realized, and a monitoring-feedback-regulation patient individual adaptive closed loop mechanism is constructed; and by monitoring the physiological signal of the movement of the uninjured eyelid, the affected eyelid is driven to move, so that auxiliary rehabilitation training which has higher physiological adaptability and conforms to the human natural eye movement rule is realized.

In the present specification, a method and an apparatus for generating ocular movement ability information are disclosed. The method for generating ocular movement ability information, according to the present specification, is a method for generating ocular movement ability information using a device including an RGB camera, a depth sensor, and an ocular tracking sensor in a housing of a predetermined shape mounted on a head of a subject, and may comprise: (a) calculating, by a processor, coordinates for a plurality of grid points in image data obtained by photographing an ocular movement indicator input from the RGB camera; (b) receiving, from the ocular tracking sensor by the processor, a measurement value for the gaze direction of one of the eyes of the subject of which the field of view is blocked, and storing the measurement value; (c) storing, by the processor, timing (hereinafter, referred to as "timing information") at which one of the eyes of the subject of which the field of view is open fixates at the grid points included in the ocular movement indicator; and (d) calculating, by the processor, a gaze error of the subject with respect to each of the grid points by using distance information between the subject and the indicator input from the depth sensor, the timing information, and gaze tracking information.

This application provides an eye-tracking trajectory analysis method and system based on hybrid clustering and temporal constraints, belonging to the field of computer vision technology. The method involves: acquiring raw eye-tracking data points; calculating the time difference and movement speed between consecutive raw eye-tracking data points; comparing the movement speed with a speed threshold to classify them into candidate fixations and saccades; extracting the spatial and temporal features of the candidate fixations, performing standardization processing, and weighted fusion into a spatiotemporal feature vector; clustering the spatiotemporal feature vector to obtain preliminary cluster labels for the candidate fixations; applying temporal constraints to each cluster; calculating the duration of the time-constrained cluster to generate the final cluster label for the candidate fixations; and combining the saccade classification results to output the final classification label for each raw eye-tracking data point.

The invention relates to the technical field of data processing, and discloses an underground transportation driver fatigue identification method based on myoelectricity and eye movement multi-modal fusion. Comprising the steps of acquiring myoelectricity characteristic data of the underground transportation driver, acquiring eye movement characteristic data of the underground transportation driver, performing significance analysis on myoelectricity characteristics and eye movement characteristics of the underground transportation driver and fatigue, training a fatigue mapping model of the underground transportation driver, monitoring the fatigue of the underground transportation driver and the like. The underground transportation driver fatigue feature recognition path and channel are added, the underground transportation driver fatigue feature recognition process and method are provided, and the recognition accuracy is greatly improved. According to the method, fatigue recognition of the underground transportation driver is supplemented through multi-modal information fusion, and the accuracy of fatigue recognition of the underground transportation driver can be improved.