6410results about "Eye diagnostics" patented technology

The invention discloses an intelligent system and method for early screening of depression based on electroencephalogram-eye movement multi-modal data fusion. According to the system, in a virtual reality situation, emotion and cognitive response of a subject are stimulated through an elaborately-designed cognitive task, and brain electrical activity and sight behavior data are synchronously collected by adopting a wearable EEG device and a high-precision eye tracker. After signal preprocessing and feature extraction, deep fusion of multi-modal data is realized by using a weighted fusion formula or a graph neural network, the weight of each modal is automatically adjusted, interaction features between electroencephalogram and eye movement are extracted, and then the depression risk of a subject is accurately judged through a classifier. Experiments show that the method can effectively capture weak physiological abnormalities of mild depression patients, has the advantages of high sensitivity, high accuracy and real-time online screening, and meets the requirements of non-invasive, portable and intelligent clinical application.

The invention relates to the technical field of sleep aiding, in particular to a method and a system for extracting feature information of a sleep state monitoring model. The method comprises the following steps: carrying out multi-mode physiological signal acquisition on a detected object to obtain original sleep monitoring data; performing noise removal and signal enhancement processing on the original sleep monitoring data, and performing signal segmentation to obtain multi-channel sleep signal fragment data; performing feature extraction based on electroencephalogram, eye movement diagram, electromyogram, electrocardiogram and accelerometer signals on the multi-channel sleep signal fragment data to obtain multi-dimensional feature data; and performing ultrasonic feature and acoustic feature extraction on the multichannel sleep signal fragment data to generate acoustic feature data. According to the method, dynamic modeling and trend prediction are carried out on the conversion process of the sleep state, potential sleep disorder events can be accurately recognized, influence evaluation is carried out on the potential sleep disorder events through deep learning, and support is provided for personalized sleep diagnosis.

The invention relates to the technical field of medical auxiliary evaluation and human-computer interaction, in particular to a cognitive level evaluation system and method applied to a caregiver of a cerebral infarction patient. Generating task parameter data based on the multi-modal feature data through a task generation module; the cognitive state decoding module decodes a comprehensive cognitive state by using a double-flow deep learning network to generate a cognitive load, an attention trend and a task response score; the feedback optimization module predicts the change trend of the cognitive ability in combination with a dynamic Bayesian network, and dynamically adjusts task parameters through a closed-loop feedback mechanism; according to the method, the dynamic adaptation of the evaluation task and the design of the intervention task are realized, and the accuracy, interestingness and applicability of evaluation are effectively improved.

The invention provides an emotion recognition method based on electroencephalogram eye movement multi-mode cross-attention feature fusion, and the method comprises the steps: firstly carrying out the preprocessing of an emotion recognition public data set, and building a corresponding training set and a test set; secondly, constructing an electroencephalogram eye movement multi-mode cross-attention feature fusion emotion recognition model; then carrying out model training and performance testing; finally, an electroencephalogram eye movement multi-mode online emotion recognition system is built, and the effectiveness of the method is verified. The method has the advantages that an electroencephalogram eye movement multi-mode cross-attention feature fusion emotion recognition model is designed, a dynamic graph convolutional network and an attention mechanism are flexibly applied, meanwhile, multi-dimensional data are used, the problem that single-mode emotion recognition information is insufficient is solved, and the recognition accuracy is improved; and an electroencephalogram eye movement multi-mode online emotion recognition system is built, so that the interactivity is enhanced. The average recognition accuracy of the method reaches 97.94% and is superior to that of an existing optimal method, and meanwhile the accuracy of online emotion recognition reaches 87.4%.

The present application provides a method for diagnosing and / or evaluating the presence or absence, severity or degree of the improvement of a retinal disease in a subject, which comprises determining and / or evaluating circulatory parameters, retinal function, retina morphology and / or visual relating quality of life (QOL).

According to an embodiment, disclosed is a system comprising a processor wherein the processor is configured to receive an input data comprising an image of an ocular region of a user, clinical data of the user, and external factors; extract, using an image processing module comprising adaptive filtering techniques, ocular characteristics, combine, using a multimodal fusion module, the input data to determine a holistic health embedding; detect, based on a machine learning model and the holistic health embedding, a first output comprising likelihood of myopia, and severity of myopia; predict, based on the machine learning model and the holistic health embedding, a second output comprising an onset of myopia and a progression of myopia in the user; and wherein the machine learning model is a pre-trained model; and wherein the system is configured for myopia prognosis powered by multimodal data.

The invention discloses a nerve disease detection and analysis method and system, and the method comprises the steps: obtaining a bracelet collection signal, a sphygmomanometer collection signal, a movement behavior image and behavior test data, and extracting tremor intensity features, gait symmetry features and autonomic nerve rhythm features through multi-band decomposition of the bracelet collection signal; analyzing the motion behavior image and the standardized motion test to obtain a motion function score; carrying out heart rate variability analysis to identify a neural function abnormality mode; constructing a neural function state map and calculating a feature weight; predicting a disease progress trend in combination with historical monitoring data; and dynamically adjusting a prediction result through subsequent feedback correction information. Through a mode of combining short-time intensive monitoring and long-term intermittent acquisition, long-term trend prediction and dynamic correction based on initial data are realized, and the reliability and practicability of nerve disease risk assessment in a home scene are remarkably improved.

The invention provides a motion fear intervention device based on virtual reality. The motion fear intervention device comprises a VR scene generation module; the subjective evaluation module presents the TSK scale to the user to obtain a TSK scale score of the user, and obtains a TSK grade corresponding to the TSK scale score according to the TSK scale score; the biological signal acquisition module is used for acquiring biological feedback of a user and calculating a comprehensive exercise fear grade according to the following formula: the comprehensive exercise fear grade = alpha * TSK grade + beta * HRV score + gamma * GSR score + delta * pupil change score + belonging to * EMG score; the dynamic intervention module is used for generating scene parameters in real time according to the comprehensive exercise fear level and inputting the scene parameters into the VR scene generation module to generate a corresponding scene; the dynamic intervention module is also used for providing forward excitation. According to the method and the device, the currently displayed VR hierarchical scene is adjusted through biological feedback, and closed-loop feedback adjustment of motion scene stimulation is realized.

The invention discloses an image processing algorithm for pupil center real-time extraction and trajectory tracking in an eye closing state. According to the algorithm, from the angle of digital image processing, graying, contrast enhancement and smoothing processing are firstly carried out on an input eye video frame by frame, and then the boundary of a pupil area can be accurately extracted by adopting an adaptive threshold segmentation technology and combining morphological operation and a Canny edge detection operator. And then, accurate fitting of ellipse parameters is performed on the pupil contour by using a geometric distance-based least square method, so that center coordinates and area information of the pupil are obtained, and the accuracy and reliability of pupil monitoring in an eye closing state are remarkably improved. Through recording and analyzing pupil center position changes in a time sequence, real-time tracking of pupil movement tracks is realized. And finally, evaluating pupil light reflex (PLR) in an eye closing state before and after white light excitation by adopting baseline correction and smooth filtering processing. The method has the advantages of high robustness, adaptability to illumination variation, high real-time performance, high precision and the like.

Disclosed are system, method, and computer program product embodiments for determining a digital marker. An embodiment operates by determining one or more oculometric parameters associated with an eye of a user using a video stream. The video stream comprises at least one image of a face of the user. The embodiment then determines a pupillary response function of brightness or a gaze response function by performing a deconvolution process based on at least the one or more oculometric parameters. The pupillary response function of brightness or the gaze response function is indicative of a response of the eye of the user to an external factor. The embodiment determines one or more digital markers of the user based on the pupillary response function of brightness or the gaze response function. The one or more digital markers are indicative of a neurological condition or a mental health condition of the user.

The invention discloses a multi-mode electroencephalogram analysis model construction method and an online processing method and system. According to the method, multi-modal physiological signals such as electroencephalogram, electrocardio, skin electrical activity and eye movement are collected, and an integrated multi-modal signal collection device is used for preprocessing and feature extraction. A modal expert system based on an adaptive Transform architecture is adopted to replace a standard feed-forward network so as to enhance the feature processing capability. By means of the method, the features can be complemented under the condition of mode or data missing, the robustness, real-time performance and accuracy of the brain-computer interface technology in multi-mode data processing are improved, and the method is particularly suitable for the fields of emotion recognition, cognitive load monitoring, neural rehabilitation and the like.

The invention discloses an operating driver fatigue detection method based on multi-source information fusion, and relates to a data acquisition module, a data analysis processing module, a fatigue state classification module, a fatigue state real-time detection and evaluation module and a fatigue early warning module, and the fatigue state classification module is used for constructing a fatigue state classification model. The fatigue state real-time detection and evaluation module is connected with the data analysis and processing module and is used for summarizing real-time driver physiology, driving behaviors and external environment characteristics; and the lower connection fatigue state classification module inputs the multi-source indexes into a fatigue state classification model to obtain a fatigue state grade. And the fatigue early warning module is used for issuing early warning information to the driver according to the fatigue state grade of the driver so as to relieve the fatigue of the driver. And an automatic driving module of the vehicle is started when necessary, so that the driving safety of a driver is protected. According to the invention, real-time monitoring, evaluation and prevention of the fatigue of the operating driver are realized.

According to the multi-mode visual stimulation and intelligent self-adaption combined myopia prevention and control system based on the AR glasses, all-weather, personalized and full-scene myopia suppression is realized through multi-module cooperation of sensory visual stimulation, visual function training, AI scene dynamic recognition and directional intervention, self-adaption closed-loop feedback, cloud / offline deployment and the like; through multi-dimensional visual stimulation and real-time physiological monitoring, eye axis growth is accurately regulated and controlled, myopia prevention and control precision is improved, out-of-focus and contrast setting are automatically optimized based on immediate refraction and adjustment feedback, personalized intervention is enhanced, lightweight AI and cloud pre-calculation are introduced, the endurance of equipment is effectively prolonged, an existing AR platform is compatible, the development process is simplified, and the development efficiency is improved. And the user compliance is enhanced.

The invention discloses an emotional disorder detection method based on a multi-modal heterogeneous graph convolutional neural network, and the method comprises the steps: collecting the electroencephalogram, expression pictures, eye movement data and gait video frame data of a subject, and aligning the data through an asynchronous time sequence cross-modal alignment and group horizontal alignment method. Thirdly, decomposing data of each mode by utilizing low-rank matrix decomposition of multi-mode characteristics, and performing alignment of different tasks with a nerve activation path; carrying out heterogeneous graph convolution channel path coding on each channel and embedding a modal; and finally carrying out lightweight cross-modal fusion on the obtained data stream. The invention further provides an emotional disorder detection device. According to the method provided by the invention, the recognition of the type of the emotional disorder is realized through the acquired multi-modal data including electroencephalogram, eye movement, micro-expression and gait information.

The invention discloses a multi-parameter dynamic intelligent judgment method for the safety state of an operator, and belongs to the technical field of operation safety monitoring. According to the method, by integrating an intelligent wearable device, a sensor and an eye movement tracking device, physiological parameters (such as heart rate, blood pressure, oxyhemoglobin saturation, electroencephalogram signals and the like), behavior parameters (such as action frequency, posture change and the like), psychological parameters (such as pressure level, fatigue degree and the like) and environmental parameters (such as temperature, humidity, noise and the like) of an operator are collected in real time; and a multi-dimensional monitoring system is constructed. Collected data is subjected to cleaning, standardization and feature extraction and then is input into a safety state judgment model based on a bidirectional long short-term memory network (BiLSTM), the real-time safety state of an operator is dynamically analyzed, and low-risk, medium-risk and high-risk three-level early warning results are output.

The present technology relates to systems and methods for objectively assessing accommodation of adjustable intraocular lenses ("AIOLs"). In some embodiments, a method of objectively assessing accommodation of an AIOL implanted within an eye of a patient includes receiving, via an accommodation assessment system, (a) first optical coherence tomography (OCT) geometric data associated with the AIOL when the eye is in a first accommodative state, and (b) second OCT geometric data associated with the AIOL when the eye is in a second accommodative state. The method next includes determining, based in part on a comparison of the OCT data, a first change to a thickness of an adjustable lens of the AIOL and / or a second change to a curvature of an optical surface of the AIOL. The method then includes determining, based in part on the first change and / or second change, a third change to an optical power of the AIOL.

The invention discloses a pupil image recognition and disease change evaluation system based on image recognition, and relates to the technical field of monitoring analysis, and the system comprises the steps: obtaining the identity information and historical pupil data of a patient, and building a pupil dynamic database corresponding to the patient; alternately irradiating the eyes of the patient, and collecting a dynamic image sequence corresponding to a pupil light reflex area of the patient; performing spatial-temporal characteristic analysis on the dynamic image sequence, establishing a pupil diameter dynamic change curve, and generating a light reflection sensitivity index of the pupil of the patient according to the pupil diameter dynamic change curve; constructing a pupil dynamic evaluation model according to the pupil diameter dynamic change curve and the light reflex sensitivity index of the patient pupil, and outputting a pupil grade evaluation result based on the pupil dynamic evaluation model; comprehensive analysis is conducted on pupil grade evaluation results, analysis results are uploaded to an intensive care central system, and a grading early warning mechanism is triggered. The application has the effect of reducing the occurrence of illness state delay.

The invention relates to the technical field of cognitive impairment detection, and discloses a cognitive impairment early warning method and device based on an electroencephalogram micro state and an eye movement trajectory, and the method comprises the following steps: S1, data acquisition, S2, electroencephalogram preprocessing, S3, electroencephalogram feature extraction, S4, eye movement feature extraction, S5, feature fusion, and S6, early warning judgment. According to the method, through independent convolution branch of electroencephalogram and eye movement features, a receptive field is expanded by utilizing cavity convolution to capture multi-scale features, and long-distance dependence is modeled by a self-attention layer; after tensor splicing, time sequence information is dynamically fused through a gating cycle unit (GRU), and cross-modal time correlation is captured. The method has the advantages that the multi-modal feature hierarchical extraction and self-adaptive modeling capability is enhanced, the complementarity fusion efficiency is optimized, meanwhile, by means of cavity convolution sparse connection, self-attention parameter sharing and GRU lightweight design, the model complexity and the calculation efficiency are balanced, and efficient feature representation is provided for cognitive impairment early warning.

Disclosed is a method for giving a feedback on a surgery, in particular an eye surgery, the feedback method comprising loading and / or receiving video data from a surgery, analyzing the video data, evaluating the analyzed video data, and outputting and / or displaying the evaluation result. Disclosed is further a feedback system for surgeries, in particular eye surgeries, the feedback system comprising a processing device for loading and / or receiving video data from a surgery, for analyzing the video data, and for evaluating the analyzed video data, and an output device for outputting and / or displaying the evaluation result.

The invention relates to the technical field of lens parameter optimization, and discloses an out-of-focus lens structure parameter optimization method and system based on multiple optical parameters, and the method comprises the steps: determining a lens wearing user and an optimization target, carrying out the eyeball parameter collection of the lens wearing user, obtaining a user eyeball parameter group, determining an optimization structure type group based on the optimization target, constructing a prevention effect function, a visual effect function and a wearing comfort function, generating a fitness function, confirming a standard structure parameter group according to the optimized structure category group, initializing a particle swarm based on the standard structure parameter group, a user eyeball parameter group and the fitness function, obtaining an initial position group and an initial fitness group, and obtaining an initial position group and an initial fitness group; and iterating the particle swarm based on the initial position group and the initial fitness group to obtain an optimal structure parameter group. According to the invention, the comprehensive performance of the out-of-focus lens can be improved, and personalized lens design can be realized according to different requirements.

The invention relates to the field of electroencephalogram signal processing, and particularly discloses a multi-mode electroencephalogram characteristic cognitive ability evaluation system based on a neural network, comprising: an acquisition module used for synchronously acquiring whole brain signals and eye movement data during pilot simulation training; the data preprocessing module is used for primarily processing the received whole brain signals and the eye movement data, monitoring the primarily processed data in real time by utilizing a preset monitoring mechanism, and triggering to generate interaction test starting signals if preset symbolic fluctuation occurs in the whole brain signals or the eye movement data; and the test interaction sub-module is used for generating arithmetic test questions according to a preset rule after receiving the interaction test starting signal, and synchronously recording answer correlation parameters of the pilot. By adopting the technical scheme of the invention, the deep-level reaction mechanism of the brain under the stimulation of the special complex cognitive task of flight training can be fully considered, and the cognitive load change of the pilot in the simulated training can be comprehensively and accurately reflected.

The invention relates to the technical field of artificial intelligence and biomedical sensing, and discloses a cognitive disorder risk identification device and method integrating multi-modal physiological data, and the device comprises an EEG module, an eye movement module, an fNIRS module, an edge calculation module and an acceleration sensor; the EEG module collects EEG data, the eye movement module collects eye movement eye-tracing data, and the fNIRS module collects near infrared spectrum fNIRS data and inputs the data to the edge calculation module; the edge calculation module runs the multi-modal space-time attention fusion model to output a cognitive impairment risk assessment result by using the multi-modal space-time attention fusion model; the acceleration sensor operates based on an operation dynamic filtering algorithm of a motion accelerometer to inhibit signal drift caused by head motion. According to the invention, through the modularized head-mounted multi-mode edge device, the physiological data is collected and edge end processing and cognitive disorder recognition and screening are carried out in combination with the edge calculation module, so that early recognition and screening of neurodegenerative diseases can be conveniently and effectively carried out at low cost.

A head-mounted display system is disclosed herein. In one or more embodiments, a system for body pose estimation in a virtual reality or augmented reality environment includes one or more egocentric cameras coupled to a user configured to capture video data from a perspective of the user; an instrumented flooring system including one or more sensors configured to detect foot placement and / or movement of the user, and generate sensor output data based on the detected foot placement and / or movement of the user; and at least one data processing device being configured to receive and integrate the video data from the one or more egocentric cameras and the sensor output data from the instrumented flooring system using sensor fusion algorithms, thereby generating a comprehensive body pose estimation for the user.

The invention relates to the technical field of medical equipment control, in particular to a control system and a control method of cornea crosslinking equipment. The control system comprises a multi-modal data fusion module which is configured to integrate a preoperative corneal topographic map curvature matrix, OCT layering thickness data and biomechanical parameters to generate a four-channel fusion tensor; the virtual cornea modeling module is configured to input a conditional generative adversarial network to generate a virtual cornea model under the condition of the four-channel fusion tensor; the dynamic energy planning module is configured to generate a thermodynamic diagram of energy distribution based on mechanical response prediction of the virtual cornea model; and an illumination control module configured to perform patterned illumination based on the thermodynamic diagram. According to the method, the virtual cornea model is constructed, the energy distribution thermodynamic diagram is generated, accurate patterning reference can be provided for irradiation treatment, and a better treatment effect is achieved.

The invention provides a depression virtual reality auxiliary diagnosis and treatment method fused with multi-modal data, and the method comprises the steps: obtaining the multi-modal data, the multi-modal data comprises the peripheral physiological data, such as heart rate, body temperature and skin electricity, monitored by a flexible sensor based on epidermal electrons; and switching to a personalized interactive virtual scene according to the emotional state of the patient in real time, and in the interaction process, obtaining eyeball movement data by the eye movement tracking instrument, and knowing attention distribution, psychological state and micro facial expression change of the user. In combination with multi-modal data of peripheral physiological data, eye movement data and micro-expression change, key features are automatically extracted from multi-modal input data, and emotion analysis and recognition are performed through a multi-modal deep learning method with an attention mechanism. Data can be collected and analyzed in real time, the emotional state of a patient is objectively reflected through the combination of peripheral physiological data and micro-expression analysis, the requirements of the patient are met in a personalized mode, and the treatment effect is improved.

Methods and systems for reductions in switching between depth planes of a multi-depth plane display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. The system may monitor the fixation points based upon the gaze of each of the user's eyes, with each fixation point being a three-dimensional location in the user's field of view. Location information of virtual objects to be presented to the user are obtained, with each virtual object being associated with a depth plane. In some embodiments, the depth plane on which the virtual object is to be presented is modified based upon the fixation point of the user's eyes. For example, where the user is switching their fixation between virtual objects on two different depth planes, the display system may be configured to modify the presentation of one of the objects such that both objects are placed on the same depth plane.

The invention relates to the technical field of pilot working state evaluation, and discloses an interface interaction cognitive load evaluation method based on pilot behavior data, which comprises the following steps: real-time data acquisition; discrete information processing; evaluating weights of perception type and mental movement type factors; and deducing and evaluating a cognitive factor weight and outputting a cognitive load value. The technical problems that the degree of association between the evaluated cognitive load and the interface is not high, and optimization design of an aviation interface cannot be effectively guided are solved, and by collecting observable visual, auditory and mental movement data, evaluation of the load level of an unobservable cognitive factor is achieved. And the accuracy of pilot real-time cognitive load quantitative evaluation is improved.

A method to prevent intoxicated operation is described. The method includes providing instructions to a vehicle user to position a vehicle user body portion posture in a predefined alignment. The method further includes obtaining the vehicle user body portion posture from a first vehicle detector, and determining whether the vehicle user body portion posture is in the predefined alignment. The method includes activating a plurality of vehicle visual indicators to illuminate in a predefined manner when the vehicle user body portion posture is in the predefined alignment. The method further includes providing instructions to the vehicle user to move vehicle user eyes to track the plurality of vehicle visual indicators, and obtaining a vehicle user eye movement from a second vehicle detector. The method further includes determining whether the vehicle user eye movement meets a predetermined condition, and actuating a control action accordingly.

The invention discloses a psychological health monitoring and self-adaptive nursing decision-making system and method based on electroencephalogram signals, and relates to the field, and the method comprises the following steps: S1, collecting electroencephalogram signals and eye movement characteristics, and generating a multi-modal original signal data set; s2, electroencephalogram alpha wave asymmetry and eye movement characteristics are extracted, and delta-alpha spectrum overlapping pathological interference is eliminated; s3, inputting an evaluation model and a rule base, matching a DLB pathological marker and a symptom association rule, and marking a mental health risk level; and S4, matching an intervention strategy, and dynamically updating the evaluation model and the symptom association rule. The multi-modal fusion strategy effectively covers an association path of a DLB patient from molecular pathology to clinical symptoms, further provides a high-reliability data basis for subsequent risk level marking, realizes extraction of pure alpha wave energy and correction of prefrontal lobe alpha wave asymmetry aiming at frequency spectrum overlapping interference of delta waves and alpha waves in electroencephalogram signals, and improves the accuracy of risk level marking. Emotion abnormal degree misjudgment caused by false increase of the FAA value is avoided.

The invention provides an eye health state monitoring and evaluation method and device based on big data, and relates to the technical field of health monitoring, and the method comprises the steps: obtaining first information, second information and third information; performing three-dimensional reconstruction processing according to the first information to obtain three-dimensional eye feature data; performing pattern recognition processing according to the second information to obtain historical health feature data; performing time sequence analysis according to the historical health feature data and the third information to obtain behavior-environment cumulative effect data; performing data fusion processing according to the historical health feature data and the behavior-environment cumulative effect data, and obtaining potential risk data by predicting potential risk factors of child eye health; and performing evaluation processing according to the three-dimensional eye feature data and the risk data to obtain an evaluation result. According to the method, the modes and trends related to the specific eye diseases are recognized by using the big data mining and mode recognition technology, and potential health problems can be recognized in advance.