3193results about "Acquiring/recognising eyes" patented technology

A system for serving personalized advertisements based on biometric and identifying data captured by sensors. The system employs pre-processing, feature extraction, and recognition modules to identify individuals. A context-aware processing module adapts recognition strategies based on environmental data and analyzes personal information to confirm the identity of the targeted individual and send targeted advertisements. The system includes display and speaker modules for visual and audio advertisements, a predictive interaction modeling module for generating personalized advertisements, and a cross-platform behavior analysis module for analyzing user behavior across offline and online platforms.

A driving state monitoring and feedback method and system based on multi-modal human factors intelligent data analysis, and an edge computing terminal device. The method comprises: receiving multi-modal human factors data of a tested driver that is collected in real time (S110); pre-processing the multi-modal human factors data, wherein the pre-processing comprises denoising processing and data normalization processing (S120); sending the pre-processed multi-modal human factors data into a pre-trained first state recognition model, so as to obtain a real-time recognized driver state, wherein driver states include a normal state and abnormal states, and the types of the abnormal states include a plurality of states such as a fatigue state, a distracted state and an angry state (S130); and when it is recognized that the driver state is an abnormal state, generating, for different categories of abnormal states, driving state feedback instructions to a driving intervention system, such that the driving intervention system performs state feedback adjustments on the driver on the basis of the received driving state feedback instructions (S140). The method and system can recognize different driving states of a driver in real time and then perform processing on the basis of different driving states, thereby avoiding the occurrence of traffic accidents.

Systems and techniques are described for performing foveated sensing. In some aspects, a method (e.g., implemented by an image sensor) can include capturing sensor data for a frame associated with a scene, obtaining information corresponding to a region of interest (ROI) associated with the scene, generating a first portion (having a first resolution) of the frame corresponding to the ROI, generating a second portion of the frame having a second resolution, and outputting (e.g., to an image signal processor (ISP)) the first portion and the second portion. In some aspects, a method (e.g., implemented by an ISP) can receive, from an image sensor, sensor data for a frame associated with a scene, generating a first version (having a first resolution) of the frame based on an ROI associated with the scene, and generating a second version of the frame having a second resolution (lower than the first resolution).

The invention provides an emotion recognition method based on electroencephalogram eye movement multi-mode cross-attention feature fusion, and the method comprises the steps: firstly carrying out the preprocessing of an emotion recognition public data set, and building a corresponding training set and a test set; secondly, constructing an electroencephalogram eye movement multi-mode cross-attention feature fusion emotion recognition model; then carrying out model training and performance testing; finally, an electroencephalogram eye movement multi-mode online emotion recognition system is built, and the effectiveness of the method is verified. The method has the advantages that an electroencephalogram eye movement multi-mode cross-attention feature fusion emotion recognition model is designed, a dynamic graph convolutional network and an attention mechanism are flexibly applied, meanwhile, multi-dimensional data are used, the problem that single-mode emotion recognition information is insufficient is solved, and the recognition accuracy is improved; and an electroencephalogram eye movement multi-mode online emotion recognition system is built, so that the interactivity is enhanced. The average recognition accuracy of the method reaches 97.94% and is superior to that of an existing optimal method, and meanwhile the accuracy of online emotion recognition reaches 87.4%.

An apparatus in one embodiment comprises at least one wearable device, with the at least one wearable device comprising at least one of a transmitter and a receiver of a wireless transmitter-receiver pair. The transmitter of the wireless transmitter-receiver pair transmits an acoustic signal, and the receiver of the wireless transmitter-receiver pair receives the acoustic signal. The received acoustic signal is processed utilizing a machine learning system to detect at least one characteristic of a user of the at least one wearable device. In some embodiments, a given wearable device comprises at least first and second wireless transmitter-receiver pairs, with the transmitter of each of the first and second wireless transmitter-receiver pairs transmitting an acoustic signal having a different carrier frequency. In such embodiments, a multi-channel echo profile may be generated using the multiple acoustic signals and classified by the machine learning system to detect the at least one characteristic.

The invention relates to the technical field of data processing, and particularly provides an operation intention recognition method, system and device based on multi-mode fusion and a medium, and the method comprises the steps: synchronously collecting interaction data of at least two modes of a user, the modes comprising at least two of gestures, voice and eye gaze; carrying out alignment processing on the interaction data, wherein the alignment processing comprises time synchronization and space mapping to a unified coordinate system; recognizing structured semantic information from each piece of aligned modal data, wherein the structured semantic information comprises a gesture type, a voice text and a fixation point coordinate; and based on a preset semantic rule and context memory, performing semantic association and anaphora resolution on the structured semantic information to obtain an operation intention. The method effectively overcomes the inherent defects of unnatural single-mode interaction, easy ambiguity and poor fault tolerance.

The invention relates to a method for evaluating the situation awareness ability of a pilot based on a Ti-SA model. The Ti-SA model comprises an eye movement trace multi-dimensional attention enhancement module, an AOI multi-scale comprehensive feature module and a fusion module. The evaluation method comprises the steps of collecting eye movement trace time sequence data and flight training videos of a pilot at continuous moments in real time, and obtaining corresponding AOI fixation time sequence data based on the eye movement trace time sequence data, the flight training videos and a preset AOI subdivision area; preprocessing the eye movement trace time sequence data and the AOI fixation time sequence data; inputting the preprocessed eye movement trace time sequence data into an eye movement trace multi-dimensional attention enhancement module to obtain a category probability value corresponding to the eye movement trace time sequence data; inputting the preprocessed AOI fixation time sequence data into an AOI multi-scale comprehensive feature module to obtain a category probability value corresponding to an AOI fusion feature; and inputting the two category probability values into a fusion module to obtain a predicted situation awareness level.

The invention relates to the technical field of ophthalmology medical diagnosis, and discloses a three-dimensional reconstruction-based fundus focus intelligent labeling method and diagnosis system. The method comprises the following steps: receiving multi-modal image data streams such as fundus color photos, OCT images and FFA images of an ophthalmological patient; performing spatial registration and feature fusion by using a pre-trained lesion feature fusion model to generate a three-dimensional lesion probability distribution diagram and a lesion category confidence matrix; constructing an adaptive annotation threshold model to generate a multi-modal annotation instruction set; based on the focus development chain model, focus development is simulated, and instruction set parameters are optimized and labeled; and iteratively optimizing through a distributed reinforcement learning framework, and outputting the focus labeling action sequence to an ophthalmology diagnosis platform. According to the method, multi-modal image information can be integrated, the diagnosis accuracy and efficiency are improved, personalized diagnosis is realized, resources are reasonably utilized, and powerful support is provided for ophthalmic disease diagnosis.

The invention relates to the field of medical care informatics, in particular to an ophthalmic disease prediction model training method based on artificial intelligence. Comprising the following steps which are executed in sequence: S1, acquiring a data set of ophthalmic diseases, and performing disease labeling on each image in the data set; s2, performing data expansion on the labeled data set to obtain a training data set of ophthalmic diseases; s3, performing feature extraction on the training data set of the ophthalmic disease by using a neural network algorithm based on dynamic weight redistribution as a feature extraction model to obtain extraction features of the ophthalmic disease; s4, using a self-encoding neural network based on feature refinement as a feature dimensionality reduction model to perform dimensionality reduction on the extracted features, and obtaining initial low-dimensional features of the ophthalmic disease after dimensionality reduction; s5, inputting the initial low-dimensional features of the ophthalmic diseases after dimension reduction into a classifier model for training, and obtaining a classification result; s6, classifying the data set by using the trained model; features of ophthalmic disease data are effectively captured.

The embodiment of the invention provides a human eye attention positioning method and device based on a pre-trained neural network, and the method comprises the steps: obtaining human eye movement data in real time through an eye movement tracking device, extracting feature information related to a human eye fixation region through a pre-trained neural network model containing an attention mechanism based on the human eye movement data, and obtaining a human eye attention positioning result. Inputting the extracted feature information into a regression model, and calculating space coordinates of a human eye fixation area according to the feature information through the regression model; according to the invention, accurate positioning of the human eye fixation area can be realized through pre-training the neural network model, and fixation tracking precision and response speed in a complex scene are significantly improved.

The invention provides a driver fatigue monitoring technology using eye movement tracking, and relates to the field of auxiliary driving, and the technology comprises the steps: collecting a face image frame through a front-end camera, carrying out Medipe (Media Pipeline) detection, recording a head posture, and recognizing an iris center region; extracting continuous eye closing time and mouth state information of the driver through the face information; a PNP algorithm is used; a world coordinate system and a camera coordinate system are established, and the sight line vector and the external environment are located in the same space; modeling is carried out on the human eyes and the fixation area; coordinates of a fixation point in a three-dimensional space are obtained through calculation of the geometric model; real-time attention analysis is carried out through the fixation point, and an analysis result is determined; the attention change condition and the fatigue state of the driver are effectively recognized through a BP neural network in combination with a fixation point analysis result. And early warning measures are taken in time to remind a driver. The system is crucial for improving the driving safety, and necessary warning and intervention can be provided when the driver is distracted or fatigued.

A line-of-sight direction tracking method includes: providing corneal reflection for an eye of a user by using a plurality of light sources, and capturing images including a face of the user by using a plurality of cameras; determining coordinates of a light source reflection point and coordinates of a pupil center in a world coordinate system by means of a human eye feature set acquired from the images including the face, based on hardware calibration parameters; determining a line-of-sight optical axis according to the coordinates of the light source reflection point and the coordinates of the pupil center, and reconstructing, based on the line-of-sight optical axis, a line-of-sight visual axis by means of a compensation angle; and determining a point of sight on a target object according to the line-of-sight visual axis and a position of the target object in the world coordinate system.

The invention provides a self-adaptive eye movement tracking method and device. The method comprises the steps that a calibration target position matrix and a historical center position are initialized; obtaining state information of the contact lenses, a calibration eye image sequence and a calibration sensing position sequence by using the intelligent head-mounted equipment, the intelligent contact lenses and the calibration target position matrix; processing the calibration target position matrix, the historical center position, the contact lens state information, the calibration eye image sequence and the calibration sensing position sequence to obtain a prediction target matrix, a prediction pupil center position matrix, a post-processing mark and an updated historical center position and calibration target position matrix; the post-processing mark is judged, and the calibration target position matrix is updated; and extracting the calibration target position matrix to obtain M predicted fixation point positions. Therefore, according to the method provided by the invention, the eye movement tracking mode can be adaptively selected, the defect of a single tracking method is overcome, and the accuracy and smoothness of eye movement tracking are improved.

The invention provides an interactive classroom student dynamic portrait generation method, a medium and a system, and belongs to the technical field of student dynamic portrays.The method comprises the steps that a student facial image sequence is collected, facial feature points are extracted to construct an expression feature matrix, and sight line focus data are obtained by combining an eye movement tracking technology to calculate an eye light concentration degree index; and establishing an individual association model, identifying specific expressions and analyzing attention fluctuation characteristics. And fusing multi-dimensional features such as an expression similarity vector and an eye light concentration index into a dynamic attention state score. Learning state evaluation is realized based on a multilayer bidirectional converter network, wherein multi-head attention mechanism parameters are determined by expression change frequency, an eye light concentration threshold and an attention fluctuation rate. And finally, generating a student dynamic portrait feature map, and forming a classroom interaction effect evaluation index through clustering analysis, thereby solving the technical problem that the dynamic change of the student classroom attention state is difficult to accurately evaluate based on a single feature.

The invention belongs to the technical field of intelligent education, and particularly relates to a classroom attention detection method and system based on multi-modal data fusion. Aiming at the problems of high equipment cost, low multi-source data fusion efficiency, insufficient privacy protection and the like in the prior art, the invention provides the following solutions: collecting face, eye movement, posture, voice signals and heart rate variability data of a student through a sensor; multi-modal data synchronization is realized by adopting a time sequence alignment algorithm; respectively extracting a visual attention feature, a voiceprint matching feature and a physiological wake-up feature by using a lightweight deep learning model; constructing a multi-modal data fusion network, and dynamically adjusting a feature weight in combination with a classroom scene; attention anomaly detection is realized by adopting a hybrid model, and real-time early warning is output through edge computing equipment. The method has the beneficial effects that the hardware cost is greatly reduced while the detection precision is ensured, and the privacy of students is effectively protected; a dynamic weight distribution mechanism improves the adaptability of different teaching scenes.

Systems, methods, and non-transitory media are provided for a vehicle and mobile device interface for vehicle occupant assistance. An example method can include determining, based on one or more images of an interior portion of a vehicle, a pose of a mobile device relative to a coordinate system of the vehicle; determine a state of an occupant of the vehicle; and send, to the vehicle, data indicating the state of the occupant and the pose of the mobile device relative to the coordinate system of the vehicle.

The present disclosure generally relates to enrolling a biometric feature for use with a peripheral device. While a computer system is connected to an external device, the computer system receives a first request to enroll a biometric feature. In response to receiving the request and in accordance with a determination that the request satisfies a first set of one or more criteria, the computer system displays a user interface object prompting a user to provide an authorization input to one or more input devices physically connected to the computer system. While displaying the first user interface object, the computer system receives user input. In response to receiving the user input and in accordance with a determination that the user input includes the authentication input, the computer system initiates a process to enroll the biometric feature using a biometric sensor that is integrated with the external device.

The invention discloses an eye movement tracking method, an eye movement tracking device, eye movement tracking equipment, computer equipment and a storage medium. The method comprises the following steps: calculating calibration gaze information and a calibration pupil position according to first event data collected by an event camera; determining a region of interest based on the calibrated pupil position; processing second event data collected by the event camera according to the region of interest to generate region of interest data, the collection time of the second event data being later than the collection time of the first event data; according to the region-of-interest data, calculating a current fixation position offset; according to the calibration gaze information and the current gaze position offset, the current gaze information is determined, the calibrated pupil position is updated based on the current gaze position offset, the gaze direction, the moving direction and the moving distance of the pupil are accurately captured in real time by utilizing the characteristics of high time resolution and the like of the event camera, and the eye movement tracking effect is effectively improved.

The invention discloses a pupil image recognition and disease change evaluation system based on image recognition, and relates to the technical field of monitoring analysis, and the system comprises the steps: obtaining the identity information and historical pupil data of a patient, and building a pupil dynamic database corresponding to the patient; alternately irradiating the eyes of the patient, and collecting a dynamic image sequence corresponding to a pupil light reflex area of the patient; performing spatial-temporal characteristic analysis on the dynamic image sequence, establishing a pupil diameter dynamic change curve, and generating a light reflection sensitivity index of the pupil of the patient according to the pupil diameter dynamic change curve; constructing a pupil dynamic evaluation model according to the pupil diameter dynamic change curve and the light reflex sensitivity index of the patient pupil, and outputting a pupil grade evaluation result based on the pupil dynamic evaluation model; comprehensive analysis is conducted on pupil grade evaluation results, analysis results are uploaded to an intensive care central system, and a grading early warning mechanism is triggered. The application has the effect of reducing the occurrence of illness state delay.

The invention relates to the technical field of screen control, in particular to a computer screen control method and system. The method comprises the following steps: collecting a computer lens image to carry out scene modeling, generating a three-dimensional usage scene model, carrying out scene privacy evaluation and generating a privacy degree parameter, analyzing a peep-proof demand according to the privacy degree parameter, generating a screen peep-proof demand, carrying out portrait positioning when peep-proof is needed, generating a portrait positioning set, and carrying out image processing according to the portrait positioning set. And calculating a virtual contact distance based on the portrait positioning set, performing contact sight simulation to obtain a figure sight track, analyzing a screen visible range according to the sight track and the virtual contact distance, and dynamically adjusting the screen brightness to realize the peep-proof control method. According to the invention, a safer computer screen control method is realized.

The invention discloses a naked-eye 3D display optimization method based on real-time eyeball tracking, and particularly relates to the technical field of naked-eye 3D display, and the method comprises the steps: capturing eyeball movement data in real time through a visual angle tracking sensor, and collecting illumination information in combination with an ambient light sensor; depth perception parameters are calculated based on pupil diameter variation and frequency, and a time sequence prediction type dynamic compensation coefficient is generated in combination with eyeball movement acceleration; acquiring an initial fixation point coordinate by using an improved spherical projection mapping model, and performing compensation coefficient correction to obtain a real-time coordinate; and finally, according to the real-time coordinates, dynamically adjusting the refractive index distribution of the nanostructure layer, the rotation angle of the polarizer, the focal length of the optical lens and other optical modulation parameters. The real-time distance is calculated through the binocular parallax algorithm, the depth mapping value is generated by combining the focal length of the camera and the baseline distance, the method can adapt to the illumination change and the user view angle, and the 3D display effect is optimized.

Apparatus, including a retaining structure (54), positioned near a subject's eye that has a pupil with a diameter, an optical combiner (52A) mounted on the structure before the eye, and a pixelated screen (60A) having an array of variably transparent pixels coating the combiner. There is an image capturing device (68A) mounted on the structure to capture an image of a scene viewed by the eye, and a projector (64A) is mounted on the structure to project at least one of a portion of the captured image and a stored image onto a section of the screen at a selected location thereof. A processor (26) renders the screen section at least partially opaque, selects the section location in response to a region of interest in the scene identified by analysis of the captured image, and determines a dimension of the section in response to the pupil diameter.

The invention relates to the technical field of biological agriculture, and discloses a live chicken state automatic identification system and method based on chicken face image analysis, and the system comprises a unique identity identification module which is used for generating the identity identification of a chicken through a chicken face image; the health monitoring module is used for setting a chicken classification health detection mechanism and creating a chicken body temperature monitoring unit of the coop; the egg quality detection module is used for identifying eggshell characteristics corresponding to the first type of eggs and the second type of eggs; the behavior tracking module is used for collecting ingestion records of the chickens and identifying abnormal behaviors of the chickens; the disease early warning module is used for outputting health indexes of the chickens according to the abnormal behaviors, the eggshell characteristics, the ingestion records and the real-time body temperature; and the live chicken automatic identification module is used for executing live chicken state automatic identification processing of the chicken coop in combination with the chicken face image, a classification health detection mechanism, a behavior tracking network and a disease early warning mechanism. The survival rate and the breeding efficiency of the chickens are improved.

The invention relates to a pilot SA ability evaluation system based on eye movement tracking and space directional flight simulation, and the system comprises a space directional flight simulator which is used for simulating a real flight environment; the eye tracker is used for recording an eyeball movement track of a pilot in real time in a simulated real flight environment to obtain eye movement track time sequence data and shooting a flight training video; the microprocessor evaluation module comprises a time sequence data processing module and a Ti-SA model; the time sequence data processing module obtains corresponding AOI fixation time sequence data based on the eye movement trace time sequence data, the flight training video and a preset AOI subdivision area; the Ti-SA model is used for predicting a current SA capability value of a pilot based on the eye movement trace time sequence data and the AOI fixation time sequence data and judging whether the predicted SA capability value is lower than a preset SA threshold value or not, and if the predicted SA capability value is lower than the preset SA threshold value, an early warning instruction is sent to the early warning and voice broadcasting module; and the early warning and voice broadcasting module is used for early warning when the SA capability value is lower than a preset SA threshold value.

A gaze estimation apparatus according to one or more embodiments may estimate the gaze direction of a target person using calibration information including feature information about the gaze of the eyes of the target person looking in a predetermined direction and true value information indicating the true value for the predetermined direction, in addition to a target image including the eye(s) of the target person. The gaze estimation apparatus may thus estimate the gaze direction of the target person reflecting individual differences, and may allow estimation of the gaze direction of the target person with higher accuracy.

The invention discloses a financial escorting process abnormal behavior identification method and system, and relates to the field of data processing. The method comprises the following steps: acquiring personnel characteristic data, and establishing a multi-modal biological characteristic database; matching features extracted from the real-time personnel feature data with features stored in a multi-modal biological feature database by using a convolutional neural network to obtain a matching result; when the matching result is that the matching is successful, continuing to obtain the video stream of the monitoring area in real time and preprocessing the video stream; performing target detection on the preprocessed video stream by using a deep learning algorithm, and identifying a tracking target; performing continuous frame tracking on the detected tracking target by using a target tracking algorithm to obtain a motion track of the tracking target; extracting attitude information of the target by using an attitude estimation algorithm, and constructing a multi-dimensional behavior feature vector in combination with a motion track and spatial-temporal features of the target; establishing a preset abnormal behavior rule base, wherein the abnormal behavior rule base comprises a known abnormal behavior set; and based on the output of the multi-dimensional behavior feature vector, in combination with a preset abnormal behavior rule base, performing real-time analysis on the identified behavior, and identifying whether the target is performing an abnormal behavior or not. The technical problems that in the prior art, risks in the escort process cannot be recognized at the same time through a single personnel recognition technology, and the requirement for timely finding out the escort risk is difficult to meet are solved, real-time comparative analysis of the movement track and the preset track is achieved through verification of personnel multi-mode biological characteristics and tracking of the movement track of the target, and the safety of the escort risk is improved. Therefore, the real-time monitoring efficiency is improved, and the technical effects of reducing the escort risk are achieved.

The invention discloses a multi-modal data glaucoma classification method and system, a terminal and a medium, and the method comprises the steps: obtaining a color fundus image and an OCT image in different image modes, and carrying out the advanced feature extraction of the color fundus image and the OCT image based on a first feature extraction module and a second feature extraction module, the mode specificity characteristics of the two modes are obtained; fusing the mode specific features of the two modes based on a shallow feature fusion module to obtain shallow fusion features; inputting the mode specific features of the two modes and the shallow fusion features into a deep feature fusion module for fusion to obtain final fusion features; and based on a multi-layer perceptron classifier, performing classification prediction on the final fusion feature to obtain a classification result, the classification result including no glaucoma, early glaucoma or mid-late glaucoma. According to the method, through two-stage feature extraction, two-stage feature fusion and the classifier, accurate glaucoma classification can be obtained.