1911results about "Acquiring/recognising eyes" patented technology

The invention relates to the technical field of data processing, and particularly provides an operation intention recognition method, system and device based on multi-mode fusion and a medium, and the method comprises the steps: synchronously collecting interaction data of at least two modes of a user, the modes comprising at least two of gestures, voice and eye gaze; carrying out alignment processing on the interaction data, wherein the alignment processing comprises time synchronization and space mapping to a unified coordinate system; recognizing structured semantic information from each piece of aligned modal data, wherein the structured semantic information comprises a gesture type, a voice text and a fixation point coordinate; and based on a preset semantic rule and context memory, performing semantic association and anaphora resolution on the structured semantic information to obtain an operation intention. The method effectively overcomes the inherent defects of unnatural single-mode interaction, easy ambiguity and poor fault tolerance.

The invention relates to the technical field of ophthalmology medical diagnosis, and discloses a three-dimensional reconstruction-based fundus focus intelligent labeling method and diagnosis system. The method comprises the following steps: receiving multi-modal image data streams such as fundus color photos, OCT images and FFA images of an ophthalmological patient; performing spatial registration and feature fusion by using a pre-trained lesion feature fusion model to generate a three-dimensional lesion probability distribution diagram and a lesion category confidence matrix; constructing an adaptive annotation threshold model to generate a multi-modal annotation instruction set; based on the focus development chain model, focus development is simulated, and instruction set parameters are optimized and labeled; and iteratively optimizing through a distributed reinforcement learning framework, and outputting the focus labeling action sequence to an ophthalmology diagnosis platform. According to the method, multi-modal image information can be integrated, the diagnosis accuracy and efficiency are improved, personalized diagnosis is realized, resources are reasonably utilized, and powerful support is provided for ophthalmic disease diagnosis.

The invention belongs to the technical field of intelligent education, and particularly relates to a classroom attention detection method and system based on multi-modal data fusion. Aiming at the problems of high equipment cost, low multi-source data fusion efficiency, insufficient privacy protection and the like in the prior art, the invention provides the following solutions: collecting face, eye movement, posture, voice signals and heart rate variability data of a student through a sensor; multi-modal data synchronization is realized by adopting a time sequence alignment algorithm; respectively extracting a visual attention feature, a voiceprint matching feature and a physiological wake-up feature by using a lightweight deep learning model; constructing a multi-modal data fusion network, and dynamically adjusting a feature weight in combination with a classroom scene; attention anomaly detection is realized by adopting a hybrid model, and real-time early warning is output through edge computing equipment. The method has the beneficial effects that the hardware cost is greatly reduced while the detection precision is ensured, and the privacy of students is effectively protected; a dynamic weight distribution mechanism improves the adaptability of different teaching scenes.

The invention discloses an eye movement tracking method, an eye movement tracking device, eye movement tracking equipment, computer equipment and a storage medium. The method comprises the following steps: calculating calibration gaze information and a calibration pupil position according to first event data collected by an event camera; determining a region of interest based on the calibrated pupil position; processing second event data collected by the event camera according to the region of interest to generate region of interest data, the collection time of the second event data being later than the collection time of the first event data; according to the region-of-interest data, calculating a current fixation position offset; according to the calibration gaze information and the current gaze position offset, the current gaze information is determined, the calibrated pupil position is updated based on the current gaze position offset, the gaze direction, the moving direction and the moving distance of the pupil are accurately captured in real time by utilizing the characteristics of high time resolution and the like of the event camera, and the eye movement tracking effect is effectively improved.

The invention discloses a pupil image recognition and disease change evaluation system based on image recognition, and relates to the technical field of monitoring analysis, and the system comprises the steps: obtaining the identity information and historical pupil data of a patient, and building a pupil dynamic database corresponding to the patient; alternately irradiating the eyes of the patient, and collecting a dynamic image sequence corresponding to a pupil light reflex area of the patient; performing spatial-temporal characteristic analysis on the dynamic image sequence, establishing a pupil diameter dynamic change curve, and generating a light reflection sensitivity index of the pupil of the patient according to the pupil diameter dynamic change curve; constructing a pupil dynamic evaluation model according to the pupil diameter dynamic change curve and the light reflex sensitivity index of the patient pupil, and outputting a pupil grade evaluation result based on the pupil dynamic evaluation model; comprehensive analysis is conducted on pupil grade evaluation results, analysis results are uploaded to an intensive care central system, and a grading early warning mechanism is triggered. The application has the effect of reducing the occurrence of illness state delay.

The invention discloses a naked-eye 3D display optimization method based on real-time eyeball tracking, and particularly relates to the technical field of naked-eye 3D display, and the method comprises the steps: capturing eyeball movement data in real time through a visual angle tracking sensor, and collecting illumination information in combination with an ambient light sensor; depth perception parameters are calculated based on pupil diameter variation and frequency, and a time sequence prediction type dynamic compensation coefficient is generated in combination with eyeball movement acceleration; acquiring an initial fixation point coordinate by using an improved spherical projection mapping model, and performing compensation coefficient correction to obtain a real-time coordinate; and finally, according to the real-time coordinates, dynamically adjusting the refractive index distribution of the nanostructure layer, the rotation angle of the polarizer, the focal length of the optical lens and other optical modulation parameters. The real-time distance is calculated through the binocular parallax algorithm, the depth mapping value is generated by combining the focal length of the camera and the baseline distance, the method can adapt to the illumination change and the user view angle, and the 3D display effect is optimized.

The invention relates to the technical field of biological agriculture, and discloses a live chicken state automatic identification system and method based on chicken face image analysis, and the system comprises a unique identity identification module which is used for generating the identity identification of a chicken through a chicken face image; the health monitoring module is used for setting a chicken classification health detection mechanism and creating a chicken body temperature monitoring unit of the coop; the egg quality detection module is used for identifying eggshell characteristics corresponding to the first type of eggs and the second type of eggs; the behavior tracking module is used for collecting ingestion records of the chickens and identifying abnormal behaviors of the chickens; the disease early warning module is used for outputting health indexes of the chickens according to the abnormal behaviors, the eggshell characteristics, the ingestion records and the real-time body temperature; and the live chicken automatic identification module is used for executing live chicken state automatic identification processing of the chicken coop in combination with the chicken face image, a classification health detection mechanism, a behavior tracking network and a disease early warning mechanism. The survival rate and the breeding efficiency of the chickens are improved.

The invention belongs to the field of artificial intelligence and fundus image enhancement, and discloses a fundus image enhancement method and system based on machine learning, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original fundus spectral image, and carrying out the preprocessing of the original fundus spectral image, and obtaining a preprocessed spectral image; constructing a backbone network based on a residual network, and extracting multi-scale features of the preprocessed spectral image in combination with cavity convolution; introducing a channel-space-spectrum multi-attention module into the backbone network, and performing multi-attention fusion on the multi-scale features to obtain an enhanced feature map; and performing adversarial training on the backbone network by using the generative adversarial network and the enhanced feature map, and performing image enhancement on the collected fundus spectral image by using the trained network to obtain an enhanced fundus spectral image. According to the method, more-dimensional image support is provided for medical diagnosis, the quality of the fundus image can be effectively improved, the diagnosis accuracy of a doctor on fundus lesions is improved, and the method has important clinical application value.

The invention discloses an access control equipment data management method and system based on multi-source fusion. The method comprises the following steps: collecting a multi-source access control data stream in real time; based on a preset feature extraction rule set, extracting a multi-modal biological feature vector, a voucher legality identifier, an abnormal behavior probability value and an equipment health degree index, and inputting the multi-modal biological feature vector, the voucher legality identifier, the abnormal behavior probability value and the equipment health degree index into a dynamic security assessment matrix generation model to generate a real-time security assessment matrix; matching the dimension safety score of the real-time safety evaluation matrix with a preset threshold strategy library, and dynamically generating an access control strategy instruction set; and issuing the access control strategy instruction set to the target access control equipment execution terminal. The method has the following advantages and effects: the fault tolerance bottleneck of a single-dimensional decision chain is broken through, and the system misjudgment rate is reduced by at least one order of magnitude on the premise of ensuring the security by establishing a dynamic coupling mechanism of the multi-source data stream.

The invention discloses an optic disk atrophy arc image segmentation method based on full supervision, and particularly relates to the technical field of medical equipment, and the method comprises the following steps: S1, multi-scale feature extraction, S2, local-global feature enhancement, S3, adaptive feature fusion, S4, gradient flow optimization, S5, mixing of a loss function, and S6, data enhancement and generalization optimization. According to the method, the long-range dependence of the optic disk and the atrophic arc is globally modeled through the PVTv2 encoder, the detail segmentation precision of the optic disk atrophic arc at the blood vessel crossing and fuzzy boundary is remarkably improved in combination with the multi-scale attention and dilated convolution of the feature enhancement module, cross-level feature fusion is realized with linear complexity based on the Mama decoder, and the accuracy of feature fusion is improved. Efficiency and structure coherence are considered; the weight is balanced by the mixed loss function, and early lesion missing detection is reduced; a data enhancement and residual module enhances the generalization ability of the model, and a high-precision and efficient quantitative analysis tool is integrally provided for early screening of blind eye diseases such as pathological myopia and glaucoma.

The invention provides a visual fatigue relieving method based on ambient light self-adaption and an AI algorithm, and relates to the technical field of visual health protection. The visual fatigue relieving method based on ambient light self-adaption and the AI algorithm comprises the following specific steps: S1, data acquisition: acquiring a user eye image sequence in real time through a high-frame-rate camera; and S2, parameter extraction: when the user uses the eye for the first time, testing the eyes by combining the adjustable light source with ambient light. Multi-dimensional features (such as eyeball movement, blinking mode, pupil function and the like) of eyes are collected in real time through a high-frame-rate camera, an individualized base line is established and a fatigue rate value is dynamically calculated in combination with data of an ambient light sensor, and conversion from passive response to active prevention is realized. Through a hierarchical intervention strategy (such as brightness adjustment, blue light control and forced rest), the visual load is remarkably reduced, the intervention efficiency is improved, and the problem of insufficient adjustment hysteresis and individual adaptability in the prior art is solved.

The invention discloses an ideological and political large-scale course teaching attention assessment data acquisition method based on sight tracking. The method comprises the following steps: S1, high-definition large-scene eye movement tracking photographing equipment captures eye images of students in a course teaching process in real time through an infrared camera; s2, performing pupil area detection and pupil center fitting on the eye image to obtain sight tracking data; s3, performing gaze hotspot analysis and gaze duration statistics according to the sight tracking data to quantify attention distribution and change trend of the students and visually display the attention distribution and change trend; and S4, generating a teaching quality evaluation report and teaching optimization suggestions according to the gazing hotspot analysis and gazing duration statistical results. The classroom teaching effect can be objectively evaluated in real time, subjective evaluation errors are reduced, the accuracy and efficiency of teaching feedback are improved, the method is suitable for various scenes such as traditional classrooms, online education and experiment teaching, and the method has the industrial application prospect of education digital transformation.

The invention discloses an orbit MRI image analysis method and system based on deep learning, and belongs to the field of image analys.The method comprises the steps that MRI image data of an orbit area of a patient are obtained, and the MRI image data comprise a conventional T1WI sequence, a T2WI plain scanning sequence, a conventional enhancement sequence and an extraocular muscle fibrillation enhancement sequence; performing preprocessing on the MRI image to obtain a standardized image; inputting the standardized image into a deep learning segmentation model, and outputting segmentation masks of extraocular muscle, lacrimal gland and intraorbital fat; calculating quantitative indexes of a target structure based on the segmentation mask, wherein the target structure comprises extraocular muscle, lacrimal gland and intraorbital fat; and generating a diagnosis report containing the quantitative index. Human experience dependence is avoided, and objective and uniform anatomical structure segmentation results are ensured.

Provided are a driving state monitoring and feedback method and system based on multi-modal human-factor intelligent data analysis, and an edge computing terminal device. The method includes: receiving multi-modal human-factor data collected in real time from a tested driver; preprocessing the multi-modal human-factor data; inputting the preprocessed multi-modal human-factor data to a pre-trained first state identification model to obtain a driver state identified in real-time, the driver state including a normal state and a plurality of abnormal states; and generating, in response to the driver state being identified as an abnormal state, a driving state feedback instruction for the category of the abnormal state and sending the driving state feedback instruction to a driving intervention system, to cause the driving intervention system to perform state feedback adjustment on the driver based on the received driving state feedback instruction.

Apparatuses, systems, and techniques are presented to estimate user gaze. In at least one embodiment, one or more neural networks are used to determine coarse and fine gaze estimates for one or more users.

The invention relates to a self-adaptive visual training method based on eye features, and belongs to the technical field of visual health and artificial intelligence. In order to solve the problems that traditional visual training equipment is heavy in structure, single in training mode and lack of personalized regulation and control and concentration state monitoring, the visual training equipment obtains eye images of a user in real time through an image acquisition module, adopts visual intelligent analysis, extracts double features of eye contours and pupil positions, calculates the visual concentration degree, and improves the visual training efficiency. And a concentration state is judged by combining a dynamic self-adaptive threshold value. When the concentration degree is insufficient, the control module triggers intervention mechanisms such as prompt or training pause and the like, and the remote interaction module uploads data to the cloud platform to generate a personalized training scheme. Intelligent and personalized regulation and control of the training process are achieved, the training effect and compliance are improved, the system is compact in structure and suitable for various scenes, and an efficient solution is provided for vision health.

This invention discloses a 2D gaze point estimation method based on the pupil-corneal reflection vector method. It utilizes the star ray method for pupil center localization and adds a classification pupil detection mechanism for error correction. Corneal feature extraction is performed to obtain the center coordinates of the corneal reflected light spot. Step 4: Establish a gaze mapping model, and assign the detected pupil-corneal reflection vector V(V) to each frame of the image. x V y The gaze coordinates G(G) mapped onto the screen x G y This invention enables 2D gaze point estimation based on the pupil-corneal reflection vector method. Compared with existing technologies, this invention can improve the accuracy of 2D gaze point estimation.

The embodiment of the invention discloses augmented reality equipment and a corresponding equipment calibration method. The method comprises the steps of obtaining a calibration request, displaying a virtual mark on a target display device, outputting matching prompt information to prompt a user to watch an entity mark and the virtual mark by using target side eyes, and changing a relative posture relationship between the entity mark and the head of the user to enable the entity mark and the virtual mark to be in a matching state in the eyes of the user, and after the matching is determined, calling the target live-action camera assembly to obtain a real scene image containing the entity mark, determining a target offset according to the first position of the virtual mark on the target display device and the second position of the entity mark in the real scene image, and calibrating the augmented reality device according to the target offset. According to the method, equipment calibration is carried out based on the inherent display device of the augmented reality equipment, the live-action camera assembly and the entity marks easily obtained in the external environment, a user can carry out calibration anytime and anywhere, the equipment calibration threshold is lowered, and the method is simple and easy to implement.

The invention provides a multi-modal fusion fatigue screen monitoring recognition and reminding method and system, and the method comprises the steps: data collection: collecting the visual data of a screen monitoring person in real time, and synchronously collecting the physiological data; multi-modal data fusion: adopting a feature weighted fusion method based on an entropy weight method to adaptively distribute weights and generate a fusion fatigue index FFI by quantifying dynamic information entropy of each visual data and physiological data; fatigue grade classification: realizing three-level fatigue state judgment based on a fusion fatigue index FFI obtained by multi-modal data fusion and a dynamic decision tree model; and dynamic intervention: based on a fatigue grade classification result, adopting a double-channel intervention mechanism of bracelet touch alarm and automatic telephone call value length. When fatigue and inattention of a monitoring screen watchman occur, the monitoring screen watchman can be timely and accurately identified and a reminding intervention mechanism is started, so that the continuity and the safety of the monitoring screen work are ensured, and bad safety production events caused by human reasons are avoided.

A drowsy driving detection method comprises: acquiring a side face image of a currently seated driver collected by a camera module; performing face recognition on the side face image to obtain side face feature parameters, and determining, according to the side face feature parameters, whether an ID file corresponding to the currently seated driver exists in a driver ID library; and if yes, periodically acquiring a side face image of the driver in the current period collected by the camera module, obtaining eye movement feature parameters of the driver in the current period according to the side face image of the current period, and determining whether the driver is driving while drowsy according to a comparison result between the eye movement feature parameters of the current period and the normal eye movement feature parameters of the driver.

The invention provides an identity authentication method which can be applied to the technical field of biological recognition. The identity authentication method comprises the following steps: after agreement or authorization of a user is obtained, collecting biological characteristic data of multiple modes of the user; preprocessing the collected biological characteristic data of various modes to generate corresponding biological characteristic vectors; performing quality evaluation on each biological feature vector to generate a corresponding quality score; dynamically calculating a weight coefficient of each biological feature vector in a feature fusion process by using a nonlinear weighting function based on the quality score; based on the weight coefficient, performing feature level fusion on each biological feature vector to generate a primary fusion feature; and performing cross-modal correlation analysis on the primary fusion features by using a multi-branch convolutional neural network based on an attention mechanism, and outputting an identity authentication result. The invention also provides an identity authentication device, equipment, a medium and a program product.

The invention belongs to the field of virtual reality teaching application, and provides a teaching atmosphere automatic adaptation method and system in an immersive teaching environment. The method comprises the following steps: acquiring an eye movement image; recognizing a fixation point; carrying out ROI tracking; carrying out ROI boundary fusion; adaptively optimizing the object; adjusting the brightness of the ROI; and watching object interaction. According to the method, the immersion and interactivity of a future classroom can be improved, the use experience of an immersive virtual environment is facilitated, and deep fusion of an intelligent teaching environment and self cognition of a user is promoted.

The invention relates to a teaching interaction method and device based on a virtual digital human, equipment and a medium. According to the method, semantic segmentation and differential coding are performed on a student video stream according to a face, a gesture and a background region, code rate distribution of each region is dynamically adjusted in combination with an attention weight generated by eye movement tracking, and head steering and 3D parallax rendering of a virtual digital human are driven based on pupil coordinate mapping. Meanwhile, according to network state self-adaptive slice distribution, the technical effects of accurate perception of attention of multiple students in a teaching scene, synchronous response of virtual image actions and real intentions and on-demand optimization of network resources are achieved, and the authenticity of immersive interaction and the utilization rate of system resources are remarkably improved.

Object description information that is transmitted by a biometric recognition apparatus is received, the object description information includes a biometric feature of a target object and location information of the target object. Identity information of the target object is obtained according to the biometric feature. A service information set in association with the identity information is obtained. From the service information set, one or more pieces of candidate service information are selected. The one or more pieces of candidate service information are transmitted to a terminal device associated with the identity information. At least a first piece of target service information returned by the terminal device is received. At least a first service corresponding to the first piece of target service information is processed. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated.

The invention relates to the technical field of large model cheating detection, in particular to an online interview large model cheating detection system and method based on eye movement tracking, and the detection method comprises the steps: S1, system initialization and environment adaptation; s2, eye movement tracking calibration and baseline establishment; s3, collecting and preprocessing real-time eye movement data; s4, executing an anomaly detection algorithm; and S5, carrying out risk assessment and outputting a result. According to the scheme, based on the WebGazer.js technology, the mapping relation between the fixation point and the screen coordinates is established through nine-point initialization calibration, and the personalized coefficient k is generated in combination with the pupil diameter and the facial features of the candidate, so that the eye movement tracking error is greatly reduced, and the large model cheating detection precision requirement can be met without special hardware. The design not only reduces the deployment cost, but also breaks through the hardware limitation, so that the technology can adapt to mainstream notebook equipment, is convenient for large-scale popularization and application, and solves the contradiction that high precision and low cost cannot be achieved at the same time in the traditional scheme.

The invention relates to the technical field of visual health management, in particular to a ciliary muscle adjustment training system and method based on eye movement tracking, and the method comprises the steps: collecting eye movement data of a fixation point track, an adjustment amplitude change rate and pupil response time in real time, and combining personalized parameters such as the age of a user, diopter and a training target; inputting a ciliary muscle training dynamic planning algorithm module, dynamically generating a stage scheme containing a training action type, a single-group duration, a stimulation interval and a focal length adjustment strategy, and adaptively adjusting the strategy according to real-time adjustment stability and a historical ability curve; through multi-dimensional matching analysis of a pre-stored standard eye movement template, a training specification score is calculated from the track goodness of fit, speed uniformity, amplitude standard-reaching rate and binocular coordination, visual fatigue parameters are obtained based on continuous training duration, pupil fluctuation and retina reflex change, a scoring result is dynamically corrected, and a visual fatigue evaluation result is obtained. Personalized, self-adaptive and quantitative safety control of the training process is realized.

Photometric stereo techniques enable using a single camera to perform an enrollment process for creating a user-specific anatomical model of an eye for gaze tracking. The user-specific anatomical model includes information about a user's center of vision at multiple dilation states of the eye, which can be used to enhance the accuracy of gaze tracking techniques. Accurate gaze tracking techniques enable the use of gaze tracking at close range, for example, gaze tracking within a head-mounted display device.

The invention discloses a premature infant retinopathy recognition system based on multi-modal data, and relates to the technical field of computers, artificial intelligence and image processing, and the system comprises a data set construction module which obtains an eye fundus image, carries out the preprocessing of the eye fundus image, and constructs a data set based on the eye fundus image obtained through the preprocessing and a structured prompt project; the primary training module is used for inputting the data set into a preset LLaVA-v1.5 model and training the LLaVA-v1.5 model based on a LoRA mechanism to obtain a primary training model; the secondary training module is used for constructing a knowledge enhanced thinking chain and training the primary training model based on the knowledge enhanced thinking chain to obtain a recognition model; and the recognition module inputs the fundus image to be recognized into the recognition model to obtain a recognition result. According to the novel intelligent diagnosis system, retina image analysis and medical knowledge reasoning are integrated, and high-precision and interpretable ROP screening is achieved.

The present disclosure provides a character recognition model training method and apparatus, a character recognition method and apparatus, a device and a medium, relating to the technical field of artificial intelligence, and specifically to the technical fields of deep learning, image processing and computer vision, which can be applied to scenarios such as character detection and recognition technology. The specific implementing solution is: partitioning an untagged training sample into at least two sub-sample images; dividing the at least two sub-sample images into a first training set and a second training set; where the first training set includes a first sub-sample image with a visible attribute, and the second training set includes a second sub-sample image with an invisible attribute; performing self-supervised training on a to-be-trained encoder by taking the second training set as a tag of the first training set, to obtain a target encoder.

The invention relates to a method and device for emotion evaluation, a medium and a program product. The method comprises the steps of obtaining multi-modal emotion evaluation data of a to-be-evaluated object in a process of performing a plurality of predetermined emotion evaluation tasks on the to-be-evaluated object; performing feature extraction processing on each item of data included in the emotion evaluation data to obtain corresponding feature information; fusing the feature information corresponding to each data to obtain corresponding fused feature information; and outputting a corresponding emotion evaluation result based on the fused feature information by using a trained emotion prediction model. According to the method, the trained emotion prediction model is used, the corresponding emotion evaluation result is output based on the feature information of the multi-modal emotion evaluation data, model pre-training and automatic iteration updating are carried out by adopting the meta-learning algorithm, individual differences are rapidly adapted, and the performance, efficiency and robustness of model prediction are improved.