557 results about "Head movements" patented technology

The invention provides a classroom behavior analysis method and system, electronic equipment and a storage medium, and relates to the technical field of educational informationization, and the method comprises the steps: collecting videos and audios, carrying out image enhancement, noise filtering and frame segmentation on the videos, carrying out noise reduction, sound source positioning and voice segmentation on the audios, and generating standardized images and voice sequences; an improved MTCNN cascade network is combined with a posture estimation technology, facial key points of students are extracted from videos, class arrival states, head postures and facial micro-expressions are recognized, audios are analyzed through a bidirectional LSTM network, and speaking duration and frequency are obtained; constructing individual behavior indexes according to attendance, head actions and expressions of the students; constructing an interaction index according to the teacher and student speaking data; the individual and interaction indexes are summarized to generate the multi-dimensional classroom behavior portrait, and the relation between the portrait and the teaching effect is analyzed, so that the classroom behavior analysis precision can be improved, the data processing capability can be enhanced, and the multi-dimensional classroom behavior comprehensive evaluation can be realized.

The invention provides an immersive concert experience system based on virtual reality, and belongs to the technical field of virtual reality, and the system comprises a dynamic adjustment module which is used for predicting the preference of a user for stage arrangement, light effects and music types through a machine learning model according to the historical preference data of the user, and outputting the preference to the stage arrangement, light effects and music types; dynamically adjusting visual elements and sound field parameters of the virtual scene and a refresh rate and a field angle of the virtual scene in combination with the user physiological indexes and the user head motion trail which are acquired in real time; the action capture and feedback module is used for capturing an interaction instruction sent to the dynamically adjusted virtual scene by the limb action of the user in real time; and the interaction feedback module is used for enabling the virtual performer to respond to the interaction instruction of the user in real time based on the expression recognition and voice synthesis technology, and generating personalized interaction feedback content. A user constructs an immersive music interaction field domain, and experience logic of a virtual music meeting is completely reconstructed.

The invention discloses a three-dimensional laser cutting method for a frame with an n-shaped section. The three-dimensional laser cutting method comprises the following steps that S1, a tool is clamped; s2, a part is preliminarily clamped, and a notch theoretical line is scribed; s3, alignment is carried out; s4, the cutting of all the round holes is compiled into a subprogram, and the cutting program of each notch is compiled into different bisection subprograms; and S5, the cutting track of each subprogram is preliminarily debugged, and each subprogram is adjusted as required according to the measurement result after the first workpiece is cut. The additional shaft of the machine tool controls the rotating table to rotate in a numerical control mode, so that the tool drives the part to rotate, it is ensured that each pair of notch cutting positions are located at the lowest point of the part clamping posture, cutting head movement is simplified into four-shaft movement, and interference of a rotating shaft is avoided; a small number of attitude control points are arranged to optimize the cutting attitude, three-dimensional five-axis laser cutting of the n-shaped section frame is achieved, the programming and debugging efficiency is improved, the equipment safety is improved, and the overlap ratio of notches is ensured.

The invention belongs to the field of eye movement data tracking, particularly relates to a pilot eye movement tracking method, system and equipment based on multi-camera fusion and a storage medium, and aims to solve the problems that in the prior art, the visual angle of a traditional eye tracker is limited, and sight estimation errors are caused by head movement. The method comprises the following steps of: constructing a multi-view visual perception network by deploying multiple cameras and eye trackers in a cockpit; multi-source data space-time alignment is achieved through space coordinate system mapping and time synchronization, and head motion errors are compensated by combining dynamic cutting of eye interested areas and three-dimensional head posture estimation. The geometric path generates a physical driving vector based on iris fitting and an eyeball model, and the data path regresses the vector through a deep learning model; and converting to a world coordinate system through a head posture after fusion. High-precision data of the eye tracker is preferentially adopted in the central area, and field-of-view data of the system is expanded in the edge area. According to the invention, the tracking precision under field angle coverage and motion interference is obviously improved.

The invention relates to the technical field of artificial intelligence and biomedical sensing, and discloses a cognitive disorder risk identification device and method integrating multi-modal physiological data, and the device comprises an EEG module, an eye movement module, an fNIRS module, an edge calculation module and an acceleration sensor; the EEG module collects EEG data, the eye movement module collects eye movement eye-tracing data, and the fNIRS module collects near infrared spectrum fNIRS data and inputs the data to the edge calculation module; the edge calculation module runs the multi-modal space-time attention fusion model to output a cognitive impairment risk assessment result by using the multi-modal space-time attention fusion model; the acceleration sensor operates based on an operation dynamic filtering algorithm of a motion accelerometer to inhibit signal drift caused by head motion. According to the invention, through the modularized head-mounted multi-mode edge device, the physiological data is collected and edge end processing and cognitive disorder recognition and screening are carried out in combination with the edge calculation module, so that early recognition and screening of neurodegenerative diseases can be conveniently and effectively carried out at low cost.

The invention relates to the technical field of medical rehabilitation, and discloses a vestibular function and cognitive function rehabilitation training system and method based on virtual reality, and the rehabilitation training system comprises a hardware platform, a software platform and a server. The hardware platform comprises VR interaction equipment, a head motion tracking sensor and calculation and display equipment; the software platform is integrated with a user management module, a visual stimulation module, a vestibular rehabilitation training module, a cognitive evaluation and training module, a dynamic visual acuity evaluation module and a data management and analysis module. By combining black and white chess grid visual stimulation, standardized vestibular rehabilitation tasks, spatial cognitive testing and DVA objective monitoring, synchronous evaluation and personalized collaborative intervention of vestibular functions and cognitive functions of dizzy patients are realized. According to the method, the problems of boring traditional rehabilitation means, subjective evaluation and lack of quantitative feedback are solved, and the scientificity, interestingness and curative effect testability of rehabilitation training are remarkably improved.

The invention discloses a flexible multi-mode hydrogel sensing patch as well as a preparation method and application thereof. The flexible multi-mode hydrogel sensing patch is composed of a bottom hydrogel-based bimodal pressure-temperature sensing layer and a top electrostatic spinning nanofiber-based non-contact detection layer. The hydrogel serves as a core base material and has high toughness and water-retaining property, multi-mode sensing of temperature, pressure and non-contact proximity is achieved through different sensing mechanisms, and crosstalk interference does not exist. In a simulated real scene, the feasibility of the multi-mode sensing function of the robot hand is verified by grabbing an object through the robot hand. A plurality of multi-mode sensing patches integrated at different positions of the pillow are used for intelligent sleep monitoring, and a one-dimensional convolutional neural network is used for acquiring and analyzing various human pillow interaction information and time-dependent evolution thereof, so that tracking of head movement and recognition of wrong postures possibly causing poor sleep are realized; and a promising method is provided for sleep monitoring.

The invention relates to the technical field of auxiliary training, and discloses an AR auxiliary training system for improving the diagnosis and treatment capability of general practitioners on undifferentiated diseases, and an immersive training environment is constructed by adopting an augmented reality technology. The system comprises an environment sensing module, a space positioning module, a case data module, a virtual rendering module, an interactive acquisition module, an intelligent response module, a state control module and an evaluation calculation module. The environment sensing module adopts an RGB-D camera and an inertial measurement unit to collect environment and head movement data of a user; the spatial positioning module determines a virtual patient placement position based on an SLAM algorithm; the virtual rendering module renders a three-dimensional virtual patient in a real environment; the interaction acquisition module supports voice and gesture multi-mode interaction; the intelligent response module generates an intelligent patient answer; the evaluation calculation module carries out comprehensive evaluation from three dimensions. The invention provides a highly immersive and intelligent training environment, and has the advantages of low cost, reusability and high standardization degree.

The invention provides a driver anti-fatigue monitoring alarm method, medium and system, and belongs to the technical field of image data processing. A vehicle-mounted camera device is used for collecting a facial image sequence, and a Dlib key point screening function is used for feature point positioning; the feature extraction accuracy is improved by adopting an adaptive illumination compensation algorithm and a minimum spanning tree algorithm, eye features such as an eye closing proportion value and a blinking frequency and head motion features such as a head posture angle value and a nodding frequency are calculated, and physiological data such as blood pressure, heart rate and body temperature are collected by utilizing a smart watch. And inputting the multi-dimensional features into a pre-trained FatgueNet model through an adaptive feature selection gating mechanism, outputting a fatigue state evaluation index, and dividing alarm levels. When the severe fatigue state is recognized and the driver enters the sleepy state, the system sends a pulse stimulation instruction to the smart watch to wake up the driver, and the adaptability and accuracy of the system are improved.

The invention discloses a concentration training method and system based on multi-modal fusion, and the method comprises the steps: collecting an electroencephalogram signal, an eye movement signal and a head movement signal through a wearable device in a concentration training process, and calculating an electroencephalogram frequency characteristic parameter, an eye movement parameter and a head movement modal parameter in a one-to-one correspondence manner; calculating a concentration score according to the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement modal parameters so as to evaluate the concentration; and adjusting a concentration training strategy according to a concentration evaluation result. According to the method, the electroencephalogram, eye movement and head movement signals are collected at the same time through the wearable device, the corresponding characteristic parameters are calculated, the concentration score is comprehensively evaluated, and a comprehensive, accurate and stable evaluation result is provided. On the basis of real-time evaluation, task difficulty and types are dynamically adjusted, challenging and interestingness are ensured, the user participation degree and the training effect are improved, and concentration research and technical progress are promoted.

To provide a vehicle control device and a vehicle control method that take into consideration individual differences and the effect of vehicle motion, not only in the horizontal direction but also in the front-rear direction, on the head movement of a passenger, and further reduce the head movement of the passenger through vehicle motion that coordinates the front-rear and horizontal directions. The vehicle control device includes: a head movement prediction model that finds, relative to the head movement of a passenger in a first direction caused by vehicle motion in the first direction, the change in the head movement of the passenger in the first direction caused by the vehicle motion in a second direction different from the first direction; and a vehicle motion generation unit that generates the vehicle motion on the basis of the head movement prediction model.

The invention discloses a multi-purpose humanoid biped robot based on a three-point visual recognition system. The multi-purpose humanoid biped robot comprises an upper body unit; the head unit is located at the upper end of the upper body unit; the arm units are located on the left and right sides of the upper body unit; the leg units are integrated on the left side and the right side below the upper body unit through hip joint units; a visual sensor is integrated in the head unit, and the shooting angle of the visual sensor is adjusted through a head movement module integrated in the head unit. The head of the biped robot has two degrees of freedom, so that the head can pitch and rotate, meanwhile, the visual sensor is carried, information of the surrounding environment can be collected in real time, and main information input is provided for the robot.

The invention is applicable to the technical field of information interaction, and provides an asymptomatic patient interaction method based on an eye-controlled staring trigger communication interface, which comprises the following steps: sight tracking: acquiring facial image data of a patient through a camera, extracting facial and eye mark points, performing image processing on an eye area, and judging a staring direction; staring triggering: presetting a coordinate area of an interaction control, judging whether a fixation point of a patient falls into the control area in real time, and if the fixation point is continuously kept for preset time, triggering corresponding operation; executing a compensation mechanism: compensating head movement and illumination change in real time, and correcting a staring point coordinate; and interface design: designing an interactive interface with a daily communication function, and broadcasting the content of the control through voice. The invention provides an efficient, convenient and easy-to-use eye-controlled staring triggering communication interface for the patient with the asymptomatic disease, aims to reduce the learning cost and the operation complexity of the patient, and has great significance in improving the life quality of the patient with the asymptomatic disease.

Various implementations disclosed herein include devices, systems, and methods for modifying display of virtual content. In some implementations, the method is performed by an electronic device including a non-transitory memory, one or more processors, a display, an eye tracker and a movement sensor. In some implementations, the method includes displaying, via the display, virtual content. In some implementations, the method includes detecting, via the eye tracker, a movement of a gaze in a first direction. In some implementations, the method includes detecting, via the movement sensor, a movement of a head in a second direction while detecting the movement of the gaze in the first direction. In some implementations, the method includes, in accordance with a determination that the movement of the gaze in the first direction and the movement of the head in the second direction are corresponding opposing movements, modifying a display characteristic of the virtual content.

The invention relates to the technical field of electroencephalogram monitoring, in particular to a non-contact electroencephalogram monitoring method and system, and the method comprises the steps: obtaining a corresponding head image of a target user at a current moment; determining a corresponding head pose of the target user at the current moment based on the head image corresponding to the current moment; based on the head pose at the current moment and the head pose corresponding to the previous moment, the pose of an electroencephalogram cap is determined, the electroencephalogram cap comprises a plurality of brain electrodes, and the brain electrodes monitor electroencephalogram data of the target user under the pose corresponding to the electroencephalogram cap. The problems that in the non-contact electroencephalogram collecting process, when the head moves, the collecting position deviates, and signals are not stable can be solved.

The invention discloses an eye movement tracking method and system based on multi-modal fusion, and particularly relates to the technical field of eye movement tracking, and the method comprises the steps: collecting a face image, an eye image and a head movement parameter of a user through an RGB camera, an infrared camera and an IMU which are synchronously arranged; the signals are processed in parallel, sight line direction vectors based on the face and the eyes and reliability measurement of the sight line direction vectors are extracted respectively, and the head space posture is calculated; dynamically selecting high-confidence cooperation, master-slave compensation or conflict arbitration strategies according to reliability measurement, and fusing to generate an initial sight direction; and mapping the direction to a display plane to obtain an initial fixation coordinate, performing optimization calibration in combination with visual saliency analysis of screen content, and outputting a final fixation point coordinate. The system comprises a signal acquisition and processing module, a parallel computing module, a multi-strategy fusion decision-making module and an output optimization module. According to the invention, through multi-modal information fusion and dynamic strategy selection, the precision and robustness of eye movement tracking are significantly improved.

The invention discloses an identity authentication system of wearable equipment based on a wireless earphone external microphone. The identity authentication system is characterized by comprising the following steps: step 1, customizing an earphone to obtain original audio data; step 2, signal noise reduction and enhancement; 3, constructing a user model; 4, a lightweight self-adaptive multi-window sliding detection algorithm is carried out; 5, interference events are screened out; step 6, feature extraction and data set expansion; and 7, model training and incremental learning. Compared with the prior art, the identity authentication system has the advantages that the head movement signal of a wearer is collected through the necessary external ear microphone on the wireless earphone, stable biological characteristics are extracted, finally training is carried out through the neural network, a legal user is recognized, the identity authentication system can accurately recognize in different environments, and the user experience is improved. And various cheating attacks can be resisted.

The invention discloses a piezoelectric film partition dynamic focusing system and method for intelligent glasses. The piezoelectric film partition dynamic focusing system comprises an optical modulation layer, a sensing layer, a control engine, a hybrid driving layer and a closed loop verification module. The optical modulation layer is integrated with an N * M piezoelectric film array, each subarea is provided with an independent electrode and a gradient thickness structure, and intervals are filled with low-modulus insulating polymers. The sensing layer is used for acquiring fixation point, diopter, temperature and head movement information of a wearer; and the control engine executes voltage mapping, temperature drift compensation and resonance suppression according to the sensing data, and drives the hybrid driving layer to complete deformation and displacement compensation. And the closed-loop verification module outputs a defocusing error in real time and feeds back a correction voltage. According to the invention, high-response, low-power-consumption and stable dynamic focusing is realized, and the visual adaptability is improved.

The invention provides a forged face video detection method and device based on multi-modal behavior consistency, electronic equipment, a storage medium and a program product. The method comprises the following steps: extracting voice features, facial expression features and head action features from a video signal to be detected; recognizing voice emotion, facial emotion and semantic emotion; based on the VAD value sequences of the various emotions, emotion consistency features and emotion synchronism features among the various emotions are calculated, and emotion semantic consistency features between the semantic content and the facial emotion and between the semantic content and the voice emotion are calculated; constructing a cross-modal time dependence graph to obtain interaction features; processing the voice features, the facial expression features and the head action features by using a hierarchical attention network to obtain time sequence features; a multi-dimensional fusion feature vector is formed; and processing the fusion feature vector by using a preset binary classifier to obtain a classification result indicating whether the to-be-detected video signal is a fake face video or not.

The invention provides a cross-view-angle sight line estimation method based on feature decoupling and an attention mechanism, double-view-angle face images are collected through double cameras, a view angle blind area under a single view angle is effectively eliminated, and the ambiguity problem caused by insufficient single-view-angle information is relieved; face features are decoupled into personal appearance features, head posture features and sight line related features, and accurate separation of different types of features is achieved; the adaptive capacity of the model in diversified scenes is enhanced through a feature decoupling strategy, and it is ensured that the model still keeps high-performance under the conditions of different individuals and large-amplitude head posture changes; then, adaptive weighting processing is carried out on the three types of features obtained through decoupling by applying a cross attention mechanism, and supplement and enhancement of sight line information are realized through fusion of double-view-angle sight line features; meanwhile, the influence of personal factors and head movement is compensated by combining the processed appearance features and head posture features, so that the precision and robustness of sight line estimation are remarkably improved.

A method for motion synchronization-based head pose estimation, by a head mounted display (HMD) device and an HMD device for performing the same are provided. The method includes receiving, by the HMD device, motion data from a plurality of motion sensors of the HMD device, receiving, by the HMD device, a plurality of image frames from at least one simultaneous localization and mapping SLAM camera of the HMD device, estimating, by the HMD device, a plurality of motion parameters of head movements of a user from the plurality of image frames received from memory, generating, by the HMD device, a filtered subset of the motion data received from the plurality of motion sensors based on the plurality of motion parameters of the head movements, synchronizing, by the HMD device, the plurality of image frames received from the memory and filtered subset of motion data, and estimating, by the HMD device, the head pose based on the synchronized plurality of image frames and motion data.

The present invention discloses a method for intervening in the subjective feeling of motion sickness in virtual reality based on perceptual theory, which relates to the technical field of virtual reality. The method comprises: acquiring proprioception parameter information of a current user, wherein the proprioception parameter information includes but is not limited to head movement data, eye movement data, and physiological data; performing real-time analysis on the currently acquired data using a data analysis algorithm to determine whether the user has early signs of motion sickness; determining, by the data analysis algorithm, that the user has early signs of motion sickness, and immediately initiating a visual perception adjustment step. Furthermore, by real-time capturing and analyzing the user's physiological signals in a virtual reality experience, it is possible to identify and intervene in the early stages of motion sickness, and to formulate an intervention method in a timely manner for users who have subjective feelings of motion sickness, thereby facilitating rapid relief of user discomfort.

The invention relates to the technical field of intelligent glasses, and discloses an intelligent glasses wearing identification method based on multi-modal perception, and the method comprises the steps: collecting the capacitance change rate, the facial contour matching degree, the body surface temperature, the vital signs, the head attitude angle and other multi-source feature data in real time, generating five groups of feature data, and carrying out the standardization processing and weight fusion, forming a comprehensive feature vector; calculating a preliminary confidence score by combining a dynamic weight adjustment mechanism and a rule engine, and correcting a deviation caused by head movement; judging a wearing state according to the corrected confidence score, outputting a label, and realizing function control in linkage with a gesture interaction module; the abnormal state triggers local or cloud alarm; the sampling frequency of the sensor is dynamically adjusted according to the wearing state, the frequency is reduced to 1Hz when the sensor is not worn, the frequency is improved to 10Hz when the sensor is worn so as to enhance the response precision, and the decision process is only triggered when the feature is suddenly changed in combination with an event-driven strategy. According to the invention, the precision and the reliability can be improved, and low-power-consumption design is realized.

Systems, apparatuses, methods, and computer program products are provided herein. For example, a method may include receiving, from a wearable device, head motion data. In some embodiments, the method includes generating eye motion data by applying the head motion data to an eye motion determination model. In some embodiments, the method includes detecting device motion data associated with a device. In some embodiments, the method includes generating vehicle related stabilization data by applying the eye motion data and the device motion data to a vehicle related stabilization model. In some embodiments, the method includes generating a stabilized vehicle related interface component based on the vehicle related stabilization data and vehicle related device display data. In some embodiments, the method includes causing the stabilized vehicle related interface component to be rendered to an interface of the device.

The invention discloses a landscape object identification and explanation method and system, a medium and a product, and relates to the technical field of augmented reality. The method comprises the following steps: when the head motion state of a user is stable, triggering a camera to collect and recognize a single-frame image, matching a recognition result with a regional knowledge base, and generating a candidate object list and coarse-grained poses which are sorted according to spatial probability and historical popularity; performing feature point matching on a current frame and an ideal frame in the video stream, and calculating a fine-grained pose of a target landscape in the candidate object list relative to a camera coordinate system according to a matched target feature point; according to the fine-grained pose and the real-time eye movement data, determining a drop point part of the sight line of the user on the target landscape, forming a multi-dimensional query vector according to the drop point part, the intention of the user and the environment context parameters, and generating multimedia explanation content; and carrying out fusion rendering on the multimedia explanation content and the real-time video stream, and outputting and displaying. By implementing the technical scheme, the personalized multimedia explanation content can be generated.

The invention provides a virtual data transmission flow control system for an MR head-mounted display. The virtual data transmission flow control system comprises a dynamic key area identification module, a self-adaptive flow scheduling module, a low-delay transmission optimization module and a cross-layer collaborative optimization module, wherein the dynamic key region recognition module is used for recognizing a complex key region in an MR environment in real time based on a user fixation point, a head movement track and scene semantic analysis, and the complex key region comprises a high-dynamic interaction object, a visual focus region and a real-time changing virtual scene element; the dynamic key region identification module obtains user behavior data through multi-modal sensor fusion, predicts the change trend of a user concerned region in combination with a deep learning model, and outputs the priority weight of a key region; the adaptive traffic scheduling module comprises a hierarchical coding unit, a dynamic bandwidth allocation unit and a redundancy control unit; and the self-adaptive flow scheduling module is connected with the dynamic key area identification module.

The invention discloses an optical and neural feedback multi-modal data analysis method for a nerve regulation target, and belongs to the technical field of cranial nerve treatment, and the method specifically comprises the steps: receiving a head movement instruction of a patient through an interactive interface, pausing transcranial magnetic stimulation after receiving the instruction, and switching to a head movement monitoring state; an infrared optical navigation device is combined with a magnetic resonance image to construct an individualized brain three-dimensional model, head position changes are tracked in real time, and autonomic nerve feedback signals are collected at the same time; synchronously aligning the head coordinates with the neural feedback signals to generate a multi-modal data set, and analyzing relevance to obtain real-time regulation and control parameters; the positioning and stimulation intensity of the transcranial magnetic stimulation coil are adaptively adjusted according to the parameters, and related information is displayed on a treatment interface; after the head of the patient finishes moving and is stable, a rapid re-calibration process is automatically triggered, high-precision monitoring is recovered, and magnetic stimulation output is reactivated; according to the invention, the treatment stability and adaptability are improved.

The invention relates to an intelligent sight interaction system for emotional communication. The system comprises a sight capture module, a head posture recognition module, an emotion analysis module, a digital human generation module, an interaction control module and a multi-mode output module. According to the system, sight line tracking is carried out through collected eye images, head posture correction is carried out based on face key points and a 3D model, sight line / expression / posture cooperative processing is carried out through a multi-mode sentiment analysis model, and the technical means of dynamically generating digital human sight line and expression parameters is combined; the problems of tracking errors and single emotion feedback caused by head movement in traditional sight line interaction are solved, high-precision sight line capture and emotional digital human real-time response are achieved, man-machine interaction reaches the naturalness degree close to real sight contact, and meanwhile the adaptability to the emotional state of a user is improved.

The invention relates to the technical field of TWS earphone low-delay audio processing methods, in particular to a TWS earphone low-delay audio processing method based on head motion prediction, which comprises the following steps: S1, collecting head motion parameters of a user, the head motion parameters comprising a head posture theta, a head rotation angular velocity omega, a head movement direction phi and a head movement velocity V; s2, establishing a relation model between the head motion parameters and the audio time delay; s3, predicting head motion parameters in a future time period based on a recurrent neural network; s4, according to the predicted head motion parameters, audio time delay is calculated, and audio rendering is carried out; s5, outputting the audio signal subjected to time delay compensation; wherein the steps from S2 to S5 are executed circularly so as to adjust audio output in real time; head motion prediction is carried out by using the recurrent neural network, and the head position and the motion trend of the user can be accurately estimated in a millisecond level, so that audio pre-rendering and accurate output are realized.

In some embodiments, a method comprises executing, by a head-tracked audio system, a first algorithm configured to dynamically update a forward head angle and a spatial orientation in response to a detected head movement, and a second algorithm configured to adjust a recentering filter based on the detected head movements, and maintaining, by the head-tracked audio system through execution of the first algorithm and the second algorithm, a spatial audio placement relative to a wearer during the detected head movements.