756 results about "Head movements" patented technology

The invention discloses an intelligent head-mounted device with a camera capable of rotating along with the camera, and the device comprises an eye movement tracking camera which is used for collecting and measuring the image data of the iris of the two eyes of a wearer; the IMU sensor is used for detecting head movement data of a wearer; the image acquisition and processing module is used for generating eyeball movement data according to the image data of the iris of the two eyes, and converting the eyeball movement data and the head movement data into a rotation instruction; the binocular camera motion control device executes the rotation instruction, drives the binocular camera to move, moves along with the iris and the eyeball, and collects image data in real time to realize line-of-sight tracking; and the image acquisition and processing module is also used for carrying out image and / or video fusion processing according to the received image acquired by the binocular camera in combination with the head movement data and the eyeball movement data. According to the invention, the problem of limitation in the aspects of acquiring depth information and expanding the visual field can be solved, and a more flexible camera tracking and adjusting mechanism is designed, so that wider and clearer image acquisition is realized.

The invention provides a classroom behavior analysis method and system, electronic equipment and a storage medium, and relates to the technical field of educational informationization, and the method comprises the steps: collecting videos and audios, carrying out image enhancement, noise filtering and frame segmentation on the videos, carrying out noise reduction, sound source positioning and voice segmentation on the audios, and generating standardized images and voice sequences; an improved MTCNN cascade network is combined with a posture estimation technology, facial key points of students are extracted from videos, class arrival states, head postures and facial micro-expressions are recognized, audios are analyzed through a bidirectional LSTM network, and speaking duration and frequency are obtained; constructing individual behavior indexes according to attendance, head actions and expressions of the students; constructing an interaction index according to the teacher and student speaking data; the individual and interaction indexes are summarized to generate the multi-dimensional classroom behavior portrait, and the relation between the portrait and the teaching effect is analyzed, so that the classroom behavior analysis precision can be improved, the data processing capability can be enhanced, and the multi-dimensional classroom behavior comprehensive evaluation can be realized.

A video perspective method based on virtual reality relates to the technical field of virtual reality, and comprises: at least two cameras for capturing video information in real time, the cameras being provided with spatial positions and angles; performing geometric correction on the video information; the eye movement tracking module monitors the gazing direction of the eyes of the user in real time and adjusts the display position; the eye movement tracking module tracks eyeball movement; reconstructing a three-dimensional model of the surrounding environment of the user; based on the three-dimensional model, determining a relative position relationship between the virtual object and the real object; the head posture sensor obtains head movement information of a user; performing semantic segmentation on the video frame through a deep learning algorithm; and synthesizing the video information and the virtual content on the head-mounted display device, and outputting the synthesized video information and virtual content. The technical problems of visual discomfort and dizziness caused by position mismatching can be solved, and natural fusion of a real scene and a virtual scene is realized.

The invention provides an immersive concert experience system based on virtual reality, and belongs to the technical field of virtual reality, and the system comprises a dynamic adjustment module which is used for predicting the preference of a user for stage arrangement, light effects and music types through a machine learning model according to the historical preference data of the user, and outputting the preference to the stage arrangement, light effects and music types; dynamically adjusting visual elements and sound field parameters of the virtual scene and a refresh rate and a field angle of the virtual scene in combination with the user physiological indexes and the user head motion trail which are acquired in real time; the action capture and feedback module is used for capturing an interaction instruction sent to the dynamically adjusted virtual scene by the limb action of the user in real time; and the interaction feedback module is used for enabling the virtual performer to respond to the interaction instruction of the user in real time based on the expression recognition and voice synthesis technology, and generating personalized interaction feedback content. A user constructs an immersive music interaction field domain, and experience logic of a virtual music meeting is completely reconstructed.

This disclosure relates to a system and method for real-time monitoring of driver in a vehicle. The method includes capturing at least one of a head movement, steering direction, and a direction of gaze of the driver. The method further includes determining a Plane of Vision (POV) for the driver based on end point of each side view mirrors, a rear-view mirror and bottom of a windshield of the vehicle. The method further includes determining a Region of Interest (ROI) for the driver based on one of a bounding box regression model and a direction of movement of the vehicle. The method further includes determining a Line of Gaze (LOG) for the driver based on the direction of gaze. The method further includes identifying a distraction level of the driver while driving the vehicle based on determining at least one of the POV, the ROI, and the LOG.

The invention discloses a three-dimensional laser cutting method for a frame with an n-shaped section. The three-dimensional laser cutting method comprises the following steps that S1, a tool is clamped; s2, a part is preliminarily clamped, and a notch theoretical line is scribed; s3, alignment is carried out; s4, the cutting of all the round holes is compiled into a subprogram, and the cutting program of each notch is compiled into different bisection subprograms; and S5, the cutting track of each subprogram is preliminarily debugged, and each subprogram is adjusted as required according to the measurement result after the first workpiece is cut. The additional shaft of the machine tool controls the rotating table to rotate in a numerical control mode, so that the tool drives the part to rotate, it is ensured that each pair of notch cutting positions are located at the lowest point of the part clamping posture, cutting head movement is simplified into four-shaft movement, and interference of a rotating shaft is avoided; a small number of attitude control points are arranged to optimize the cutting attitude, three-dimensional five-axis laser cutting of the n-shaped section frame is achieved, the programming and debugging efficiency is improved, the equipment safety is improved, and the overlap ratio of notches is ensured.

The invention belongs to the field of eye movement data tracking, particularly relates to a pilot eye movement tracking method, system and equipment based on multi-camera fusion and a storage medium, and aims to solve the problems that in the prior art, the visual angle of a traditional eye tracker is limited, and sight estimation errors are caused by head movement. The method comprises the following steps of: constructing a multi-view visual perception network by deploying multiple cameras and eye trackers in a cockpit; multi-source data space-time alignment is achieved through space coordinate system mapping and time synchronization, and head motion errors are compensated by combining dynamic cutting of eye interested areas and three-dimensional head posture estimation. The geometric path generates a physical driving vector based on iris fitting and an eyeball model, and the data path regresses the vector through a deep learning model; and converting to a world coordinate system through a head posture after fusion. High-precision data of the eye tracker is preferentially adopted in the central area, and field-of-view data of the system is expanded in the edge area. According to the invention, the tracking precision under field angle coverage and motion interference is obviously improved.

The invention relates to the technical field of artificial intelligence and biomedical sensing, and discloses a cognitive disorder risk identification device and method integrating multi-modal physiological data, and the device comprises an EEG module, an eye movement module, an fNIRS module, an edge calculation module and an acceleration sensor; the EEG module collects EEG data, the eye movement module collects eye movement eye-tracing data, and the fNIRS module collects near infrared spectrum fNIRS data and inputs the data to the edge calculation module; the edge calculation module runs the multi-modal space-time attention fusion model to output a cognitive impairment risk assessment result by using the multi-modal space-time attention fusion model; the acceleration sensor operates based on an operation dynamic filtering algorithm of a motion accelerometer to inhibit signal drift caused by head motion. According to the invention, through the modularized head-mounted multi-mode edge device, the physiological data is collected and edge end processing and cognitive disorder recognition and screening are carried out in combination with the edge calculation module, so that early recognition and screening of neurodegenerative diseases can be conveniently and effectively carried out at low cost.

The invention relates to the technical field of medical rehabilitation, and discloses a vestibular function and cognitive function rehabilitation training system and method based on virtual reality, and the rehabilitation training system comprises a hardware platform, a software platform and a server. The hardware platform comprises VR interaction equipment, a head motion tracking sensor and calculation and display equipment; the software platform is integrated with a user management module, a visual stimulation module, a vestibular rehabilitation training module, a cognitive evaluation and training module, a dynamic visual acuity evaluation module and a data management and analysis module. By combining black and white chess grid visual stimulation, standardized vestibular rehabilitation tasks, spatial cognitive testing and DVA objective monitoring, synchronous evaluation and personalized collaborative intervention of vestibular functions and cognitive functions of dizzy patients are realized. According to the method, the problems of boring traditional rehabilitation means, subjective evaluation and lack of quantitative feedback are solved, and the scientificity, interestingness and curative effect testability of rehabilitation training are remarkably improved.

The invention discloses a flexible multi-mode hydrogel sensing patch as well as a preparation method and application thereof. The flexible multi-mode hydrogel sensing patch is composed of a bottom hydrogel-based bimodal pressure-temperature sensing layer and a top electrostatic spinning nanofiber-based non-contact detection layer. The hydrogel serves as a core base material and has high toughness and water-retaining property, multi-mode sensing of temperature, pressure and non-contact proximity is achieved through different sensing mechanisms, and crosstalk interference does not exist. In a simulated real scene, the feasibility of the multi-mode sensing function of the robot hand is verified by grabbing an object through the robot hand. A plurality of multi-mode sensing patches integrated at different positions of the pillow are used for intelligent sleep monitoring, and a one-dimensional convolutional neural network is used for acquiring and analyzing various human pillow interaction information and time-dependent evolution thereof, so that tracking of head movement and recognition of wrong postures possibly causing poor sleep are realized; and a promising method is provided for sleep monitoring.

The invention relates to the technical field of auxiliary training, and discloses an AR auxiliary training system for improving the diagnosis and treatment capability of general practitioners on undifferentiated diseases, and an immersive training environment is constructed by adopting an augmented reality technology. The system comprises an environment sensing module, a space positioning module, a case data module, a virtual rendering module, an interactive acquisition module, an intelligent response module, a state control module and an evaluation calculation module. The environment sensing module adopts an RGB-D camera and an inertial measurement unit to collect environment and head movement data of a user; the spatial positioning module determines a virtual patient placement position based on an SLAM algorithm; the virtual rendering module renders a three-dimensional virtual patient in a real environment; the interaction acquisition module supports voice and gesture multi-mode interaction; the intelligent response module generates an intelligent patient answer; the evaluation calculation module carries out comprehensive evaluation from three dimensions. The invention provides a highly immersive and intelligent training environment, and has the advantages of low cost, reusability and high standardization degree.

The invention discloses an automobile rearview mirror angle automatic adjusting and angle metering method and device, and belongs to the field of automobile rearview mirror automatic adjusting.The method comprises the following steps that coordinates of the head position and the eye position of a driver are obtained, and the eye height of the driver is obtained through the eye movement tracking technology; calculating an optimal reference angle of the rearview mirror according to the height of the eye position, and taking the optimal reference angle as a reference angle theta base of the rearview mirror; the head action of a driver and the dynamic state of a vehicle are sensed in real time through sensor data, data of various sensors are integrated through a fusion algorithm, the fused head angle of the driver, vehicle body posture information and adjustment gain factors are obtained, and the angle of a rearview mirror is dynamically adjusted according to the information. And the driver can always obtain the optimal view angle through closed-loop feedback and a self-calibration mechanism. By means of a personalized adjusting mode, it is ensured that a driver can always see the road condition behind at the most comfortable position.

The invention provides a driver anti-fatigue monitoring alarm method, medium and system, and belongs to the technical field of image data processing. A vehicle-mounted camera device is used for collecting a facial image sequence, and a Dlib key point screening function is used for feature point positioning; the feature extraction accuracy is improved by adopting an adaptive illumination compensation algorithm and a minimum spanning tree algorithm, eye features such as an eye closing proportion value and a blinking frequency and head motion features such as a head posture angle value and a nodding frequency are calculated, and physiological data such as blood pressure, heart rate and body temperature are collected by utilizing a smart watch. And inputting the multi-dimensional features into a pre-trained FatgueNet model through an adaptive feature selection gating mechanism, outputting a fatigue state evaluation index, and dividing alarm levels. When the severe fatigue state is recognized and the driver enters the sleepy state, the system sends a pulse stimulation instruction to the smart watch to wake up the driver, and the adaptability and accuracy of the system are improved.

The invention discloses a concentration training method and system based on multi-modal fusion, and the method comprises the steps: collecting an electroencephalogram signal, an eye movement signal and a head movement signal through a wearable device in a concentration training process, and calculating an electroencephalogram frequency characteristic parameter, an eye movement parameter and a head movement modal parameter in a one-to-one correspondence manner; calculating a concentration score according to the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement modal parameters so as to evaluate the concentration; and adjusting a concentration training strategy according to a concentration evaluation result. According to the method, the electroencephalogram, eye movement and head movement signals are collected at the same time through the wearable device, the corresponding characteristic parameters are calculated, the concentration score is comprehensively evaluated, and a comprehensive, accurate and stable evaluation result is provided. On the basis of real-time evaluation, task difficulty and types are dynamically adjusted, challenging and interestingness are ensured, the user participation degree and the training effect are improved, and concentration research and technical progress are promoted.

To provide a vehicle control device and a vehicle control method that take into consideration individual differences and the effect of vehicle motion, not only in the horizontal direction but also in the front-rear direction, on the head movement of a passenger, and further reduce the head movement of the passenger through vehicle motion that coordinates the front-rear and horizontal directions. The vehicle control device includes: a head movement prediction model that finds, relative to the head movement of a passenger in a first direction caused by vehicle motion in the first direction, the change in the head movement of the passenger in the first direction caused by the vehicle motion in a second direction different from the first direction; and a vehicle motion generation unit that generates the vehicle motion on the basis of the head movement prediction model.

The invention relates to a motion capture optimization method of a tactical AR helmet. Based on a motion model and dynamic changes of muscle activities and biological signals; collecting data in real time through a sensor, and establishing a time-dependent model of motion data by using a time sequence analysis method; in combination with an LSTM (Long Short Term Memory) model in deep learning, capturing head motion and time dynamic change of motion, and predicting and correcting an error in motion capture; intelligent processing is carried out based on continuous time sequence data, and short-term errors of capturing data drift are avoided; designing a multi-level sensor correction algorithm, and performing time alignment and error correction on IMU data and visual data; drift phenomena including position errors and rotation errors in motion capture are eliminated; optimizing the helmet capture system using a ground sensor including a gait monitoring device and external vision calibration data; a target-oriented motion trail optimization algorithm is designed by combining a motion target and a tactical task of a user, and a capture strategy is dynamically adjusted according to the change of the task.

The invention discloses a multi-purpose humanoid biped robot based on a three-point visual recognition system. The multi-purpose humanoid biped robot comprises an upper body unit; the head unit is located at the upper end of the upper body unit; the arm units are located on the left and right sides of the upper body unit; the leg units are integrated on the left side and the right side below the upper body unit through hip joint units; a visual sensor is integrated in the head unit, and the shooting angle of the visual sensor is adjusted through a head movement module integrated in the head unit. The head of the biped robot has two degrees of freedom, so that the head can pitch and rotate, meanwhile, the visual sensor is carried, information of the surrounding environment can be collected in real time, and main information input is provided for the robot.

The invention is applicable to the technical field of information interaction, and provides an asymptomatic patient interaction method based on an eye-controlled staring trigger communication interface, which comprises the following steps: sight tracking: acquiring facial image data of a patient through a camera, extracting facial and eye mark points, performing image processing on an eye area, and judging a staring direction; staring triggering: presetting a coordinate area of an interaction control, judging whether a fixation point of a patient falls into the control area in real time, and if the fixation point is continuously kept for preset time, triggering corresponding operation; executing a compensation mechanism: compensating head movement and illumination change in real time, and correcting a staring point coordinate; and interface design: designing an interactive interface with a daily communication function, and broadcasting the content of the control through voice. The invention provides an efficient, convenient and easy-to-use eye-controlled staring triggering communication interface for the patient with the asymptomatic disease, aims to reduce the learning cost and the operation complexity of the patient, and has great significance in improving the life quality of the patient with the asymptomatic disease.

Various implementations disclosed herein include devices, systems, and methods for modifying display of virtual content. In some implementations, the method is performed by an electronic device including a non-transitory memory, one or more processors, a display, an eye tracker and a movement sensor. In some implementations, the method includes displaying, via the display, virtual content. In some implementations, the method includes detecting, via the eye tracker, a movement of a gaze in a first direction. In some implementations, the method includes detecting, via the movement sensor, a movement of a head in a second direction while detecting the movement of the gaze in the first direction. In some implementations, the method includes, in accordance with a determination that the movement of the gaze in the first direction and the movement of the head in the second direction are corresponding opposing movements, modifying a display characteristic of the virtual content.

A personal display system with which a user may adjust the configuration of displayed media is provided. The personal display system may include an electronic device operative to provide media to a personal display device operative to display the received media. Using one or more optical and digital components, the personal display device may adjust displayed media to overlay features of a theater, thus giving the user of the personal display device the impression of being in the theater. In some embodiments, the personal display device may detect the user's movements using one or more sensors and may adjust the displayed image based on the user's movements. For example, the device may detect a user's head movement and cause the portion of media displayed to reflect the head movement.

The invention relates to the technical field of electroencephalogram monitoring, in particular to a non-contact electroencephalogram monitoring method and system, and the method comprises the steps: obtaining a corresponding head image of a target user at a current moment; determining a corresponding head pose of the target user at the current moment based on the head image corresponding to the current moment; based on the head pose at the current moment and the head pose corresponding to the previous moment, the pose of an electroencephalogram cap is determined, the electroencephalogram cap comprises a plurality of brain electrodes, and the brain electrodes monitor electroencephalogram data of the target user under the pose corresponding to the electroencephalogram cap. The problems that in the non-contact electroencephalogram collecting process, when the head moves, the collecting position deviates, and signals are not stable can be solved.

The invention discloses an eye movement tracking method and system based on multi-modal fusion, and particularly relates to the technical field of eye movement tracking, and the method comprises the steps: collecting a face image, an eye image and a head movement parameter of a user through an RGB camera, an infrared camera and an IMU which are synchronously arranged; the signals are processed in parallel, sight line direction vectors based on the face and the eyes and reliability measurement of the sight line direction vectors are extracted respectively, and the head space posture is calculated; dynamically selecting high-confidence cooperation, master-slave compensation or conflict arbitration strategies according to reliability measurement, and fusing to generate an initial sight direction; and mapping the direction to a display plane to obtain an initial fixation coordinate, performing optimization calibration in combination with visual saliency analysis of screen content, and outputting a final fixation point coordinate. The system comprises a signal acquisition and processing module, a parallel computing module, a multi-strategy fusion decision-making module and an output optimization module. According to the invention, through multi-modal information fusion and dynamic strategy selection, the precision and robustness of eye movement tracking are significantly improved.

The invention discloses an identity authentication system of wearable equipment based on a wireless earphone external microphone. The identity authentication system is characterized by comprising the following steps: step 1, customizing an earphone to obtain original audio data; step 2, signal noise reduction and enhancement; 3, constructing a user model; 4, a lightweight self-adaptive multi-window sliding detection algorithm is carried out; 5, interference events are screened out; step 6, feature extraction and data set expansion; and 7, model training and incremental learning. Compared with the prior art, the identity authentication system has the advantages that the head movement signal of a wearer is collected through the necessary external ear microphone on the wireless earphone, stable biological characteristics are extracted, finally training is carried out through the neural network, a legal user is recognized, the identity authentication system can accurately recognize in different environments, and the user experience is improved. And various cheating attacks can be resisted.

The invention discloses a piezoelectric film partition dynamic focusing system and method for intelligent glasses. The piezoelectric film partition dynamic focusing system comprises an optical modulation layer, a sensing layer, a control engine, a hybrid driving layer and a closed loop verification module. The optical modulation layer is integrated with an N * M piezoelectric film array, each subarea is provided with an independent electrode and a gradient thickness structure, and intervals are filled with low-modulus insulating polymers. The sensing layer is used for acquiring fixation point, diopter, temperature and head movement information of a wearer; and the control engine executes voltage mapping, temperature drift compensation and resonance suppression according to the sensing data, and drives the hybrid driving layer to complete deformation and displacement compensation. And the closed-loop verification module outputs a defocusing error in real time and feeds back a correction voltage. According to the invention, high-response, low-power-consumption and stable dynamic focusing is realized, and the visual adaptability is improved.

The invention belongs to the related technical field of grinding robots, and discloses a servomotor cylinder inner wall weld joint robot grinding multi-objective optimization method and system based on multi-task reinforcement learning. The method comprises the following steps: constructing simulation models of a moving unit, a polishing robot and a servomotor cylinder body; the grinding process is divided into the process that a grinding head moves towards a weld joint starting point and the process that the grinding head moves after reaching the starting point; respectively constructing respective multi-task reinforcement learning optimization models for the two processes, wherein the multi-task reinforcement learning optimization models take a state space as input and an action space as output; and changing the length and the position of the welding seam to perform polishing simulation, and training the multi-task reinforcement learning optimization model of the two processes until the preset training times are reached and the reward function of the optimization target is converged. Through the method, the problem that multi-target optimization such as collision avoidance, grinding accessibility and rigidity maximization cannot be considered at the same time in the robot grinding process is solved.

The invention provides a forged face video detection method and device based on multi-modal behavior consistency, electronic equipment, a storage medium and a program product. The method comprises the following steps: extracting voice features, facial expression features and head action features from a video signal to be detected; recognizing voice emotion, facial emotion and semantic emotion; based on the VAD value sequences of the various emotions, emotion consistency features and emotion synchronism features among the various emotions are calculated, and emotion semantic consistency features between the semantic content and the facial emotion and between the semantic content and the voice emotion are calculated; constructing a cross-modal time dependence graph to obtain interaction features; processing the voice features, the facial expression features and the head action features by using a hierarchical attention network to obtain time sequence features; a multi-dimensional fusion feature vector is formed; and processing the fusion feature vector by using a preset binary classifier to obtain a classification result indicating whether the to-be-detected video signal is a fake face video or not.

The invention provides a cross-view-angle sight line estimation method based on feature decoupling and an attention mechanism, double-view-angle face images are collected through double cameras, a view angle blind area under a single view angle is effectively eliminated, and the ambiguity problem caused by insufficient single-view-angle information is relieved; face features are decoupled into personal appearance features, head posture features and sight line related features, and accurate separation of different types of features is achieved; the adaptive capacity of the model in diversified scenes is enhanced through a feature decoupling strategy, and it is ensured that the model still keeps high-performance under the conditions of different individuals and large-amplitude head posture changes; then, adaptive weighting processing is carried out on the three types of features obtained through decoupling by applying a cross attention mechanism, and supplement and enhancement of sight line information are realized through fusion of double-view-angle sight line features; meanwhile, the influence of personal factors and head movement is compensated by combining the processed appearance features and head posture features, so that the precision and robustness of sight line estimation are remarkably improved.

The invention discloses a spatial audio processing method and device, computer equipment and a storage medium, and the method comprises the steps: responding to an operation of a user for audio equipment, and initializing a gyroscope sensor in the audio equipment; starting a gyroscope sensor, obtaining head motion original data of a user, and then performing data preprocessing to obtain current head angle information of the user; determining a corresponding head-related transfer function based on the head angle information; performing conversion processing on the original audio data based on the head-related transfer function to obtain target audio data; and performing spatial reverberation simulation on the target audio data to obtain a corresponding spatial reverberation audio. According to the scheme, high-precision and high-real-time audio output can be provided while low resource occupation is kept, and more real and immersive auditory experience is provided for a user.

The invention provides AR glasses based on integrated imaging three-dimensional display. According to the AR glasses, the integrated imaging technology is adopted, a three-dimensional light field is reconstructed through the micro-lens array, and multi-view three-dimensional image display is achieved. The display module adopts a Micro-LED display screen and is combined with a micro-lens array to realize superposition of a virtual image and a real scene. The optical waveguide system uses a holographic waveguide or a free-form surface optical device to guide virtual image light to the visual field of a user, so that the equipment is light and the display definition is ensured. Meanwhile, the system is provided with a head tracking and parallax adjusting module, and an inertial measurement unit (IMU) and an external camera are used for tracking the head movement of a user in real time, dynamically adjusting three-dimensional image display and ensuring the consistency of a virtual image and a real scene. According to the design, through precise matching of the display screen and the micro lens array, high-quality reconstruction of a three-dimensional image is ensured, and the cruising ability of equipment is improved by optimizing power consumption management.

A method for motion synchronization-based head pose estimation, by a head mounted display (HMD) device and an HMD device for performing the same are provided. The method includes receiving, by the HMD device, motion data from a plurality of motion sensors of the HMD device, receiving, by the HMD device, a plurality of image frames from at least one simultaneous localization and mapping SLAM camera of the HMD device, estimating, by the HMD device, a plurality of motion parameters of head movements of a user from the plurality of image frames received from memory, generating, by the HMD device, a filtered subset of the motion data received from the plurality of motion sensors based on the plurality of motion parameters of the head movements, synchronizing, by the HMD device, the plurality of image frames received from the memory and filtered subset of motion data, and estimating, by the HMD device, the head pose based on the synchronized plurality of image frames and motion data.