663 results about "Head posture" patented technology

The invention discloses a fatigue driving monitoring and early warning method based on multi-dimensional feature fusion, and relates to the field of intelligent traffic and safe driving, and the method comprises the steps: carrying out the feature extraction of a collected face image of a vehicle driver through a feature extraction model based on a convolutional neural network, and obtaining a face key point; then calculating the eye closing rate, the mouth opening rate, the head posture data and the facial muscle data to finally obtain multi-dimensional features, and fusing the multi-dimensional features by using a multi-layer perceptron to obtain fused facial features; and then, a fatigue state detection model obtained based on recurrent neural network training is utilized to detect the facial feature time sequence at a plurality of continuous sampling moments so as to detect whether a fatigue driving state occurs and trigger alarm feedback. According to the method, the fatigue state of the driver can be identified and judged more accurately through combination of multi-dimensional data fusion and a time sequence analysis algorithm, and the monitoring and early warning accuracy and reliability are improved. And the system can keep high robustness even in an illumination change or complex driving environment.

The invention discloses an intelligent control method and system for AR equipment, and the method specifically comprises the steps: determining a pupil center position coordinate and a cornea reflection point position coordinate of a user according to an eye image of the user, and inputting the pupil center position coordinate and the cornea reflection point position coordinate into a geometric mapping model for matching, obtaining user sight direction information; acquiring image brightness value distribution of an area near a user fixation point, determining a plurality of candidate saliency areas according to the image brightness value distribution, and comparing the candidate saliency areas with a preset saliency area template to determine a user attention focus; and acquiring head posture information of the user in real time by adopting a head posture tracking algorithm, and dynamically controlling the shooting direction of a camera of the AR equipment according to the head posture information, the sight direction information of the user and the attention focus of the user. According to the method and the device, accurate alignment of the AR content and the visual focus of the user is realized, and the immersion and interaction experience of the AR application are greatly improved.

The present application relates to the technical field of image processing. Disclosed are a texture mapping method and apparatus based on three-dimensional (3D) face reconstruction, a device and a storage medium. The method comprises: registering a standard 3DMM model with a 3D grid model, and performing region division on the 3D grid model; obtaining a texture image of a target object, and matching 2D face key points of the texture image with 3D face key points of the registered 3DMM model to obtain head posture information of the texture image; and on the basis of whether the head posture information meets preset requirements or not, determining whether to use the texture image for mapping. According to the technical solution, a standard 3DMM model is used to perform region division on a 3D grid model of a target object, and on the basis of head posture information of texture images, a texture image with the head posture meeting preset requirements is selected to perform mapping processing on a corresponding region, thereby making the texture of the mapping more consistent, and constructing a 3D face model having a better visual effect.

A video perspective method based on virtual reality relates to the technical field of virtual reality, and comprises: at least two cameras for capturing video information in real time, the cameras being provided with spatial positions and angles; performing geometric correction on the video information; the eye movement tracking module monitors the gazing direction of the eyes of the user in real time and adjusts the display position; the eye movement tracking module tracks eyeball movement; reconstructing a three-dimensional model of the surrounding environment of the user; based on the three-dimensional model, determining a relative position relationship between the virtual object and the real object; the head posture sensor obtains head movement information of a user; performing semantic segmentation on the video frame through a deep learning algorithm; and synthesizing the video information and the virtual content on the head-mounted display device, and outputting the synthesized video information and virtual content. The technical problems of visual discomfort and dizziness caused by position mismatching can be solved, and natural fusion of a real scene and a virtual scene is realized.

The invention provides a driver fatigue monitoring technology using eye movement tracking, and relates to the field of auxiliary driving, and the technology comprises the steps: collecting a face image frame through a front-end camera, carrying out Medipe (Media Pipeline) detection, recording a head posture, and recognizing an iris center region; extracting continuous eye closing time and mouth state information of the driver through the face information; a PNP algorithm is used; a world coordinate system and a camera coordinate system are established, and the sight line vector and the external environment are located in the same space; modeling is carried out on the human eyes and the fixation area; coordinates of a fixation point in a three-dimensional space are obtained through calculation of the geometric model; real-time attention analysis is carried out through the fixation point, and an analysis result is determined; the attention change condition and the fatigue state of the driver are effectively recognized through a BP neural network in combination with a fixation point analysis result. And early warning measures are taken in time to remind a driver. The system is crucial for improving the driving safety, and necessary warning and intervention can be provided when the driver is distracted or fatigued.

The invention relates to the technical field of computer vision security and protection, and discloses a helmet normative wearing identification method and system considering head key point positions, and the method comprises the steps: constructing a human body detection model and a helmet normative wearing classification model, and carrying out the training; collecting a monitoring video, performing frame extraction on a video stream according to a specific time interval to obtain a video frame, and positioning a human body in the video frame by using the trained human body detection model to output a human body image; reasoning each human body image through a human body key point recognition algorithm, positioning a plurality of key point positions of a human body, judging a head deflection posture by taking mutual positions among key points in a head region, and cutting a head region image according to the key points in the head region; and inputting the head region image into the standard helmet wearing classification model for reasoning and recognition, judging the head posture through head key point recognition, and judging whether the helmet is worn in a standard manner through the classification model, so that the method has relatively high generalization ability and recognition precision.

The embodiment of the invention provides a character animation generation method and device based on voice driving, equipment and a medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: acquiring an audio signal input by a user, a target human portrait picture and a target scene reference video; encoding the audio signal to obtain an audio code; generating a three-dimensional face model motion coefficient according to the portrait picture of the target person and the audio code; extracting blink information and head posture information from the target scene reference video; correcting the motion coefficient of the three-dimensional face model according to the blinking information and the head posture information to obtain a corrected motion coefficient of the three-dimensional face model; and taking a three-dimensional deformable head model as a potential vector, mapping the three-dimensional deformable head model to the corrected three-dimensional face model motion coefficient frame by frame, converting the three-dimensional face model motion coefficient into a video frame, and generating a head speech video of the target person. According to the embodiment of the invention, the quality and reality of character animation generation driven by voice can be improved.

The invention relates to the technical field of image recognition and storage, and discloses a driving video recording method and system based on four-way monitoring. The method comprises the following steps: synchronously acquiring and processing the surrounding environment of a vehicle and the head posture of a driver through a video stream to obtain driving attention reference data; compressing and storing the sight blind area data according to the head posture features to obtain hierarchical video data; performing enhanced mapping on the graded video data to obtain an attention compensation image; performing threat identification and sorting on the compensation image to obtain threat level data; establishing a threat level index structure; and performing hierarchical storage on the data according to the attention coverage degree to obtain a storage data set. According to the method and the device, the technical problem that key scene data may be lost due to lack of consideration of driver attention distribution in a multi-channel video real-time recording process is solved.

A method of predicting neural activation and psychopathology, indicative of ADHD and mood and anxiety disorders in a subject, such as children, is disclosed. The method may be implemented via computer software on a tablet computer, laptop or desktop computer, an input device, such as touchscreen, keyboard or mouse, and a camera. The camera is positioned to capture the subject's face in the frame of the camera. A frustration-inducing activity, such as a game, to the subject, where the subject received positive or negative feedback based on the selections they make in a game. The subject's reactions to the feedback are captured and parsed by the software, to determine the subject's emotional reaction to feedback. Reactions such as eye gaze, head pose, and facial expression may be used. Over a number of trials of the game, a prediction may be made for the neural activation and psychopathology of the subject.

In various examples, head pose prediction for automotive occupant sensing systems and applications is presented. The systems and methods described herein provide for a machine learning model trained using a dataset that comprises ground truth head pose data computed using a registered head model of a training subject. While operating a vehicle, one or more cameras and a depth sensor capture synchronized images of the training subject. To compute a ground truth 3D head pose, angular deviations between a 3D point cloud and the registered head model may be computed to obtain a 3D ground truth head pose measurement. Using an extrinsic calibration transform, the head pose measurement may be mapped into the sensor coordinate frame. Training samples may be produced for training the machine learning model that comprise an optical image frame and the head pose measurement transposed into the frame of reference for that optical image frame.

The invention provides a driving behavior intelligent identification method and system based on video analysis, and the method comprises the steps: obtaining a driver face video stream and a road environment video stream collected by a vehicle-mounted camera, and reading the driving information recorded by a whole vehicle communication network; recognizing an eyelid closing state, a sight line direction and a head posture in the driver face video stream based on a posture recognition model, and performing fatigue distraction analysis to obtain driver state information; performing motion trail analysis on the road environment video stream and the driving information, and performing driving risk assessment in combination with the driver state information to obtain driving assessment information; and performing early warning construction according to the driving evaluation information, generating early warning prompt information, and synchronously writing the early warning prompt information, the driving evaluation information and the driver state information into a safety data protection memory. The fatigue and distraction states of the driver can be recognized more accurately, and the accuracy of state judgment is improved.

A method includes determining at an extended reality (XR) device, a first set of vertex adjustment values of a distortion mesh and receiving image frame data of a scene captured at a first time and at a first head pose using a see-through camera of the XR device. The method further includes applying the first set of vertex adjustment values of the distortion mesh to the image frame data to obtain intermediate image data, and predicting a second head pose at a second time subsequent to the first time. The method also includes generating, based on the predicted second head pose, a second set of vertex adjustment values of the distortion mesh, applying the second set of vertex adjustment values of the distortion mesh to the intermediate image data to generate a rendered virtual frame and displaying the rendered virtual frame by the XR device at the second time.

The invention discloses a classroom student attention assessment method and system based on multi-source feature fusion. According to the invention, face recognition is carried out through the face detection unit of the system, occurrence frequencies of different emotional states of students are obtained through a trained student expression recognition neural network model containing four expression discrimination blocks in the expression recognition unit, and fatigue times of the students are obtained through the fatigue detection unit. The number of times of abnormal head postures of the students is obtained through the head posture estimation unit, finally attention evaluation parameters of the students are obtained through an extreme learning machine model of the attention evaluation unit, and then the attention states of the students in the class are evaluated and displayed. According to the method, a more efficient dynamic student expression recognition method can be provided, real-time and accurate classroom feedback is provided for teachers, optimization and implementation of personalized teaching strategies are supported, longitudinal extension of teaching of the teachers and transverse extension of learning modes of the students are facilitated, and new talents are expanded to cultivate a new normal form.

The invention provides a driver anti-fatigue monitoring alarm method, medium and system, and belongs to the technical field of image data processing. A vehicle-mounted camera device is used for collecting a facial image sequence, and a Dlib key point screening function is used for feature point positioning; the feature extraction accuracy is improved by adopting an adaptive illumination compensation algorithm and a minimum spanning tree algorithm, eye features such as an eye closing proportion value and a blinking frequency and head motion features such as a head posture angle value and a nodding frequency are calculated, and physiological data such as blood pressure, heart rate and body temperature are collected by utilizing a smart watch. And inputting the multi-dimensional features into a pre-trained FatgueNet model through an adaptive feature selection gating mechanism, outputting a fatigue state evaluation index, and dividing alarm levels. When the severe fatigue state is recognized and the driver enters the sleepy state, the system sends a pulse stimulation instruction to the smart watch to wake up the driver, and the adaptability and accuracy of the system are improved.

The invention relates to the technical field of computers, and discloses an interaction control method of intelligent glasses. The method comprises the following steps: synchronously acquiring multi-modal data such as eye movement, voice, gestures and head postures and environment and application context information; carrying out independent time sequence feature coding on each modal data; generating a modulation vector in combination with the context, and outputting probability distribution of user intentions through a cross-modal attention fusion network; and a unique execution instruction is determined through an instruction arbitration module based on rules and a state machine. The system comprises corresponding function modules. According to the method, the accuracy, robustness and naturalness of interaction are improved through multi-modal synchronous fusion and a context self-adaption mechanism.

Disclosed is a system with at least one server that is communicably coupled to at least one display apparatus, wherein the at least one server is configured to detect a start of a Vestibulo-Ocular Reflex (VOR) movement (T1) based on a gaze tracking-data and a pose tracking-data received from the at least one display apparatus; control remote rendering of an extended reality video stream at the at least one display apparatus by modifying a head pose data indicated in the pose tracking-data and a gaze direction data indicated in the gaze tracking-data such that a total retinal angular velocity of a focus target approaches a zero value during a VOR movement (T2); detect an end of the VOR movement (T3); and stop the modification and revert video rendering parameters to pre-set default settings after the end of the VOR movement (T3).

The invention provides a target distance estimation method and system based on optical characteristics of a holder, and the method comprises the steps: carrying out the automatic target detection and tracking shooting of a to-be-detected target at a current visual angle, and obtaining a target video of the to-be-detected target; meanwhile, the cradle head posture and the camera focal length are collected in real time; performing target detection according to the target video to obtain key feature points of the target; performing distortion correction on the key feature points through image coordinate mapping to form a unified image coordinate system; combining the image coordinate system with the attitude information of the holder, and mapping the image coordinate system to a three-dimensional virtual space to construct a virtual coordinate system; and calculating the target distance of the to-be-measured target according to the geometric position of the to-be-measured target in the virtual coordinate system. According to the invention, the target distance can be accurately estimated in a complex scene and at different visual angles based on the holder system with a dynamic attitude adjustment function, and the method has the technical effects of real-time performance and robustness.

The invention discloses a device with sleep assessment and intervention functions, which is arranged on a pillow or a back cushion, and comprises an information acquisition module for acquiring physiological data, head posture data and sound data of a human body, and sending the acquired data to a sleep assessment module, the sleep assessment module is used for extracting multi-modal sleep features according to the collected data and performing sleep assessment according to the multi-modal sleep features; wherein the source of the physiological data is analyzed by using the head posture data, and a dynamic weight is given, so that a self-adaptive sleep staging algorithm is completed; and the sleep intervention module is used for acquiring a sleep intervention scheme according to the sleep evaluation result, and activating different working modes of a vibration motor and / or a loudspeaker according to the sleep intervention scheme so as to achieve the effect of intervening the sleep of the user. According to the method, the source of the physiological data is analyzed by using the head posture data, and the dynamic weight is given, so that a self-adaptive sleep staging algorithm is completed, and the sleep staging evaluation accuracy of a real scene is improved.

The present disclosure provides a digital human synthesis method, device, electronic device and storage medium, which relate to the field of artificial intelligence technology. The method includes: obtaining a target image, a driving video and a target audio; extracting three-dimensional face data and three-dimensional implicit key points in the target image, as well as the audio features of the target audio; predicting the three-dimensional face data mapped by the audio features based on the three-dimensional face data and the audio features; obtaining the head posture data of the character in the driving video, and predicting the motion optical flow of the target image based on the head posture data, the three-dimensional implicit key points and the three-dimensional face data mapped by the audio features; synthesizing the video frame image of the digital human based on the motion optical flow to obtain a video of the digital human broadcasting the target audio. The present disclosure ensures that the detailed description of the digital human's face and the head swing are continuous and natural, and improves the effect and stability of the digital human broadcasting video.

The invention provides a head posture estimation method and system based on a neural implicit field, and belongs to the technical field of image processing. According to the method, discretization processing is carried out on a head posture label, and a Gaussian function and a Gumbel function are used for joint modeling to generate an asymmetrically distributed label. Sampling points are arranged at fixed intervals in a three-dimensional space, and global features of an input RGB image are extracted through a RepVang network. And establishing a mapping relationship between image features and spatial sampling points by means of a neural implicit field, mapping face information to a point cloud space, and performing binary classification on each sampling point by using a multi-layer perceptron to generate human face point cloud data. Geometric features of the point cloud data are extracted and fused with global features of the RGB image, and the head posture is predicted through combination of a classification module and a regression module. By adopting the method and the system, the precision and the robustness of head posture estimation can be effectively improved, and a reliable solution is provided for practical application.

Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, totem, and / or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The wearable system can detect when different inputs converge together, such as when a user seeks to select a virtual object using multiple inputs such as eye gaze, head pose, hand gesture, and totem input. Upon detecting an input convergence, the wearable system can perform a transmodal filtering scheme that leverages the converged inputs to assist in properly interpreting what command the user is providing or what object the user is targeting.

The invention relates to a target detection method of a tactical AR helmet. Comprising the following steps: a head posture sensing and space positioning module: after a helmet is started, an inertial measurement unit IMU sensor acquires acceleration and angular velocity data of a head through a gyroscope, an accelerometer and a magnetometer; integrating the data of the multiple sensors by using a sensor fusion algorithm including Kalman filtering; creating space mapping between the virtual scene and the real scene; triggering of a target detection module: inputting a real-time video frame captured by a camera into a pre-trained convolutional neural network (CNN) model; the CNN model performs feature extraction according to different detection tasks including target classification and object boundary detection; performing multi-scale detection on a target through multi-layer convolution and pooling operation; generating a plurality of candidate boxes using a regional proposal network (RPN); real-time target tracking and calibration: a detected target position is transmitted to a target tracking algorithm, and tracking is initialized; the size and the position of the tracking frame are adjusted according to the visual angle change of a helmet wearer.

The invention relates to a multi-posture portrait animation generation method. The method comprises the following steps: acquiring a portrait image and converting the portrait image into a three-dimensional deformation model; acquiring audio data, inputting the audio data into an audio-driven facial motion parameter extraction network, and calculating facial motion parameters according to the facial expression parameters; calculating a head posture motion parameter and a hand posture motion parameter according to the head posture parameter, the hand posture parameter and the audio data by using a posture generation variational auto-encoder; extracting appearance features of the portrait image, and performing standard key point estimation on the portrait image to generate standard key points; mapping the head posture parameter, the hand posture parameter, the facial expression parameter, the facial motion parameter, the head posture motion parameter and the hand posture motion parameter into parameter representation under a 2D view; and finally, inputting the mapped parameter representation, the standard key points and the appearance features into an image generator, and generating a multi-pose portrait animation containing hand postures and dynamic expressions.

The invention discloses an intelligent interaction system applied to AI glasses, relates to the technical field of man-machine interaction, and is used for solving the problems that resource utilization redundancy is increased and equipment power consumption efficiency is reduced. Fusing to form a head posture variable quantity, and analyzing the dispersion degree of the head posture variable quantity based on an MAD algorithm to judge the complexity of an interaction scene; selecting to execute resource optimization or enhance an interaction response strategy according to the complexity, such as improving the speech recognition accuracy and adapting to a complex sound field; further introducing a proportionality coefficient of illumination intensity, movement speed and noise intensity, constructing a long-short-term memory neural network model to predict interaction complexity, realizing dynamic interaction mode adjustment through a gating mechanism and a regularization method, and judging whether to improve the interaction mode according to prediction output. And the self-adaption and high-reliability interaction capability of the AI glasses in a complex environment is ensured.

The invention discloses a distance measurement system based on artificial intelligence, and the system can achieve the automatic grabbing of size parameters of glasses. According to the system, a camera is adopted to capture a glasses image, a gyroscope and an artificial intelligence algorithm are adopted to recognize the spatial position of glasses, spatial coordinates of feature points are obtained, the pixel scale # imgabs0 # of the image is obtained subsequently based on the physical geometrical relationship of the glasses and the real pupil distance of a user, and finally the size parameters of the glasses are obtained. The glasses parameters comprise a glasses width, a pupil distance, a front inclination angle, a surface bending angle and a glasses-eye distance; the camera is an apple panel built-in camera; the gyroscope can obtain the pitch angle # imgabs1 #, the deflection angle # imgabs2 # and the inclination angle # imgabs3 #, so that the head posture of a glasses wearer is obtained, and the size measurement precision of the glasses is greatly improved. According to the system, automatic measurement of glasses parameters is realized, the working intensity of manual glasses matching is greatly reduced, and digital upgrading of high-end customized glasses is realized.

A method for training a driver to operate a vehicle includes identifying a first head pose direction of a driver, at a first time period, in accordance with monitoring the driver. The method also includes activating one or more first magnets of a first set of magnets to interact with one or more second magnets of a second set of magnets in accordance with a difference between the first head pose direction of the driver and an expected head pose direction. The first set of magnets may be integrated with a headrest of the vehicle, and the second set of magnets may be integrated with a helmet worn by the driver. The method further includes deactivating the one or more first magnets in accordance with a second head pose direction of the driver, at a second time period, being within a range of the expected head pose direction.

The invention relates to an automatic emergency braking threshold value adjusting method and system based on in-cabin multi-mode information, and relates to the technical field of auxiliary driving, the method comprises the steps that visual information and audio information of a driver and at least one passenger in a cabin are obtained, the visual information comprises facial expressions and head postures, and the audio information comprises audio information of the driver and the at least one passenger; the audio information comprises voice features and non-voice sounds; respectively calculating a driver state coefficient and a passenger state coefficient based on the visual information and the audio information; fusing the driver state coefficient and the passenger state coefficient to obtain an in-cabin safety situation coefficient; and according to the in-cabin safety situation coefficient, a triggering threshold value of the automatic emergency braking system is adjusted in a self-adaptive mode. According to the method, multi-modal information such as vision and audio in the cabin is fused, and state analysis of all passengers is combined, so that intelligent self-adaptive adjustment of the AEB braking threshold value can be realized, and the sensing accuracy and decision reliability of an emergency braking system in a complex scene are improved.

The invention relates to an intelligent screening system for children with autism. The intelligent screening system comprises a data acquisition module, wherein the data acquisition module acquires face videos of children completing various normal form tasks; the data preprocessing module acquires basic data related to sight drop points, facial expressions and head postures of the children according to the facial video; the feature extraction module adopts a deep learning technology to extract sight drop point features, facial expression features and head posture features from the basic data obtained in the data preprocessing module; the feature fusion module fuses the sight drop point features, the facial expression features and the head posture features; the classification module comprises an LSTM (Long Short Term Memory) model, integrates the fused features into time sequence data and inputs the time sequence data into the LSTM model for classification analysis; according to the arrangement, the feature extraction module and the feature fusion module are used for integrating extrinsic behavior features under various typical paradigms, time sequence analysis is carried out on the features through the LSTM model, and accurate classification of ASD children is achieved.

The invention provides a man-machine interaction sight tracking method, which comprises the following steps of: intercepting and amplifying eye positions in a whole image, training a Yolov5 algorithm by adopting a data set to mark pupil positions and light spot positions from the image, and determining the position of a pupil according to a central position coordinate and a height value and a width value of a detection frame. Marking the pupil position and the light spot position; then, according to a better embodiment of the invention, a 3 * 3 rotation matrix is adopted, the head posture of the interaction object is marked, and the rotation angle of the head posture is obtained; finally, in the mapping process of the coordinates of the display area, the rotation angle of the head posture and the difference value of the pupil light spot position are fitted, so that the tracking precision is remarkably improved, and especially the positioning accuracy of the edge position of the display area is improved.

The invention discloses a method and device for recalibrating the gazing direction of a driver, and belongs to the technical field of intelligent driving. The method comprises the steps that a driver face image in a natural driving scene is collected for head posture estimation, and head posture data of a driver standardized image are obtained; on the basis of a gaze area prediction model, gaze data corresponding to the head posture data of the driver standardized image is obtained, the gaze data is processed, and a calibration image and a target image are further obtained; and respectively inputting the calibration image and the target image into a feature extraction network to extract image features, inputting the calibration image features and the target image features into a direction classification network to find an association relationship between the target image and the calibration image, and completing sight line calibration based on the association relationship. The method can be used for self-adaptive deployment of the pre-training sight estimation model, implicit gaze direction mapping under the condition of few samples and even zero samples is achieved, and extra hardware calibration or model training is not needed.