66 results about "Facial movement" patented technology

The invention discloses an auxiliary decision-making system for cognitive competence assessment of old people, which relates to the technical field of cognitive auxiliary decision-making and comprises an environmental noise acquisition and feature extraction module, a time sequence fluctuation analysis module, a dynamic threshold reconstruction and judgment module, a cross-domain consistency calibration module, a multi-modal synchronous verification module and a dynamic threshold regulation and control module. According to the method, the environmental noise is dynamically analyzed, so that the defects of a fixed noise suppression threshold and a static feature extraction model in the traditional technology are overcome. A sound field energy distribution model is constructed in real time and time sequence fluctuation analysis is combined so that a noise fluctuation interval can be identified, a voice starting point detection threshold value is dynamically adjusted and evaluation precision is enhanced. Through multi-mode synchronous verification combining a reaction time curve and a facial movement track, a time offset error is corrected, a judgment standard is adjusted in real time according to noise changes, misjudgment and virtual high risk early warning are avoided, and therefore the accuracy and reliability of cognitive ability assessment of the old are improved.

Systems and methods are disclosed for driver assistance. In one example a method for an advanced driver alert system for a vehicle comprises monitoring, via a user-facing camera, facial movement of a user over time. The method includes determining a plurality of facial movement metrics based on the facial movement over an adjustable time window, transforming the plurality of facial movement metrics into a machine readable representation of the adjustable time window, and determining, via a code reading model, one or more user requests based on the machine readable representation of the facial movement metrics. The method includes executing one or more user assistance operations based on the determined user requests.

The invention relates to a face forgery identification method, a face forgery identification device, equipment and a medium. The method comprises the following steps: determining a first frame image from a to-be-identified video; wherein the first frame image is a frame image of which the face action amplitude change exceeds a threshold value; acquiring first optical flow motion information of the first frame image and second optical flow motion information of the second frame image; wherein the sampling time of the second frame image is before the sampling time of the first frame image; performing gradient calculation on the first frame image to obtain first gradient information corresponding to the first frame image; and based on the first optical flow motion information, the second optical flow motion information and the first gradient information, obtaining a face forgery identification result of the to-be-identified video.

The present disclosure relates to a device and method for generating avatar lip-sync animation based on multimodal biosignals, The device comprises a multimodal data collection unit configured to collect data including biosignal data including brain waves when a user imagines speaking and image data; a preprocessing unit configured to preprocess the multimodal data; a feature extraction unit configured to extract feature vectors including the user's biosignal feature and facial feature from the preprocessed multimodal data; an avatar generation unit configured to generate an avatar; a lip-sync reconstruction unit configured to predict the mouth shape and facial movement when the user imagines speaking by inputting the extracted feature vectors to a pre-prepared lip-sync reconstruction model; and a lip-sync animation implementation unit for implementing an avatar lip-sync animation by applying the mouth shape and facial movement predicted by the lip-sync reconstruction unit to the avatar generated by the avatar generation unit.

This application provides a method, apparatus, and electronic device for detecting attacks on identity authentication, relating to the field of identity authentication technology. The method for detecting attacks on identity authentication includes: performing specified segmentation processing on target audio and target video to obtain multiple data segments, wherein the multiple data segments include various audio segments and various video segments, and the target audio and target video are content recorded during user authentication based on reading specified verification text; extracting speech features from each audio segment to obtain feature vectors for each audio segment, and extracting facial motion features from each video segment to obtain feature vectors for each video segment; determining the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors; and determining the detection result based on the deviation result corresponding to each target data segment. Therefore, this solution can improve the accuracy of attack detection.

The application discloses a speech-driven lip shape generation method, device and equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: extracting facial motion parameters and face identification features based on target image frames of a video sequence; encoding a driving audio sequence to obtain a time sequence audio feature sequence; performing prediction based on the facial motion parameters and the time sequence audio feature sequence to obtain target implicit key point expression coefficients; and generating target lip shape video frames based on the target implicit key point expression coefficients and the time sequence audio feature sequence. The application further generates target lip shape video frames by calculating target implicit key point expression coefficients, simplifies the overall process of lip shape generation, and improves the efficiency of speech-driven lip shape generation.

The application discloses a face video generation method and system based on a diffusion model and capable of continuously controlling emotions, and the method comprises the following steps: obtaining a speaker face video, performing 3D face reconstruction on video frames, and extracting expression vectors; a corresponding mapping relationship between the extracted expression vectors and emotion labels is established, an emotion editing condition generator is constructed and trained, and corresponding expression vectors are generated according to the emotion labels; a diffusion model is constructed and trained, noise frames, identity frames and motion frames are spliced in the channel dimension to serve as the input of the diffusion model, and time embedding, audio embedding and expression embedding are fused for regulation and control; a reference feature network is constructed, and features output by each layer of the reference feature network are integrated into the diffusion model based on a cross attention and a spatial attention mechanism; and the face video frames output by the diffusion model are subjected to super-resolution processing to obtain a speaker face video. The application can explicitly and continuously edit the emotions of a speaker face video and realize fine-grained control.

Implementations described herein relate to methods, systems, and computer-readable media to generate animations for a 3D avatar from input video and audio captured at a client device. A camera may capture video of a face while a trained face detection model and a trained regression model output a set of video FACS weights, head poses, and facial landmarks to be translated into the animations of the 3D avatar. Additionally, a microphone may capture audio uttered by a user while a trained facial movement detection model and a trained regression model output a set of audio FACS weights. Additionally, a blending term is provided for identification of lapses in audio. A modularity mixing component fuses the video FACS weights and the audio FACS weights based on the blending term to create final FACS weights for animating the user's avatar, a character rig, or another animation-capable construct.

The invention relates to the field of image reconstruction, and provides a face image cross-modal modeling method and device based on voice features. The method comprises the following steps: for a to-be-reconstructed object, collecting real-time voice data of the to-be-reconstructed object; identity information of the to-be-reconstructed object is confirmed based on the real-time voice data, and real-time voice features of the to-be-reconstructed object are extracted from the real-time voice data; selecting a reference face model matched with the identity information from a reference face model library; predicting a real-time face motion state of the object to be reconstructed in combination with the real-time voice features, and mapping the real-time face motion state to a face motion space of the reference face model to form a motion state sequence of the reference face model; and adaptively generating a face image of the to-be-reconstructed object based on the motion state sequence. The identity feature stability of cross-modal modeling, the synchronization rate between face images and voice changes and the quality of generated results can be improved in diversified application scenes and large-scale user environments.

The application provides a real-time digital human video generation method and device, electronic equipment and storage medium, and relates to the technical field of digital human video generation. The method can realize real-time and low-computing-cost digital human video generation by using a predefined face motion template and a material database. The computationally intensive tasks such as design and optimization of the face motion template, customization of the digital human template, and generation of the face motion material are completed in the pre-production stage, which can greatly reduce the computing burden of the real-time digital human video generation stage, simplify the process of the real-time generation stage, and accordingly reduce the performance requirements of the hardware device. The method does not need to rely on expensive computing resources such as high-end GPUs, thereby significantly reducing the overall computing cost and hardware investment of the real-time digital human video generation. Meanwhile, the face motion material obtained in the pre-production stage can be reused, which further improves the resource utilization efficiency and reduces the long-term operating cost.

The application discloses an intelligent vocal music learning and assisting method and system based on artificial intelligence, and relates to the technical field of vocal music teaching.The method comprises the following steps: constructing a teacher pronunciation expression database comprising original pronunciation audio of multiple teachers, frame-level aligned real face videos and international phonetic transcription sequences, and generating teacher 3D face animation data for associated storage; acquiring learner audio data and extracting audio feature vectors; selecting the most similar target teacher based on audio feature similarity; inputting the learner audio into an audio-driven 3D face animation generation model to generate learner 3D face animation data; extracting the key frame sequences of the learner and the teacher for pronunciation actions; performing three-dimensional model difference calculation on the corresponding key frames to obtain frame-level difference quantization data, and generating visualized difference images.The application visualizes abstract vocal music pronunciation skills into intuitive 3D facial movements, realizes precise personalized guidance highly consistent with the individual characteristics of learners, and improves teaching efficiency.

Provided in the present application are a video generation method and a related apparatus. The embodiments of the present application can be applied to various scenarios such as artificial intelligence and computer vision. An embodiment of the present application comprises: first acquiring a face image and a video frame sequence comprising facial motion information of a talking face; then, extracting facial feature information from the face image, and extracting from the video frame sequence head pose features and facial expression features of the talking face; then, performing rendering on the basis of the facial feature information, head pose feature information and facial expression feature information, so as to obtain target frame images; and finally, on the basis of the target frame images, generating a target synthetic video, facial features of the target synthetic video being the same as facial features of the face image. The method provided by the embodiments of the present application introduces a video frame sequence comprising facial motion information of a talking face, so as to add to a static face image dynamic information missing during talking, thereby improving the quality of a target synthetic video.

The embodiment of the invention discloses a video generation method and device, computer equipment, a medium and a program product, and the method comprises the steps: obtaining a face corresponding relation and an original video, obtaining face movement tracks corresponding to all historical faces included in previous i-1 original video frames, and according to a plurality of face movement tracks, obtaining a plurality of original video frames; predicting a prediction area corresponding to each historical face in the i-th original video frame, detecting the i-th original video frame to obtain a detection area of the target face in the i-th original video frame, determining a target area of the target face in the i-th original video frame according to the detection area and the plurality of prediction areas, and fusing the reference face to the target area of the ith original video frame based on the indication of the face corresponding relation, and generating a target video. Therefore, the face changing accuracy is improved, and the face changing stability is guaranteed.

The invention discloses a non-contact real-time heart rate detection method and system based on rPPG and wavelet enhancement, and belongs to the technical field of medical health. Face video signals are collected through the camera, a sensor or an electrode does not need to be worn, real non-contact physiological signal detection is achieved, and the comfort level and use convenience of a user are improved; discrete wavelet transform denoising and enhancement processing is introduced on the basis of pulse signals extracted by a POS algorithm, and high-frequency noise caused by illumination variation, camera quantization errors and facial movement is removed through multi-scale decomposition and threshold filtering, so that the signal-to-noise ratio of the pulse signals is remarkably increased; the fast Fourier transform is combined with a sliding window mechanism, so that real-time updating and continuous calculation of the heart rate can be realized; the signal after wavelet enhancement is clearer in periodicity, so that the heart rate calculation error is remarkably reduced, a dynamic visual interface of a heart rate change curve and a pulse waveform is provided, a user can observe physiological state changes in real time, and good interactive experience is achieved.

The invention relates to the technical field of artificial intelligence and human-computer interaction, and discloses a rehabilitation training adjustment method and system based on multi-modal feature fusion and a storage medium, and the method comprises the steps: obtaining voice, face and physiological signals, constructing a multi-modal data stream, extracting features, and outputting a speech error classification identifier and a physiological signal quality index; splitting the facial action net displacement into healthy and affected sides, correcting affected side features based on healthy side features, and constructing visual representation; fusing each feature to output a comprehensive state vector; updating a dynamic baseline in a task gap, mapping a state vector, and screening to obtain an effective action set; calculating a composite reward and storing the composite reward in an experience playback pool to update the strategy network; and based on the action set, re-weighting the large model output probability, and generating a target interaction corpus. According to the method, the state sensing precision is improved by correcting the deviation of the affected side through the uninjured side, and a closed loop of self-adaptive adjustment of the rehabilitation difficulty and safe interaction text generation is realized by combining the dynamic baseline and penalty mechanism optimization reinforcement learning.

A face mask apparatus configured to more closely adhere to the face of the user such that it is fitted to the bridge of the nose to minimize any vision obstruction by a wearer. The face mask is equipped with a nose bridge segment, a mouth segment, and a chin segment, each disposed in serial communication at obtuse angles. A nose bridge support is disposed within a material of the nose bridge segment and is configured to ensure that the mask remains in position during movement of the face. Additionally, a secondary support is present within the mouth segment to ensure the mask is fitted to the face of the user for comfort and to ensure efficacy. The material of the mask is configured to filter bacteria, viruses, pollen, mold, dust, and other debris, and is preferably removably affixed to the face of the user via retention straps.

The invention provides a multi-source motion fusion speaker video generation system and method, belongs to the technical field of video generation, and aims to solve the problems of lip shaking and motion blurring in traditional speaker video generation. The system comprises an input processing and static modeling module, an initial and final motion conversion module, a multi-source adaptive fusion module and a dynamic rendering and synthesizing module. The method comprises the following steps: establishing a unique figure three-dimensional representation and an audio and video feature representation of a figure identity through an input processing and static modeling module, and analyzing a processing result through an initial and final motion conversion module to generate a control signal; the control signals are input into a multi-source adaptive fusion module for intelligent fusion to generate a high-fidelity control instruction for finally driving the three-dimensional face to move, and finally the high-fidelity control instruction for driving the three-dimensional face to move is efficiently and vividly converted into continuous high-fidelity speaker video frames for output through a dynamic rendering and synthesis module.

The invention discloses a medical monitoring alarm system, which comprises a multi-modal data acquisition module, a multi-modal fusion analysis module and a three-level intelligent alarm module, and is characterized in that emotion, facial action, language and physiological data of a patient are acquired through a medical camera, a microphone array and a physiological sensor; and carrying out multi-modal fusion analysis by using an improved deep learning model, and realizing three-level intelligent alarm based on comprehensive scoring. The system integrates a dynamic weighted fusion algorithm and a hierarchical response mechanism, significantly improves the accuracy and timeliness of medical monitoring, and is suitable for various nursing scenes.

The invention provides a real-time digital human video generation method and device, electronic equipment and a storage medium, and relates to the technical field of digital human video generation, and the method can realize real-time and low-calculation-cost digital human video generation by using a predefined facial movement template and a material database. Computational intensive tasks such as design and optimization of a facial motion template, customization of a digital human template and generation of facial motion materials are completed in a pre-production stage in a centralized manner, so that the calculation burden of a digital human video real-time generation stage can be greatly reduced, and the flow of the real-time generation stage is simplified; and the performance requirement on hardware equipment is correspondingly reduced, and expensive computing resources such as a high-end GPU (Graphics Processing Unit) are not needed, so that the overall computing cost and the hardware investment of the real-time generation of the digital human video are remarkably reduced. Meanwhile, the facial exercise materials obtained in the pre-production stage can be reused, the resource utilization efficiency is further improved, and the long-term operation cost is reduced.

The invention discloses a digital human video generation method and device, equipment, a storage medium and a product, and relates to the technical field of artificial intelligence. The method comprises the following steps: inputting an original face image into an appearance and motion extractor of a target video generation model to obtain an appearance feature map and a face motion parameter; inputting the original audio into a facial expression generator of the target video generation model to obtain facial expression parameters; inputting the original audio into a head posture generator of a target video generation model to obtain head posture parameters; and inputting the appearance feature map, the facial motion parameter, the facial expression parameter and the head posture parameter into a video generation network of the target video generation model to obtain a digital human video generated by the video generation network. Decoupling is carried out through the determination process of the facial expression parameter and the head posture parameter, and the synchronism of the facial expression and the audio content in the generated video is improved.

The invention discloses a robot facial expression generation control method based on a multi-particle swarm optimization algorithm, and relates to the technical field of robot control. According to the method, motor parameters corresponding to facial expressions of a robot are abstracted into particles, populations and parameters are initialized, expressions are generated through physical mapping, and an RGB camera captures and processes images; the method comprises the following steps: firstly, obtaining a Pareto solution set, then respectively calculating expression accuracy, key motion unit activation degree and motion unit coordination degree by using three objective functions to obtain particle fitness, then updating the Pareto solution set and particle information, finally judging whether an end condition is met, and carrying out loop iteration until a natural and vivid target expression is generated. Manual calibration is not needed, the automation degree is high, the universality is good, computing resources can be saved, and the generated expressions conform to the human face movement rule and are vivid and real.

The present disclosure provides a video generation method, a model training method and related products. The video generation method comprises: performing feature extraction on an object image of a target object to obtain a first image feature; performing semantic feature extraction on a target audio to obtain a first audio feature; performing sound event feature extraction on the target audio to obtain a second audio feature; determining a facial motion feature of the target object according to the first audio feature and the second audio feature; and generating a target video corresponding to the target object according to the first image feature and the facial motion feature. According to the embodiments of the present disclosure, the sound events contained in the audio can be extracted and processed, so that the digital person can respond to the sound events accurately.

The embodiment of the invention discloses a video processing method and device, electronic equipment and a storage medium. The method comprises the following steps: extracting a standard image from a reference face video; extracting a time deformation field of each video frame relative to the standard image from the reference face video, wherein the time deformation field comprises face motion information; performing specified style processing on the standard image to obtain a stylized image; and generating a face animation video according to the stylized image and the time deformation field. According to the embodiment of the invention, the generation efficiency of the face animation video is improved, the incoherence and violation of the face animation video are avoided, and the overall fluency of the video is improved.

The application relates to the technical field of biometric feature recognition, and discloses a smart signboard verification management method based on biometric features, which comprises the following steps: establishing a living body recognition model based on historical living body feature data; combining the acquired living body feature data of a current user and the living body recognition model to generate a living body recognition result H3 of the current user; establishing a biometric feature information base based on the fingerprint feature information and the face feature information of all users; searching the biometric feature information base based on the acquired biometric feature information of the current user to generate an identity recognition result F of the current user; and combining the identity recognition result F and the living body recognition result H3 of the current user to generate a signboard verification result. The application can accurately recognize living bodies according to the comprehensive features of facial movements and illumination, and improves the adaptability of the model in various practical application scenarios.

The invention discloses a micro-expression recognition method based on optical flow guidance and local-global representation fusion, and belongs to the technical field of computer vision and emotion calculation. Comprising the following steps: step 1, designing an optical flow coding module, and extracting micro-expression motion information through TV-L1 optical flow and strain features; and step 2, designing an optical flow guided double-attention Transform mechanism, using optical flow motion information to significantly modulate an image feature learning process, highlighting a face motion area, and inhibiting identity and background interference. 3, designing a local-global representation learning module, dividing a human face into 9 AU semantic regions, extracting local features through a local Transform sharing a weight, modeling a cooperative relationship between motion regions through a global Transform, and realizing collaborative optimization of local-global representation, and 4, designing a multi-objective loss function, promoting double-branch deep collaboration, and realizing collaborative optimization of local-global representation. And the complementarity and discrimination of the features and semantic alignment of the expression labels are enhanced, so that the recognition performance of micro-expression recognition is further improved.

This invention discloses an AI-based online teaching management method and system, relating to the field of teaching management technology. It involves acquiring an image dataset of a target user, enhancing the images in the dataset to obtain a target image set, which contains multiple target images. Three-dimensional construction is performed on the target images in the target image set to obtain structural features, and feature extraction is performed on the target images in the target image set to obtain feature groups. The feature groups include global features and local features. The feature groups and structural features are input into a state recognition model to obtain a state score, and warnings are issued to the target user based on the state score. By acquiring and enhancing images, structural features and global and local features of facial movement can be accurately extracted and input into the state recognition model, enabling real-time and accurate identification of the user's learning state, thus improving the recognition success rate.

The present disclosure provides an upper limb movement rehabilitation system based on emotional interaction, comprising: a sensing unit for acquiring physiological signals and behavioral signals of a patient; in the central controller, a data processing module determines the physical state and psychological state of the patient in real time, an interaction strategy generation module simulates the professional knowledge and thinking mode of a rehabilitation physician to understand and integrate the current state of the patient for reasoning and decision-making based on a large language model, and outputs a multi-dimensional interaction strategy with the patient; in the execution unit, a loudspeaker is used to play a voice with a familiar timbre feature and a positive emotional style of the patient according to the voice interaction content during training; a visual interaction device generates a virtual character according to the visual interaction content, and makes it produce corresponding facial movements and expressions with the voice generated by the loudspeaker; the upper limb rehabilitation robot guides the patient to complete the upper limb rehabilitation training action according to the upper limb rehabilitation training content. The present disclosure provides emotional support for patient training, enhancing the rehabilitation effect.

The application provides a long video micro-expression detection and recognition model construction and application method and an electronic device. A pupil dynamic feature is introduced to cooperatively modulate a facial motion feature, and a long video micro-expression detection and recognition model is constructed, so as to solve the problems that in the prior art, long video micro-expression analysis mainly depends on a single facial motion mode, a weak micro-expression key moment is not sensitive enough, cross-modal response is not synchronous, and video level interval recovery stability is poor.

The invention provides a video generation method and a related device. The embodiment of the invention can be applied to various scenes such as artificial intelligence and computer vision. According to the embodiment of the invention, firstly, a face image and a video frame sequence containing face motion information during speaking are acquired, then, face feature information is extracted from the face image, and head posture features and facial expression features of a face during speaking are extracted from the video frame sequence; the method comprises the steps of obtaining facial feature information, head posture feature information and facial expression feature information, performing rendering according to the facial feature information, the head posture feature information and the facial expression feature information to obtain a target frame image, and finally generating a target synthetic video according to the target frame image, the face features of the target synthetic video being the same as the face features in the face image. The video with the natural speaking face can be generated, the reality sense and attraction of people in the target synthetic video during speaking are improved, and the quality and integrity of the target synthetic video are improved.

A facial paralysis evaluation system based on a three-dimensional perception technology relates to the technical field of facial paralysis evaluation, and mainly comprises a three-dimensional perception module, a three-dimensional key point prediction network model construction and training module, and a facial paralysis identification and rating module. The three-dimensional sensing module comprises a three-dimensional texture map acquisition sub-module, a point cloud data extraction and preprocessing sub-module and a position map generation sub-module; the three-dimensional key point prediction network model construction and training module is used for constructing a three-dimensional key point prediction network model and performing training by using a normal person three-dimensional face data set, namely, a Boscorus data set; the facial paralysis recognition and rating module comprises a facial paralysis patient position map prediction sub-module, a key point connecting line selection sub-module and a facial paralysis evaluation sub-module. According to the invention, objective and quantitative evaluation of the facial movement of the facial paralysis patient is realized based on the three-dimensional perception technology.