84 results about "Facial affect" patented technology

This disclosure describes techniques for creating real-time interactive videos. A source image is generated by a first machine learning model based on capturing an image of a user. The image includes a face of the user. One or more facial images of the user are captured. The one or more facial images depict one or more facial expressions. The source image and information extracted from the one or more facial images are input into a second machine learning model. The second machine learning model is configured and trained to transfer the creator’s facial expressions to the machine-generated images in real-time. A real-time interactive video is created by dynamically driving the source image based on the one or more facial expressions.

The application provides a high-efficiency expression migration method using base expression space transformation, comprising a preprocessing stage and a facial expression migration stage; the preprocessing stage comprises: obtaining a source model O, the source model O being a base expression model comprising expression meshes under multiple different expressions; performing facial expression reconstruction on the source model O and a target model T to obtain expression description parameters of the two; performing similarity estimation according to the expression description parameters to obtain an expression parameter conversion matrix and a similarity matrix; the facial expression migration stage comprises: inputting a character picture into the base expression model, calculating expression description parameters of a target expression model according to the expression description parameters in the base expression model and the similarity matrix, and completing expression migration.

This invention relates to the fields of computer vision and intelligent robotics, specifically disclosing a facial expression capture method and robot that integrates an edge-locking mechanism and a lightweight SAM-Tiny model. The method includes: generating a facial mask sequence using an improved lightweight SAM-Tiny model and calculating an edge confidence index; triggering mechanical locking matching the inference latency or calculating multi-axis drive parameters for pose compensation based on the index; recognizing facial expression modalities based on the mask sequence to generate interactive commands; and the robot comprising an electric chassis, a multi-degree-of-freedom support, a vision terminal integrating the improved SAM-Tiny model, a mechanical gripper, and a controlled locking mechanism. This invention, through an algorithm-mechanical timing closed loop, deeply couples edge confidence with mechanical locking frequency, solving the problem of the trade-off between tracking stability and real-time performance in edge computing environments, and achieving non-contact semantic interaction, suitable for medical monitoring and intelligent interaction scenarios.

The application discloses a kind of multi-task wisdom robots based on multi-modal data, comprising: personalized interaction module obtains scene image and obtains face image by face recognition, and judge whether it is student, if yes, the identity information of student is obtained, and corresponding historical data is obtained;Face expression extraction is carried out based on face image and generates face label and conversation theme in combination with historical behavior data;Conversation content is generated based on face label, conversation theme and historical data, whether student is in speaking state is judged, if yes, conversation is stopped, and new conversation content is generated;Multi-task module pre-set task mode, regional function and schedule, obtain current time and current location, obtain current task mode and collect student dynamic data, generate evaluation report based on multi-task data, at least including student dynamic data and historical data.The efficient and personalized conversation and multi-task service with student are realized by the application.

We provide the system. [Solution] A processing device that uses a facial recognition algorithm to detect human features from video data and compares them with a subject list, A device that monitors abnormalities in entrances and windows using a vibration detection unit and an opening / closing detection unit, A system that analyzes a person's behavior and facial expressions to quantify the degree of danger and issues a warning based on that evaluation, A means by which the information processing unit shares information about abnormal individuals with multiple external organizations, A means of transmitting a warning to a mobile communication device when an anomaly is detected, A system that includes this.

The application discloses a face expression capturing model training method based on unsupervised face expression capturing model training method system and medium, obtains a group of preset prompt words for image generation, and randomly extracts a preset prompt word to generate a first two-dimensional face image; obtains a first face expression score vector and a face parameter based on the first two-dimensional face image; calculates three-dimensional face key points, and obtains a second two-dimensional face image according to the three-dimensional face key points; inputs the second two-dimensional face image into an emotion regression network to obtain a second face expression score vector; calculates a preset loss according to a preset loss calculation mode, and trains the face expression capturing model in a gradient back propagation mode. The application can train a model with a driving effect closer to a real face by generating the first two-dimensional face image corresponding to the preset prompt word, combining the face expression score vector, the face parameter and a large amount of unsupervised data, and simplifying the data acquisition process and reducing the cost.

A head-mountable device renders a graphical user interface and causes user-interface changes responsive to facial expressions of a user. The device receives sensor data from user-facing sensor assemblies, including imaging of left and right eye regions, and eye-tracking data indicative of eye movement. The device derives eye-openness data for each eye from the sensor data, including measurements based on eyelid geometry. In response to a facial expression, the device causes a user-interface change that is conditional, at least in part, on the eye-tracking data and the eye-openness data. Facial expressions may include winking, blinking, and squinting. In some embodiments, the device determines an amplitude and / or duration of the facial expression and varies a magnitude or speed of the user-interface change based on the determined parameter(s), enabling text entry, selection, navigation, and manipulation of displayed objects and applications.

The application discloses a kind of facial expression muscle dynamic coordination evaluation system based on deep learning, including data acquisition module, for collecting the multi-object high frame rate facial dynamic video of preoperative baseline period of subject and the multi-object high frame rate facial dynamic video of postoperative review period;Data preprocessing module is used to generate preoperative three-dimensional facial coordinate dataset and postoperative three-dimensional facial coordinate dataset;Vector field extraction module, extract preoperative full-face expression muscle three-dimensional displacement vector field and postoperative full-face expression muscle three-dimensional displacement vector field;Feature extraction module, extract the physiological asymmetry reference feature set of bilateral corresponding muscle area before operation, and the motion feature set of bilateral corresponding muscle area after operation;Model inference module, output eliminates the quantitative value of diagnosis and treatment related muscle movement symmetry deviation and dynamic coordination degree change quantitative value after innate physiological asymmetry;Result generation module generates the expression muscle dynamic coordination change evaluation result of corresponding subject before and after oral diagnosis and treatment.

This specification discloses a method and system for collecting facial expression data. The method includes: controlling a plurality of camera devices to be arranged in a ring around a shooting point, and controlling the plurality of playback devices to play video clips; when a target object at the shooting point watches the video clips, capturing and recognizing the facial expression of the target object; and triggering a target camera device among the plurality of camera devices to start recording the facial expression data of the target object, using a change in the facial expression of the target object as a first trigger condition.

The application provides a humanoid robot facial expression mapping and calibration method, the facial expression mapping method constructs a self-supervised discrete expression data set, trains a multilayer perceptron model based on the data set, realizes preliminary mapping from expression parameters to rudder control signals, extracts expression parameters in a real person expression data set, generates robot rudder control signals through the model, constructs a self-supervised time sequence expression data set based on a redirection method, trains a long short-term memory network model, and realizes time sequence mapping of continuous expressions. The facial expression calibration method uses a visual capture device to obtain robot facial expression parameters in real time, gradually optimizes the rudder control signal, and makes the expression parameter response approach the target value. Through data-driven modeling and visual feedback optimization, the application realizes high-fidelity mapping from semantic expression parameters to multi-rudder collaborative control and systematic calibration, effectively improving the naturalness, accuracy and long-term stability of the robot facial expression.

This application relates to the field of speech analysis technology and proposes a speech emotion analysis method driven by facial expressions of virtual characters. The method includes: acquiring speech signals in different emotional states to obtain different speech signal frames; extracting different speech emotion features from the speech signal frames, pre-setting different time scales, and constructing inter-frame difference coefficients for the speech signal frames; associating the speech signals with emotional states to obtain the probability distribution of all emotional states and the dominant emotion of the speech signal frames, calculating the prior probability of the dominant emotion of the speech signal frames, and calculating the emotion transfer coefficient of the speech signal frames; and obtaining an adjustment probability distribution based on the inter-frame difference coefficients and emotion transfer coefficients of the speech signal frames, wherein the adjustment probability distribution is the result of speech emotion analysis. This application aims to improve the coherence and naturalness of facial expressions of virtual characters driven by speech emotion recognition results.

The application provides a lamp adaptive adjustment method and system based on driver interaction, and relates to the technical field of vehicle driving. The method comprises the following steps: constructing display schemes of a lamp group and display atmospheres corresponding to each display scheme based on sample pictures uploaded by a user; monitoring facial expressions and gesture changes of a driver in real time through a visual acquisition subsystem; matching a target display scheme and a target display atmosphere from the display schemes according to the facial expressions and gesture changes, and determining a corresponding target lamp group control instruction; and adjusting the display color of each LED lamp in the vehicle lamp group according to the target lamp group control instruction, so that the lamp group display matches the target display scheme and the display atmosphere. By implementing the method, the display scheme and the display atmosphere of the lamp group can be automatically adjusted to match the mood and environmental requirements of the driver, the individualization and comfort of the vehicle interior environment are increased, and the interactivity and intelligent degree of the driving experience are improved.

The application provides a cabin adjustment method, device, equipment, medium and program. It relates to the technical field of cabin adjustment. The method comprises the following steps: acquiring video data and pressure data of passengers in a vehicle within a preset time length; performing feature extraction on each frame of image in the video data to obtain a target seat of a child among the passengers, a head posture, a facial expression and a continuous closed-eye time length; obtaining a pressure sum, a pressure center and a pressure change of the child based on the target seat and the pressure data; performing weighted processing on the head posture, the facial expression and the continuous closed-eye time length, the pressure sum, the pressure center and the pressure change according to a fusion weight to obtain fusion data of the child; taking the fusion data as an input of a state estimation model to obtain a child state corresponding to the child; and adjusting the cabin of the vehicle to a state conforming to the child state according to the child state and a preset cabin adjustment rule. The application achieves the effect of improving the safety and comfort of the child during the ride.

The present disclosure provides a facial expression simulation method and apparatus, a device, and a storage medium. The method comprises: collecting a local facial image to be processed of a target object, and generating an expression coefficient corresponding to the local facial image to be processed, wherein the local facial image to be processed belongs to an expression image sequence, and the expression coefficient is determined on the basis of the position of the local facial image to be processed in the expression image sequence; and simulating a facial expression of the target object according to the expression coefficient.

This invention discloses an AI-based physical education exam scoring method, belonging to the field of exam scoring methods, including the following steps: (1) activating the cameras deployed in the exam room to collect motion images and facial expression images of the examinee during pull-ups; (2) analyzing the collected facial expression images using AI image recognition technology to divide the exam process into three stages; (3) for each stage, analyzing the examinee's motion images using AI technology to determine whether each pull-up motion is a qualified motion; (4) counting the number of qualified motions in each stage and scoring them separately, adding the scores of the three stages to obtain the examinee's final exam score. This invention can divide the exam into three stages based on facial expression recognition, perform differentiated analysis on the motion characteristics of each stage, better fit the examinee's physical fitness change patterns, and make the evaluation results more targeted.

The application discloses a facial expression labeling method and device and a medium, and relates to the technical field of data processing. The method comprises the following steps: determining a plurality of expression data, determining a control group from the plurality of expression data, wherein the control group comprises first expression data and second expression data; determining a labeling type of the plurality of expression data, determining an expression difference of the control group according to the labeling type; and performing expression labeling on the control group according to the expression difference. The application compares facial expression data two by two, compares the differences of the two in terms of "positive degree" and "awakening degree", and adopts a simple selection method to determine the expression with a higher positive degree or a higher awakening degree in the paired facial expression. The application greatly reduces the difficulty of the labeling task, thereby greatly reducing the intra-individual variability and inter-individual variability.

The method of quantification of an occurrence probability of a stroke condition, comprises the steps of:providing, by a computer interface, instructions to a user,capturing a series of images and sounds of the user during the execution of each instruction,extracting, by a computing device, features from each series captured,providing the extracted features for each instruction to a dedicated trained machine learning model, said trained machine learning model being trained to associate an intermediate quantified value of an occurrence probability of a stroke condition,receiving the several intermediate quantified value of an occurrence probability of a stroke condition,providing the several received intermediate quantified value of an occurrence probability of a stroke condition to a trained machine learning model, said trained machine learning model being trained to associate a final quantified value of an occurrence probability of a stroke condition with several intermediate quantified values of an occurrence probability of a stroke,receiving the final quantified value of an occurrence probability of a stroke condition, andproviding the final quantified value of an occurrence probability of a stroke condition.

A computer-implemented system for generating video contents associated with digital interactive platforms to optimize user experience in digital interactive platforms, is disclosed. The computer-implemented system is configured to: obtain first inputs from user devices of users; obtain second inputs from the user devices of the users; correlate real-time feedback from the users on actions performed by the users during the digital interactive platforms and the current state of the digital interactive platforms, with face expressions assigned to characters in digital interactive platforms; and generate animated media contents in the video contents to optimize user experience in the digital interactive platforms, based on correlation between the real-time feedback from the users on the actions performed by the users during digital interactive platforms and current state of the digital interactive platforms, and face expressions assigned to characters in digital interactive platforms.

The present application relates to a kind of facial rehabilitation action training method and computer program product, belong to medical rehabilitation technical field.The method is by obtaining user video stream, extracts facial key point and generates facial expression parameter;From facial expression parameter, determine the target parameter associated with preset rehabilitation action, and the target parameter is compared with preset evaluation standard, obtain evaluation result;According to evaluation result, execute feedback operation and / or adjust training target.The method can be for closed-lip action, cheek-puffing action, tongue muscle stretch action, tongue muscle around action and neck massage action etc. many kinds of rehabilitation training are accurately evaluated, by monitoring the duration of facial expression parameter, repeat frequency, facial key point displacement etc. index, combined with training management model generates individualized training target and evaluation threshold.The present application provides a kind of intelligent, individualized facial rehabilitation training solution, can effectively improve the accuracy and effectiveness of facial muscle function rehabilitation training.

The application discloses a facial expression tracking method, a facial expression tracking model training method, an electronic device, a storage medium and a computer program product, relates to the technical field of image processing, and comprises the following steps: regarding each frame of facial image in a plurality of frames of facial images continuously collected for a target object as a first target image respectively, and acquiring a first facial image sequence corresponding to the first target image; inputting the first facial image sequence into a preset facial expression tracking model for prediction, so as to obtain facial expression features corresponding to the first target image, wherein the convolution operation in the facial expression tracking model is a 3D convolution operation performed in three dimensions of time, width and height. The application improves the accuracy of the facial expression features extracted in the facial expression tracking technology.

The application provides a multi-stage semantic emotion-driven digital human action generation method, and relates to the technical field of digital human action generation. The application extracts audio features and text semantic features and completes time alignment and fusion through multi-modal preprocessing of input audio and text to obtain a multi-modal feature sequence. Then, the sequence is used to identify semantic intent and analyze emotion to obtain a semantic intent sequence and an emotion curve containing categories, intensity and change trend. Subsequently, a global action script containing action type, intensity, duration and priority is generated according to a preset action mapping rule. The script is encoded into a feature vector and input into a lightweight neural network to obtain an action parameter sequence containing facial expressions, mouth shapes, skeletal actions and gaze directions. After time sequence smoothing and physical constraint checking, the final digital human driving parameters are output, which can realize low-power real-time operation, smooth action, accurate mouth shape synchronization and emotion and semantic matching.

A joint expression coding system based on static and dynamic expression images is disclosed, including an image preprocessing module, a dynamic expression image generation module, a dynamic weight image generation module, and a joint expression coding image generation module. A joint expression coding method based on the joint expression coding system is also disclosed. A static expression image and a dynamic expression image are combined into one image according to the coding method by adopting the joint expression coding system and method based on the static and dynamic expression images, whereby static expression information and dynamic expression information can be represented at the same time, thus improving the emotion recognition capability based on facial expressions.

This application discloses a music recommendation method, device, and vehicle. The music recommendation method includes: acquiring passenger images, with the number of passengers being no less than two; identifying the behavioral state characteristics of each passenger based on the acquired images, including the passenger's facial expression, whether the passenger is asleep, and the passenger's body posture; and analyzing the human body state data of each passenger based on the behavioral state characteristics, whereby the human body state data can characterize the user's cognitive and physiological state; recommending music that matches the human body state data of the same category when the number of passengers corresponding to the same category of human body state data reaches a first threshold; and recommending music that matches the human body state data of the category with the highest priority based on the priority of each category of human body state data when the number of passengers corresponding to the same category of human body state data does not reach the first threshold, thereby improving the user experience.

Some implementations disclosed herein present a computer-generated reality (CGR) environment in which a user participates in an activity, identify a cognitive state of the user (e.g., working, learning, resting, etc.) based on data regarding the user's body (e.g., facial expressions, hand movements, physiological data, etc.), and update the environment with a surrounding environment that promotes a cognitive state of the user for the activity.

The present invention sets forth a technique for simulating wrinkles under dynamic facial expression. The technique includes receiving a wrinkle graph, including a plurality of nodes associated with a plurality of pores included in a three-dimensional (3D) representation of a facial structure and a plurality of edges associated with a plurality of wrinkles included in the 3D representation of a facial structure. The technique also includes assigning one or more of the plurality of wrinkles associated with edges in the wrinkle graph to one of a plurality of bins and generating, for each of the bins, a plurality of pre-computed displacement texture maps. The technique further includes generating a per-frame displacement texture map and modifying an animation frame based on the per-frame displacement texture map, such that the modified animation frame depicts the plurality of pores and the plurality of wrinkles included in the 3D representation of the facial structure.

This invention provides a method and system for generating virtual digital humans for performances. The method includes: real-time acquisition of raw audio and video streams from live actors' performances, and generating multimodal data packets that are synchronized with the live actors' performance states in real time through data preprocessing; identifying the textual semantic information corresponding to the voice data and labeling it with corresponding emotional tags to form input data packets; inputting the input data packets into a character replication model to analyze the performance style characteristics under the current performance context; generating subsequent target dialogue text that conforms to the target actor's performance style based on the target actor's historical dialogue data; inputting the subsequent target dialogue text into a response generation model to obtain virtual digital human driving instruction packets to generate the virtual digital human's audio stream, facial expressions, and body movement animations, and rendering them on a display terminal. This enables highly realistic virtual digital human performances with creative capabilities, promoting the digital innovation and industrialization of traditional stage arts.

This invention relates to the fields of computer and virtual reality technology, and discloses a method for virtual reality reproduction of opera performances based on motion capture technology. This method uses inertial and optical motion capture, eye tracking, and facial expression acquisition to fuse multi-source heterogeneous data into a 4D temporal motion dataset; it combines a stylized opera motion library for semantic annotation, and uses a graph convolutional neural network to construct a mapping model from motion to digital human driving parameters; it drives a high-fidelity opera digital human in a virtual reality environment, synchronously rendering costumes, makeup, dynamic fabrics, and stage lighting and sound effects that conform to the characteristics of different opera roles, achieving high-precision reproduction of hand, eye, body, and footwork.

We provide the system. [Solution] A computer processing unit for generating advertising information, and means for generating advertising expressions by analyzing past advertising history information and market trend information, A communication means for transmitting and displaying the generated advertising content on the user's mobile device, A means for analyzing gaze information and facial expression information acquired by the user's visual device to identify the user's emotional state, A means to evaluate the effectiveness of advertising expressions generated based on sentiment information analyzed in real time, and to optimize the advertising content as needed. A means by which users can select and display advertising content while using visual devices, A system that includes this.

The utility model discloses a kind of doll assembly mechanism, comprising: mounting seat, face part, front hair part, rear hair part, torso part and clamping structure;Clamping structure is set to the mounting seat, and the face part and the mounting seat are detachably connected by the clamping structure, and at least one of the front hair part, the rear hair part and torso part is detachably connected to the mounting seat and / or the face part by the clamping structure.When assembling, face part, torso part and rear hair part can be clamped to mounting seat in sequence by clamping structure, and the connection mode of clamping has the advantage of easy disassembly and assembly.Since front hair part, rear hair part and face part can be independently disassembled and replaced, users can independently match different front hair, rear hair, facial expression and torso assembly, so that the playability and interaction value of the product can be effectively improved.

An electronic device, a method and a computer program product enable selection of an animation to present during image capturing based on subject facial expression. The method includes presenting a first animation on a display to attract the attention of an individual whose image is to be captured. The method includes extracting a preview image from a preview image stream and determining if the preview image contains the face of the individual with a facial expression that is an acceptable facial expression for initiating capture of a persistent image. In response to determining that the facial expression of the individual is not an acceptable facial expression, the method includes selecting a second animation from among a plurality of animations. The second animation is selected to prompt the individual to respond with a different facial expression. The method includes presenting the second animation on the display.