11 results about "Expression - action" patented technology

The invention relates to the field of human-computer interaction, and discloses a human-computer interaction robot facial expression implementation control method, which comprises the following steps: dynamically capturing the face, sound and posture of a user in a human-computer interaction scene, and generating an initial emotion perception sequence; performing hierarchical cross mapping on the initial emotion perception sequence, and performing combinatorial analysis on facial expressions, voices and intonations and body movement features to form an emotion matching vector; based on the emotion matching vector, utilizing a priority regulation model to predict the emotion trend of the user at the next moment, and identifying an expression enhancement point and an emotion attenuation area; mapping the generated facial micro-expression adjusting instruction to a robot facial driving unit, and performing real-time correction and conflict resolution on an expression action sequence; and reversely fusing a user fixation point, facial muscle micro-motion and voice emotion which are acquired in real time into an emotion matching vector, and dynamically updating a perception weight and expression regulation and control parameters. The method has the advantage of improving the emotion matching degree of the robot expressions.

The application discloses a kind of virtual image model construction method and system based on image clone, it is related to computer graphics field, including: from real-time audio and video stream of real person, the multimodal data of fusion vision, voice and action are obtained;Respectively extract facial expression, voice and action emotional data;Adopt dynamic time warping algorithm to calculate the similarity between each modal emotional data, generate the modal correlation mapping matrix of quantitative modal synchronization relationship;Adopt machine learning model to construct emotional label to limb posture expression action mapping model;According to the deviation of expression and action, the feature is adjusted back;Finally, based on the feature after adjustment, mapping matrix and mapping model, generate the virtual image that expression, voice and action are accurately aligned on time axis.The application establishes the quantitative correlation and feedback adjustment mechanism between modal, significantly improves the real sense and coordination of virtual image when cloning real person in multidimensional emotional expression.

The invention relates to the technical field of artificial intelligence, and provides an old-age care robot content output method and device based on mental health safety modeling, and the method comprises the steps: carrying out the mapping of expression motion features, voice features and physiological denoising information, and obtaining a mental state feature vector; obtaining a content candidate set corresponding to the voice information, extracting an output content candidate feature matrix corresponding to the content candidate set according to a natural language processing model and historical preference data, screening output contents in the content candidate set, and obtaining a final psychological security risk matrix and optimal output content feature parameters, and obtaining the target output content and the dynamic adjustment strategy through psychological safety verification for dynamic output. The method can be applied to a content interaction system in the business fields of medical health, old-age care and the like, psychological safety assessment modeling is carried out on the output content, it is guaranteed that content output is matched with the psychological state, and interaction adaptability and content output reliability are improved.

The invention relates to a multi-mode synchronous control method for driving a robot to narrate, which comprises the following steps: receiving to-be-described text data, and standardizing the to-be-described text data to obtain preprocessed text data; performing hierarchical decoding on the preprocessed text data based on a large language model to obtain structured feature annotation data; generating a multi-modal expression sequence according to the structured feature annotation data, and performing timestamp alignment synchronization on the multi-modal expression sequence to obtain a collaborative multi-modal instruction; and carrying out bandwidth allocation and execution scheduling on the collaborative multi-modal instruction by adopting a dynamic priority strategy, and driving robot hardware to execute a corresponding multi-modal expression action. The diversity of robot emotion expression content is improved, and interaction delay is reduced.

The invention discloses a vehicle-mounted digital human voice interaction system and method based on an unreal engine. The system comprises a voice acquisition module which adopts a microphone array and supports noise reduction processing; the natural language processing module is used for integrating the fine-tuned large language model, judging the emotion of the user through the text subjected to voice recognition, and outputting an emotion label and an answer text; the streaming speech synthesis module is used for performing speech synthesis according to the emotion label by adopting a neural network so as to realize low-delay output; the digital human rendering module is constructed based on an unreal engine MetaHuman technology, realizes synchronization of voice data and animation driving through a WebSocket protocol, and supports expression action driving of mouth shape matching and emotion labels; and the vehicle-mounted display module is used for displaying the digital human image in the screen of the vehicle-mounted terminal and synchronously playing the audio. According to the invention, the problem that an existing vehicle-mounted voice assistant lacks naturalness and emotion feedback is solved, and low-delay and high-immersion vehicle-mounted digital human interaction experience is realized.

The invention discloses holographic interaction equipment based on expression driving. The holographic interaction equipment comprises a holographic display module, a face capture module, an expression recognition module, an interaction decision module, a system control module and a feedback module. Through high-precision three-dimensional face capture, dynamic expression recognition and interactive decision-making mechanisms, facial expression changes of a user can be analyzed in real time, natural expression actions are mapped into virtual interface operation instructions, and then a holographic display interface is dynamically controlled. According to the method, the naturalness, fluency and immersion of interaction are improved, the perception effect of a user on operation is greatly improved through a multi-mode feedback mechanism, such as combination of tactile feedback and visual feedback, and the non-contact and humanized interaction requirements under different application scenes are met. The method is suitable for various application fields such as smart home, rehabilitation assistance, virtual display, immersive entertainment and the like, and has good practical value and wide popularization prospect.

The application relates to a data allocation permission adjustment method and device based on video expression actions, an electronic device and a computer readable medium. The method comprises the following steps: obtaining initial data allocation permission and access information of a target; determining video text content through the access information; establishing a real-time video connection with the target, and displaying the video text content according to the video connection to generate video data; identifying the expression actions of a user in the video data to determine a corresponding permission adjustment coefficient; and adjusting the data allocation permission of the user according to the initial permission and the permission adjustment coefficient. The data allocation permission adjustment method and device based on video expression actions, the electronic device and the computer readable medium can identify micro expressions and micro actions of a user, check the security level of the user from multiple angles, quickly and accurately adjust the access permission of the user, and ensure system safety, data safety and transaction safety.

The application relates to the technical field of artificial intelligence, and relates to a child social accompanying robot interaction system based on emotion recognition, which comprises a multi-source perception and feature extraction module, a cross-modal correlation analysis module, a multi-dimensional emotion fusion module, an emotion strategy mapping and generation module, a double-channel instruction compiling module and a collaborative execution and interaction response module. Multi-scale feature extraction is performed on a face image and a voice signal to obtain a face feature vector and a voice emotion vector. A correlation weight matrix of a target child is constructed. The face feature vector and the voice feature vector are coupled in multiple dimensions to obtain a composite emotion feature. The emotion state is mapped in a strategy mode to obtain an emotion interaction strategy. The composite emotion feature is compiled in an instruction mode to obtain a voice content instruction and an expression action instruction. The voice content instruction and the expression action instruction are input into an execution terminal to obtain an emotion accompanying interaction response of the robot. The application can improve the efficiency of child emotion interaction response.

The application provides a face camouflage method and system based on expression action migration. The method comprises the following steps: extracting a driving video made by a third person according to a random instruction from a face recognition system, taking the difference between the key points of each frame image in the driving video and the key points of a target face, generating an action video of the target face, migrating the action of the third person in the driving video to the target face, retaining the appearance of the target face, adding the motion characteristics of the face of the third person in the driving video, obtaining an action video of the target face, and completing face recognition.

The application discloses a micro-expression feature extraction method based on a motion unit prototype template, and comprises the following steps: in the preprocessing, in view of the face difference problems among different subjects and the head translation problems of the same subject in the video, an organ-based face position correction and cutting method is used to ensure the accuracy of the optical flow feature extraction; in order to accurately analyze the motion unit in the micro-expression, a motion unit-oriented prototype template is proposed. The representative face motion unit dynamic information is recorded in each prototype template, and the AU recognition of the face action with certain difference has strong robustness; in order to accurately capture the micro-expression action, the optical flow map sequence of the micro-expression video is matched with the AU motion template, the complex micro-expression action in the video is deeply analyzed, and the accuracy and the explainability of the micro-expression feature are improved. The application has the advantages that the subtle face action when the micro-expression occurs can be effectively captured, and the application can be used for micro-expression detection, recognition and generation.

The invention discloses a video editing processing method based on artificial intelligence, and relates to the technical field of video processing, and the method comprises the following steps: building a light flash feature description rule for an illumination rapid jump scene, converting each brightness sudden change in a video frame into a light flash event, forming the continuous flash events into a flash event sequence to construct a flash track; and screening a time slice with the maximum brightness change amplitude on the light flash track, extracting a slice with the character expression action kept stable from the time slices, and taking the screened slice as an emotion reference slice to form an emotion baseline. According to the method, through compression of a light flash track, an emotion baseline and an emotion curve, emotion change expression is closer to a real state of a person, and emotion interference caused by illumination jump is weakened; and meanwhile, self-adjusting editing rhythm control is added into the smooth emotion band, so that the rhythm can be adjusted along with the emotion change and the illumination disturbance is avoided, and the rhythm continuity and the emotion expression accuracy of a piece are improved.