Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

1125 results about "Facial affect" patented technology

Intelligent multi-mode virtual digital human interaction system based on AI language large model, interaction method and application

The invention discloses an intelligent multi-modal virtual digital human interaction system based on an AI language large model. The system comprises a high-authenticity face generation module; the high-authenticity face generation module uses an AdaAN network, based on adaptive feature fusion and voice driving and time sequence modeling of voice features, feature information related to voice is extracted, the extracted voice features are processed through a deep neural network, it is ensured that the voice and facial expressions are highly aligned in time and space, and the face recognition accuracy is improved. Collecting a bio-electricity signal, mapping the signal to facial muscle movement, generating a final facial expression, and interacting with a user; the system further comprises an intelligent interaction module, a training optimization and efficient generation module, an efficient integration module, a multi-modal data acquisition module, an AI large model core processing module, a digital human image generation and driving module, an interaction scene adaptation module and a feedback optimization module. The invention further discloses a multi-mode digital human interaction method which has wide application value.
Owner:EAST CHINA NORMAL UNIV

Intelligent pre-examination triage method based on multi-modal data and edge calculation

The invention provides an intelligent pre-examination triage method based on multi-modal data and edge calculation, and relates to the technical field of multi-modal data, and the method comprises the steps: collecting the facial expression, wound part, sound and physiological signals of a patient through medical equipment; multi-modal features are extracted by using a deep learning network and dynamic weighted fusion is carried out to obtain a patient state evaluation result; performing data time sequence alignment and feature extraction based on edge nodes, dynamically adjusting calculation precision and realizing task migration; multi-source data are collected for time sequence alignment calibration and feature fusion, the emergency degree is calculated, and group damage distribution is predicted. According to the method, real-time processing and dynamic optimization of multi-modal data are realized, and the accuracy and efficiency of pre-examination triage are improved.
Owner:北京紫云智能科技有限公司

Emotion recognition method based on large model and related device

The invention relates to the field of data processing, in particular to an emotion recognition method based on a large model and a related device. The method comprises the following steps: firstly, acquiring to-be-processed multi-modal data; the multi-modal data at least comprises text information, audio information and image information input by a user; the image information at least comprises a facial expression image and / or a limb action image. And inputting the multi-modal data into an emotion recognition model for emotion state analysis to obtain a predicted emotion state corresponding to the user. And finally, performing confidence evaluation based on the predicted emotional state, and generating a user emotion recognition result based on the predicted emotional state and a corresponding evaluation result. According to the method, richer emotion details are captured from the multi-modal data through the emotion recognition model, cross-modal emotion intelligent recognition is realized, the accuracy of emotion recognition is further improved by combining confidence evaluation, the emotion recognition efficiency is improved, and the application range of emotion recognition is expanded.
Owner:BEIJING LIXIN INTELLIGENT TECH CO LTD

Man-machine interaction method and system based on multiple modes

The invention provides a multi-mode-based man-machine interaction method and system, and relates to the technical field of man-machine interaction. Voice input of a user is received, environment information is collected, facial expressions of the user are captured, and voice features, environment features and facial features are obtained based on the input; utilizing a pre-trained emotion recognition model to generate emotion features of the user, and utilizing a pre-trained user behavior model to generate behavior features according to the current operation information of the user; after the features are obtained, the first weights and the second weights of the features are dynamically adjusted, feature weighted fusion is carried out, and comprehensive features are obtained; the multiple candidate responses are generated based on the comprehensive characteristics, the final response is screened out through a preset scoring mechanism and output, comprehensive understanding of the intention and the emotional state of the user is achieved through comprehensive utilization of multiple data sources, and meanwhile the weight of each characteristic can be dynamically adjusted so that more intelligent and humanized interaction experience can be generated.
Owner:GUANGZHOU DAZZLE VIEW INTELLIGENT TECH CO LTD

Multi-sensory autonomous multimodal emotion-synchronized environmental control architecture and regulation system (amesecar)

An autonomous environmental regulation and behavioral monitoring system is disclosed, configured to adapt temperature, lighting, and acoustic conditions based on real-time emotional and physiological data. The system includes a dual-redundant central processor, hierarchical communication networks, multi-angle visual acquisition units, infrared thermometers, and modular environmental subsystems. It detects posture, gestures, facial expressions, and thermal signals to classify user states and apply individualized airflow, light, and sound modulation without relying on external internet connectivity. The system also monitors connected appliances using voltage-based pressure analysis to forecast device degradation. With integrated gesture recognition, privacy-preserving data handling, and predictive adaptation, the invention enables multi-user personalization, long-term learning, and uninterrupted operation within residential, administrative, or healthcare infrastructures.
Owner:SEYEDKHAMOUSHI FAEZEHALSADAT +1

Virtual digital human interaction system based on AI

The invention provides an AI-based virtual digital human interaction system, and relates to the technical field of data processing, and the system is used for obtaining a user feature data set by performing feature extraction on an interaction data set, calculating an audio change rate and speech speed stability, obtaining speech emotion data, calculating syntactic complexity and counting the number of emotion words. Calculating a facial expression symmetry degree; obtaining facial emotion data; calculating body stability and identifying a visual staying area; obtaining attention state data; adjusting tone and speed of the virtual digital person according to the virtual digital person text, optimizing sentence pattern complexity and sentiment words in the virtual digital person text, adjusting changes of eyes, eyebrows and mouth corners of the virtual digital person, and adjusting information display sequence and visual focus guidance. According to the method, the interaction efficiency of the virtual digital human is improved by analyzing the facial expressions and the body actions of the user.
Owner:GANSU YUSU TECH DEV CO LTD

Dynamic calibration method and system of vehicle-mounted emotion recognition system

The invention provides a dynamic calibration method and system for a vehicle-mounted emotion recognition system, and the method comprises the steps: S1, obtaining multi-source data which comprises a facial image, a voice signal and a physiological signal; the obtained multi-source data are preprocessed, and preprocessed multi-source data are obtained; s2, performing feature extraction based on the preprocessed facial image, the voice signal and the physiological signal to obtain a facial expression feature vector, an audio feature vector and a physiological state feature vector; s3, evaluating the current environment credibility based on an environment credibility evaluation function; s4, dynamically distributing the weight of the multi-source data according to the credibility of the current environment and the real-time scene; and S5, constructing a multi-modal fusion vector based on the dynamically distributed weight of the multi-source data, the facial expression feature vector, the audio feature vector and the physiological state feature vector, and performing emotion recognition by using the constructed emotion recognition model based on the multi-modal fusion vector.
Owner:SHANGHAI PUFAFEN ELECTRONIC TECH CO LTD

Self-learning multi-modal emotion recognition method based on multi-scale cavity attention

The invention provides a self-learning multi-modal emotion recognition method based on multi-scale cavity attention, and solves the problem of low recognition precision caused by different importance of basic action units of a face and different distances between key action units, and the problem of different modality confidence during decision-level fusion. The method comprises the following steps: preprocessing a facial expression image, inputting the facial expression image into a multi-scale cavity attention convolution module, extracting features through a parallel three-branch convolution structure, splicing the features, calibrating through an attention mechanism to obtain an enhanced feature map, and sending the enhanced feature map to a full connection layer to recognize emotion; an original electroencephalogram signal is input into a time-frequency-space three-dimensional feature extraction network, the signal is decomposed, differential entropy features are calculated and processed by a global attention module comprising a frequency spectrum attention module, a space attention module and a time attention module, time-frequency-space multi-dimensional feature representation is output, and emotions are recognized by a full connection layer; and finally, inputting the emotion recognition result of the facial expression and the electroencephalogram signal into a self-learning weight module, and obtaining a final emotion recognition result through dynamic weighted fusion.
Owner:DALIAN UNIV

Mental disorder-oriented multi-mode large model identification and intelligent intervention method and system

The invention discloses a multi-modal large model identification and intelligent intervention method and system for mental disorder, and belongs to the technical field of intelligent identification and intervention of mental disorder, and the method comprises a multi-modal data acquisition stage, a data preprocessing stage, a feature extraction stage, a cross-modal feature fusion stage, a mental disorder identification stage, and a personalized non-drug intervention stage. In the multi-modal data acquisition stage, skin electricity, electrocardio, electroencephalogram and facial expression multi-modal data are acquired in real time, and a dialogue-feedback-adjustment closed-loop mechanism is constructed in combination with a cognitive behavior therapy theory. According to the multi-modal large model recognition and intelligent intervention method and system for mental disorder, accurate recognition and personalized intervention of mental disorder are achieved by integrating electroencephalogram, electrocardio, skin electricity, expression analysis and text data multi-modal physiological and behavior data and combining a large language model and a cross-modal attention mechanism.
Owner:BEIJING INST OF TECH

Dynamic self-adaptive multi-modal sentiment analysis fusion method and system

The invention provides a dynamic self-adaptive multi-modal sentiment analysis fusion method and system, and relates to the technical field of multi-modal sentiment analysis. The method comprises the following steps: synchronously acquiring voice, text, facial expression and limb movement data of a target user to form a multi-modal data set; the method comprises the following steps: firstly, extracting emotional characteristics of each mode, and constructing a cross-mode correlation model to capture a collaborative and complementary relationship among different modes; and calculating a real-time confidence score and a complementarity index of each modal based on the weight matrix of the cross-modal correlation model. Then, according to the scores and the indexes, a weighted average or maximum entropy algorithm is dynamically selected to fuse multi-modal emotion features, and a comprehensive emotion feature vector is generated; and finally, inputting the vector into a pre-training deep learning model, and outputting an emotional state category of the user. According to the method, the user emotion can be accurately and comprehensively captured, efficient emotion recognition and classification are realized, and the robustness and flexibility of an emotion analysis system in a complex scene are improved.
Owner:HUNAN OPEN UNIV (HUNAN PROVINCIAL CADRE EDUCATION & TRAINING ONLINE COLLEGE)

Touch and talk pen intelligent children education method based on emotion recognition

The invention discloses a touch and talk pen intelligent children education method based on emotion recognition. The method comprises the steps that S1, facial expression data, voice and intonation data and touch behavior data of children are collected; s2, multi-modal high-quality emotion data matched with the actual emotion state is generated; s3, generating deep emotional features related to facial expressions, voices and tones and touch behaviors; s4, generating an emotional state label corresponding to the current emotional state of the child; s5, the generation model generates voice feedback content, image animation content and interactive game content which are adaptive to the current emotional state; s6, generating a personalized recommendation content list conforming to the emotional state of the child; and S7, displaying the education content in the personalized recommendation content list through a display screen, a voice output unit and a touch interaction unit of the touch and talk pen. According to the method, the fuzzy classification problem of the emotional state is solved in a touch and talk pen emotion recognition scene, and the adaptability of the model to the complex emotional state is remarkably improved.
Owner:FOSHAN CHUANGZHI XINGKONG CLOUD INFORMATION TECH CO LTD

Psychological accompanying method based on multi-modal emotion recognition

The invention discloses a psychological accompanying method based on multi-modal emotion recognition, and belongs to the technical field of psychological health services, and the method comprises the steps: S1, synchronously collecting physiological signals, voice features, facial expressions and interactive behavior data of a user through a multi-modal sensor; s2, performing fusion analysis on the multi-modal data by using a deep learning model, and identifying a current emotional state and an emotional intensity level of the user; and S3, dynamically generating an adaptive psychological accompanying intervention scheme based on a preset emotion-intervention strategy mapping rule in combination with historical emotion data and personalized preferences of the user. According to the method, the physiological signals, the voice features, the facial expressions and the interactive behavior data are synchronously collected through the multi-modal sensor, the deep learning model is used for fusion analysis, and compared with single-modal recognition, the emotion state and the intensity level of the user can be judged more comprehensively and accurately, the emotion misjudgment risk is reduced, and a reliable basis is provided for subsequent intervention.
Owner:刘梓宸

Facial expression and context-based academic emotion recognition method and device

The invention discloses an academic emotion recognition method and device based on facial expressions and contexts. According to the method, accurate emotion modeling is realized by utilizing a visual language pre-training model CLIP and combining student facial expressions in a video sequence and contextual information of a learning situation. Specifically, according to the method, firstly, collaborative modeling is carried out on facial expressions and context information in a learning scene, so that the ability of recognizing the learning state (such as focusing on learning and distracting) of students is remarkably improved; and secondly, the strong language and visual feature alignment capability of the CLIP pre-training model is utilized, so that the dependence on large-scale video training data is effectively reduced, and the generalization capability and practicability of the model are improved. An accurate and feasible solution is provided for academic emotion recognition, and the method is suitable for monitoring and analysis of academic emotions of students in an education scene.
Owner:ZHEJIANG GONGSHANG UNIVERSITY

Vehicle control method, device, equipment and medium

The invention relates to a vehicle control method and device, equipment and a medium, and the method comprises the steps: obtaining voice data and video data of a target user in a vehicle, inputting the voice data and the video data into an instruction generation model, and obtaining a target instruction text outputted by the instruction generation model; and then the vehicle is controlled through the target instruction text. Wherein the instruction generation model is used for fusing audio features corresponding to the voice data and image features corresponding to the video data so as to generate a target instruction text, the audio features comprise a first text sub-feature and a first emotion sub-feature, and the image features comprise a second text sub-feature and a second emotion sub-feature; the first emotion sub-feature is used for representing at least one of the tone, intonation and volume of speaking of the target user, and the second emotion sub-feature is used for representing the facial expression and / or lip change of the target user. According to the invention, the accuracy of vehicle control is improved, and the driving experience of a user is improved.
Owner:CHONGQING CHANGAN AUTOMOBILE CO LTD

Intelligent accompanying human-type robot based on multi-modal emotion interaction and sensing method

The invention relates to the technical field of robot perception, in particular to an intelligent accompanying human-type robot based on multi-mode emotion interaction and a perception method, and the method comprises the following steps: collecting facial expression and body posture data of a user through a visual perception unit, collecting intonation, speed, volume and keyword data through a voice perception unit, and carrying out the recognition of the speech, speed, volume and keyword data; data preprocessing and feature extraction are carried out; secondly, carrying out time sequence modeling on the extracted visual and voice features based on a recurrent neural network, and generating visual and voice time sequence feature vectors; the method comprises the steps that firstly, time sequence features are extracted, then, a multi-mode fusion model based on an attention mechanism is adopted, the time sequence features are fused into a comprehensive emotion feature vector used for emotion state classification, and finally, personalized interaction content such as voice, expressions and limb actions matched with the emotion state of a user is generated according to an emotion classification result. According to the method, the accuracy of robot emotion perception and the naturalness of man-machine interaction are remarkably improved, and the method is suitable for multi-scene application of education, medical treatment, old-age care and the like.
Owner:BEIJING HAIBAICHUAN TECH CO LTD

Employment and entrepreneurship support system based on artificial intelligence

The invention discloses an employment and entrepreneurship support system based on artificial intelligence, relates to the technical field of occupational planning, and aims to solve the problem that a traditional support mode is insufficient in individuation and accuracy. The system comprises a multi-modal data acquisition module used for acquiring facial expressions, whole body dynamics and voice data of a user in at least 30 minutes of video dialogue in real time; a dialogue text is processed through a large language model based on a Transform architecture, continuous time sequence modeling is carried out on multi-modal dynamic behaviors by applying a liquid time constant network, and deep fusion reasoning is carried out in combination with multiple groups of multi-head Transform attention mechanisms. The core analyzes the real thought, psychological state, behavior pattern and core ability of the user through consistency verification, generates a structured dynamic user insight abstract, and customizes personalized vocational development or entrepreneurship planning according to the structured dynamic user insight abstract. According to the method, the potential of the user can be deeply informed, high-precision personalized planning is provided, the decision-making quality and success rate are improved, and the method has dynamic adaptation and continuous learning capabilities.
Owner:青岛市军队离休退休干部活动中心

System for real-time analysis of emotional feedback during motivational presentations

A system for real-time analysis of emotional feedback during motivational speeches, consisting of: a series of multimodal sensors, including at least one visual sensor configured to capture facial expressions of spectators, at least one directional microphone configured to capture the audio responses of the audience, and optionally one or more physiological sensors configured to capture biometric signals from spectators; an edge-based processing unit that is communicatively coupled to the arrangement of multimodal sensors, wherein the edge-based processing unit comprises the following: (a) a feature extraction module configured to extract visual features from captured facial images, acoustic features from voice responses, and physiological features from biometric signals; (b) an emotion inference machine configured to process the features using a deep learning-based emotion recognition model comprising a convolutional neural network (CNN) for classifying facial expressions, a recurrent neural network (RNN) for classifying voice emotions, and a multimodal late fusion layer configured to compute a composite emotion state vector representing the aggregated emotions of the audience; (c) a timestamp and speech alignment module configured to correlate the calculated composite emotion state vector with segmented portions of a live motivational speech based on real-time speech-to-text transcription and semantic analysis; and (d) a session-based storage unit configured to log time-indexed emotional state vectors and corresponding speech segments for post-event analysis; A speaker feedback interface comprising a portable display or a podium-mounted visualization panel, wherein the interface is configured to display visual indicators of emotional feedback in real time, the indicators being derived from the emotional state vector and including at least emotional trend graphs, threshold alerts, or engagement indices.
Owner:1XL LLC FZ +2

Emotion recognition method based on electroencephalogram and expression data

The invention discloses an emotion recognition method based on electroencephalogram (EEG) and expression data, which combines fusion analysis of electroencephalogram (EEG) signals and facial expression data, and provides an innovative solution for the complexity and accuracy of emotion recognition. The method comprises the following specific implementation steps: 1, finding an optimal frame in an expression video by using multi-instance learning, extracting global visual features by adopting Swin Transform, and realizing dimension reduction and balance representation of cross-modal information through a cross attention mechanism; 2, preprocessing the electroencephalogram signal by using band-pass filtering and a fast ICA algorithm, and extracting effective features by using PSD; and 3, designing a fusion module based on Transform, combining the characteristics of the two modes, and giving full play to the complementary advantage of the modes. Experimental results show that 94.85% of classification accuracy is achieved in four types of emotion classification tasks, and compared with an existing method, the accuracy is remarkably improved.
Owner:BEIJING UNIV OF TECH

Generating a modified digital image utilizing a human inpainting model

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.
Owner:ADOBE INC

Three-dimensional digital human generation method and system capable of voice interaction

The invention belongs to the technical field of three-dimensional reconstruction, and discloses a three-dimensional digital human generation method and system capable of voice interaction. According to the invention, brand new speaking audios in different languages are automatically generated according to different languages of the input target text and the sampled human voice audios; the sequential stability and detail reduction capability of three-dimensional human motion are guaranteed by using multi-model joint estimation and a sequential loss function, and facial expression details and hand postures in the image can be accurately estimated. After the high-precision three-dimensional human body model is obtained through estimation, human body action and expression generation is carried out based on voice driving, accurate synchronization of actions and expressions generated through voice is achieved, and facial expression movement and body posture movement, namely a whole-body three-dimensional human body model, conforming to brand-new speaking audio are accurately generated; and finally, rendering the whole-body three-dimensional human body model into a real digital human capable of voice interaction by using a three-dimensional neural rendering model. According to the invention, the realization of single person picture input, high-precision three-dimensional digital person generation and voice interaction is facilitated.
Owner:NANJING UNIV OF SCI & TECH

Multi-modal data fused virtual reality assisted diagnosis and treatment method for depression

The invention provides a depression virtual reality auxiliary diagnosis and treatment method fused with multi-modal data, and the method comprises the steps: obtaining the multi-modal data, the multi-modal data comprises the peripheral physiological data, such as heart rate, body temperature and skin electricity, monitored by a flexible sensor based on epidermal electrons; and switching to a personalized interactive virtual scene according to the emotional state of the patient in real time, and in the interaction process, obtaining eyeball movement data by the eye movement tracking instrument, and knowing attention distribution, psychological state and micro facial expression change of the user. In combination with multi-modal data of peripheral physiological data, eye movement data and micro-expression change, key features are automatically extracted from multi-modal input data, and emotion analysis and recognition are performed through a multi-modal deep learning method with an attention mechanism. Data can be collected and analyzed in real time, the emotional state of a patient is objectively reflected through the combination of peripheral physiological data and micro-expression analysis, the requirements of the patient are met in a personalized mode, and the treatment effect is improved.
Owner:CAPITAL NORMAL UNIVERSITY

Real-time interaction 3D digital holographic cabin method based on deep learning and sound cloning

The invention discloses a real-time interaction 3D digital holographic cabin method based on deep learning and sound cloning. The method comprises the following steps that S1, user data are collected and preprocessed; s2, extracting feature vectors of facial expressions and limb actions; s3, generating speech synthesis data by using the improved GE2E network and a preset target speech text; s4, generating a synthesized voice audio based on the voice synthesis data; s5, generating a three-dimensional digital human motion sequence according to the facial expression feature vector and the body motion feature vector; s6, performing timestamp alignment on the three-dimensional digital human action sequence and the synthesized voice audio, and constructing a synchronous output stream; and S7, rendering the synchronous output stream, and performing three-dimensional visual output. According to the method, the improved GE2E network, deep learning and sound cloning methods are fused, three-dimensional virtual human voice action synchronous control is achieved, and the method has the advantages of being high in real-time performance, high in immersion and natural in interaction.
Owner:HANGZHOU SECOND LIFE TECH CO LTD

Conversation marketing strategy optimization method and system based on artificial intelligence

The invention relates to the technical field of intelligent dialogues, and discloses a dialogue marketing strategy optimization method based on artificial intelligence, and the method comprises the following steps: collecting the voice, text, facial expression and physiological signals of a user in real time through a multi-modal perception assembly of a terminal device, constructing a dynamic emotion map, and extracting a multi-modal feature vector; a cross-modal information processing module is utilized to align the multi-modal data through a comparative learning algorithm, and causal relationship description of user behaviors and strategies and strategy risk scores are generated; cooperatively training a global strategy model through differential privacy and homomorphic encryption technologies; combining the dynamic emotion map and a causal model to generate an emotion adaptive dialogue script, and optimizing strategy selection through a reinforcement learning algorithm; and dynamically updating the emotion map, the risk score and the global model through a closed-loop feedback mechanism to form a real-time optimized strategy generation system. According to the invention, the practicability of artificial intelligence to real-time services can be improved.
Owner:SHENZHEN SKYCRANE TECH CO LTD

System for delivering personalized motivational content using biometric signals

A system for the real-time delivery of personalized motivational content based on biometric information; the system includes: a biometric acquisition module configured to capture a variety of physiological signals from a user, wherein the physiological signals include at least heart rate variability, electrodermal activity, facial expressions and electroencephalographic (EEG) signals; a preprocessing module that is operationally coupled with the biometric acquisition module, wherein the preprocessing module is configured to remove noise, normalize and extract signal features from the physiological signals in real time; a multimodal biometric fusion engine configured to temporally align and synchronize the extracted features across signal modalities using dynamic time distortion and confidence-weighted interpolation; a motivational state inference model with a hybrid neural architecture comprising a Convolutional Neural Network (CNN) for spatial pattern recognition and a Recurrent Neural Network (RNN) for temporal sequence modeling, wherein the inference model is configured to output a motivational input score and an affective state classification; an engine for recommending motivational content, configured to select and prioritize content from a content repository based on motivational uptake score, user profile metadata, contextual signals including time of day and geolocation, and historical content effectiveness profiles; and a content delivery subsystem comprising one or more output modalities selected from an acoustic actuator, a visual display, a haptic actuator or an environmental controller, wherein the content delivery subsystem is capable of presenting the selected motivational content in a modality that is dynamically adapted to the user's current psychophysiological state.
Owner:1XL LLC FZ +3

Method and system for latent-space facial feature editing in deep learning based face swapping

A computer-implemented method of changing a face within an output image or video frame that includes: receiving an input image that includes a face presenting a facial expression in a pose; processing the image with a neural network encoder to generate a latent space point that is an encoded representation of the image; decoding the latent space point to generate an initial output image in accordance with a desired facial identity but with the facial expression and pose of the face in the input image; identifying a feature of the facial expression in the initial output image to edit; applying an adjustment vector to a latent space point corresponding to the initial output image to generate an adjusted latent space point; and decoding the adjusted latent space point to generate an adjusted output image in accordance with the desired facial identity but with the facial expression and pose of the face in the input image altered in accordance with the adjustment vector.
Owner:LUCASFILM ENTERTAINMENT COMPANY LTD +1

Qt interactive game role action response control method and system fused with AI emotion recognition

The invention relates to the technical field of game development and artificial intelligence, and discloses a Qt interactive game role action response control method fused with AI emotion recognition, which comprises the following steps: S1, acquiring multi-modal data such as facial expression, voice intonation and limb action of a player by using sensors such as a camera and a microphone on game equipment; s2, the collected multi-modal data are transmitted to an AI emotion recognition module, a deep learning algorithm is adopted to analyze and process the data, and the current emotion state of the player is recognized; and S3, obtaining current game scene information and other operation instructions input by the player through the Qt framework. According to the Qt interactive game role action response control method and system fused with AI emotion recognition, through the AI emotion recognition technology, a game role can sense the emotion state of a player and make the emotion interaction action matched with the game role, the emotion resonance between the game role and the player is enhanced, and the interaction experience and immersion of a game are greatly improved.
Owner:张敏飞

Automated nonverbal analysis system

Examples relate to computer-implemented methods for analyzing communication in digital evaluation. A computing device accesses multimodal data comprising video and audio information of human subjects and configures a computational model using this data to identify patterns in communication that correlate with assessment metrics. The configuring implements processing techniques that preserve relationships between features across different modalities. When a video recording of a candidate is received, the computing device processes the video using the configured computational model to extract communication features. These features may include facial expressions, gestures, eye movements, posture, vocal tone, and speech patterns. The device generates an evaluation of the candidate based on the extracted communication features and outputs a representation of the evaluation.
Owner:LIGHT STEVEN PATRICK

Sign language recognition method and device based on multi-modal deep learning

The invention discloses a sign language recognition method and device based on multi-modal deep learning, and the method comprises the steps: multi-modal data input: capturing hand motions, gesture tracks and facial expressions at the same time through a camera, a motion capture sensor and other devices, and forming multi-modal data input; sign language action recognition: precise recognition of sign language actions is realized through a combined model of a deep convolutional neural network and a long-short-term memory network; facial expression and gesture track combined recognition: realizing understanding and translation of complex sign language sentences by combining the captured facial expressions and gesture tracks; and context natural language processing: generating a target statement in combination with context semantic understanding, and outputting a translation result. Through hand motion capture, facial expression analysis, gesture trajectory tracking and context natural language processing, complex sign language motions can be recognized more accurately and translated into characters or voices in real time, and low delay and high accuracy are achieved.
Owner:MIANYANG CITY UNIV

Old-age service robot based on emotion adjusting system

The invention relates to the technical field of service robots, and discloses an elderly care service robot based on an emotion adjustment system, and the robot comprises a multi-mode emotion recognition module which is used for carrying out the comprehensive perception of the emotional state of the elderly through combining various emotion signals of voice, facial expression and limb movement; the personalized emotion adjusting module is used for generating personalized emotion responses meeting the requirements of the old people; the emotion memory module is used for recording emotion changes, important life events and personal preferences of the elderly; the scene self-adaption module is used for sensing the living environment of the old people in real time and adjusting the emotional expression mode of the robot according to the environment information; the health intervention module is used for generating a health management scheme through long-term monitoring of the emotional state and health of the old people; and a safety monitoring module. According to the invention, the emotion state of the elderly is analyzed through the multi-modal fusion network, the emotion misjudgment probability is reduced, and the man-machine interaction experience and the emotion support effect of the elderly are remarkably improved.
Owner:王彦