507 results about "Facial expression" patented technology

The invention discloses a scene interactive AI rehabilitation evaluation training and health monitoring system, and relates to the technical field of rehabilitation medical treatment and artificial intelligence, a semantic perception module is used for collecting and recognizing voice input, facial expressions, action tracks and eye movement paths of a user in a training process, and extracting context parameters; the knowledge-driven training generation module is used for calling a rehabilitation knowledge graph constructed by a graph neural network based on context parameters and individual training history, and generating a multi-path training scheme; training a feedback regulation engine, collecting posture offset, physiological stress and emotion feedback, and dynamically adjusting task difficulty, rhythm and prompt mode based on a dual-channel reinforcement learning model; the prediction module fuses training and monitoring data, and predicts a network identification function degradation risk through degradation driving; the cloud edge fusion platform is used for realizing task quick response and graph strategy iterative updating; according to the invention, the individuation, self-adaption and intelligent prediction capabilities of rehabilitation training are improved, and the rehabilitation effect and the system practicability are obviously optimized.

The invention relates to the technical field of AI interaction, and discloses an AI multi-modal emotion interaction memory terminal, which realizes microsecond-level synchronization of voice, facial expression and text data through a multi-thread acquisition engine, dynamically allocates each modal weight by adopting a multi-head cross attention mechanism, and adaptively adjusts modal importance based on a conversation context hidden state; when the cross-modal confidence difference exceeds a threshold value, a gating LSTM conflict resolution module is activated, and the multi-source data collaboration problem is solved; the emotional memory modeling constructs an emotional state transition topology based on a graph convolutional network, protects user privacy in combination with a differential privacy mechanism, and realizes associated event storage of millisecond backtracking of short-term memory and long-term memory. The technology integrates multi-modal dynamic perception, privacy security calculation and adaptive learning ability, significantly improves the real-time performance and personification degree of emotion interaction, and can be applied to the fields of intelligent customer service, emotion accompanying, health monitoring and the like.

An AI-driven system for personalized advertising in real time, where: ◯ an analytics unit to monitor and analyze user behavior in real time across multiple digital platforms such as websites, mobile applications, social media, and smart devices; the unit collects data on user engagement, browsing patterns, time spent, and content preferences to enable targeted, personalized advertising; ◯ a prediction module that predicts preferences and interests of a user, operatively connected to the user interaction data collection unit, wherein the prediction module uses machine learning models such as deep learning, recurrent neural networks (RNNs) and transformer-based architectures to predict interests of users based on historical interactions and inferred preferences; ◯ an emotion and sentiment analysis unit that assesses the user's mood in real time through computer vision, natural language processing (NLP) and voice analysis, whereby the analysis of facial expressions, voice pitch and linguistic mood is used to determine emotional states and receptivity to advertising content; ◯ an embodiment of a context awareness component in operational communication with the emotion analysis and mood unit, in some cases further augmented by various environmental and situational data such as the device type and its physical location, date and time, and the content processed in the device, processing methods, etc., in establishing adaptability and automatic ad placement to be as relevant as possible to the user and their status as prescribed; ◯ Use reinforcement learning algorithms and generative AI models to drive advertising with personalization engines. Creative elements, messaging, and presentations are dynamically adjusted in real time based on user responses to ensure advertising is personalized and always optimized for best performance; o a privacy-focused AI system with federated learning, differential privacy methods, and on-device AI processing to reduce targeted advertising while complying with international data protection laws such as the General Data Protection Regulation (GDPR) and California consumer privacy laws; o an operationally adapted ad delivery mechanism to engage with real-time bidding (RTB) networks, programmatic advertising exchanges and demand-side platforms (DSPs) and place advertisements through digital advertising networks, which guarantees the delivery of tailored advertising to the most relevant audience in real time; and o a contextual feedback loop in which the machine learning models used in the preference and interest prediction module and in advertising personalization The engine is continuously updated to reflect the latest user engagement data, improving personalization over time and optimizing advertising performance.

The embodiment of the invention provides a vehicle-mounted emotion interaction method and device based on multi-dimensional recognition, and the method and device achieve the precise judgment of the emotion of a driver through innovatively constructing an emotion fusion recognition model and integrating the facial expression, voice emotion, driving behavior and physiological state features. And designing a scene-based self-adaptive interaction strategy, and establishing an interaction triggering threshold value for intelligent matching in combination with external environment data and a danger level. An interaction effect evaluation mechanism is introduced, an interaction strategy model is continuously optimized through an online learning module, and dynamic adjustment of personalized interaction content is achieved. According to the method, the defects of the traditional technology in the aspects of emotion recognition, interaction strategies, effect evaluation and the like are effectively overcome, and the intelligent level and the user experience of vehicle-mounted emotion interaction are remarkably improved.

The invention discloses a personalized digital human generation method based on a single video, and relates to the field of virtual digital human modeling and driving, and the method aims at the video and voice data of a target person, through introducing a multi-modal alignment constrained voice driving synchronization mechanism and combining semantic understanding and an emotion label expression generation model, a personalized digital human model is generated. High synchronization, nature and vividness of digital human facial expressions and voice contents are realized. According to the method, a self-supervised style consistency constraint is added in model training to ensure that the style of a generated character image is stable, and a 3D semantic mask is added in an image fusion stage to improve the fusion precision and realistic effect of a synthetic facial expression and a reference face. According to the method, the digital human can be rapidly cloned and driven to synthesize the expression through the voice only through a single video sample, the generation process is efficient, and the obtained digital human video has excellent sense of reality and interactivity.

The invention discloses an aircraft cabin environment personalized adjustment method based on sentiment analysis, and the method comprises the steps: collecting the facial expression, voice waveform and environment parameters of a passenger in real time through a cabin multi-source sensor, and generating a standardized physiological signal matrix and anonymized voice features through noise reduction and feature extraction; inputting the physiological signal matrix and the voice features into a pre-trained deep learning model, outputting an emotion index and a classification label, and updating model parameters through a federal learning framework; dynamically generating temperature, humidity and oxygen concentration adjusting instructions and dynamic weights by adopting a fuzzy reasoning system in combination with the emotion indexes and passenger preset preferences; environment adjustment is executed through a closed-loop control system, and environment parameter errors are fed back; synchronously updating a fuzzy inference system rule base and deep learning model parameters by utilizing reinforcement learning in combination with environmental parameter errors and emotion index changes, and completing optimization of a closed loop; according to the invention, real-time dynamic regulation and control and continuous adaptation optimization of the personalized cabin environment can be realized.

The invention discloses a multi-mode driven virtual digital human face animation generation method and system, and relates to the field of computer graphics, and the method comprises the steps: obtaining voice input and text input, and extracting voice features and text features; dynamically fusing the two features through an attention fusion model to generate facial expression and head posture control parameters, and dynamically adjusting contribution weights of a voice mode and a text mode to the control parameters by adopting a driving strategy of differentiation of facial upper expression and facial lower expression; and performing local deformation on the virtual digital human face image based on the control parameter to generate an initial animation frame, and performing refining processing on the initial animation by using a generative adversarial network to obtain a refined face animation. By means of the technical scheme, the natural and vivid virtual human face animation matched with the voice content and the text semantics can be generated.

The embodiment of the invention provides a digital human interaction system and method based on multi-modal emotion recognition, and belongs to the technical field of digital human interaction. The system comprises a multi-modal sensing module used for collecting multi-modal data and preprocessing the multi-modal data to generate a standardized data stream; the cross-modal fusion and emotion recognition module is used for carrying out interactive modeling on the multi-modal features and outputting a current emotion label and emotion intensity; the reaction planning module is used for generating a composite reaction strategy; and the digital human rendering module is used for mapping the composite reaction strategy into control signals corresponding to the voice, the facial expression and the action respectively, and driving a digital human to execute corresponding voice output, facial expression change and limb action through the control signals so as to realize interaction. According to the method, multi-modal data are deeply fused through the cross-modal graph neural network and comparative learning, the weight is dynamically adjusted in combination with the modal confidence, and the emotion recognition accuracy and robustness are improved.

The invention discloses a psychological state pre-screening system and method based on multi-modal data. The psychological state pre-screening system comprises a data acquisition module, a data preprocessing module, a psychological state analysis module, a psychological state evaluation and screening module and a feedback and early warning module. The system collects multi-modal data such as facial expressions, voices, texts, physiological signals and the like, and performs psychological state analysis by using a deep learning algorithm. The psychological state score is calculated through a multi-modal data fusion algorithm, and feedback or early warning is provided, so that the user or related mechanisms can perform further intervention. According to the method, the accuracy and the real-time performance of psychological state screening can be improved, and the problem of misjudgment of a single data source is avoided. The system can be integrated to a smart phone, a smart bracelet, a computer and other equipment, realizes low-cost and non-perceptual mental health monitoring, is suitable for personal health management, enterprise employee mental monitoring, psychological counseling auxiliary diagnosis and school psychological screening, and is beneficial to early discovery and intervention of mental health problems.

The embodiment of the invention provides an elderly emotion recognition method and device based on multi-modal perception, and the method and device achieve the optimization and enhancement of the signal quality through innovatively constructing a multi-modal data preprocessing mechanism and integrating the facial expression, voice and posture features. And designing a personalized feature mapping model based on historical emotion expression data, and establishing an adaptive feature fusion strategy for intelligent matching in combination with a cross-modal attention network. A hierarchical time sequence classification mechanism is introduced, dynamic modeling of the emotional development trend is realized through a long-short term memory network, and accurate prediction of the emotional state is supported. According to the method, the defects of the traditional technology in the aspects of multi-modal processing, personalized modeling, time sequence analysis and the like are effectively overcome, and the accuracy and reliability of sentiment recognition of the old people are remarkably improved.

A user workload state processing method and apparatus based on multimodal data, and a device. The method comprises: acquiring state information of a first user; determining workload information of the first user on the basis of the acquired state information, wherein the state information comprises at least two of the following information: physiological information, eye movement information, electroencephalogram information, brain function imaging information, motion capture information, spatiotemporal acquisition information, behavior acquisition information, and facial expression and state information; and feeding back the workload information to a first management account, and formulating a training scheme matched with the workload information. According to the embodiments of the present application, workload of users can be identified on the basis of multimodal data of users.

The invention relates to an intelligent video editing method fusing human face features and human voice features, which comprises the steps of input preprocessing, multi-modal analysis, fusion scoring and automatic editing, and adopts a multi-modal mode for analysis, so that the recall rate of key segments is improved, and the efficiency of editing is improved. A personalized editing strategy is supported, facial expression changes and voice emotion peak values are aligned through dynamic time warping, a CLIP-like structure is used for training a bimodal encoder, the human face and the voice are mapped to a unified vector space, the association weight of the human face and the voice is automatically learned, and the intelligent degree of editing is improved.

The invention provides a pain assessment system and method based on multi-modal physiological signals. The system comprises a multi-modal signal acquisition module, a signal preprocessing module, a multi-modal feature extraction module, a deep fusion analysis module, an individualized calibration module and a result output and early warning module. By synchronously collecting and analyzing multi-dimensional data such as facial expressions, sound features, physiological signs and behavior responses and combining deep learning and multi-modal information fusion technologies, objective quantitative evaluation and real-time monitoring of the pain degree are achieved, and accurate decision support is provided for clinical pain management.

The invention discloses a cross-domain expression motion unit detection method based on text bridging, and the method comprises the steps: extracting AU visual features of different scales through employing a visual encoder, introducing a group of learnable prompt vectors, and combining with AU text description to obtain AU text features; the source domain features and the text features are aligned through comparative learning, and the discrimination of the source domain AU visual features is improved; a cross-modal attention mechanism is adopted to realize preliminary interaction of vision and texts, and AU interaction feature representation is obtained; further promoting deep interaction and fusion of the text and the visual information through a graph neural network to obtain a fused AU feature; and on the basis of a negative sample AU description-based comparative learning method, target domain features are aligned with text features, and feature distribution differences between a source domain and a target domain are reduced. According to the method, the field invariance of the text is utilized, the visual features of the source domain and the target domain are unified, the text features are aligned, and the generalization performance of an AU detection model and the accuracy of the AU detection model on the target domain are remarkably improved.

The invention relates to a multi-modal virtual human real-time generation method based on an expression control embedding space, and belongs to the field of artificial intelligence. According to the method, an expression control embedding space is constructed and used for fusing voice semantics, a rhythm structure and multi-dimensional emotion information, and continuous and controllable multi-modal driving vectors are generated. The whole system has an end-to-end linkage mechanism from audio input to expression and action output. Semantic features, rhythm structures and emotional states jointly act on generation paths of lip and upper body postures and expression modalities, and all modal features are fused and expressed in a unified control space through a collaborative coding and time sequence alignment mechanism. And finally, a high-consistency and high-fidelity virtual human video is generated in real time through an output scheduling mechanism. The method has remarkable advantages in the aspects of modal fusion consistency, generation expression naturalness and emotion control flexibility, and can be widely applied to key scenes such as virtual human broadcasting, voice interaction agency and meta-universe digital identity construction.

The invention belongs to the technical field of computer vision and image recognition, and particularly relates to a fine-grained image block embedded multi-task emotion recognition method, which comprises the following steps of: constructing a golden snub monkey multi-modal emotion data set for wild primate animals, and covering emotion, individual and gender multi-dimensional labels; the method comprises the following steps: preprocessing an input wild primate image, dividing the input wild primate image into non-overlapping local image blocks with fixed sizes through blocking and feature extraction, and mapping the non-overlapping local image blocks to a high-dimensional feature space through linear projection to form a series of image block embedding vectors; performing local feature modeling on the image block embedded vector based on a fine-grained local scanning module to enhance fine-grained perception of local features such as facial expression and hair texture of the golden snub monkey, and performing global feature modeling on the image block feature vector by a global scanning module to enhance global semantic representation; according to the invention, the performance and generalization ability of multi-task identification of golden snub monkeys are improved.

The invention discloses an intelligent accompanying robot system, and the system comprises the following modules: a multi-mode interaction module which is composed of a voice recognition unit, a voice synthesis unit, an emotion recognition unit, and a visual recognition unit, and is used for collecting the voice, facial expression, motion, and environment information of a user; the localized AI decision module is used for deploying a lightweight DeepSeekR1 large language model based on an ESP32-S3 edge computing chip, performing real-time processing on the multi-modal data and generating a social guidance strategy and an emotion intervention instruction; the data management module comprises a user behavior database and a privacy protection unit and is used for desensitizing the data and realizing sensitive data isolation through local storage; according to the method, the single-person intervention cost is reduced, and the time consumption of manual scene simulation is reduced.

The invention discloses an elderly emotion evaluation system based on multi-modal physiological and behavior signals. The elderly emotion evaluation system is composed of a multi-modal signal acquisition module, an emotion induction task design module and a multi-modal feature processing module. By collecting multi-mode signals such as electroencephalogram, electrocardio, voice and facial expressions, the system can synchronously record physiological and behavior changes of a user in a specific situation. The emotion induction task adopts scientifically designed audio and video contents, and six typical emotions such as happiness, tension, boring, anger, calm and sadness are guided. Then, the system extracts emotion related features from the multi-modal signals by using a deep learning method, models a cooperative relationship among the multi-modal signals by using an attention mechanism fusion technology, and generates unified emotion feature representation; by comprehensively analyzing and identifying the emotional state of the elderly user, the system provides important reference for early monitoring of emotional health of the elderly, psychological state evaluation and personalized intervention, and has social and clinical values.

The invention provides an intelligent interaction system and method based on a multi-stage cognitive mode, and the system comprises a multi-modal data collection module which is used for collecting user interaction data through a multi-modal sensor, and the data comprise language input, non-language behaviors, interface operation data, expressions, eye movement tracks and the like; and the cognitive feature analysis module is used for calling a deep learning model to perform feature extraction on the interaction data. According to the method, language, behavior, interaction, physiology and other data are fused through the multi-modal sensor, the cognitive driving vector is generated by using the deep learning model, the real-time cognitive state of the user is effectively captured, then the probability distribution of the cognitive stage is constructed in combination with Bayesian reasoning, the problems that in the prior art, only the cognitive level can be statically judged, and real-time updating is difficult are solved, and the user experience is improved. The accuracy and timeliness of user state perception are remarkably improved, and dynamic accurate recognition and continuous modeling in the cognitive stage are achieved.

The invention discloses a robot interaction system and method based on machine learning and emotion calculation, and the method comprises the steps: a robot collects the voice, facial expression, body language and physiological signal data of a user through a multi-mode perception device, and carries out the noise elimination, signal enhancement and feature extraction of the data; utilizing a deep neural network and an emotion classification algorithm to identify the emotion state of the user, identifying mixed emotion and performing label classification; a personalized emotion model is established through machine learning in combination with historical emotion data of the user, an emotion change rule is reflected, and long-term learning and adaptation are carried out; based on the sentiment analysis result, a response conforming to the user sentiment state is generated; through user feedback and interaction, the system continuously optimizes emotion understanding and reaction ability and updates emotion archives; emotion computing tasks are shared through edge computing or cloud computing resources, and real-time response to complex emotion computing tasks is ensured.

The invention discloses a depression classification method and system based on multi-modal fusion, and relates to the technical field of multi-modal data processing and intelligent identification. Comprising a data acquisition module used for acquiring an electrocardiosignal, an electroencephalogram signal, a facial expression video and a gastrointestinal environment expiration signal; the pre-processing module is used for performing pre-processing operation on the four modal signals; the data fusion module is used for performing high-order feature extraction and dimensionality reduction on the four modal signals by using different deep learning sub-networks, considering the real-time performance and the mutual relation between different modals, and performing feature fusion on the four modal signals after dimensionality reduction based on an attention soft fusion strategy; and the classification detection module is used for performing classification detection on the comprehensive features by using a deep learning classification model. According to the method, real-time signals of four modes are collected, and efficient and accurate recognition and dynamic monitoring of the depression state are achieved in combination with multi-mode signal preprocessing, multi-domain feature extraction, multi-mode fusion and a deep learning classification model.

The invention discloses a senile dementia early warning, evaluation and health management system and method based on artificial intelligence, and relates to the field of artificial intelligence and medical health. The system comprises a data acquisition module for acquiring Chinese speech, electroencephalogram signals, facial expression images and eye movement data of a patient; the data preprocessing module is used for preprocessing and storing various data; the feature extraction and modeling module is used for extracting a feature set and constructing a one-dimensional classification model; the intelligent diagnosis module inputs the feature set to a one-dimensional classification model to obtain a disease probability and a preliminary diagnosis result, and a final result is obtained through comprehensive diagnosis after a preliminary diagnosis threshold value is compared; and the user interaction module realizes diagnosis visualization, allows a user to define a preliminary diagnosis threshold value, imports data to retrain and updates a one-dimensional classification model. Through multi-modal data fusion and intelligent analysis, potential relations among different modal data are fully mined, and the accuracy of senile dementia diagnosis and the system adaptability are effectively improved.

The invention belongs to the technical field of personality prediction, and relates to a large five personality prediction method based on quantum multi-modal fusion and a space-time diagram network. The method comprises the following steps: acquiring an image video, and extracting an image sequence, an audio signal and a text after audio transcription from the image video; extracting facial expression data, limb movement data, voice signal data and text semantic data; extracting feature vectors of time-space relevance from the data respectively; establishing a cross-modal dynamic mapping relationship between the feature vectors, and generating a weight of each piece of data; optimizing the weight of each piece of data based on a quantum optimization algorithm, and establishing a big five-personality trait prediction model according to the optimized weight of each piece of data; collecting data in real time and inputting the data into a big five-personality trait prediction model; and outputting a five-dimensional personality traits vector, and generating each data contribution degree thermodynamic diagram and anti-fact explanation. According to the method, efficient multi-modal feature fusion and accurate spatial-temporal feature modeling are realized, and the method has good interpretability.

The invention relates to an automatic emergency braking threshold value adjusting method and system based on in-cabin multi-mode information, and relates to the technical field of auxiliary driving, the method comprises the steps that visual information and audio information of a driver and at least one passenger in a cabin are obtained, the visual information comprises facial expressions and head postures, and the audio information comprises audio information of the driver and the at least one passenger; the audio information comprises voice features and non-voice sounds; respectively calculating a driver state coefficient and a passenger state coefficient based on the visual information and the audio information; fusing the driver state coefficient and the passenger state coefficient to obtain an in-cabin safety situation coefficient; and according to the in-cabin safety situation coefficient, a triggering threshold value of the automatic emergency braking system is adjusted in a self-adaptive mode. According to the method, multi-modal information such as vision and audio in the cabin is fused, and state analysis of all passengers is combined, so that intelligent self-adaptive adjustment of the AEB braking threshold value can be realized, and the sensing accuracy and decision reliability of an emergency braking system in a complex scene are improved.

Techniques for generating modified images using facial content information are disclosed. First image data comprising first facial content information is received, and a facial content encoder generates a first embedding by extracting the first facial content information from the first image data. Second image data comprising second facial content information and non-facial content information (e.g., style information, pose, facial expression) is received, and a non-facial content encoder generates a second embedding comprising the non-facial content information. A decoder generates a modified image using the first embedding and the second embedding, the modified image comprising the first facial content information of the first image data and the non-facial content information of the second image data.

The invention discloses an abnormal driving behavior monitoring method for an automobile data recorder based on a hierarchical dynamic feedback mechanism. The abnormal driving behavior monitoring method comprises the following steps: S1, acquiring a facial image, an automobile speed and acceleration data of a driver; s2, identifying a driving abnormity type, and analyzing facial expressions, eye movement features and vehicle dynamic data; s3, selecting a task specific network architecture based on the exception type; s4, the hybrid neural architecture searches and optimizes the network architecture, and meta-learning training is carried out; s5, performing layered dynamic feedback according to the severity of the driver behavior; s6, dynamically adjusting the feedback intensity and mode according to the response of the driver; and S7, performing continuous optimization and real-time adjustment, and providing personalized driving monitoring. Through the intelligent monitoring system based on a layered dynamic feedback mechanism, abnormal behaviors of a driver are recognized and intervened in real time, the driving safety is improved, and personalized driving behavior monitoring is provided.

The invention provides an image processing method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring image data of a facial expression; extracting a first expression feature and a first identity feature from the image data; performing feature splicing on the first expression feature and the first identity feature to obtain a first joint feature, and extracting joint information between the first expression feature and the first identity feature from the first joint feature; estimating mutual information between the first expression feature and the first identity feature, and adjusting the first expression feature into a second expression feature by taking minimization of the mutual information as a target; predicting a classification result of the facial expression based on the second expression feature and the joint information; therefore, expression and identity collaborative modeling is realized.

The invention relates to an interaction device, system and method for a virtual digital human, and the method comprises the steps: obtaining environment information and user information based on a multi-mode sensor of the virtual digital human; based on the big language cue word project, the virtual digital human interacts with the user; and reloading the virtual digital human on the basis of character settings preset in the cue word project of the large language model. Real-time natural language communication between the virtual human and the user in the characteristic environment can be realized, and the technical effect of improving fusion of the virtual digital human and a real scene is achieved by matching with facial expressions, actions and occupational dress-up of the virtual digital human.

The present disclosure describes techniques for creating a real-time interactive video. A source image is generated by a first machine learning model based on capturing an image of a user. The image comprises a face of the user. One or more facial images of the user are captured. The one or more facial images depict one or more facial expressions. The source image and information extracted from the one or more facial images are input into a second machine learning model. The second machine learning model is configured and trained to transfer facial expressions of creators to machine-generated images in real-time. The real-time interactive video is created by dynamically driving the source image based on the one or more facial expressions.

The invention relates to the field of education, and discloses an interactive feedback system for modern education, which comprises a state monitoring module, a feedback optimization module, a learning recommendation module, a teaching analysis module and a long-term tracking module, the state monitoring module is used for collecting expression, voice and eye movement data of the student in real time to judge the emotional state and understanding condition of the student; the feedback optimization module dynamically adjusts a feedback strategy according to the learning state and presents the feedback strategy to the teacher; the learning recommendation module generates a learning resource list according to individual demands of students and pushes the learning resource list to the student terminals; the teaching analysis module analyzes the data of the student group and provides teaching strategy optimization suggestions for teachers; and the long-term tracking module records learning data of the students and performs time sequence analysis to form a long-term feedback report. According to the invention, real-time feedback, personalized learning support and data-based teaching strategy optimization for students can be realized, and the teaching efficiency and the learning effect of the students are improved.