Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

2070 results about "Emotion recognition" patented technology

Emotion recognition is the process of identifying human emotion, most typically from facial expressions as well as from verbal expressions. This is both something that humans do automatically but computational methodologies have also been developed.

Transform-based cross-modal fusion multi-modal emotion recognition method

The invention discloses a Transform-based cross-modal fusion multi-modal emotion recognition method and device, which are used for solving the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in a multi-modal emotion recognition task, and the method takes the accuracy and robustness of emotion recognition as performance evaluation indexes. Firstly, feature information of three modes of vision, voice and text is obtained, feature extraction is performed on each mode through a deep learning model, then features of different modes are fused by using a cross-mode Transform module, and a complex dependency relationship between the modes is dynamically modeled through a multi-head self-attention mechanism, so that more accurate emotion recognition is realized, and the emotion recognition efficiency is improved. And finally, performing emotion prediction on the fused features based on time sequence modeling and an emotion classification module. According to the method, the problems of modal isomerism, difficulty in time alignment and insufficient dynamic emotion modeling in multi-modal emotion recognition can be effectively solved.
Owner:SOUTHEAST UNIV

Intelligent emotion intervention and personalized recommendation method and system based on large model in medical industry

The invention discloses an emotion intelligent intervention and personalized recommendation method and system based on a large model in the medical industry, belongs to the technical field of natural language processing, and aims to solve the technical problem of how to improve the efficiency and accessibility of psychological health services and ensure the safety and privacy of user data. According to the technical scheme, the method comprises the following steps: data acquisition and preprocessing: acquiring related data in multiple dimensions, performing data cleaning and structured processing on the acquired related data, and extracting emotion keywords and themes; sentiment analysis: performing fine tuning by using pre-training models such as BERT or GPT-3 to adapt to medical sentiment recognition; constructing a personalized model: customizing a user emotion intervention strategy through time sequence analysis, social network analysis, behavior pattern mining and user portrait construction; generating an emotion intervention strategy; intelligent recommendation; a feedback and learning mechanism; privacy protection and security are realized; designing a user interface and interaction; and integrating and deploying the architecture.
Owner:SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Emotional evolution method and terminal for virtual avatar in educational metaverse

Disclosed are an emotional evolution method and terminal for a virtual avatar in educational metaverse. By collecting expression data and audio data of the virtual avatar, and performing emotional feature extraction based on the expression data and the audio data; fusing an extracted sound emotional feature with an extracted expression emotional feature by using an emotional feature fusion model, and performing emotion recognition on a multi-modal emotional feature fusion result obtained by fusion to obtain an emotional category corresponding to the multi-modal emotional feature fusion result; labeling the multi-modal emotional feature fusion result based on a semantic vector of the emotional category to generate an emotional evolution sequence; and driving the virtual avatar to perform emotional expression according to a target emotional evolution pattern extracted from the emotional evolution sequence, a unified and united multi-modal emotional feature fusion result is formed, the emotional self-evolution of the virtual avatar is achieved.
Owner:FUJIAN TQ DIGITAL +1

Gait emotion recognition method, system, storage medium, and computer equipment based on spatiotemporal graph convolution.

This invention relates to a gait emotion recognition method, system, storage medium, and computer device based on spatiotemporal graph convolution. The method includes the following steps: S1, data augmentation by reversing the temporal direction of gait; S2, obtaining deep emotion features and prior emotion features respectively through a spatiotemporal graph convolutional network and prior feature statistical methods; S3, performing nonlinear mapping on the prior emotion features using a feature mapping layer; S4, inputting the fused features of the deep emotion features and prior emotion features into an emotion classifier to obtain the emotion category. The feature mapping layer of this invention achieves more effective feature fusion by performing nonlinear mapping on prior features; it also introduces causal temporal convolution to replace general temporal convolution, effectively extracting fine-grained temporal features by enhancing temporal correlation and cross-period feature fusion. Furthermore, a walking direction recognition auxiliary task is designed to accelerate the training and convergence speed of the model, enhancing the ability to extract temporal-dependent features and the performance of emotion recognition.
Owner:SOUTH CHINA UNIV OF TECH

Digital human emotion recognition and feedback system based on large model

The invention discloses a digital human emotion recognition and feedback system based on a large model, and relates to the technical field of large models.During operation of the system, multi-modal data of a driver are collected in real time through an embedded sensor network in various driving environments, and the emotion, driving behaviors and driving environment parameters of the driver are obtained; the method comprises the steps of constructing fatigue, anger and other multi-dimensional emotion features of a driver to form an emotion feature vector, analyzing an emotion change trend through threshold comparison, obtaining an evaluation state, dynamically generating a feedback strategy for optimizing a driving state based on an emotion state and a driving scene of the driver, and embedding a generative large model into a module. Diversified feedback responses including audio reminding, driving mode adjustment and entertainment content recommendation are automatically generated, feedback strength and form are adaptively adjusted through multi-dimensional parameters, a feedback generation effect is dynamically adjusted, and an emotion state analysis result and vehicle behavior monitoring data are integrated, so that a user experience is improved. And evaluating the potential dangerous state of the driver in real time.
Owner:SUZHOU LVHUA TECH CO LTD

Multi-modal classroom emotion recognition method and system based on modal adaptive learning

The invention belongs to but not limited to the technical field of emotion recognition, and discloses a multi-modal classroom emotion recognition method and system based on modal adaptive learning, and the method comprises the steps: building a multi-modal data collection system, collecting the multi-modal data, such as texts, audios and videos, in a classroom teaching process in real time, and carrying out the real-time collection of the multi-modal data; and preprocessing and feature extraction are carried out on data of different modes. For emotion expression feature differences of different classroom scenes, a multi-modal data fusion method based on adaptive modal scoring (AMS) is provided, and a time sequence modeling and emotion classification model is constructed. The multi-modal fusion and time sequence modeling mechanism simulates the multi-dimensional perception of the teacher on the emotional states of the students in the class, and enhances the interpretability of the emotion recognition system. According to the system, weighted fusion is carried out on data features from texts, audios and videos through fusion of multi-modal data, and complete emotion feature representation is formed.
Owner:HUAZHONG NORMAL UNIV

Multi-modal digital employee reception system and method based on edge calculation

The invention relates to a multi-modal digital employee reception system and method based on edge calculation, and relates to the technical field of artificial intelligence, and the system comprises a multi-modal data collection module, an edge calculation module, a data fusion and decision module, a trigger activation module, an emotion recognition module, an interaction response module and a behavior prediction module. The method comprises the following steps: acquiring voice, image and action information of a user through a high-resolution camera, a microphone array and an infrared sensor; locally processing data by using an edge calculation module, and extracting voice, vision and thermal imaging features; a Transformer model is dynamically optimized through Test Time Training (TTT), multi-modal data are dynamically fused in combination with a multi-head attention mechanism, a comprehensive decision vector is generated, the system can accurately judge the emotional state of a user, user behaviors are predicted based on a fused Token sequence and historical records, personalized services are actively provided, and by dynamically adjusting a trigger threshold value and a weight distribution mechanism, the emotion state of the user can be accurately judged. The false triggering rate and the missed triggering rate are reduced, and the stability and the robustness of the system are enhanced.
Owner:BEIJING HUAYUAN TECHNOLOGY CO LTD

Intelligent emotional interaction method for service robot

The invention relates to the technical field of intelligent robots, in particular to a service robot intelligent emotion interaction method, which comprises the following steps: acquiring emotion related data of a user through a multi-modal sensor on a service robot, analyzing the acquired data based on a deep learning model, and extracting emotion features of the user; future emotion changes are predicted through a time sequence analysis algorithm, a service robot generates a personalized interaction strategy through an emotion decision-making system according to the current and predicted emotion states, and the system adopts a self-adaptive algorithm based on reinforcement learning, so that the interaction strategy can be continuously optimized along with the emotion changes of a user; the robot performs modeling and optimization on long-term interaction data by utilizing a self-learning mechanism in combination with emotional feedback of the user, so that the precision and interaction effect of emotional response are gradually improved; according to the method, the problems of inaccurate emotion recognition, inflexible interaction strategy and lack of adaptive optimization in the prior art are effectively solved, and the interaction experience of the service robot and the user is remarkably improved.
Owner:BEIJING HAIBAICHUAN TECH CO LTD

Emotion recognition method based on visual language pre-training and multi-modal collaborative fusion

The invention discloses an emotion recognition method based on visual language pre-training and multi-modal collaborative fusion, and belongs to the technical field of multi-modal emotion recognition. According to the method, a visual encoder is pre-trained by using a large-scale visual-text pair data set through a visual language pre-training normal form, so that the visual feature extraction capability of the visual encoder is improved; image frames, skeleton nodes and audios are extracted from the video and are respectively conveyed into encoders of various modes, and features of the multiple modes are extracted; the skeleton node features and the features extracted from the video frames are fused through a cross attention mechanism, so that visual features are further enhanced through information such as motion postures of people; finally, through a learnable query vector and a cross attention mechanism, the visual features and the audio features are fused into the query vector; in the prediction stage, sentiment classification is carried out by using query vector features obtained by learning.
Owner:TIANJIN UNIV

Robot anthropomorphic interaction method based on multi-modal emotion recognition and customized portrait generation

The invention discloses a robot anthropomorphic interaction method based on multi-modal emotion recognition and customized portrait generation. The method comprises the following steps: S1, dynamically fusing multi-modal emotions; the method comprises the following steps: S1, synchronously acquiring voice, visual and text signals through a multi-source heterogeneous sensor, capturing a user voice stream by a high-fidelity microphone array, and extracting acoustic characteristics such as intonation and speed, S2, performing cross-modal reasoning; s3, synchronously generating contents; step S4: style migration; step S5, anthropomorphic voice and expression generation; according to the method, man-machine interaction emotion is analyzed and generated by utilizing a large language model and multi-modal information fusion, the singleness of interaction emotion and the deficiency of emotional sharing ability are avoided, a strong emotion interaction characteristic is achieved, the image of the robot is obtained through a generative technology and can be migrated to any image, the limitation that a specific image is independently made is broken through, and the interaction effect of the robot is improved. The advantage that one robot can be suitable for different scenes is achieved.
Owner:JIANGSU YUNMU ZHIZAO TECH CO LTD

Multi-modal sentiment analysis method and device based on multi-agent collaboration

The invention provides a multi-modal sentiment analysis method and device based on multi-agent collaboration, and the method comprises the steps: carrying out the clock synchronization collection and preprocessing of multi-modal data, and obtaining a multi-modal data package with a time sequence mark; and performing feature extraction on the multi-modal data packet by using a plurality of intelligent agents, and mapping the extracted feature vector and a preset emotion ontology to obtain a multi-modal emotion vector. And detecting the difference degree between the vectors in the multi-modal emotion vectors, and starting a conflict detection algorithm based on feature credibility to perform evaluation when a significant difference is detected to obtain an evaluation result. And performing feature fusion operation on the multi-modal emotion vector according to the evaluation result and a preset weight coefficient to obtain an emotion recognition result, and performing timestamp association storage on the emotion recognition result and the corresponding multi-modal data. By introducing a difference detection and evaluation mechanism, the abnormal feature vector is effectively prevented from influencing the fusion result, and the reliability of modal sentiment analysis is improved.
Owner:SHENZHEN FUTURE QINGYAN INTELLIGENT TECHNOLOGY CO LTD

Psychological counseling robot based on multi-modal data and large vertical model

The invention relates to a psychological counseling robot based on multi-modal data and a large vertical model. The robot is used for providing psychological counseling services for scenes such as families, schools and enterprises, and the front end depends on an intelligent robot terminal, adopts a main mode of virtual digital man-machine interaction and is assisted by multi-modal emotion recognition to enhance the psychological counseling effect; the rear end takes an intelligent agent as a control core and takes a psychological big model as a computing platform. The strong semantic understanding and dialogue interaction capability of the large model is utilized, and the multi-modal mode recognition of expressions and voices is combined, so that the psychological assessment and psychological healing functions are realized, and meanwhile, the knowledge supplementing and checking capability of an external knowledge base is realized; according to the method, a strategy of combining structured prompt with multi-modal emotion data enhancement is used, performance advantages of a large model are fully utilized by means of a retrieval enhancement generation and knowledge graph technology, meanwhile, occurrence of illusion problems is reduced, a digital human interaction mode is adopted, interpersonal barriers are reduced, and consultation quality is improved.
Owner:THE SECOND XIANGYA HOSPITAL OF CENT SOUTH UNIV +1

Man-machine interaction method and system based on multiple modes

The invention provides a multi-mode-based man-machine interaction method and system, and relates to the technical field of man-machine interaction. Voice input of a user is received, environment information is collected, facial expressions of the user are captured, and voice features, environment features and facial features are obtained based on the input; utilizing a pre-trained emotion recognition model to generate emotion features of the user, and utilizing a pre-trained user behavior model to generate behavior features according to the current operation information of the user; after the features are obtained, the first weights and the second weights of the features are dynamically adjusted, feature weighted fusion is carried out, and comprehensive features are obtained; the multiple candidate responses are generated based on the comprehensive characteristics, the final response is screened out through a preset scoring mechanism and output, comprehensive understanding of the intention and the emotional state of the user is achieved through comprehensive utilization of multiple data sources, and meanwhile the weight of each characteristic can be dynamically adjusted so that more intelligent and humanized interaction experience can be generated.
Owner:GUANGZHOU DAZZLE VIEW INTELLIGENT TECH CO LTD

Voice generation method and device based on multi-modal fusion, equipment and medium

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of medical health, financial science and technology, cultural transmission and the like, and discloses a voice generation method based on multi-modal fusion, which comprises the following steps: collecting audio data to extract timbre features, and training a field feature timbre generation model; analyzing text semantic recognition emotion information, adjusting speech synthesis parameters, combining personalized information to construct a parameter mapping table, fusing to generate a synthesis control parameter sequence, aligning the synthesis control parameter sequence with character labels, visual elements and background music data, driving a domain feature timbre generation model, and generating synthesis data of synchronous speech, text, vision and music. Domain timbres are generated through timbre feature training, speech expression is optimized by combining semantic analysis and emotion recognition, user requirements are matched based on personalized information, and time alignment is performed by fusing text, vision and music data, so that the synthesized speech has domain features, emotion adaptability and personalization, and the speech quality is improved. And the voice immersion and the information transmission capability are improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Method for real-time generation of empathy expression of virtual human based on multimodal emotion recognition and artificial intelligence system using the method

Provided are a conversational artificial intelligence (AI) system and method based on real-time multimodal emotion recognition. The system includes a model server configured to provide a machine learning-based conversational model, a terminal configured to perform a conversation with the machine learning-based conversational model through the model server, display a virtual human responding to a user during a conversation with the user, and capture a facial image of the user during the conversation, and a multimodal empathetic conversation-generation system configured to access the model server and receive a response to a question of the user from the terminal, and assess an emotion of the user from the facial image of the user and control, based on the assessed emotion, an expression of the virtual human displayed on the terminal.
Owner:SANGMYUNG UNIV IND ACAD COOP FOUND

Multi-modal sentiment analysis and interactive adjustment method and system based on large model

The invention discloses a multi-modal sentiment analysis and interactive adjustment method and system based on a large model, and relates to the technical field of artificial intelligence and natural language processing. Comprising the steps of 1, collecting dialogue video data and preprocessing the video data, 2, respectively carrying out image feature extraction, voice feature extraction and text feature extraction according to video frames, audios and texts, 3, carrying out interactive fusion on every two modal features through a cross attention mechanism, and step 4, generating a corresponding dialogue strategy according to the emotion label in the recognition result and the requirement of appointed emotion adjustment, and carrying out interaction according to the dialogue strategy, so as to obtain the emotion recognition result of the multi-layer perceptron after the feature after the interaction fusion is subjected to average pooling processing and then is input into the multi-layer perceptron to carry out emotion classification, and obtaining a classification corresponding to the highest probability as the emotion label recognition result. Dynamically adjusting a dialogue strategy according to the change of the emotion label; the method is suitable for enhancing the emotion understanding and response ability of the intelligent dialogue system and improving the user experience.
Owner:SHANDONG INSPUR SCI RES INST CO LTD

Emotion recognition method and system based on electroencephalogram eye movement multi-mode cross-attention feature fusion

The invention provides an emotion recognition method based on electroencephalogram eye movement multi-mode cross-attention feature fusion, and the method comprises the steps: firstly carrying out the preprocessing of an emotion recognition public data set, and building a corresponding training set and a test set; secondly, constructing an electroencephalogram eye movement multi-mode cross-attention feature fusion emotion recognition model; then carrying out model training and performance testing; finally, an electroencephalogram eye movement multi-mode online emotion recognition system is built, and the effectiveness of the method is verified. The method has the advantages that an electroencephalogram eye movement multi-mode cross-attention feature fusion emotion recognition model is designed, a dynamic graph convolutional network and an attention mechanism are flexibly applied, meanwhile, multi-dimensional data are used, the problem that single-mode emotion recognition information is insufficient is solved, and the recognition accuracy is improved; and an electroencephalogram eye movement multi-mode online emotion recognition system is built, so that the interactivity is enhanced. The average recognition accuracy of the method reaches 97.94% and is superior to that of an existing optimal method, and meanwhile the accuracy of online emotion recognition reaches 87.4%.
Owner:BEIHANG UNIV

Multi-mode interaction method of accompanying robot

The invention discloses a multi-mode interaction method for an accompanying robot, and relates to the technical field of robot interaction.The multi-mode interaction method comprises the steps that voice, visual and tactile signals are converted into quantum states through multi-mode quantum state coding, emotion weights are dynamically fused, and unified quantum representation is constructed; the dynamic quantum decision engine analyzes environmental noise and user emotion intensity in real time based on a quantum measurement theory, generates an adaptive strategy through dynamic modal weight distribution, and solves a multi-modal instruction conflict; holographic reinforcement learning optimization is combined with quantum acceleration calculation and classical reinforcement learning, the reward function weight is dynamically adjusted, cross-modal knowledge migration is achieved through the quantum tunneling effect, and the system strategy is continuously optimized. The problems that a traditional multi-mode interaction system is rigid in mode switching, high in emotion recognition error rate, lack of self-adaptive optimization of a feedback strategy and the like are solved, and the interaction efficiency and the user experience are improved.
Owner:WIRELESS TAG TECH CO LTD

Audio and video dual-mode emotion recognition method and system based on adapter fusion

The invention relates to the technical field of artificial intelligence and emotion calculation, in particular to an audio and video dual-mode emotion recognition method and system based on adapter fusion. The method comprises the following steps: acquiring a video frame sequence and an audio signal, and preprocessing the video frame sequence and the audio signal; constructing an emotion recognition model; based on a bimodal feature extraction module, a space adapter and a global adapter are embedded in sequence, and corresponding modal enhanced space features and global features are obtained in sequence; generating intermediate representations of the corresponding modes based on the global features, and performing feature fusion according to the intermediate representations to obtain fusion features of the corresponding modes; the fusion features are spliced, time sequence features are extracted, and final features are obtained; inputting the final features into a classifier to obtain a predicted emotion category, training an emotion recognition model by adopting a loss function, and determining an optimal emotion recognition model; and inputting a to-be-recognized video frame sequence and an audio signal into the emotion recognition model, and outputting a recognition result.
Owner:NANJING MEDICAL UNIV

Psychological counseling strategy generation method and device based on intelligent agent, equipment and medium

The invention discloses a psychological counseling strategy generation method and device based on an intelligent agent, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: collecting audio and video signals of a target user who is performing psychological counseling; performing face emotion recognition and voice recognition on the audio and video signals by using a computer vision technology and a voice recognition technology to obtain multi-mode signals; inputting the multi-modal signal into a decision-making large model created based on a decision-making tree and a multi-modal large model to generate a psychological counseling strategy; obtaining a user portrait model of the target user generated based on the historical consultation record of the target user; and adjusting the psychological consultation strategy according to the user portrait model and the professional psychological knowledge to obtain a personalized consultation strategy for the target user. According to the method, the emotional state and the personal demand of the user are deeply understood, the personalized psychological counseling strategy generation demand of the user is met, and the psychological counseling strategy generation accuracy is improved.
Owner:ATHENAEYES CO LTD

Relieving lamp intelligent control method and system based on voice interaction

The invention discloses a soothing lamp intelligent control method and system based on voice interaction, and relates to the technical field of data encryption, and the method comprises the steps: receiving a voice command inputted by a user through a voice interaction device, awakening a control system, and starting a voice emotion recognition unit; the voice emotion recognition unit is used for performing emotion analysis on the voice command, extracting emotion characteristics and judging real-time emotion of the user; on the basis of continuous voice input of a user, voice signals are collected regularly, the emotional state is updated dynamically, and light setting is optimized in combination with environmental data; by recording historical voice interaction data and environment data of a user, an emotion mode and light preference of the user are learned, and light setting is continuously optimized. Light setting is continuously optimized through a self-learning mechanism, it is ensured that the system can conduct self-adaptive adjustment along with changes of user requirements, and light control is more intelligent and personalized.
Owner:SHENZHENSHI KAIXIN GUANGDIAN CO LTD

Personalized English education system and method based on multi-modal sentiment analysis

The invention provides a personalized English education system and method based on multi-modal sentiment analysis. The system comprises a multi-modal interaction module for receiving and processing multi-modal data of students, an emotion recognition module for performing emotion analysis on the input multi-modal data, and a personalized module for evaluating real-time learning states of the students based on historical learning data of the students, and the core brain module is used for dynamically adjusting interactive feedback according to the emotional state and the real-time learning state. The emotion recognition module comprises emotion information fusion, the emotion information fusion adopts a weighting strategy, and the final output emotion state is adjusted through emotion consistency constraint and a conflict correction mechanism. According to the invention, through an emotion consistency loss function, a conflict correction mechanism and a knowledge graph-based super-outline control mechanism, the emotion and cognitive states of the students are accurately identified.
Owner:XIAMEN UNIV

Customer voice analysis system based on large model

The invention discloses a customer voice analysis system based on a large model, and the system comprises a voice data collection and preprocessing module which is used for collecting and preprocessing customer voice data; the voice recognition module is used for recognizing the preprocessed customer voice data; the feature fusion module is used for extracting semantic features of the text data and fusing customer voice data features to generate voice text representation; the large model fine tuning module is used for performing fine tuning on the pre-trained starry fire cognition large model; the semantic analysis and emotion recognition module is used for processing the voice text representation, extracting semantic information, recognizing an emotion state and generating an analysis result; and the dialogue context management module is used for associating customer historical voice data, establishing dialogue context information and updating a voice analysis result. According to the method, the stellate fire cognitive large model and the improved quantitative low-rank adaptation are utilized, and the method has the advantages of high precision, low calculation cost and high intelligent adaptability.
Owner:HEFEI UNIV OF TECH

Dynamic self-adaptive multi-modal sentiment analysis fusion method and system

The invention provides a dynamic self-adaptive multi-modal sentiment analysis fusion method and system, and relates to the technical field of multi-modal sentiment analysis. The method comprises the following steps: synchronously acquiring voice, text, facial expression and limb movement data of a target user to form a multi-modal data set; the method comprises the following steps: firstly, extracting emotional characteristics of each mode, and constructing a cross-mode correlation model to capture a collaborative and complementary relationship among different modes; and calculating a real-time confidence score and a complementarity index of each modal based on the weight matrix of the cross-modal correlation model. Then, according to the scores and the indexes, a weighted average or maximum entropy algorithm is dynamically selected to fuse multi-modal emotion features, and a comprehensive emotion feature vector is generated; and finally, inputting the vector into a pre-training deep learning model, and outputting an emotional state category of the user. According to the method, the user emotion can be accurately and comprehensively captured, efficient emotion recognition and classification are realized, and the robustness and flexibility of an emotion analysis system in a complex scene are improved.
Owner:HUNAN OPEN UNIV (HUNAN PROVINCIAL CADRE EDUCATION & TRAINING ONLINE COLLEGE)

AI-assisted limb rehabilitation system

The invention discloses an AI auxiliary limb rehabilitation system, and belongs to the field of communication control systems, the AI auxiliary limb rehabilitation system combines a sensing module, an AI analysis module, a VR interaction module, a feedback module and the like, comprehensively collects limb movement and physiological data, introduces gamification elements, improves the patient participation degree through target setting, an award mechanism and interesting tasks, and improves the rehabilitation effect. A personalized rehabilitation plan is generated by using a deep learning algorithm, an emotion recognition unit can adjust the plan according to a psychological state, environment self-adaption and real-time adjustment of training difficulty to ensure training safety and is challenging, data cloud storage facilitates remote monitoring, third-party equipment is compatible with extended monitoring ability, and the training efficiency is improved. A brand-new, efficient and personalized solution system scheme is provided for limb rehabilitation.
Owner:PEKING UNIV THIRD HOSPITAL QINHUANGDAO HOSPITAL

Touch and talk pen intelligent children education method based on emotion recognition

The invention discloses a touch and talk pen intelligent children education method based on emotion recognition. The method comprises the steps that S1, facial expression data, voice and intonation data and touch behavior data of children are collected; s2, multi-modal high-quality emotion data matched with the actual emotion state is generated; s3, generating deep emotional features related to facial expressions, voices and tones and touch behaviors; s4, generating an emotional state label corresponding to the current emotional state of the child; s5, the generation model generates voice feedback content, image animation content and interactive game content which are adaptive to the current emotional state; s6, generating a personalized recommendation content list conforming to the emotional state of the child; and S7, displaying the education content in the personalized recommendation content list through a display screen, a voice output unit and a touch interaction unit of the touch and talk pen. According to the method, the fuzzy classification problem of the emotional state is solved in a touch and talk pen emotion recognition scene, and the adaptability of the model to the complex emotional state is remarkably improved.
Owner:FOSHAN CHUANGZHI XINGKONG CLOUD INFORMATION TECH CO LTD

Feature attention and bilinear gating fused speech emotion recognition method and device

The invention discloses a feature attention and bilinear gating fused speech emotion recognition method and device, and the method comprises the following steps: 1, collecting an audio file, obtaining corresponding label information, generating audio waveform and time frequency representation data through preprocessing, and constructing an audio waveform mask and a time frequency mask to mark an effective information region; 2, constructing a dual-path feature extraction module which comprises a time-frequency feature extraction module and a pre-training acoustic feature coding module; wherein the time-frequency feature extraction module models emotion correlation through local convolution and a multi-dimensional attention mechanism, and performs global time sequence modeling based on a bidirectional gated loop network; the pre-training acoustic feature coding module extracts high-level speech representation with high expression ability for emotion distinguishing by using a pre-training model; and step 3, constructing a feature fusion module and an emotion classification module, and combining with a dual-path feature extraction module to form a speech emotion recognition model.
Owner:SICHUAN UNIV

Emotion recognition method based on EEG space-time frequency feature fusion

The invention discloses an emotion recognition method based on EEG space-time frequency feature fusion, which comprises the following steps: S1, carrying out a series of preprocessing steps on an original EEG signal, including baseline correction, standardization and time window addition; s2, performing form conversion on the electroencephalogram signals processed in the S1 to obtain spatial information and time-frequency information of the electroencephalogram signals; s3, multi-aspect feature extraction is carried out; s4, fusing the extracted emotion features, and performing multi-feature emotion recognition; s5, further extracting time features of the electroencephalogram signals by using BILSTM added with a channel attention mechanism, and performing deep fusion of the features; and S6, obtaining an emotion classification result. According to the emotion recognition method based on EEG space-time-frequency feature fusion, feature information of the time dimension, the frequency dimension and the space dimension of the EEG signals is fully considered, the highest recognition accuracy is achieved on the awakening dimension and the titer dimension, the emotion recognition accuracy and robustness are improved, and the effectiveness of the method is verified.
Owner:NORTHEAST DIANLI UNIVERSITY

AI interaction method and system based on emotion recognition

The invention belongs to the technical field of AI interaction, and discloses an AI interaction method and system based on emotion recognition. The method comprises the following steps that in a cloud data center, an emotion relieving database is built, and an emotion AI interaction engine is built; according to the real-time multi-mode monitoring data, using an emotion AI interaction engine to generate a real-time emotion recognition result and real-time AI interaction data; according to the real-time emotion recognition result, a real-time AI response strategy is generated, and real-time emotion relieving data is extracted; generating real-time AI response data according to the real-time AI response strategy, the real-time AI interaction data and the real-time emotion relieving data; and optimizing the emotion AI interaction engine according to the real-time feedback data to obtain an optimized emotion AI interaction engine. According to the method, the problems of low emotion recognition accuracy, low AI intelligence degree and lack of an evolution function in the prior art are solved.
Owner:ZHENJIANG BERET NETWORK TECHNOLOGY CO LTD

Information processing system and information processing method

To easily create a diary. [Solution] The information processing system is characterized by comprising a conversation data acquisition unit that acquires conversation data between a user, an emotion identification unit that analyzes the conversation data to identify the user's emotions, and a diary generation unit that generates diary data for the user based on the conversation data and the emotions.
Owner:IMBESIDEYOU INC