167 results about "Emotion detection" patented technology

The invention discloses an emotion detection system based on facial recognition, and relates to the technical field of computer vision and emotion calculation. A video stream time sequence analysis module is used for extracting a facial micro-expression image sequence of continuous frames, a time sequence feature vector containing a micro-expression intensity gradient, an illumination robustness coefficient and a facial action unit cooperation feature is generated, and a multi-mode dynamic sensing module is combined to carry out real-time analysis on an emotion classification probability, voice emotion parameters and physiological signals. And the fusion decision module performs dynamic weighted fusion on the multi-modal data based on the scene adaptive weight, and finally generates a comprehensive emotion score. Through multi-modal time sequence modeling and a dynamic weight optimization mechanism, the accuracy and environmental adaptability of emotion recognition are remarkably improved, and real-time perception and accurate decision making of customer emotion are realized in a target scene.

A method of providing emotive text-to-speech includes obtaining input text characterizing a natural language response generated by an assistant LLM to a query input by a user during a conversation between the user and the assistant LLM, and processing, using the assistant LLM, the input text conditioned on an emotion detection task prompt to predict, as output from the assistant LLM, an emotional state of the natural language response. The method also includes determining, based on the emotional state of the natural language response predicted as output from the assistant LLM, an emotional embedding for the input text and instructing a TTS model to process the input text and the emotional embedding to generate a synthesized speech representation of the natural language response conveying the emotional state of the natural language response as specified by the emotional embedding.

The present invention proposes an AI dynamic emotion recognition multi-dimensional targeted training system, which relates to the fields of computer vision technology and artificial intelligence technology. It includes multiple emotion collection and training terminals at the front end and a background server. The emotion collection and training terminals are used to detect the emotions of the tested personnel. The terminals analyze the face images to obtain the multi-dimensional emotion data of the tested personnel. The multi-dimensional emotion data is uploaded to the background server. The background server analyzes the current emotion state of the tested personnel according to the received multi-dimensional emotion data, and pushes different targeted training tasks to the corresponding emotion collection and training terminals for the tested personnel in different states. After receiving the targeted training tasks, the emotion collection and training terminals, the tested personnel complete the corresponding training content according to the guidance. The present invention can specifically improve the psychological quality of the tested personnel or improve the bad emotion state.

An AI-driven emotion recognition system for learning adaptation in educational institutions, comprehensive: • an emotion recognition unit that tracks students' facial expressions in real time and categorizes the resulting expressions into a specific emotional class from a given set using a Convolutional Neural Network (CNN) trained on a large emotion dataset; • a real-time processing unit that extracts facial features and dynamic emotional characteristics of students through adaptive filter algorithms; • an LMS integration unit, adapted to combine discovered emotional knowledge with online and offline learning environments for customizable learning adjustments; • a personalization engine programmed to intelligently adapt training content, response approaches, and training materials according to the identified emotional states, so that each learner receives individually tailored and more intensive learning experiences; • a feedback loop that sends educators real-time alerts and recommended videos to address emotional feedback or lesson gaps, allowing teachers to tackle negative signals in the moment (e.g., frustrated face, etc.); • AI data storage and adaptive learning model unit that stores anonymized data for emotion recognition, updates the AI ​​model and improves it through learning over time, and ensures that the use of the information is in compliance with data protection regulations; • A user interface module specifically designed for educators, providing a visual overview of students' emotional patterns and real-time analysis of their emotions and engagement as a basis for important decisions that lead to improvements in teaching processes.

The invention provides a complex emotion representing and sensing method based on an emotion vector, and the method employs the emotion vector to represent a complex emotion containing two or more emotions, the emotion vector is composed of a plurality of element emotion elements corresponding to human emotions, the value of each element represents the intensity of the element emotion, and the value of each element represents the intensity of the element emotion for the sensing of the emotion vector. According to the method, the approximation degree of the perceived emotion and the actual emotion is improved through a meta-emotion detection algorithm (ISL-MED) based on iterative self-learning. According to the invention, the confidence score of the element emotion examination is utilized to approximately represent the intensity of the element emotion, a plurality of element emotion detectors are designed to detect the complex emotion in the voice, and the detection results are combined into an emotion vector to represent the perceived emotion result. Evaluation of the emotion perception result is based on the distance or loss measurement between the labeled emotion vector and the perception emotion vector, and the actual voice emotion can be described more accurately.

The invention relates to the technical field of artificial intelligence, and particularly provides a multi-modal information fusion emotion detection method based on visible light and voiceprint. The method comprises the following steps: respectively acquiring video images and environment sounds of old people in a monitoring environment through a visible light image module and an audio voiceprint acquisition module; a facial expression detection module is used for recognizing a human face in the video image, and a visual anomaly signal is output; recognizing an audio voiceprint signal in the environmental sound according to a voiceprint detection module, and outputting an audio abnormal signal; according to the method, the detection accuracy is effectively improved, the missing report is reduced, the method is suitable for home and old-age care institution environments, and the method is of great significance to guarantee the safety of old people and alleviate serious consequences caused by falling down.

The invention discloses a high-precision facial expression emotion detection method based on improved YOLOv8n, which optimizes a facial expression recognition task while retaining a basic framework of the YOLOv8n, and comprises the following steps of: firstly, replacing a standard convolution module in a backbone network and a neck structure of the YOLOv8n with a self-adaptive multi-branch convolution module; a large-scale separable nuclear attention module is integrated to the seventh layer and the tenth layer of a YOLOv8n backbone network, dynamic attention on key facial features is enhanced, interlayer feature injection connection is applied to the YOLOv8n backbone network and a neck structure, cross-layer connection between the sixth layer and the seventeenth layer and cross-layer connection between the ninth layer and the nineteenth layer are established, and therefore the dynamic attention on the key facial features is enhanced. By establishing the direct connection between the shallow and deep features, the information loss is reduced, the multi-scale feature fusion is optimized, and the precision and efficiency of facial expression emotion detection are significantly improved.

The invention discloses a wearable emotion monitoring and evaluating system based on brain-heart-enteric nervous system electric signals, and belongs to the field of bio-electricity signal processing and recognition. According to the system, electroencephalogram (EEG) and electrocardio (ECG) are integrated, and an EGG signal closely related to nervous system activity is creatively integrated, so that the system can provide comprehensive emotion evaluation. The signals respectively represent different emotional response mechanisms, the emotion change can be more accurately captured through the fusion of the multi-modal signals, and the accuracy and reliability of emotion detection are improved. The system can collect and analyze physiological signals of a user in real time, provide instant emotion feedback, help the user to recognize emotion fluctuation in an early stage and take corresponding measures, and avoid emotion out-of-control or long-term emotion lowering.

The invention provides an emotion recognition method and device. The method comprises the following steps: acquiring face video data and environment data; extracting image data, voice data and text data according to the collected face video data; extracting image features and physiological signal features from the extracted image data; extracting audio features from the extracted voice data; extracting text features from the extracted text data; environment features are extracted from the collected environment data; splicing and fusing the extracted image features, physiological signal features, audio features, text features and environment features by using an attention mechanism to generate multi-modal fusion features; and inputting the generated multi-modal fusion features into a pre-trained emotion detection model for emotion recognition, and outputting emotion contents to which the multi-modal fusion features belong. The emotion recognition method can solve the problem that an existing emotion recognition method only uses a single-mode picture and ignores other mode data.

The invention relates to a medical emotion support large model and a generation method thereof, and belongs to the field of medical data processing. Comprising an emotion detection module which is used for recognizing the current negative emotion of a patient through dialogue history and inputting a detection result into a response generation module; the short-term memory storage module is used for storing the dialogue between the patient and the response generation module as a dialogue history record, compiling the dialogue history exceeding the upper limit into background information through a large language model, and inputting the background information into the response generation module; the dialogue strategy guidance module is used for judging a dialogue communication strategy which should be used by the response generation module according to the dialogue historical record; the response generation module is used for setting personal background information and current confronted conditions and generating proper response; and the correction module is used for performing self-reflection according to the feedback of the patient to the response of the medical emotion support framework, and updating the recommendation experience of the dialogue strategy guidance module. According to the method, the medical emotional dialogue data set can be generated through the constructed framework.

The invention relates to the field of medical signal processing and the field of artificial intelligence, in particular to an electroencephalogram signal emotion detection method based on multi-stage feature fusion, which comprises the following steps of: constructing a first electroencephalogram signal feature extraction network by taking a time sequence electroencephalogram signal as input to extract frequency domain features; a second electroencephalogram signal feature extraction network is constructed with differential entropy features as input, the network takes EEGNet as a trunk network, and space attention mechanism extraction space and time domain feature extraction space, channel and time domain features are introduced. And constructing a third electroencephalogram signal feature extraction network by taking the first statistical feature as input to extract a second statistical feature, fusing the five features to obtain a fused feature, and performing recognition by using an electroencephalogram signal recognition network to obtain an emotion detection result and output the emotion detection result. According to the method, the problem that information such as space, time domain, frequency domain and channel in the electroencephalogram signals cannot be fully utilized in existing electroencephalogram signal emotion detection is solved.

The invention relates to the technical field of intelligent cabins, and discloses a vehicle-mounted interaction system which comprises the steps that when a driver monitoring system recognizes a negative expression of a driver based on a collected first video stream, an emotion recognition request is sent to an in-vehicle passenger detection system, and the first video stream is sent to an emotion interaction system; when the in-vehicle passenger monitoring system receives an emotion recognition request and recognizes a negative body language of the in-vehicle passenger based on a collected second video stream, an emotion detection request is sent to a voice system, and the second video stream is sent to an emotion interaction system; when the voice system receives the emotion detection request and recognizes a negative language based on the collected voice signal, the voice system sends a voice signal to the emotion interaction system; the emotion interaction system determines the current emotion of the passenger in the vehicle based on the first video stream, the second video stream and the voice signal, and adjusts the interaction information in the vehicle based on the current emotion, so that the possibility of misreporting negative emotion is reduced, and the intelligent ability of the vehicle-mounted interaction system is improved.

The present invention discloses a control method and system based on child emotion recognition, the method comprising: detecting the pressure of the rear seat to obtain the rear seat pressure value; judging whether the rear seat pressure value reaches a preset pressure threshold; if so, detecting the temperature of the target object on the rear seat to judge whether the target object is alive; if so, determining that there is a child in the rear seat, triggering the central lock to lock and the window opening to rise to the preset opening value; determining the current emotional state of the child according to the sound characteristic value and temperature value of the child in the rear seat; and regulating the child's emotion through the intelligent cockpit system according to the current emotional state of the child. The control method and system based on child emotion recognition of the present invention can realize child emotion recognition and relief, does not require the additional arrangement of rear row cameras, has low cost, and does not involve user privacy; and does not require hardware upgrades, only the existing system in the car needs to be used to realize low-cost child emotion detection and care.

The invention relates to the technical field of emotion analysis, in particular to an emotion detection model training method and system based on Transform architecture and a storage medium. The emotion detection model training method based on the Transform architecture provided by the invention has an excellent self-attention mechanism, and can process complex language features and emotion expressions. According to the method, at least two data enhancement operations are defined, independent application probability and adjustment parameters are set for each operation, and the enhancement operations are dynamically selected and applied in the training process, so that the diversity of training data can be increased in a more targeted manner, and the model is more accurate and better in robustness when facing different emotion expression modes. By periodically calculating emotion prediction results and confidence difference values on a verification set and adjusting initial hyper-parameters and updating an enhanced data set according to the difference values, the mechanism enables the model to be adaptively optimized and continuously improved, and high efficiency of the training process is ensured.

Techniques for image scaling based on emotion detection are described. In some embodiments, the techniques include acquiring one or more images of a user, processing the one more images to generate emotion-specific three-dimensional (3D) positions of ears of the user based on a 3D head geometry and an emotion of the user, where the emotion is identified based on the one or more images of the user, and processing one or more audio signals to generate one or more processed audio signals based on the three-dimensional positions of the ears. Further embodiments include systems and non-transitory computer-readable media that perform the steps of the method.

The invention discloses a driver road rage emotion detection method based on machine learning. The method comprises the steps that a, a driver facial expression image is collected through an in-vehicle camera; b, analyzing a voice signal of the driver through a voice recognition system; c, physiological signals such as the heart rate and the breathing rate of the driver are collected through a sensor; d, collecting traffic condition information around the vehicle by using an environment sensing module; e, performing feature extraction on the data obtained in the steps a to d; f, inputting the features extracted in the step e into a pre-trained multi-modal fusion model to identify whether the driver is in a road rage emotional state; g, if the result of the step f indicates that the driver is in the road rage emotion state, corresponding intervention measures are triggered, the multi-modal data fusion and feature extraction technology is used, subtle changes in facial expressions and voice can be recognized more accurately, and the accuracy and reliability of road rage emotion detection are improved.

The invention provides a family-centered child development platform designed to support and enhance a child's academic, emotional, social, and cultural growth. By utilizing an AI-driven system for personalized learning, immersive VR / AR technologies for engaging educational experiences, and a community-building module for peer interaction, it offers an integrated solution for holistic child development. The system incorporates emotion detection, biometric feedback, and emotion-driven content adaptation, which dynamically adjusts the child's learning experience based on real-time emotional and physiological data. Additionally, the platform features culturally relevant content, promotes intergenerational collaboration, and includes a gamified learning environment to foster motivation and engagement. A secure interaction system ensures safe online communication, while the adaptive challenge system maintains an optimal level of difficulty tailored to the child's evolving needs. By combining these technologies, the platform provides a personalized, interactive, and supportive learning journey for children, supported by parents and educators.

Disclosed in the present invention are an emotional haptic regulation system and method based on multi-modal fusion. The system comprises a haptic optimal parameter adjustment module, a haptic generation module, a visual-auditory generation module, a multi-physiological signal collection module, a multi-sensory signal collection module and a multi-modal fusion emotion recognition module. The system collects multiple physiological signals of a user, fuses features of the multiple physiological signals with audio and haptic modality features, accurately recognizes the current emotional state of the user in real time by using advanced data processing and analysis techniques, automatically searches for haptic parameters by means of an optimization theory, and actively regulates the emotional state by applying haptic stimuli to the user; and the limitations of conventional methods such as subjective scales are overcome, the impact of unstable physiological signals on emotion recognition results is effectively reduced, and the accuracy of emotion detection in the emotional haptic regulation system is significantly improved. The emotional haptic regulation system has broad application potential in the fields of personal emotion management, medical rehabilitation, audio-video entertainment, etc.

The invention is suitable for the technical field of emotion detection, and provides an anxiety risk detection system based on multi-scale time sequence feature fusion, and the system comprises a short-term feature obtaining module which is used for obtaining short-term branch features of multi-modal data of a user; the short-term branch features are used for describing psychological state changes of the user in a recent short time period; the long-term feature acquisition module is used for acquiring long-term branch features of the multi-modal data; the long-term branch features are used for describing a stable psychological trend of the user in a long time period; the attention fusion module is used for fusing the short-term branch features and the long-term branch features based on an attention mechanism to obtain a fusion result; and the detection module is used for processing the fusion result to obtain an anxiety risk detection result of the user. The accuracy and interpretability of anxiety risk detection can be improved.

The embodiment of the invention provides an emotion detection method based on a multi-mode electroencephalogram eye movement signal. According to the method, a dual-path electroencephalogram modal encoder is used for determining time sequence characteristics of an electroencephalogram in a time domain path, determining frequency domain characteristics of an amplitude spectrum of the electroencephalogram in a frequency domain path, and obtaining dual-path time domain and frequency domain fused electroencephalogram characteristics based on the time sequence characteristics and the frequency domain characteristics; the dual-path eye movement modal encoder is used for determining pupil features of the eye movement data in the pupil diameter path, determining eye movement fixation features of the eye movement data in the fixation point path, and obtaining fused eye movement features based on the pupil features and the eye movement fixation features; and the multi-modal expert mixing module is used for dynamically performing modal calling on the multi-modal mixed characteristics which are subjected to modal alignment and are fused with the electroencephalogram characteristics and the eye movement characteristics, performing emotion detection by utilizing an expert with a modal corresponding to the multi-modal mixed characteristics, and outputting an emotion classification result. According to the embodiment of the invention, alignment is realized through multi-task pre-training, and the depressive emotion is accurately recognized.

The embodiment of the invention provides a speech synthesis method, a speech synthesis device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the steps of performing mask processing on an original Mel spectrum of a sample based on sample emotion information to obtain a masked Mel spectrum and a masked Mel spectrum of the sample; detecting the sample emotion information and the original Mel spectrum of the sample based on an emotion detection sub-model to obtain a sample target emotion feature; performing frequency spectrum generation on the sample masked Mel frequency spectrum, the sample target emotion features and the sample text based on a frequency spectrum generation sub-model to obtain a predicted Mel frequency spectrum; and adjusting the initial synthesis model based on the mask Mel spectrum, the predicted Mel spectrum and the sample target emotion feature to obtain a speech synthesis model, so as to perform speech synthesis on the target text, the target object speech with the target speech expression feature and the target emotion information with the target non-language emotion feature. According to the embodiment of the invention, synthetic speech with more accurate emotion expression can be generated.

The invention provides an infant emotion detection method based on an improved Transform, and relates to the field of emotion recognition, and the method comprises the steps: constructing a data set; an improved Transform model is established, the improved Transform model comprises an improved encoder and an improved decoder, the improved encoder at least comprises a cross convolution layer, and the improved decoder at least comprises an encoder-decoder attention mechanism; training and optimizing and improving a Transform model through a data set; acquiring a to-be-detected infant face image; through the optimized improved Transform model, the emotion of the infant is recognized based on the face image of the infant to be detected, and the method has the advantage of more accurate and reliable infant emotion recognition.

The invention provides an in-vehicle emotion detection method and device and electronic equipment. The in-vehicle emotion detection method comprises the steps of obtaining visual data, voice data and physiological data; splicing and fusing the visual data, the voice data and the physiological data after the redundant data is removed to obtain long connection fusion data; extracting a first emotional feature in the long connection fusion data, wherein the first emotional feature comprises a first visual feature, a first voice feature and a first physiological feature; obtaining weights respectively corresponding to the first visual feature, the first voice feature and the first physiological feature; processing the first emotional feature through weight to obtain a second emotional feature; and outputting an emotion detection result for the second emotion feature. According to the embodiment of the invention, through fusion and weight adjustment of the multi-modal data, the adaptability and robustness in a complex driving environment are enhanced, so that the driving safety and comfort are effectively improved.

The invention relates to an emotion detection method combining metaphor recognition and a syntactic structure, and belongs to the technical field of natural language processing and emotion analysis. The implementation method comprises the following steps: 1, tagging part-of-speech coarseness of a to-be-analyzed text by utilizing a natural language processing library; extracting and identifying a labeled result to obtain an emotion keyword; 2, performing fine-grained marking on the emotion keywords, and performing serial number marking on positions of the emotion keywords; 3, obtaining a feature vector, a segment vector and a mask vector; 4, screening to obtain a target word of the metaphor; 5, constructing a node relationship of the dependency tree; meanwhile, sorting and splicing are carried out, and then clauses are obtained; 6, performing emotion classification evaluation; the evaluation is used in emotional natural language text data processing; compared with the prior art, the method has the advantages that through cooperative work of the metaphor recognition model and the emotion detection model, the recognition precision of the expression emotion of the metaphor is improved, introduction of irrelevant information is effectively reduced, and high efficiency of emotion analysis is ensured.

The present invention discloses a customer service intelligent quality inspection method, device and storage medium. The method comprises: obtaining customer service voice data; inputting the customer service voice data into a voice keyword detection model to obtain a voice keyword detection result, including: S1, obtaining a first acoustic feature of the customer service voice data; S2, performing a first two-dimensional convolution, maximum pooling, and a first activation function processing on the first acoustic feature; S3, inputting the processing result into a multi-head attention mechanism for keyword capture, and processing the keyword to obtain a voice keyword detection result; inputting the customer service voice data into a voice emotion detection model to obtain a voice emotion detection result, including: A, constructing a three-channel feature map; B, performing a second two-dimensional convolution and batch normalization processing on the feature map; C, obtaining a voice emotion detection result. Performing voice keyword detection and voice emotion detection simultaneously realizes multi-task parallel detection, and improves the quality inspection accuracy by combining the detection results of the two.

There is provided a method for training text-based emotion detection language model. The method comprises receiving text data; generating clean data by removing errors included in the received text data; detecting emotion data related to a specific emotion from the clean data and generating a training dataset including the detected emotion data; and training the language model using the training dataset.

Apparatuses, systems, and methods for low-latency audio-to-face animation with emotion detection are disclosed herein. The system may receive a first audio stream associated with a first device and a second audio stream associated with a second device, and provide, concurrently, a first segment of the first audio stream and a second segment of the second audio stream as inputs to an emotion detection artificial intelligence (AI) model to obtain first emotion data and second emotion data. The system may then provide, concurrently, a third segment of the first audio stream with the first emotion data and a fourth segment of the second audio stream with the second emotion data as inputs to a face animation AI model to obtain first face pose data and second face pose data, and provide the first face pose data to the first device and the second face pose data to the second device.

The invention relates to the technical field of artificial intelligence, in particular to an edge end real-time voice emotion detection method and system and a storage medium, and the method comprises the steps: obtaining real-time voice data of a user end; performing feature extraction on the voice data by adopting a preset integrated code module to obtain a multi-dimensional acoustic feature vector; inputting the multi-dimensional acoustic feature vector into an emotion recognition model embedded in the preset integrated code, and detecting the emotion of a user to which the user side belongs to obtain a real-time emotion state; and based on the real-time emotional state, executing an interaction operation matched with the user side. According to the method, the voice data is locally processed and does not need to be uploaded to the cloud server, so that the risk of user privacy disclosure is greatly reduced, the computing and bandwidth resource requirements on the cloud server are reduced, and the operation cost of a service provider is reduced.

An object of a system according to an embodiment is to detect an emotion of a user and provide appropriate advice on the basis of the detected emotion.SOLUTION: A system according to an embodiment includes a smart emotion detection speaker, an emotion detection AI, and a mentoring generation AI. The smart speaker detects emotions from the user's voice and speech. The emotion detection AI generates words based on emotions. The mentoring generation AI listens to the user's distress and provides advice.SELECTED DRAWING: Figure 1

Techniques for image scaling based on emotion detection are described. In some embodiments, the techniques include acquiring one or more images of a user, processing the one more images to generate emotion-specific three-dimensional (3D) positions of ears of the user based on a 3D head geometry and an emotion of the user, where the emotion is identified based on the one or more images of the user, and processing one or more audio signals to generate one or more processed audio signals based on the three-dimensional positions of the ears. Further embodiments include systems and non-transitory computer-readable media that perform the steps of the method.