103 results about "Affective computing" patented technology

The invention relates to the technical field of artificial intelligence and emotion calculation, and discloses an emotion state evaluation method based on multi-modal data, and the method comprises the steps: obtaining and decoupling an implicit signal, an explicit signal and situation data of an evaluated object; and respectively generating implicit features, explicit features and context vectors through double-flow decoupling coding and context knowledge graph coding. And the core engine determines the internal emotional state and the extrinsic emotional representation in parallel, and dynamically regulates and controls a bridging conversion process from the internal state to the predicted extrinsic representation by utilizing the situation vector. Finally, the emotion regulation intensity is calculated by comparing actual and predicted extrinsic representations, and the emotion regulation intensity, the internal state and the extrinsic representations are jointly used as a multi-dimensional evaluation result to be output. According to the method, the dynamic relationship and the adjustment strategy between the internal feeling and the external expression can be disclosed, a deeper and more three-dimensional analysis view angle is provided for the fields of mental health, human-computer interaction and the like, and the method has remarkable application value.

The invention relates to the technical field of computer vision and emotion calculation, in particular to an emotion real-time recognition and long-term monitoring method based on facial expressions. According to the method, an emotion recognition result is obtained by recognizing a high-definition facial image, and the emotion recognition result, environment information and physiological state data are fused to obtain time-space aligned multi-modal data; performing emotional causal analysis based on the multi-modal data, and judging emotional causes by combining a rule engine and a machine learning model: outputting a real-time emotional state recognition result and a periodic emotional report according to the emotional causes, and performing differentiated feedback according to the emotional causes. According to the method, through multi-source data fusion and a causal inference mechanism, the accuracy and interpretability of emotion recognition are effectively improved, and the technical span from passive recognition to personalized active intervention is realized.

The invention discloses a multi-modal emotion calculation method and system, and mainly relates to the technical field of artificial intelligence and human-computer interaction. Comprising the following steps: synchronously collecting voice, visual and tactile multi-modal data of a user, and carrying out feature extraction on each modal data; performing cross-modal fusion on the extracted multi-modal features, including time sequence alignment and semantic association modeling, and generating a global emotion feature vector; performing dynamic emotion reasoning based on the global emotion feature vector, and outputting an emotion category and emotion intensity; generating a tactile feedback signal according to the emotion category and the emotion intensity, and driving an actuator to output; and updating the emotion memory graph based on user interaction data to complete personalized model self-adaption. The method has the beneficial effects that high-precision and low-delay emotion recognition and natural tactile feedback in a cross-culture scene are realized, and the core bottleneck of dynamic drift adaptation failure and emotion-behavior feedback decoupling in the prior art is solved.

The invention discloses a student psychological risk perception method based on multiple modes, and relates to the technical field of emotion calculation and intelligent education. The method comprises the following steps: firstly, extracting a facial expression feature vector and a voice intonation feature vector respectively by using a convolutional neural network and Fourier transform through a collected video stream and an audio stream; then adaptive denoising processing is carried out on environmental interference, timestamp alignment and dynamic time warping are carried out on the denoised multi-modal data, time sequence synchronization is ensured, and corrected multi-modal sequence data are formed; then, dynamic emotion track features are extracted from the sequence data, a preliminary emotion state label is generated by comparing the dynamic emotion track features with a baseline threshold value, and the threshold value is adaptively updated in combination with historical data so as to improve the judgment accuracy; and finally, aggregating the emotional state labels of a plurality of students to generate a visual group emotional thermodynamic diagram so as to realize macroscopic perception of group psychological risks. The accuracy, robustness and visualization degree of student psychological state analysis are effectively improved, and an efficient technical means is provided for campus psychological early warning.

The invention provides an emotion and cognition driven children education interaction system and method, and relates to the technical field of man-machine interaction. Emotion is quantified through an emotion calculation module by using a titer awakening degree two-dimensional continuous model, and an emotion migration slope is predicted in combination with a time sequence convolutional network; accurate description and prospective prediction of the emotion dynamics of children are realized, and the accuracy and initiative of emotion guidance are remarkably improved; secondly, the constructed knowledge graph endows the nodes with three-dimensional dynamic attributes of mastery degree, interest intensity and associated confusion degree, and real-time updating is carried out according to interaction data, so that the cognitive development trajectory of children is dynamically described, and a solid foundation is provided for personalized teaching; and finally, proposals are actively pushed by setting collaborative windows of interest, cognitive correction and the like, and a structured knowledge injection interface is provided, so that parents are converted from passive supervisors to common constructors of education contents, and deep fusion of family intelligence and artificial intelligence is realized.

The invention discloses a multi-user emotion recognition and digital human feedback method based on behavior data, and belongs to the field of artificial intelligence emotion calculation. Sensing user approaching, identifying and confirming the identity of a registered user, distributing an identifier, distributing an identifier for a visitor, and establishing an independent session channel in a multi-user scene; capturing time sequence dialogue behavior data containing voice rhythm features, dialogue interaction modes and linguistic features in real time; inputting the data into a sequence information processing model based on a state space theory and provided with a data dependence selection mechanism and an emotion analysis model spliced by static context features, and outputting multi-dimensional emotion and cognitive state tags; generating an emotion support strategy through a decision-making model in combination with the personal file of the user; and generating a multi-mode control instruction, and driving the digital human to synchronously present special effects such as expressions and the like. According to the method, privacy concerns are eliminated through non-intrusive collection, the method is adaptive to multi-user scenes, deep cognitive states can be recognized, and interaction effectiveness is improved for a long time.

The invention relates to the technical field of digital human voice interaction, and discloses a digital human live voice interaction system fused with emotion calculation. The system constructs an emotional response time window by acquiring a user voice input data stream, the starting point of the emotional response time window is the end moment of the user voice input data stream, and the end point is obtained by subtracting the necessary duration of voice response synthesis from the preset maximum response cut-off moment. The system obtains an emotional state vector of the user in real time and judges whether the emotional state vector reaches an emotional intensity threshold value or not; and if so, predicting the total generation duration of the digital human voice response. And when the residual duration of the emotional response time window is equal to the total generation duration, taking the window starting point as a voice response starting moment, and controlling the digital human to start voice response generation. According to the system, the voice response matched with the emotional state of the user is generated through refined time window management and emotional state perception, invalid response and interaction delay are reduced, and the naturalness of digital human live broadcast voice interaction and the user experience are improved.

The invention provides a customer service response system and method based on artificial intelligence. The customer service response system comprises a client, an intelligent access network, a core AI engine system, a multi-modal understanding system, a federated learning coordinator, a knowledge management system, an emotion calculation engine system, a dynamic decision system and a digital ethical arbitration system. According to the method, multi-dimensional data such as texts, voices and vision are fused through a multi-modal understanding system, more accurate user intention recognition is achieved in combination with a cross-modal entanglement network and intention conflict detection, and the misjudgment problem caused by traditional single-modal analysis is reduced; the emotion calculation engine is combined with physiological signals, voice emotion analysis and micro-expression recognition to dynamically sense the emotion state of the user, so that customer service interaction is more similar.

The invention provides an intelligent triage method and system for outpatient and emergency treatment, and relates to the technical field of medical information. The method comprises the steps that physiological data, symptom description text data, facial image data and voice audio data of a patient are collected through a multi-mode sensor; analyzing the image and audio data by using an emotion calculation engine to generate emotion state data; processing the physiological and text data through an analysis model to generate illness state emergency degree data; fusing emotion and illness state data to form a preliminary triage result; in combination with the real-time medical resource state data, a triage decision-making scheme including department allocation, processing priority and personalized pacifying strategies is generated through a dynamic decision-making device; finally, guiding and pacifying are executed through the execution terminal and the interaction robot, automation, intelligence and humanization of the triage process are achieved, the triage efficiency and accuracy are remarkably improved, medical resource configuration is optimized, and care for psychological needs of patients is enhanced.

The invention relates to the technical field of vocational ability evaluation, in particular to a student-based vocational ability evaluation system and method, and the system comprises a data collection module which is used for collecting multi-source data and evaluation data of students; the vocational ability modeling module is used for establishing a vocational ability index system based on a vocational ability model comprising profession, method and social ability dimensions; and the data processing and calculating module is used for preprocessing the multi-source data, calculating ability scores of professional and method dimensions according to the vocational ability index system, and calculating ability scores of social ability dimensions through natural language processing, emotion calculation or behavior log analysis. In the invention, the multi-source data and the evaluation data of the trainees are collected, and the vocational ability model including profession, method and social ability dimensions is established, so that the problem that the comprehensive vocational ability of the trainees is difficult to comprehensively reflect due to the fact that most traditional vocational ability evaluation adopts questionnaires or single evaluation is solved.

The invention relates to the technical field of brain-computer interfaces and emotion calculation, in particular to an electroencephalogram emotion recognition method based on adaptive multi-stream graph fusion and gated space-time Transform, which comprises the following steps: constructing and training an electroencephalogram emotion recognition model, and inputting to-be-detected signal data into the trained electroencephalogram emotion recognition model to obtain a detection result; the electroencephalogram emotion recognition model comprises a self-adaptive multi-stream graph fusion network, a gating enhanced time sequence block and an emotion classification head; the average accuracy and the F1 score of the method are superior to those of the existing mainstream model, and the superiority of the method in the aspect of improving the personalized emotion recognition performance is proved.

The invention discloses a multi-modal sentiment analysis system and method based on dynamic routing and feature decoupling, and relates to the field of artificial intelligence and sentiment calculation. The system comprises a feature decoupling module, a decoupling guide routing module and a dynamic attention fusion module. The method comprises the following steps: firstly, decomposing an original feature of each input mode into a mode sharing feature and a mode specific feature through a feature decoupling module; aggregating the modal sharing features to obtain enhanced modal sharing features; a decoupling-guided routing module generates a group of modal weights for each sample in a self-adaptive manner on the basis of the decoupled modal importance weights; the dynamic attention fusion module performs dynamic weighting and cross-modal attention fusion on the modal specific features by using the modal weight to obtain enhanced specific features; and finally, fusing the enhanced specific features and the enhanced modal sharing features to carry out emotion prediction. According to the method, the defect that an existing dynamic fusion method makes a decision on entanglement features is overcome, and more accurate modal importance evaluation and fusion are achieved.

The invention relates to the technical field of intelligent education, in particular to a learning resource recommendation method, system and device, a medium and a product. According to the method, basic information and historical learning behavior data of a target object are obtained, an initial interest map is constructed in combination with a graph neural network algorithm, emotion calculation is performed according to multi-modal emotion data of the target object, a comprehensive emotion vector is generated, and therefore the target object is obtained according to a first interest vector of the initial interest map and the comprehensive emotion vector. And determining a learning resource recommendation result so as to improve the accuracy and adaptability of learning resource recommendation and further improve the learning efficiency of target objects such as children.

The invention discloses a mode-missing-oriented multi-mode small sample emotion recognition method and device, and relates to the technical field of emotion computation.The method comprises the steps that S1, electroencephalogram data and eye movement data are obtained; s2, constructing a multi-modal small sample emotion recognition model comprising a feature extraction module, a missing modal reconstruction module, an uncertainty perception adaptive fusion module and a small sample classification module; s3, training the multi-modal small sample emotion recognition model by using the electroencephalogram data and the eye movement data to obtain a trained multi-modal small sample emotion recognition model; and S4, performing emotion recognition by using the trained multi-modal small sample emotion recognition model. According to the method, the multi-modal small sample emotion recognition model is constructed, and the technical effect of remarkably improving the accuracy, robustness and generalization ability of multi-modal emotion recognition in complex reality scenes such as modal deficiency, labeled sample scarcity and cross-subject distribution offset is achieved.

The invention discloses an intelligent music recommendation method based on emotional perception and acoustic features, and relates to the technical field of intelligent recommendation systems and emotional computation.The method comprises the steps that physiological signals, music acoustic features and historical behavior data of a user are collected; preprocessing the multi-source features and mapping the multi-source features to the same dimension to construct a fusion matrix; building a double-branch deep learning model, fusing features through an attention mechanism and training parameters; an evolutionary algorithm is adopted to optimize hyper-parameter screening optimal combination; generating a recommendation list matched with the real-time emotion and preference; and continuously collecting user interaction data, and regularly and incrementally training the dynamic update model. According to the method, emotion and behavior dual-drive recommendation is achieved by fusing physiological signals and acoustic features, emotion perception is accurate, recommended content fits the real-time mood, model optimization is efficient, recommendation precision and diversity are remarkably improved, and the music consumption experience of a user is greatly improved.

The invention discloses an emotion classification method and system for heart electromagnetic signals of heart rate variability based on topology analysis, and relates to the technical field of biomedical signal processing and emotion calculation, and the emotion classification method comprises the following steps: preprocessing magnetocardiogram signals or electrocardiogram signals; positioning R waves, extracting an RR interval sequence, and extracting statistics of topological feature points of the persistent graph as topological features; and performing fusion through the time sequence branch, the topological branch and the fusion layer to generate a fusion vector, and classifying the emotional state based on the fusion vector. According to the method, the signal quality and the anti-interference capability are remarkably improved, topological data analysis (TDA) is innovatively introduced, quantitative characterization of a heart rate variability nonlinear dynamic structure is achieved, and the capturing capability of the model to a key physiological mode is enhanced; the problem of overfitting under small samples is effectively relieved, the accuracy, robustness and cross-individual generalization ability of emotion classification are remarkably improved, and a new method is provided for emotion calculation.

The invention relates to the technical field of artificial intelligence, and discloses a child emotion calculation device and method, a storage medium and an accompanying toy, the child emotion calculation device comprises a sensing module configured to collect modal data including facial expressions, voices, contact behaviors, motion postures and environment information; a calculation module configured to receive the modal data and convert the modal data into digital features; the emotion understanding module is configured to identify a user emotion state and an interaction intention according to the digital characteristics; the decision reasoning module is configured to generate a personalized behavior strategy according to the emotional state and the interaction intention; and the interactive execution module is configured to output a physical feedback signal to an execution terminal according to the personalized behavior strategy. According to the method, simultaneous analysis of modes such as facial expressions, voices, intonations and tactile behaviors of children is realized through a multi-mode cooperation mechanism, the emotion recognition accuracy is greatly improved, and professional psychological accompanying support can be provided for the children.

The application discloses a kind of based on bidirectional attention mechanism and multi-level fusion adolescent emotion recognition method and system, it is related to artificial intelligence and affective computing technical field.The method includes: obtaining and preprocessing the audio and video data of adolescent, extracts initial feature vector;Characteristics are input into the multi-level fusion model based on the architecture of Transformer, sequentially early, mid and late fusion are carried out by bidirectional cross attention mechanism, to capture subtle signs, semantic conflict and decision weighting respectively;Dynamic weighting is carried out to anti-interference by introducing gate mechanism to fusion characteristics;Finally, output emotion category.Corresponding system contains data acquisition, feature processing, dynamic gate and output module.The application solves the problem of inconsistent expression of adolescent emotion and modal heterogeneity by multi-level progressive fusion and dynamic loss optimization, significantly improves the recognition accuracy, robustness and generalization ability of complex emotion.

The invention relates to the technical field of child emotion recognition, in particular to an emotion calculation method and device for child expressions, and the method comprises the steps: collecting and marking the multi-modal dynamic expression data of a plurality of Asian children, and constructing an enhanced mixed expression database according to the multi-modal dynamic expression data of the Asian children; performing iterative training on a pre-constructed deep convolutional neural network model by using the enhanced mixed expression database until a preset iterative period is reached, so as to obtain a deep time sequence emotion calculation model; and inputting the multi-modal dynamic expression of the child to be recognized into the depth time sequence emotion calculation model to calculate the current emotion of the child. Therefore, the problems that an existing child emotion recognition model is mostly based on European and American adult database training and mostly adopts single-frame static image analysis, and the complete dynamic process of expressions cannot be captured, so that the accuracy is remarkably reduced, the model robustness is poor, and effective application in real and continuous interaction scenes is difficult are solved.

This application provides a joint emotion recognition method and system based on action unit-driven attention, relating to the fields of computer vision and affective computing. The method includes: inputting a feature map into an action unit (AU)-driven attention branch, processing it through a sigmoid activation function to obtain an activation vector, and inputting this vector into a learnable mapping layer; projecting the AU information back into the spatial geometric space to generate a single-channel spatial attention map, which is then multiplied element-wise with the feature map to obtain a weighted feature map; determining continuous values ​​for discrete emotion category probabilities, valence, and arousal; constructing a total loss function; and training the model using the acquired AffectNet and Aff-wild2 datasets to determine the target model for emotion recognition. This application achieves end-to-end joint optimization of AU detection and emotion recognition, significantly enhancing the interpretability and generalization ability of the model while improving emotion recognition accuracy, enabling it to better adapt to the emotion recognition needs in complex scenarios.

The invention provides a personalized voice cloning accompanying robot system and method based on a large model, and relates to the technical field of artificial intelligence voice interaction and emotion computing, the system comprises an end side and a cloud end, and the method comprises the following steps: collecting a user voice sample, and carrying out preliminary feature extraction; finely adjusting the TTS model by adopting a feed-shot learning algorithm to generate personalized sound; inputting multi-modal data to an emotion fusion LLM engine, and outputting a response text with an emotion tag; voice response is synthesized through local reasoning; and outputting a voice response. According to the method, the problems of low cloning accuracy, insufficient emotion fusion, high privacy risk, high response delay and the like of an existing voice cloning technology in an accompanying robot are solved, and high accuracy (gt; 95%) few sample cloning, emotional response generation, end-side privacy protection and low latency (lt; and the applicability of the accompanying robot in children education and elderly nursing scenes is improved.

The invention discloses intelligent teaching equipment integrating emotion calculation and feedback adjustment, and relates to the technical field of intelligent teaching. The data acquisition module is used for acquiring multi-modal emotion related data of the learner; the emotion calculation module is used for receiving the multi-modal emotion related data, preprocessing the multi-modal emotion related data, performing emotion state recognition through a preset emotion recognition model, and outputting an emotion recognition result; the teaching control module comprises a teaching content database and a feedback adjustment strategy library, and is used for receiving the emotion recognition result, matching a corresponding teaching adjustment parameter from the feedback adjustment strategy library according to the emotion recognition result, adjusting the teaching content in the teaching content database based on the teaching adjustment parameter, and generating an adjusted teaching instruction; and the feedback output module is used for receiving the teaching instruction and displaying the adjusted teaching content to the learner in a preset output form. According to the method, the accuracy and comprehensiveness of emotion recognition are remarkably improved.

This application relates to the field of speech analysis technology and proposes a speech emotion analysis method driven by facial expressions of virtual characters. The method includes: acquiring speech signals in different emotional states to obtain different speech signal frames; extracting different speech emotion features from the speech signal frames, pre-setting different time scales, and constructing inter-frame difference coefficients for the speech signal frames; associating the speech signals with emotional states to obtain the probability distribution of all emotional states and the dominant emotion of the speech signal frames, calculating the prior probability of the dominant emotion of the speech signal frames, and calculating the emotion transfer coefficient of the speech signal frames; and obtaining an adjustment probability distribution based on the inter-frame difference coefficients and emotion transfer coefficients of the speech signal frames, wherein the adjustment probability distribution is the result of speech emotion analysis. This application aims to improve the coherence and naturalness of facial expressions of virtual characters driven by speech emotion recognition results.

The application provides an intelligent interaction experience optimization method and system combined with emotional computing, and belongs to the technical field of intelligent interaction. First, original behavior data flow during multi-round task conversation of a user and a smart terminal is collected, including interface touch pressure distribution trajectory, voice instruction frequency spectrum feature sequence and eye gaze heat map change record. Then, the interface touch pressure distribution trajectory and the eye gaze heat map change record are double-mode fusion coded to generate an operation intention focus area distribution graph. The space emotion distribution atlas is obtained by superimposing the voice emotion color label, and the emotional negative experience blind area is identified therefrom. Finally, interface element strengthening prompt instructions are generated according to the blind area position, the blind area interface element is triggered to visually protrude, and the intelligent interaction experience is optimized.

An emotion support robot man-machine interaction method based on multi-modal emotion recognition belongs to the field of man-machine interaction technology, artificial intelligence and emotion calculation, and comprises the following steps: step 1, carrying out multi-modal emotion recognition on a user; 2, performing personalized emotion support dialogue generation based on an emotion recognition result in the step 1; and 3, transmitting the emotion recognition result in the step 1 and the emotion support dialogue in the step 2 to an interaction robot, determining a corresponding interaction action by the interaction robot according to the emotion recognition result, and outputting a personalized emotion support dialogue to a user through a TTS technology. According to the method, the multi-modal emotion data set for the college student group is constructed, the multi-modal emotion recognition model based on text enhancement is adopted, various complex emotions possibly possessed by the user at the same time can be recognized, quantitative calculation is conducted on the complex emotions, and therefore the current emotion state of the user can be recognized more accurately and more robustly, and the user experience is improved. And the limitation of single-mode identification is overcome.

The invention discloses a double-branch knowledge distillation lightweight micro-expression recognition method and system, relates to the technical field of artificial intelligence and emotion calculation, and realizes efficient migration of macro-expression knowledge to a micro-expression recognition task through constructing a macro-micro-expression double-branch network framework and through hierarchical feature decoupling and a gating attention mask mechanism. Global semantics irrelevant to micro-expressions are dynamically isolated by using a gating attention mask, and the problem of information loss caused by semantic gaps in cross-domain knowledge migration is relieved; designing a time sequence feature sensing module, fusing optical flow time sequence features and a residual attention structure, and enhancing the local dynamic feature extraction capability; a lightweight convolution module is adopted; and the model parameter quantity and the calculation complexity are reduced on the premise of ensuring the recognition precision.

The application discloses a video emotion recognition method based on multi-modal, relates to the technical field of computer vision, deep learning and emotion computing, and comprises the following steps: constructing a micro-motion label system, performing secondary labeling on a CAER data set based on the micro-motion label system, and labeling micro-motion labels; constructing a dual-modal feature extraction network, including a face macro-emotion encoder for extracting macro facial features and a time-series micro-motion feature encoder for extracting time-series micro-motion features; the face macro-emotion encoder outputs a Logits vector for face emotion classification for the face in the video; the time-series micro-motion feature encoder outputs a Logits vector for micro-motion emotion classification for the body in the video; and the dual-modal posterior Logits are fused, and the final emotion category is output based on the fused Logits vector. The application significantly improves the accuracy and robustness of video emotion recognition.

This invention discloses a personalized digital family member replication, persona reshaping, and immersive interactive system based on AI and VR, belonging to the fields of artificial intelligence and virtual reality technology. The invention first collects multi-source data such as images, voice, text, and feature tags to complete 3D digital human reconstruction, personality logic, and unique voiceprint replication. Then, it supports user-customizable switching between various personas such as poet, singer, and storyteller. Combined with affective computing technology, it achieves anthropomorphic two-way interaction, enabling functions such as dialogue, singing, recitation, and knowledge explanation. Next, a VR engine is used to build virtual scenes, achieving immersive face-to-face interaction. Finally, the data is encrypted and stored in the cloud with tiered access permissions. This invention highly restores the real characteristics of the person, offers flexible and configurable personas, rich interactive forms, and a strong immersive experience. It can realize family remembrance and emotional comfort, as well as the digital inheritance of family knowledge and traditions, with a wide range of applications and high security.

This invention relates to a method and system for EEG emotion recognition based on an adaptive multi-view graph neural network, belonging to the field of brain-computer interface and emotion computing technology. The method includes: dividing multi-channel EEG signals into continuous time windows, and using four adjacent time windows as temporal input samples; extracting multi-band differential entropy features of each time window as initial node features; fusing prior knowledge of electrode spatial proximity and brain biological symmetry to construct a basic matrix, and modulating and applying sparse constraints through a learnable attention mechanism to generate an individualized brain functional connectivity topology; designing a parallel bi-branch deep network, where a graph convolutional branch extracts global spatiotemporal features from the graph structure sequences corresponding to the four time windows, and a one-dimensional convolutional branch extracts and fuses local frequency-spatial features; and during training, comprehensively applying node-level domain adversarial and graph structure collaborative regularization to output the emotion category. This invention is beneficial for improving cross-subject recognition performance.

The present invention relates to the work environment adaptation method for buildings and a system for realization the adaptation method that creates a certain effect and analyses levels of a person's performance and happiness taking into account the current work environment in a fixed or mobile workplace. Such systems make it possible to analyse and improve the overall labour efficiency, and in particular the maintenance and extension of the period of valuable human performance, and allow making changes to work environment data to adjust them to an employee's short- and medium-term responses. This system is based on long-term research and makes use of the advances in artificial intelligence (Al) and affective computing systems.