Emotion recognition method and device based on artificial intelligence

Abstract

The embodiments of the present application provide an artificial intelligence-based emotion recognition method and device, which realizes identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to an emotion recognition method and device based on artificial intelligence. Background Art

[0002] Existing emotion recognition methods have significant shortcomings. Traditional systems rely primarily on single-modal data for emotion judgment, making it difficult to accurately capture complex emotional states and lacking mechanisms to protect patient privacy and data security.

[0003] Furthermore, existing technologies face bottlenecks in multimodal feature fusion. Most systems employ simple feature concatenation or weighted averaging, failing to fully utilize the complementary information between modalities, resulting in suboptimal fusion results.

[0004] Existing systems have technical shortcomings in emotion assessment. They lack temporal modeling of patients' historical emotional changes, making it difficult to integrate physiological indicators for comprehensive analysis, impacting the accuracy of medical monitoring decisions. Addressing these issues is crucial for improving emotion recognition in medical settings. Summary of the Invention

[0005] In response to the problems in the existing technology, this application provides an artificial intelligence-based emotion recognition method and device, which can effectively solve the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improve the accuracy and practical value of emotion recognition in medical scenarios.

[0006] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an emotion recognition method based on artificial intelligence, comprising:

[0008] After obtaining authorization, facial image data, voice data, and body movement data of the target object are collected, a facial image sequence of the target object is obtained through an image acquisition device, a voice signal of the target object is acquired through a sound acquisition device and voiceprint features are extracted, a body movement sequence of the target object is acquired through a motion acquisition device, the identity of the target object of the voice signal is verified based on the voiceprint features, and the facial image sequence, the voice signal, and the body movement sequence are respectively input into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and action emotion features;

[0009] Constructing a multi-level attention fusion network to perform multimodal feature fusion on the facial emotion features, the voice emotion features, and the action emotion features. The multi-level attention fusion network calculates the attention weights of the modal internal features and the cross-modal features through the feature attention layer and the modal attention layer, performs weighted fusion on the features based on the attention weights to obtain a cross-modal feature representation, and extracts the emotional state features from the cross-modal feature representation;

[0010] An emotion change time series model is established based on the previous emotion records contained in the medical history data of the target object, the emotion state characteristics are input into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, the emotion state evaluation result is corrected based on the physiological indicator data of the target object, the corrected emotion state evaluation result is input into the medical monitoring model to generate monitoring instructions, and the medical monitoring equipment is controlled to perform corresponding monitoring operations according to the monitoring instructions.

[0011] Furthermore, the method further includes: acquiring a facial image sequence of a target object through an image acquisition device disposed in a medical monitoring location, sampling the facial image sequence at a preset frame rate, performing face detection and alignment on the sampled images, extracting facial region images to construct facial image data, and extracting coordinates of facial feature points in the facial image data based on a deep neural network to generate facial feature point trajectory data;

[0012] The voice signal of the target object is collected by a sound collection device installed in the medical monitoring place, the voice signal is preprocessed to remove background noise, the voiceprint features and acoustic feature parameters of the voice signal are extracted, and the target object identity is authenticated by the voiceprint features based on a deep neural network. The limb movement image of the target object is collected by a motion collection device installed in the medical monitoring place, the limb movement image is detected with skeleton key points, and the key point coordinates are extracted to generate motion sequence data.

[0013] Furthermore, the method further includes: inputting the voiceprint features into a voiceprint verification model to identify the target object, the voiceprint verification model calculating a similarity score between the input voiceprint features and pre-stored target object voiceprint features based on a twin neural network structure, and determining the target object's identity based on the similarity score; inputting the facial feature point trajectory data into a facial emotion recognition model, the facial emotion recognition model extracting facial expression temporal features using a spatiotemporal convolutional network structure, and weighting the facial expression temporal features based on an attention mechanism to obtain facial emotion features;

[0014] The authenticated voice signal is input into the voice emotion recognition model, and the voice emotion recognition model uses a recurrent neural network structure to perform time series modeling on the acoustic feature parameters of the voice signal to obtain voice emotion features. The action sequence data is input into the action emotion recognition model, and the action emotion recognition model uses a graph convolutional network structure to extract spatiotemporal features of the skeleton key point sequence to obtain action emotion features.

[0015] Furthermore, the method further includes: constructing a feature attention layer to perform attention calculation on the facial emotion feature, the voice emotion feature, and the action emotion feature respectively, the feature attention layer using a self-attention mechanism to calculate the correlation between internal features of the same modality to obtain a feature attention weight matrix, weighting each modality feature based on the feature attention weight matrix to obtain a modality internal feature representation, and normalizing the modality internal feature representation to generate a feature-level fusion result;

[0016] A modal attention layer is constructed to perform cross-modal feature fusion on the feature-level fusion results. The modal attention layer adopts a cross-attention mechanism to calculate the interaction relationship between different modal features to obtain a modal attention coefficient matrix. The feature-level fusion results are weighted based on the modal attention coefficient matrix to obtain a cross-modal feature representation. The cross-modal feature representation is reduced in dimension through a nonlinear transformation to obtain a fused feature vector.

[0017] Furthermore, the method further includes: calculating the fusion weights of facial emotion features, voice emotion features, and action emotion features based on the modal attention coefficient matrix, performing weighted summation of the fusion weights and the corresponding feature-level fusion results to obtain modal fusion features, combining the modal fusion features with the original features through residual connections, and performing nonlinear transformation on the combined features using a multi-layer perceptron to obtain a cross-modal feature representation;

[0018] An emotion feature knowledge graph encoder is constructed to perform graph structure encoding on the cross-modal feature representation. The emotion feature knowledge graph encoder models the relationship between feature nodes based on a graph neural network, updates the node feature representation through a message passing mechanism, and uses a graph pooling operation to aggregate the updated node features to obtain an emotion state feature vector.

[0019] Furthermore, the method further includes: extracting past emotional records of the target subject from a medical record database, sorting the past emotional records in chronological order to construct a time series data set, constructing a time series model of emotional change based on a long short-term memory network, inputting the time series data set into the time series model of emotional change for training, learning the temporal dependency of emotional states through a gating mechanism, and optimizing model parameters using a gradient descent method to obtain a trained time series model of emotional change;

[0020] The emotional state feature vector is spliced ​​with the historical emotional state sequence to obtain sequence input data, and the sequence input data is input into the trained emotional change time series model. The emotional change time series model extracts key time series features in the sequence data based on the attention mechanism, and dynamically models the emotional state through the hidden state transfer mechanism to obtain the emotional state evaluation result.

[0021] Furthermore, the method further includes: collecting physiological indicator data of heart rate, blood pressure, and body temperature of the target subject, normalizing the physiological indicator data to obtain a physiological feature vector, fusing the physiological feature vector with the emotional state assessment result, constructing a physiological indicator correction model based on a deep neural network, wherein the physiological indicator correction model uses a multi-layer fully connected structure to perform nonlinear transformation on the fused features, and outputting the corrected emotional state assessment result through a soft maximization function;

[0022] The corrected emotional state assessment result is input into a medical monitoring model, which classifies the emotional state based on a decision tree structure, selects corresponding monitoring instructions from a preset monitoring strategy library according to the classification result, converts the monitoring instructions into device control signals, and sends the device control signals to the medical monitoring device through a communication interface for monitoring operation control.

[0023] In a second aspect, the present application provides an emotion recognition device based on artificial intelligence, comprising:

[0024] a data acquisition module configured to, after obtaining authorization, collect facial image data, voice data, and body movement data of a target object, obtain a facial image sequence of the target object through an image acquisition device, obtain a voice signal of the target object through a voice acquisition device and extract voiceprint features, obtain a body movement sequence of the target object through a motion acquisition device, verify the identity of the target object of the voice signal based on the voiceprint features, and input the facial image sequence, the voice signal, and the body movement sequence into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and motion emotion features;

[0025] a feature processing module, configured to construct a multi-level attention fusion network to perform multimodal feature fusion on the facial emotion features, the voice emotion features, and the action emotion features; the multi-level attention fusion network calculates the attention weights of the modal internal features and the cross-modal features through the feature attention layer and the modal attention layer, performs weighted fusion on the features based on the attention weights to obtain a cross-modal feature representation, and extracts the emotional state features from the cross-modal feature representation;

[0026] An emotion recognition module is used to establish an emotion change time series model based on the previous emotion records contained in the medical history data of the target object, input the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, correct the emotion state evaluation result based on the physiological indicator data of the target object, input the corrected emotion state evaluation result into a medical monitoring model to generate a monitoring instruction, and control the medical monitoring equipment to perform corresponding monitoring operations according to the monitoring instruction.

[0027] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the artificial intelligence-based emotion recognition method when executing the program.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the artificial intelligence-based emotion recognition method.

[0029] In a fifth aspect, the present application provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the artificial intelligence-based emotion recognition method.

[0030] It can be seen from the above technical solutions that the present application provides an emotion recognition method and device based on artificial intelligence, which realizes identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 Schematic diagram of the process of emotion recognition method based on artificial intelligence in an embodiment of the present application;

[0033] Figure 2This is a structural diagram of an artificial intelligence-based emotion recognition device in an embodiment of the present application;

[0034] Figure 3 Schematic diagram of the structure of the electronic device in the embodiment of the present application.

[0035] Reference numerals:

[0036] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0039] Taking into account the problems existing in the prior art, the present application provides an emotion recognition method and device based on artificial intelligence, which realizes identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios.

[0040] In order to effectively solve the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improve the accuracy and practical value of emotion recognition in medical scenarios, this application provides an embodiment of an emotion recognition method based on artificial intelligence, see Figure 1 , the emotion recognition method based on artificial intelligence specifically includes the following contents:

[0041] Step S101: After obtaining authorization, facial image data, voice data, and body movement data of the target object are collected, a facial image sequence of the target object is obtained through an image acquisition device, a voice signal of the target object is obtained through a sound acquisition device and voiceprint features are extracted, and a body movement sequence of the target object is obtained through a movement acquisition device. The identity of the target object of the voice signal is verified based on the voiceprint features, and the facial image sequence, the voice signal, and the body movement sequence are respectively input into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and movement emotion features;

[0042] Optionally, this embodiment deploys a multimodal data acquisition system in a medical monitoring facility, and first obtains the informed consent authorization of the target subject through the electronic medical record system. After the authorization is confirmed, the multimodal data acquisition process is started. The process adopts a distributed architecture to ensure the real-time and reliability of data acquisition. In the medical monitoring ward, a high-definition camera array is used to collect facial image sequences of the target subject. The camera installation position has been optimized to ensure that facial expression changes can be captured from multiple angles. During the acquisition process, the image resolution is set to 1920×1080 pixels and the frame rate is 30fps. The selection of these parameters not only meets the accuracy requirements of facial feature extraction, but also ensures the real-time processing capability of the system.

[0043] This embodiment uses a medical-grade sound acquisition device to collect voice data. This device includes an anti-interference microphone array that effectively filters background noise generated by medical equipment. The voice signal sampling rate is set to 16kHz, with 16-bit quantization precision. This configuration ensures the accuracy of voiceprint feature extraction while meeting real-time processing requirements. Voiceprint feature extraction uses an improved MFCC (Mel-Frequency Cepstral Coefficient) algorithm. The specific calculation process is as follows:

[0044] F(m) = Σ|X(k)|×H_m(k) ,

[0045] Where X(k) is the spectrum of the speech signal, H_m(k) is the Mel filter bank, and m is the filter number. This algorithm fully considers the auditory characteristics of the human ear, and the extracted features have good discriminability.

[0046] This example uses depth cameras deployed at various locations in the ward to capture the target subject's body movement sequences. The depth cameras utilize binocular stereo vision technology to accurately capture three-dimensional spatial information. The motion data acquisition frequency is set to 60Hz to ensure rapid changes in body movements. The depth image resolution is 640×480 pixels, and the depth accuracy can reach ±1cm. These parameter settings ensure the accuracy of motion capture.

[0047] This embodiment innovatively designs a voiceprint verification mechanism, using an improved twin neural network structure for identity verification. The network input is the currently collected voiceprint features and the pre-stored registered voiceprint features, and the feature similarity is calculated through comparative learning:

[0048] S = cos(f(x1), f(x2)) ,

[0049] Where f(x1) and f(x2) are feature extraction functions, x1 and x2 are the voiceprint features to be compared, and cos represents cosine similarity. This verification mechanism effectively prevents identity theft and ensures the reliability of emotion recognition results.

[0050] After completing identity verification, this embodiment parallelizes multimodal data input into the corresponding emotion recognition model. This facial emotion recognition model utilizes an improved 3D-CNN architecture, which includes a spatiotemporal attention module and is capable of capturing the dynamic characteristics of facial expressions. The model's convolutional kernel design fully considers the scale variations of facial features and employs a multi-scale feature extraction strategy to ensure sensitivity to subtle changes in facial expressions.

[0051] The speech emotion recognition model in this embodiment is based on the BiLSTM-Attention architecture, which effectively models long-term temporal dependencies in speech signals. It captures contextual speech features through a bidirectional recurrent neural network and uses an attention mechanism to highlight important acoustic features. Model inputs include acoustic parameters such as fundamental frequency, energy, and zero-crossing rate, which have a direct physical correlation with emotional states.

[0052] The action emotion recognition model in this embodiment uses a spatiotemporal graph convolutional network (ST-GCN) architecture, which models human skeleton points as a graph structure and extracts action features through graph convolution operations. This model considers the spatial connectivity and temporal variation between human joints, accurately capturing emotion-related action patterns. Each skeletal joint point contains 3D spatial coordinates and confidence information, ensuring reliable feature extraction.

[0053] Through these technological innovations, this embodiment effectively addresses several key issues in emotion recognition in medical monitoring scenarios: First, the simultaneous collection and preprocessing of multimodal data ensures the quality of the original data; second, the identity verification mechanism ensures the reliability of emotion recognition results; and finally, a specially designed emotion recognition model accurately extracts emotional features from each modality. In practical applications, this solution has significantly improved the accuracy and stability of emotion recognition.

[0054] The innovations of this embodiment primarily lie in its data collection strategy, authentication mechanism, and model design. Through a carefully designed processing flow, high-quality emotion feature extraction is achieved in medical monitoring scenarios. This solution provides a reliable data foundation for subsequent multimodal feature fusion and has significant application value in the field of medical monitoring. Experimental verification demonstrates that this method can accurately capture changes in patient emotions, showing significant advantages, particularly in medical scenarios requiring long-term monitoring.

[0055] Step S102: constructing a multi-level attention fusion network to perform multimodal feature fusion on the facial emotion features, the voice emotion features, and the action emotion features. The multi-level attention fusion network calculates the attention weights of the modal internal features and the cross-modal features through the feature attention layer and the modal attention layer, performs weighted fusion on the features based on the attention weights to obtain a cross-modal feature representation, and extracts the emotional state features from the cross-modal feature representation.

[0056] Optionally, this embodiment innovatively designs a multi-level attention fusion network to address the multimodal feature fusion problem of emotion recognition in medical monitoring scenarios. This network uses a hierarchical attention calculation structure. First, a self-attention mechanism is constructed at the feature level to calculate the correlation weights of features within the same modality. For facial emotion features, attention calculation uses key-value pairs:

[0057] A(Q,K) = softmax(QK^T / √d) ,

[0058] Where Q and K are the query matrix and key matrix respectively, and d is the feature dimension. This calculation method can capture the dependency between expressions in different facial regions.

[0059] This embodiment designs an adaptive weight calculation mechanism in the feature attention layer. Aiming at the speech emotion features, an improved multi-head attention structure is adopted:

[0060] MultiHead(X) = Concat(head_1,...,head_h)W_o,

[0061] Where head_i represents the output of the i-th attention head, and W_o is the projection matrix. This multi-head mechanism enables the model to simultaneously focus on different semantic dimensions of speech features, such as pitch, rhythm, and energy. This design significantly improves the expressiveness of speech emotion.

[0062] This embodiment introduces a spatiotemporal attention module when processing action emotion features. This module considers feature dependencies in both spatial and temporal dimensions:

[0063] A_st(F) = α×A_s(F) + (1-α)×A_t(F) ,

[0064] Where A_s and A_t are spatial attention and temporal attention respectively, and α is the adaptive weight coefficient. This design can effectively capture the emotional expression characteristics of body movements in both spatial and temporal dimensions.

[0065] This embodiment innovatively constructs a modal attention layer to achieve dynamic fusion of cross-modal features. Modal attention uses a cross-attention mechanism to calculate the interaction between different modal features:

[0066] C(F_i,F_j) = softmax(W_q×F_i×(W_k×F_j)^T) ,

[0067] Where F_i and F_j are features of different modalities, and W_q and W_k are learnable projection matrices. This mechanism can adaptively adjust the importance weights of features of different modalities.

[0068] This embodiment implements an innovative feature fusion strategy. Based on the calculated attention weights, multimodal features are weightedly fused:

[0069] F_fusion = Σ(w_i×T_i(F_i)),

[0070] Where w_i is the modal weight, T_i is the feature conversion function, and F_i is the feature of each modality. The conversion function includes a nonlinear activation layer and a normalization layer to ensure the consistency of feature distribution.

[0071] This embodiment designs a residual connection mechanism to preserve the original feature information:

[0072] F_res = F_fusion +λ×F_orig,

[0073] Where F_orig is the original feature and λ is the residual coefficient. This design ensures sufficient feature fusion while avoiding the loss of useful information. The introduction of residual connections significantly improves the training stability of the model.

[0074] This embodiment adopts a hierarchical processing strategy in the feature extraction stage. First, the fusion feature is reduced in dimension by a multi-layer perceptron:

[0075] F_reduced = MLP(F_res),

[0076] The MLP is a multi-layer perceptron structure. The dimensionality reduction process retains the key information of the features while reducing the computational complexity of subsequent processing.

[0077] This embodiment innovatively constructs an emotion feature knowledge graph to provide a structured representation of the features after dimensionality reduction. The nodes in the graph represent different types of emotion features, and the edges represent the semantic associations between features. Feature propagation is performed through graph neural networks:

[0078] H (l+1) =σ(AH (l)W (l)),

[0079] Where A is the adjacency matrix, H is the node feature, and W is the learnable weight matrix.

[0080] Through the above technological innovations, this embodiment effectively addresses the critical issue of multimodal emotion feature fusion in medical monitoring scenarios. This solution adaptively adjusts the importance of different modal features while maintaining semantic consistency across them. In practical applications, this approach significantly improves the accuracy and robustness of emotion recognition.

[0081] The innovations of this embodiment primarily lie in the design of the attention mechanism, the feature fusion strategy, and the construction of a knowledge graph. Through a carefully designed multi-level fusion network, high-quality emotion feature fusion is achieved in medical monitoring scenarios. This solution provides important support for improving the performance of emotion recognition systems and has broad application prospects in the medical monitoring field. Experimental verification demonstrates that this method can accurately capture the complex correlations between multimodal emotion features, showing significant advantages in scenarios where patients' emotional states are complex and changeable.

[0082] Step S103: Establish an emotion change time series model based on the previous emotion records contained in the medical history data of the target object, input the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, correct the emotion state evaluation result based on the physiological indicator data of the target object, input the corrected emotion state evaluation result into the medical monitoring model to generate a monitoring instruction, and control the medical monitoring equipment to perform corresponding monitoring operations according to the monitoring instruction.

[0083] Optionally, this embodiment addresses the dynamic and complex nature of emotional state assessment in medical monitoring scenarios by innovatively constructing a time-series model of emotional changes based on medical records. First, the target subject's historical emotional records are extracted from the electronic medical record system, including key information such as emotional state description, onset time, and duration. These records are structured and processed by a natural language processing module, converting them into standardized time-series feature sequences.

[0084] This embodiment designs an improved long short-term memory network structure as the core of time series modeling. The network contains three control units: forget gate, input gate, and output gate:

[0085] ,

[0086] ,

[0087] ,

[0088] Where f_t, i_t, and o_t are the state values ​​of the forget gate, input gate, and output gate, respectively; h_(t-1) is the hidden state at the previous moment; and x_t is the current input. This gating mechanism can effectively capture long-term dependencies between emotional states.

[0089] This embodiment implements an innovative time series feature extraction strategy. The model training process uses a dynamic time window mechanism, and the window size is adaptively adjusted according to the frequency of emotion changes:

[0090] W_size = base_size×(1+β×V) ,

[0091] Where base_size is the base window size, V is the rate of change of sentiment, and β is the adjustment coefficient. This design ensures the model’s sensitivity to sentiment changes at different time scales.

[0092] This example introduces an attention enhancement module into the time series model. This module calculates the importance weights of features at different time steps:

[0093] α_t = softmax(v T×tanh(W_h×h_t + W_s×s_t)),

[0094] Where h_t is the hidden state, s_t is the current state, W_h and W_s are weight matrices. The attention mechanism enables the model to focus on the historical information that is most relevant to the current emotional state assessment.

[0095] This embodiment innovatively designs a physiological indicator correction mechanism. First, the target object's heart rate, blood pressure, body temperature and other physiological data are collected in real time through medical monitoring equipment. These data are converted into physiological feature vectors after normalization. Feature normalization uses the adaptive normalization method: x_norm = (x-μ_rolling) / σ_rolling,

[0096] Where μ_rolling and σ_rolling are the mean and standard deviation within the sliding window. This processing method takes into account individual differences in physiological indicators.

[0097] This example constructs a deep neural network as a physiological indicator correction model. This model uses a multilayer perceptron architecture, with batch normalization and dropout layers added between each layer to improve generalization. The model input is the concatenation of emotional state assessment results and physiological feature vectors, and a nonlinear transformation is performed to obtain the corrected assessment results.

[0098] This embodiment optimizes the decision-making mechanism of the medical monitoring model. The model is based on an improved decision tree structure, and the division criteria of decision nodes take into account multiple dimensions of emotional state:

[0099] Split_score = I_emotion + λ × I_physio,

[0100] Where I_emotion is the emotional information gain, I_physio is the physiological information gain, and λ is the balance factor. This decision-making mechanism can comprehensively consider emotional state and physiological condition.

[0101] This embodiment implements an intelligent monitoring instruction generation strategy. A monitoring strategy library contains multiple preset monitoring scenarios, each corresponding to a different combination of emotional states. Based on the decision-making results, the system selects the most appropriate monitoring instruction and converts it into a control signal recognizable by the device. This control signal is transmitted to the medical monitoring device via a standardized communication interface.

[0102] Through the above technological innovations, this embodiment effectively addresses the challenges of accurate and real-time emotional state assessment in medical monitoring scenarios. This solution accurately predicts emotional trends and, combined with physiological indicators, provides reliable state assessment. In practical applications, this approach significantly enhances the intelligence and targeted nature of medical monitoring.

[0103] The innovations of this embodiment primarily lie in time series modeling, state assessment, and monitoring decision-making. Through a carefully designed processing flow, intelligent emotion management is achieved in medical monitoring scenarios. This solution provides important support for improving the automation level of medical monitoring and has broad application prospects in the field of smart healthcare. Experimental verification has demonstrated that this method can accurately assess a patient's emotional state and promptly provide appropriate monitoring plans, demonstrating significant advantages in medical scenarios requiring long-term, continuous monitoring.

[0104] From the above description, it can be seen that the artificial intelligence-based emotion recognition method provided in the embodiment of the present application can realize identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios.

[0105] In one embodiment of the emotion recognition method based on artificial intelligence of the present application, the following contents may also be specifically included:

[0106] Step S201: Acquire a facial image sequence of a target subject using an image acquisition device installed in a medical monitoring facility, sample the facial image sequence at a preset frame rate, perform face detection and alignment on the sampled images, extract facial region images to construct facial image data, and extract facial feature point coordinates from the facial image data based on a deep neural network to generate facial feature point trajectory data;

[0107] Step S202: The target object's voice signal is collected by a sound collection device installed in the medical monitoring place, the voice signal is preprocessed to remove background noise, the voiceprint features and acoustic feature parameters of the voice signal are extracted, the target object's identity is authenticated by the voiceprint features based on a deep neural network, the target object's limb movement image is collected by a motion collection device installed in the medical monitoring place, the limb movement image is detected with skeleton key points, and the key point coordinates are extracted to generate motion sequence data.

[0108] Optionally, this embodiment constructs a multimodal data acquisition environment in a medical monitoring facility. The image acquisition device uses a high-precision camera array, with multiple acquisition nodes strategically arranged in the ward to ensure full coverage of the target subject's face. The camera uses high dynamic range imaging technology, which can adapt to image acquisition requirements under different lighting conditions. The original image sequence is sampled at a frame rate of 30fps, and the sampling process uses an adaptive frame rate adjustment mechanism:

[0109] R_adj = R_base×(1 + α×M),

[0110] Where R_base is the base frame rate, M is the motion intensity index, and α is the adjustment coefficient. This mechanism ensures that more details can be captured when the face moves violently.

[0111] This embodiment innovatively designs a face detection and alignment algorithm. The detection module uses an improved YOLOv3 architecture and improves detection accuracy through a multi-scale feature pyramid:

[0112] P_face = σ(Conv(Concat(F1, F2, F3))),

[0113] Where F1, F2, and F3 are feature maps of different scales, Conv is the convolution operation, and σ is the activation function. The alignment process uses a spatial transformer network to achieve normalization of facial poses.

[0114] This embodiment achieves accurate positioning of facial feature points. A cascade regression network is used to extract the coordinates of 68 key points. The network structure includes multiple cascade stages:

[0115] S_t+1 = S_t + S_t,

[0116] Where S_t is the position of the feature point in the tth stage, S_t is the position increment. Each stage includes local feature extraction and global shape constraint to ensure the accuracy of feature point positioning.

[0117] This embodiment uses a medical-grade noise-reduction microphone array for voice collection. The microphone uses beamforming technology to suppress background noise through spatial filtering:

[0118] Y(ω) = W^H(ω)X(ω),

[0119] Where X(ω) is the microphone array signal, W(ω) is the optimal weight vector, and Y(ω) is the output signal. This method effectively improves the signal-to-noise ratio of the speech signal.

[0120] This embodiment optimizes the voiceprint feature extraction process. It uses an improved x-vector architecture, including a time-domain convolutional network (TDNN) layer and a statistical pooling layer:

[0121] E = Pool(TDNN(S)),

[0122] Where S is the speech signal, TDNN is the time-domain convolution operation, and Pool is the statistical pooling. This structure can capture the time-varying characteristics of voiceprints.

[0123] This embodiment innovatively constructs an acoustic feature parameter set, including fundamental frequency contour, resonance peak, energy envelope, etc.

[0124] F0 = ACF(frame),

[0125] ACF is the autocorrelation function. These parameters have a direct physical correlation with emotional states and provide a reliable basis for emotion recognition.

[0126] This embodiment implements a multi-view solution for motion capture. Multiple depth cameras are used to synchronously capture the target object's body movements, and the cameras are synchronized through a time synchronization module to maintain frame synchronization:

[0127] T_sync = T_base + T_i ,

[0128] Where T_base is the base timestamp, T_i is the time deviation of the i-th camera. The synchronization accuracy can reach millisecond level.

[0129] This embodiment designs an efficient skeleton key point detection algorithm. Using the improved OpenPose architecture, the connection relationship between key points is established through the Part Affinity Field (PAF):

[0130] C = argmax(PAF(p1, p2)) ,

[0131] Among them, p1 and p2 are the key points to be connected. This method can accurately reconstruct the human skeleton structure.

[0132] This embodiment innovatively implements spatiotemporal modeling of action sequences. The detected key point sequence is converted into a standardized action description:

[0133] A = {[x_i, y_i, z_i, c_i]},

[0134] Where (x_i, y_i, z_i) are spatial coordinates and c_i is the confidence level. The sequence data is smoothed in the time dimension to reduce the impact of detection noise.

[0135] Through the above technological innovations, this embodiment effectively addresses the key issue of multimodal data acquisition in medical monitoring scenarios. This solution can stably and reliably collect facial, voice, and motion data, and extract features using advanced algorithms. In practical applications, this approach significantly improves the quality and usability of data acquisition.

[0136] The innovations of this embodiment primarily lie in its data acquisition strategy, feature extraction algorithm, and synchronous control. Through a carefully designed acquisition system, high-quality multimodal data acquisition is achieved in medical monitoring scenarios. This solution provides a reliable data foundation for subsequent emotion recognition tasks and has significant application value in the field of medical monitoring. Experimental verification demonstrates that this method can accurately capture patients' facial expressions, voice characteristics, and body movements, demonstrating significant advantages, particularly in complex medical environments.

[0137] In one embodiment of the emotion recognition method based on artificial intelligence of the present application, the following contents may also be specifically included:

[0138] Step S301: Inputting the voiceprint features into a voiceprint verification model to identify the target object. The voiceprint verification model calculates a similarity score between the input voiceprint features and the pre-stored target object voiceprint features based on a twin neural network structure, and determines the identity of the target object according to the similarity score. Inputting the facial feature point trajectory data into a facial emotion recognition model, the facial emotion recognition model uses a spatiotemporal convolutional network structure to extract facial expression temporal features, and weighting the facial expression temporal features based on an attention mechanism to obtain facial emotion features.

[0139] Step S302: Input the authenticated voice signal into the voice emotion recognition model, and the voice emotion recognition model uses a recurrent neural network structure to perform time series modeling on the acoustic feature parameters of the voice signal to obtain voice emotion features. Input the action sequence data into the action emotion recognition model, and the action emotion recognition model uses a graph convolutional network structure to extract spatiotemporal features of the skeleton key point sequence to obtain action emotion features.

[0140] Optionally, this embodiment innovatively designs a voiceprint verification model to meet the needs of identity authentication and multimodal emotion recognition in medical monitoring scenarios. This model uses an improved twin neural network structure and includes two main modules: feature extraction and similarity calculation. The feature extraction network F(x) uses a deep CNN structure:

[0141] F(x) = CNN(x) / ||CNN(x)||,

[0142] Where x is the input voiceprint feature, and ||×|| represents the L2 norm. This normalization process ensures the scale consistency of the feature vector and improves the reliability of the similarity calculation.

[0143] This embodiment implements an innovative similarity measurement mechanism, using a combination of cosine distance and Euclidean distance:

[0144] S(x1,x2) = α×cos(F(x1),F(x2)) + (1-α)×exp(-||F(x1)-F(x2)||) ,

[0145] Where α is an adaptive weight coefficient, which is dynamically adjusted based on the validation set. This hybrid metric improves the robustness of identity verification.

[0146] This embodiment designs a spatiotemporal feature extraction mechanism for a facial emotion recognition model. The model uses a 3D convolutional network structure, and the convolution kernel design takes the time dimension into consideration:

[0147] F_t = Conv3D(X_t) + ResConn(X_(t-1)),

[0148] Where X_t is the facial feature at the current moment, and ResConn is the residual connection. This structure can effectively capture the dynamic changes in facial expressions.

[0149] This embodiment optimizes the facial attention calculation process. The calculation of attention weight adopts a dual attention mechanism:

[0150] A = Spatial_Attention(F)×Temporal_Attention(F) ,

[0151] Among them, Spatial_Attention and Temporal_Attention calculate the attention weights of spatial and temporal dimensions respectively. This design enables the model to focus on key areas and key moments of the face at the same time.

[0152] This embodiment innovatively constructs a speech emotion recognition model. It uses a BiLSTM structure to perform bidirectional modeling of acoustic features:

[0153] h_f = LSTM_forward(x_t, h_(t-1))

[0154] h_b = LSTM_backward(x_t, h_(t+1)) ,

[0155] Where h_f and h_b are the forward and backward hidden states respectively. Bidirectional processing ensures full utilization of context information.

[0156] This embodiment implements dynamic acoustic feature modeling. The model input includes multi-dimensional acoustic parameters:

[0157] X = [F0, Energy, MFCC, Formants],

[0158] These parameters describe the acoustic characteristics of speech from different perspectives and are closely related to emotional expression. Time series modeling is used to capture the dynamic changes of these parameters.

[0159] This embodiment designs an innovative action emotion recognition architecture. It uses a spatiotemporal graph convolutional network (ST-GCN) to process skeleton sequences:

[0160] H (l+1) = σ(A×H l× W^l),

[0161] Where A is the adjacency matrix, H is the node feature, and W is the learnable weight. The adjacency matrix is ​​predefined based on the human skeletal structure to ensure the rationality of feature propagation.

[0162] This embodiment optimizes the feature extraction strategy of graph convolution. It adds modeling of the time dimension based on the standard graph convolution:

[0163] F_st = TemporalConv(SpatialConv(X)),

[0164] SpatialConv and TemporalConv process the features of spatial and temporal dimensions respectively. This decomposition design improves computational efficiency.

[0165] This embodiment implements an adaptive feature aggregation mechanism. For different types of action features, attention weighting is used for fusion:

[0166] F_motion = Σ(w_i×F_i) ,

[0167] Where w_i is the attention weight and F_i is the different types of action features. The weights are automatically learned by the network to adapt to the importance of features under different emotional states.

[0168] This embodiment, through the above technological innovations, effectively addresses the accuracy and reliability issues of emotion recognition in medical monitoring scenarios. This solution first ensures the authenticity of the data source through strict authentication, and then uses a specially designed deep learning model to extract emotional features from multiple modalities. In practical applications, this approach has significantly improved the accuracy and stability of emotion recognition.

[0169] The innovations of this embodiment primarily lie in its authentication mechanism, multimodal feature extraction, and time series modeling. Through a carefully designed algorithmic architecture, high-quality emotion feature extraction is achieved in medical monitoring scenarios. This solution provides important support for improving the intelligence of medical monitoring systems and has broad application prospects in the healthcare sector. Experimental verification demonstrates that this method can accurately identify patients' emotional states, showing significant advantages in medical scenarios requiring long-term continuous monitoring.

[0170] In one embodiment of the emotion recognition method based on artificial intelligence of the present application, the following contents may also be specifically included:

[0171] Step S401: constructing a feature attention layer to perform attention calculations on the facial emotion features, the voice emotion features, and the action emotion features, respectively. The feature attention layer uses a self-attention mechanism to calculate the correlation between internal features of the same modality to obtain a feature attention weight matrix. Based on the feature attention weight matrix, each modal feature is weighted to obtain a modal internal feature representation. The modal internal feature representation is normalized to generate a feature-level fusion result.

[0172] Step S402: Construct a modal attention layer to perform cross-modal feature fusion on the feature-level fusion result. The modal attention layer uses a cross-attention mechanism to calculate the interaction relationship between different modal features to obtain a modal attention coefficient matrix. The feature-level fusion result is weighted based on the modal attention coefficient matrix to obtain a cross-modal feature representation. The cross-modal feature representation is reduced in dimension through a nonlinear transformation to obtain a fused feature vector.

[0173] Optionally, this embodiment innovatively designs a feature attention layer to address the fusion problem of multimodal emotional features in medical monitoring scenarios. This layer uses an improved self-attention mechanism to first map the features of each modality to the query and key-value space:

[0174] Q = W_q× F, K = W_k× F, V = W_v× F,

[0175] Where F is the input feature, W_q, W_k, and W_v are learnable transformation matrices. This mapping mechanism makes it easier to calculate the correlation of features in high-dimensional space.

[0176] This embodiment implements the feature correlation calculation within the modality. The calculation of attention weight uses scaled dot product attention:

[0177] A = softmax(QK^T / √d_k) ,

[0178] Where d_k is the feature dimension. The introduction of the scaling factor √d_k avoids the vanishing gradient problem and ensures training stability. For facial emotion features, this mechanism can capture the dependencies between different facial regions.

[0179] This embodiment optimizes the feature weighting strategy. The features are weighted based on the calculated attention weight matrix:

[0180] F_weighted = AV + λF,

[0181] Where λ is the residual coefficient. The introduction of residual connections ensures the preservation of original feature information and improves the model's expressiveness. For speech emotion features, this weighting method can highlight important acoustic features.

[0182] This embodiment innovatively designs a feature normalization method, using adaptive layer normalization processing:

[0183] F_norm = γ×(F - μ) / σ + β,

[0184] Where μ and σ are the mean and standard deviation of the features, and γ and β are learnable scaling and bias parameters. This normalization method ensures the consistency of the distribution of features of different modalities.

[0185] This embodiment builds an efficient modality attention layer. This layer uses a cross-attention mechanism to calculate the interaction between different modalities:

[0186] C(i,j) = softmax(W_1×F_i×(W_2×F_j) T),

[0187] Where F_i and F_j are the features of different modalities, and W_1 and W_2 are projection matrices. This mechanism can capture the complementary information between modalities.

[0188] This embodiment implements an adaptive modality fusion strategy. Feature fusion is performed based on the modality attention coefficient matrix:

[0189] F_cross = Σ(C_ij×T_j(F_j)),

[0190] Where T_j is the feature conversion function. This fusion method takes into account the relative importance of different modal features and adapts to the modal contribution under different emotional states.

[0191] This embodiment optimizes the representation learning of cross-modal features. A multi-layer perceptron is used for nonlinear transformation:

[0192] F_transform = MLP(F_cross) + Shortcut(F_cross),

[0193] The shortcut is a quick connection. This structure not only ensures the nonlinear expression ability of features, but also maintains the transmission of original information.

[0194] This embodiment designs a dimensionality reduction processing module. It uses pooling operation with attention to perform dimensionality compression:

[0195] F_reduced = Attention_Pool(F_transform),

[0196] Attention_Pool is an attention-weighted pooling function. This approach can retain important feature information during the dimensionality reduction process.

[0197] This embodiment implements the normalization of the feature vector. L2 regularization is used to ensure the stability of the feature vector:

[0198] F_final = F_reduced / ||F_reduced||,

[0199] This processing method facilitates subsequent similarity calculation and classification tasks.

[0200] Through the above technological innovations, this embodiment effectively addresses the critical issue of multimodal emotion feature fusion in medical monitoring scenarios. This solution accurately captures the correlation and complementarity between features from different modalities, generating high-quality fused feature representations. In practical applications, this approach significantly improves the accuracy and robustness of emotion recognition.

[0201] The innovations of this embodiment primarily lie in the design of the attention mechanism, feature fusion strategy, and dimensionality reduction. Through a carefully designed feature processing process, efficient feature fusion is achieved in medical monitoring scenarios. This solution provides important support for improving the performance of emotion recognition systems and has broad application prospects in the medical monitoring field. Experimental verification demonstrates that this method can effectively integrate multimodal emotion features, showing significant advantages in scenarios where patients' emotional states are complex and changeable.

[0202] In one embodiment of the emotion recognition method based on artificial intelligence of the present application, the following contents may also be specifically included:

[0203] Step S501: Calculating the fusion weights of facial emotion features, voice emotion features, and action emotion features based on the modal attention coefficient matrix, performing weighted summation on the fusion weights and the corresponding feature-level fusion results to obtain modal fusion features, combining the modal fusion features with the original features through residual connections, and performing nonlinear transformation on the combined features using a multi-layer perceptron to obtain a cross-modal feature representation;

[0204] Step S502: Construct an emotion feature knowledge graph encoder to perform graph structure encoding on the cross-modal feature representation. The emotion feature knowledge graph encoder models the relationship between feature nodes based on a graph neural network, updates the node feature representation through a message passing mechanism, and uses a graph pooling operation to aggregate the updated node features to obtain an emotion state feature vector.

[0205] Optionally, this embodiment innovatively designs an attention-based modal weight calculation mechanism to meet the deep integration requirements of emotional features in medical monitoring scenarios. First, the modal attention coefficient matrix is ​​normalized in the row direction:

[0206] W_m = softmax(Σ_j C_ij) where C_ij is the attention coefficient of the i-th modality to the j-th modality in the modality attention coefficient matrix. This normalization process ensures the rationality of the weight distribution of different modal features.

[0207] This embodiment implements an adaptive feature weighted fusion strategy. The fusion process takes into account the dynamic importance of modal features:

[0208] F_fused = Σ(W_m×G_m(F_m)),

[0209] Where G_m is the feature conversion function and F_m is the feature of each modality. The conversion function contains learnable parameters that can adjust the feature expression according to the specific scenario.

[0210] This embodiment optimizes the residual connection mechanism. The residual connection uses a gating unit to perform feature combination:

[0211] F_res = g×F_fused + (1-g)×F_orig,

[0212] Where g is the gating coefficient, which is calculated by the sigmoid function. This design ensures full utilization of the fusion features while avoiding the loss of the original feature information.

[0213] This embodiment innovatively designs a multi-layer perceptron structure. The network adopts a progressive feature transformation strategy:

[0214] H_i = PReLU(BN(W_i×H_(i-1))),

[0215] BN is the batch normalization layer, and PReLU is the parameterized ReLU activation function. This structure can learn complex nonlinear relationships between features.

[0216] This embodiment constructs an innovative emotional feature knowledge graph. The graph nodes contain basic emotion categories and complex emotional states, and the edges represent the conversion relationship and influence strength between emotions. The graph structure is constructed by combining prior knowledge and data-driven methods:

[0217] G = (V, E, A) ,

[0218] Where V is the node set, E is the edge set, and A is the adjacency matrix.

[0219] This embodiment implements a feature propagation mechanism for graph neural networks. Node feature updates at each layer use two steps: message aggregation and feature conversion:

[0220] M_v = AGG({h_u: u∈N(v)}) h_v' = UPDATE(h_v, M_v),

[0221] Where N(v) is the neighbor set of node v, AGG and UPDATE are message aggregation and node update functions respectively.

[0222] This embodiment optimizes the message delivery strategy. Message aggregation takes into account edge weights and direction information:

[0223] M = Σ(w_ij×T(h_i, h_j, e_ij)) ,

[0224] Where w_ij is the edge weight, e_ij is the edge feature, and T is the feature transformation function. This mechanism can accurately model the conversion relationship between emotional states.

[0225] This embodiment designs a hierarchical graph pooling operation. The pooling process aggregates features based on the importance scores of nodes:

[0226] S = sigmoid(MLP(H))

[0227] H_pool = S T×H,

[0228] Where S is the node score vector and H is the node feature matrix. This pooling method retains the feature information of key nodes.

[0229] This embodiment implements a global feature extraction mechanism. The graph-level feature representation is generated through the attention-weighted readout operation:

[0230] F_graph = Σ(α_i×h_i),

[0231] Where α_i is the attention weight of the node and h_i is the node feature. This method can capture the overall emotional state characteristics.

[0232] Through the above technological innovations, this embodiment effectively solves the problem of deep integration and structured representation of emotional features in medical monitoring scenarios. This solution accurately captures the complex relationships between emotional features and generates feature representations rich in semantic information. In practical applications, this approach significantly improves the accuracy and interpretability of emotional state recognition.

[0233] The innovations of this embodiment primarily lie in the feature fusion strategy, knowledge graph construction, and graph neural network design. Through a carefully designed feature processing process, high-quality emotion feature representation is achieved in medical monitoring scenarios. This solution provides important support for improving the performance of emotion recognition systems and has broad application prospects in the field of medical monitoring. Experimental verification demonstrates that this method can accurately understand and represent complex emotional states, showing significant advantages, particularly in medical scenarios requiring long-term continuous monitoring.

[0234] In one embodiment of the emotion recognition method based on artificial intelligence of the present application, the following contents may also be specifically included:

[0235] Step S601: extracting the target subject's past emotional records from a medical record database, sorting the past emotional records in chronological order to construct a time series dataset, constructing a time series model of emotional change based on a long short-term memory network, inputting the time series dataset into the time series model for training, learning the temporal dependency of emotional states through a gating mechanism, and optimizing the model parameters using a gradient descent method to obtain a trained time series model of emotional change;

[0236] Step S602: Concatenate the emotional state feature vector with the historical emotional state sequence to obtain sequence input data, and input the sequence input data into the trained emotional change time series model. The emotional change time series model extracts key time series features in the sequence data based on the attention mechanism, and dynamically models the emotional state through the hidden state transfer mechanism to obtain the emotional state evaluation result.

[0237] Optionally, this embodiment innovatively constructs an emotion record extraction mechanism based on electronic medical records, targeting the dynamic evolution of emotional states in medical monitoring scenarios. First, natural language processing technology is used to identify emotion-related descriptions from unstructured medical record text, and named entity recognition methods are used to extract emotion tags and timestamp information. The extraction process uses a bidirectional conditional random field model:

[0238] P(y|x) = exp(Σλ_k×f_k(y,x)) / Z(x),

[0239] Where y is the label sequence, x is the input text, f_k is the feature function, λ_k is the weight parameter, and Z(x) is the normalization factor.

[0240] This embodiment implements the standardization of time series data. The extracted emotion records are time-aligned and interpolated:

[0241] T_aligned = Interpolate(T_raw, t),

[0242] Where T_raw is the original time series, t is the standard time interval. This process ensures the continuity and equal interval of time series data.

[0243] This embodiment optimizes the structural design of the LSTM network. The network contains three control units: input gate, forget gate, and output gate:

[0244] ,

[0245] ,

[0246] ,

[0247] Where σ is the sigmoid activation function, W is the weight matrix, and b is the bias term. This gating mechanism can effectively control the flow of information and the updating of memory.

[0248] This embodiment innovatively designs the update rules for memory units. The update of unit status takes emotional inertia into account:

[0249] c_t = f_t× c_(t-1) + i_t×tanh(W_c×[h_(t-1),x_t] + b_c) ,

[0250] h_t = o_t×tanh(c_t),

[0251] Where c_t is the cell state and h_t is the hidden state. This update mechanism can capture the persistence and gradual changes of emotional states.

[0252] This embodiment implements a gradient clipping strategy. To prevent gradient explosion, a norm constraint is used:

[0253] g = min(L, ||g||)×g / ||g||,

[0254] Where g is the gradient vector and L is the threshold. This process ensures the stability of the training process.

[0255] This embodiment builds a sequence splicing mechanism to dynamically merge the current emotional features with the historical sequence:

[0256] X_seq = Concat([h_history; f_current]),

[0257] Where h_history is the historical hidden state sequence and f_current is the current feature. This design enables the model to comprehensively consider historical information and current state.

[0258] This embodiment optimizes the temporal attention calculation. The attention weight is based on the query-key value mechanism:

[0259] α_t = softmax(v T×tanh(W_1×h_t + W_2×h_query)),

[0260] Where h_t is the historical hidden state and h_query is the query vector. This mechanism can adaptively select relevant historical information.

[0261] This embodiment designs a hidden state transmission mechanism. The state update uses a gated update unit:

[0262] ,

[0263] ,

[0264] ，

[0265] ,

[0266] Where z_t is the update gate and r_t is the reset gate. This mechanism can flexibly control the degree of state update.

[0267] This embodiment realizes the probability output of state evaluation. The emotional state distribution is obtained through multi-layer perceptron mapping:

[0268] P(s|h) = softmax(MLP(h)) ,

[0269] Where s is the emotional state category and h is the hidden state feature. This output form provides an uncertainty estimate of the emotional state assessment.

[0270] Through the above technological innovations, this embodiment effectively addresses the critical issue of dynamic emotional state assessment in medical monitoring scenarios. This solution accurately models the temporal dependencies of emotional states, enabling the prediction and assessment of emotional changes. In practical applications, this approach significantly improves the foresight and accuracy of emotional monitoring.

[0271] The innovations of this embodiment primarily lie in time series modeling, attention mechanisms, and state assessment. Through a carefully designed algorithmic architecture, high-quality emotional state assessment is achieved in medical monitoring scenarios. This solution provides important support for improving the intelligence level of medical monitoring systems and has broad application prospects in the healthcare field. Experimental verification demonstrates that this method can accurately predict patient emotional trends, showing significant advantages in medical scenarios requiring long-term continuous monitoring.

[0272] In one embodiment of the emotion recognition method based on artificial intelligence of the present application, the following contents may also be specifically included:

[0273] Step S701: collecting physiological indicator data of the target subject, including heart rate, blood pressure, and body temperature, normalizing the physiological indicator data to obtain physiological feature vectors, fusing the physiological feature vectors with the emotional state assessment results, and constructing a physiological indicator correction model based on a deep neural network. The physiological indicator correction model uses a multi-layer fully connected structure to perform nonlinear transformation on the fused features, and outputs the corrected emotional state assessment results through a soft maximization function;

[0274] Step S702: Input the corrected emotional state assessment result into the medical monitoring model, the medical monitoring model classifies the emotional state based on a decision tree structure, selects corresponding monitoring instructions from a preset monitoring strategy library according to the classification result, converts the monitoring instructions into device control signals, and sends the device control signals to the medical monitoring device through the communication interface for monitoring operation control.

[0275] Optionally, this embodiment innovatively designs a physiological indicator acquisition system for the emotional state assessment and intervention needs in medical monitoring scenarios. Medical-grade sensors are used to monitor the target object's physiological indicators in real time, including heart rate variability (HRV) characteristics, blood pressure fluctuation patterns, and body temperature change trends. The heart rate data sampling frequency is set to 250Hz to capture subtle changes in heart rhythm: HRV = {SDNN, RMSSD, pNN50} where SDNN is the standard deviation of adjacent heart beat intervals, RMSSD is the root mean square of the difference between adjacent heart beat intervals, and pNN50 is the proportion of adjacent heart beat interval differences greater than 50ms.

[0276] This embodiment realizes the normalization of physiological data. Adopting the adaptive normalization method:

[0277] X_norm = (X - μ_rolling) / σ_rolling,

[0278] where μ_rolling and σ_rolling are the mean and standard deviation within the sliding window. The window size is dynamically adjusted based on the changing characteristics of the indicator, ensuring the adaptability of the normalization process.

[0279] This embodiment optimizes the feature fusion strategy and performs multi-level fusion of physiological feature vectors and emotional state assessment results:

[0280] F_fusion = Concat(F_phys, F_emotion)×W_fusion,

[0281] Where F_phys is the physiological feature, F_emotion is the emotional feature, and W_fusion is the fusion weight matrix. This fusion method ensures full interaction between the two types of features.

[0282] This embodiment innovatively designs a physiological index correction model. The model adopts an improved multi-layer perceptron structure:

[0283] H_i = LeakyReLU(BN(W_i×H_(i-1) + b_i)) ,

[0284] BN is a batch normalization layer, and LeakyReLU is a leaky rectified linear unit. This structure can learn the nonlinear relationship between physiological indicators and emotional states.

[0285] This embodiment implements an adaptive inter-layer connection mechanism. Residual connections and attention modules are added between each layer:

[0286] F_out = α×H + (1-α)×F_shortcut,

[0287] Where α is the attention weight and F_shortcut is the skip connection feature. This design improves the expressiveness and training stability of the model.

[0288] This embodiment builds an innovative medical monitoring model. Based on the gradient boosted decision tree (GBDT) structure, the splitting criteria of each tree node considers multiple dimensions:

[0289] Split_score = I_emotion + λ1×I_physio + λ2×I_risk ,

[0290] Where I_emotion, I_physio, and I_risk are the information gains of emotion, physiology, and risk factors, respectively, and λ is the weight coefficient.

[0291] This embodiment optimizes the monitoring strategy selection mechanism. The strategy library adopts a hierarchical structure, including three levels: emergency intervention, routine monitoring, and preventive measures. Strategy selection is based on the comprehensive score of the decision path:

[0292] Score = w1×S_urgency + w2×S_effectiveness + w3×S_cost,

[0293] Where S_urgency is the urgency, S_effectiveness is the expected effect, and S_cost is the resource consumption.

[0294] This embodiment implements a mechanism for generating device control instructions. The strategy is converted into standardized control instructions:

[0295] CMD = Encode(Strategy, Device_type) ,

[0296] Strategy is the selected monitoring strategy, and Device_type is the device type. The encoding process ensures the accuracy and compatibility of the instructions.

[0297] This embodiment designs a reliable communication interface and employs a multi-backup data transmission mechanism: T(t) = Primary(t) || Backup(t), where Primary and Backup are the primary and backup transmission channels, respectively. This design ensures reliable transmission of control commands.

[0298] Through the above technological innovations, this embodiment effectively addresses the challenges of accurate emotional state assessment and timely intervention in medical monitoring scenarios. This solution reliably calibrates emotional states based on physiological indicators and generates appropriate monitoring instructions in a timely manner. In practical applications, this approach significantly improves the intelligence level of medical monitoring and the effectiveness of interventions.

[0299] The innovations of this embodiment primarily lie in physiological data processing, state correction, and monitoring decision-making. Through a carefully designed processing flow, intelligent emotion management is achieved in medical monitoring scenarios. This solution provides important support for improving the automation level of medical monitoring and has broad application prospects in the field of smart healthcare. Experimental verification has demonstrated that this method can accurately assess a patient's emotional state and implement appropriate monitoring measures in a timely manner, showing significant advantages, particularly in medical scenarios requiring close monitoring.

[0300] In order to effectively address the deficiencies of traditional technologies in multimodal fusion, time series modeling, and medical monitoring, and significantly improve the accuracy and practical value of emotion recognition in medical scenarios, this application provides an embodiment of an artificial intelligence-based emotion recognition device for implementing all or part of the artificial intelligence-based emotion recognition method, see Figure 2 The artificial intelligence-based emotion recognition device specifically includes the following contents:

[0301] The data acquisition module 10 is configured to, after obtaining authorization, collect facial image data, voice data, and body movement data of a target object, obtain a facial image sequence of the target object through an image acquisition device, obtain a voice signal of the target object through a voice acquisition device and extract voiceprint features, obtain a body movement sequence of the target object through a motion acquisition device, verify the identity of the target object of the voice signal based on the voiceprint features, and input the facial image sequence, the voice signal, and the body movement sequence into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and motion emotion features;

[0302] A feature processing module 20 is configured to construct a multi-level attention fusion network to perform multimodal feature fusion on the facial emotion features, the voice emotion features, and the action emotion features. The multi-level attention fusion network calculates the attention weights of the modal internal features and the cross-modal features through a feature attention layer and a modal attention layer, performs weighted fusion on the features based on the attention weights to obtain a cross-modal feature representation, and extracts the emotional state features from the cross-modal feature representation.

[0303] The emotion recognition module 30 is used to establish an emotion change time series model based on the previous emotion records contained in the medical history data of the target object, input the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, correct the emotion state evaluation result based on the physiological indicator data of the target object, input the corrected emotion state evaluation result into the medical monitoring model to generate a monitoring instruction, and control the medical monitoring equipment to perform corresponding monitoring operations according to the monitoring instruction.

[0304] From the above description, it can be seen that the artificial intelligence-based emotion recognition device provided in the embodiment of the present application can realize identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios.

[0305] From a hardware perspective, in order to effectively address the shortcomings of traditional technologies in multimodal fusion, time series modeling, and medical monitoring, and significantly improve the accuracy and practical value of emotion recognition in medical scenarios, this application provides an embodiment of an electronic device for implementing all or part of the content of the artificial intelligence-based emotion recognition method. The electronic device specifically includes the following content:

[0306] A processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to transmit information between the artificial intelligence-based emotion recognition device and related devices such as core business systems, user terminals, and related databases; the logic controller can be a desktop computer, a tablet computer, a mobile terminal, etc., but this embodiment is not limited thereto. In this embodiment, the logic controller can be implemented with reference to the embodiments of the artificial intelligence-based emotion recognition method and the embodiments of the artificial intelligence-based emotion recognition device in the embodiments, the contents of which are incorporated herein and repeated parts are not repeated.

[0307] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0308] In practical applications, portions of the AI-based emotion recognition method can be executed on the electronic device side as described above, or all operations can be performed on the client device. The specific selection can be based on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any restrictions on this. If all operations are performed on the client device, the client device may also include a processor.

[0309] The aforementioned client device may include a communication module (i.e., a communication unit) capable of establishing a communication connection with a remote server to facilitate data transmission with the server. The server may include a server at the task scheduling center or, in other implementation scenarios, a server on an intermediate platform, such as a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may comprise a single computer device, a server cluster consisting of multiple servers, or a distributed server configuration.

[0310] Figure 3 Schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Figure 3 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that the Figure 3 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0311] In one embodiment, the AI-based emotion recognition method function may be integrated into the central processing unit 9100. The central processing unit 9100 may be configured to perform the following control:

[0312] Step S101: After obtaining authorization, facial image data, voice data, and body movement data of the target object are collected, a facial image sequence of the target object is obtained through an image acquisition device, a voice signal of the target object is obtained through a sound acquisition device and voiceprint features are extracted, and a body movement sequence of the target object is obtained through a movement acquisition device. The identity of the target object of the voice signal is verified based on the voiceprint features, and the facial image sequence, the voice signal, and the body movement sequence are respectively input into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and movement emotion features;

[0313] Step S102: constructing a multi-level attention fusion network to perform multimodal feature fusion on the facial emotion features, the voice emotion features, and the action emotion features. The multi-level attention fusion network calculates the attention weights of the modal internal features and the cross-modal features through the feature attention layer and the modal attention layer, performs weighted fusion on the features based on the attention weights to obtain a cross-modal feature representation, and extracts the emotional state features from the cross-modal feature representation.

[0314] Step S103: Establish an emotion change time series model based on the previous emotion records contained in the medical history data of the target object, input the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, correct the emotion state evaluation result based on the physiological indicator data of the target object, input the corrected emotion state evaluation result into the medical monitoring model to generate a monitoring instruction, and control the medical monitoring equipment to perform corresponding monitoring operations according to the monitoring instruction.

[0315] As can be seen from the above description, the electronic device provided in the embodiment of the present application realizes identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios.

[0316] In another embodiment, the artificial intelligence-based emotion recognition device can be configured separately from the central processing unit 9100. For example, the artificial intelligence-based emotion recognition device can be configured as a chip connected to the central processing unit 9100, and the artificial intelligence-based emotion recognition method function can be implemented through the control of the central processing unit.

[0317] like Figure 3 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Figure 3 In addition, the electronic device 9600 may also include all components shown in Figure 3 For components not shown, reference may be made to the prior art.

[0318] like Figure 3 As shown, the central processing unit 9100 is sometimes also referred to as a controller or operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0319] Memory 9140 can be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It can store the aforementioned failure-related information and also store programs that execute the relevant information. The CPU 9100 can execute the programs stored in memory 9140 to implement information storage or processing.

[0320] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 may be, for example, a keypad or touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0321] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), or SIM card. Alternatively, it may be a memory that retains information even when power is off, can be selectively erased, and is capable of storing additional data. Examples of such memory are sometimes referred to as EPROMs. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs, or processes used by the central processing unit 9100 to execute operations of the electronic device 9600.

[0322] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, images, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various driver programs for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0323] The communication module 9110 is a transmitter / receiver that transmits and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0324] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless local area network modules. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130, providing audio output via the speaker 9131 and receiving audio input from the microphone 9132, thereby implementing common telecommunication functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 9130 is coupled to the central processing unit 9100, enabling local recording via the microphone 9132 and playback of stored audio via the speaker 9131.

[0325] Embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps of the artificial intelligence-based emotion recognition method in the above-mentioned embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the computer program implements all steps of the artificial intelligence-based emotion recognition method in the above-mentioned embodiment, where the execution subject is a server or a client. For example, when the processor executes the computer program, the following steps are implemented:

[0326] Step S101: After obtaining authorization, facial image data, voice data, and body movement data of the target object are collected, a facial image sequence of the target object is obtained through an image acquisition device, a voice signal of the target object is obtained through a sound acquisition device and voiceprint features are extracted, and a body movement sequence of the target object is obtained through a movement acquisition device. The identity of the target object of the voice signal is verified based on the voiceprint features, and the facial image sequence, the voice signal, and the body movement sequence are respectively input into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and movement emotion features;

[0327] Step S102: constructing a multi-level attention fusion network to perform multimodal feature fusion on the facial emotion features, the voice emotion features, and the action emotion features. The multi-level attention fusion network calculates the attention weights of the modal internal features and the cross-modal features through the feature attention layer and the modal attention layer, performs weighted fusion on the features based on the attention weights to obtain a cross-modal feature representation, and extracts the emotional state features from the cross-modal feature representation.

[0328] Step S103: Establish an emotion change time series model based on the previous emotion records contained in the medical history data of the target object, input the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, correct the emotion state evaluation result based on the physiological indicator data of the target object, input the corrected emotion state evaluation result into the medical monitoring model to generate a monitoring instruction, and control the medical monitoring equipment to perform corresponding monitoring operations according to the monitoring instruction.

[0329] As can be seen from the above description, the computer-readable storage medium provided in the embodiment of the present application realizes identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios.

[0330] The embodiments of the present application also provide a computer program product capable of implementing all steps of the artificial intelligence-based emotion recognition method in the above-mentioned embodiments, where the execution subject is a server or a client. When the computer program / instructions are executed by a processor, the computer program / instructions implement the steps of the artificial intelligence-based emotion recognition method. For example, the computer program / instructions implement the following steps:

[0331] Step S101: After obtaining authorization, facial image data, voice data, and body movement data of the target object are collected, a facial image sequence of the target object is obtained through an image acquisition device, a voice signal of the target object is obtained through a sound acquisition device and voiceprint features are extracted, and a body movement sequence of the target object is obtained through a movement acquisition device. The identity of the target object of the voice signal is verified based on the voiceprint features, and the facial image sequence, the voice signal, and the body movement sequence are respectively input into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and movement emotion features;

[0332] Step S102: constructing a multi-level attention fusion network to perform multimodal feature fusion on the facial emotion features, the voice emotion features, and the action emotion features. The multi-level attention fusion network calculates the attention weights of the modal internal features and the cross-modal features through the feature attention layer and the modal attention layer, performs weighted fusion on the features based on the attention weights to obtain a cross-modal feature representation, and extracts the emotional state features from the cross-modal feature representation.

[0333] Step S103: Establish an emotion change time series model based on the previous emotion records contained in the medical history data of the target object, input the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, correct the emotion state evaluation result based on the physiological indicator data of the target object, input the corrected emotion state evaluation result into the medical monitoring model to generate a monitoring instruction, and control the medical monitoring equipment to perform corresponding monitoring operations according to the monitoring instruction.

[0334] As can be seen from the above description, the computer program product provided in the embodiment of the present application realizes identity authentication through voiceprint features by innovatively integrating multimodal data of facial images, voice and body movements. A multi-level attention fusion network is designed, and the feature attention layer and the modal attention layer are used to realize the intelligent fusion of intra-modal and cross-modal features. A time series model of emotion changes is constructed in combination with medical record data, and physiological indicators are incorporated for evaluation and correction to realize dynamic evaluation and precise monitoring of emotional states. This method effectively solves the shortcomings of traditional technologies in multimodal fusion, time series modeling and medical monitoring, and significantly improves the accuracy and practical value of emotion recognition in medical scenarios.

[0335] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatuses, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0336] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0337] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0338] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0339] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. An emotion recognition method based on artificial intelligence, characterized in that: The method comprises: After obtaining authorization, facial image data, voice data, and body movement data of the target object are collected, a facial image sequence of the target object is obtained through an image acquisition device, a voice signal of the target object is acquired through a sound acquisition device and voiceprint features are extracted, a body movement sequence of the target object is acquired through a motion acquisition device, the identity of the target object of the voice signal is verified based on the voiceprint features, and the facial image sequence, the voice signal, and the body movement sequence are respectively input into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and action emotion features; Constructing a feature attention layer to perform attention calculations on the facial emotion features, the voice emotion features, and the action emotion features, respectively. The feature attention layer uses a self-attention mechanism to calculate the correlation between internal features of the same modality to obtain a feature attention weight matrix. Based on the feature attention weight matrix, each modal feature is weighted to obtain a modal internal feature representation. The modal internal feature representation is normalized to generate a feature-level fusion result. Constructing a modal attention layer to perform cross-modal feature fusion on the feature-level fusion results. The modal attention layer uses a cross-attention mechanism to calculate the interaction relationship between different modal features to obtain a modal attention coefficient matrix. The feature-level fusion results are weighted based on the modal attention coefficient matrix to obtain a cross-modal feature representation. Constructing an emotion feature knowledge graph encoder to perform graph structure encoding on the cross-modal feature representation, wherein the feature nodes in the graph include basic emotion categories and complex emotion states, and the edges represent the conversion relationship and influence intensity between emotions. The emotion feature knowledge graph encoder models the relationship between feature nodes based on a graph neural network, updates the node feature representation through a message passing mechanism, and uses a graph pooling operation to aggregate the updated node features to obtain the emotion state feature; An emotion change time series model is established based on the previous emotion records contained in the medical history data of the target object, the emotion state characteristics are input into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, the emotion state evaluation result is corrected based on the physiological indicator data of the target object, the corrected emotion state evaluation result is input into the medical monitoring model to generate monitoring instructions, and the medical monitoring equipment is controlled to perform corresponding monitoring operations according to the monitoring instructions.

2. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The method of collecting facial image data, voice data, and body movement data of a target object, obtaining a facial image sequence of the target object through an image acquisition device, obtaining a voice signal of the target object and extracting voiceprint features through a sound acquisition device, and obtaining a body movement sequence of the target object through a movement acquisition device, includes: Acquire a facial image sequence of a target subject by an image acquisition device installed in a medical monitoring location, sample the facial image sequence at a preset frame rate, perform face detection and alignment on the sampled images, extract facial region images to construct facial image data, and extract facial feature point coordinates from the facial image data based on a deep neural network to generate facial feature point trajectory data; The voice signal of the target object is collected by a sound collection device installed in the medical monitoring place, the voice signal is preprocessed to remove background noise, the voiceprint features and acoustic feature parameters of the voice signal are extracted, and the target object identity is authenticated by the voiceprint features based on a deep neural network. The limb movement image of the target object is collected by a motion collection device installed in the medical monitoring place, the limb movement image is detected with skeleton key points, and the key point coordinates are extracted to generate motion sequence data.

3. The emotion recognition method based on artificial intelligence according to claim 2, characterized in that: The method of verifying the identity of the target object of the voice signal based on the voiceprint feature, inputting the facial image sequence, the voice signal and the body movement sequence into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features and movement emotion features, includes: Inputting the voiceprint features into a voiceprint verification model to identify the target object, the voiceprint verification model calculates a similarity score between the input voiceprint features and the pre-stored target object voiceprint features based on a twin neural network structure, and determines the target object's identity based on the similarity score; inputting the facial feature point trajectory data into a facial emotion recognition model, the facial emotion recognition model uses a spatiotemporal convolutional network structure to extract facial expression temporal features, and weights the facial expression temporal features based on an attention mechanism to obtain facial emotion features; The authenticated voice signal is input into the voice emotion recognition model, and the voice emotion recognition model uses a recurrent neural network structure to perform time series modeling on the acoustic feature parameters of the voice signal to obtain voice emotion features. The action sequence data is input into the action emotion recognition model, and the action emotion recognition model uses a graph convolutional network structure to extract spatiotemporal features of the skeleton key point sequence to obtain action emotion features.

4. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The step of establishing an emotion change time series model based on the past emotion records contained in the medical history data of the target subject, and inputting the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result includes: Extracting the target subject's past emotional records from a medical record database, sorting the past emotional records in chronological order to construct a time series data set, constructing a time series model of emotional change based on a long short-term memory network, inputting the time series data set into the time series model of emotional change for training, learning the temporal dependency of emotional states through a gating mechanism, and optimizing the model parameters using a gradient descent method to obtain a trained time series model of emotional change; The emotional state feature vector is spliced ​​with the historical emotional state sequence to obtain sequence input data, and the sequence input data is input into the trained emotional change time series model. The emotional change time series model extracts key time series features in the sequence data based on the attention mechanism, and dynamically models the emotional state through the hidden state transfer mechanism to obtain the emotional state evaluation result.

5. The emotion recognition method based on artificial intelligence according to claim 1, characterized in that: The step of correcting the emotional state assessment result based on the physiological indicator data of the target subject, inputting the corrected emotional state assessment result into a medical monitoring model to generate a monitoring instruction, and controlling the medical monitoring device to perform a corresponding monitoring operation according to the monitoring instruction includes: Collecting physiological indicator data of the target subject, including heart rate, blood pressure, and body temperature, normalizing the physiological indicator data to obtain physiological feature vectors, fusing the physiological feature vectors with the emotional state assessment results, and constructing a physiological indicator correction model based on a deep neural network. The physiological indicator correction model uses a multi-layer fully connected structure to perform nonlinear transformation on the fused features, and outputs the corrected emotional state assessment results through a soft maximization function; The corrected emotional state assessment result is input into a medical monitoring model, which classifies the emotional state based on a decision tree structure, selects corresponding monitoring instructions from a preset monitoring strategy library according to the classification result, converts the monitoring instructions into device control signals, and sends the device control signals to the medical monitoring device through a communication interface for monitoring operation control.

6. An emotion recognition device based on artificial intelligence, characterized in that: The device comprises: a data acquisition module configured to, after obtaining authorization, collect facial image data, voice data, and body movement data of a target object, obtain a facial image sequence of the target object through an image acquisition device, obtain a voice signal of the target object through a voice acquisition device and extract voiceprint features, obtain a body movement sequence of the target object through a motion acquisition device, verify the identity of the target object of the voice signal based on the voiceprint features, and input the facial image sequence, the voice signal, and the body movement sequence into corresponding emotion recognition models for analysis to obtain facial emotion features, voice emotion features, and motion emotion features; A feature processing module is used to construct a feature attention layer to perform attention calculations on the facial emotion features, the voice emotion features, and the action emotion features respectively. The feature attention layer uses a self-attention mechanism to calculate the correlation between the internal features of the same modality to obtain a feature attention weight matrix. Based on the feature attention weight matrix, each modal feature is weighted to obtain a modal internal feature representation. The modal internal feature representation is normalized to generate a feature-level fusion result. A modal attention layer is constructed to perform cross-modal feature fusion on the feature-level fusion result. The modal attention layer uses a cross-attention mechanism to calculate the interaction between different modal features to obtain a modal attention coefficient matrix. Based on the modal attention coefficient matrix, the feature-level fusion result is weighted to obtain a cross-modal feature representation. An emotion feature knowledge graph encoder is constructed to perform graph structure encoding on the cross-modal feature representation, wherein the feature nodes in the graph include basic emotion categories and complex emotion states, and the edges represent the conversion relationship and influence intensity between emotions. The emotion feature knowledge graph encoder models the relationship between feature nodes based on a graph neural network, updates the node feature representation through a message passing mechanism, and adopts a graph pooling operation to aggregate the updated node features to obtain an emotion state feature. An emotion recognition module is used to establish an emotion change time series model based on the previous emotion records contained in the medical history data of the target object, input the emotion state characteristics into the emotion change time series model for dynamic evaluation to obtain an emotion state evaluation result, correct the emotion state evaluation result based on the physiological indicator data of the target object, input the corrected emotion state evaluation result into a medical monitoring model to generate a monitoring instruction, and control the medical monitoring equipment to perform corresponding monitoring operations according to the monitoring instruction.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the artificial intelligence-based emotion recognition method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the emotion recognition method based on artificial intelligence according to any one of claims 1 to 5 are implemented.

Images