An electroencephalogram emotion segment segmentation method based on human-in-the-loop

Abstract

The application discloses a human-in-the-loop electroencephalogram emotion segment segmentation method, comprising the following steps: step 1, electroencephalogram signal acquisition and preprocessing; step 2, initial manual annotation of emotion segments; step 3, emotion segment segmentation model training based on space-time joint modeling; step 4, active learning human-in-the-loop feedback and model iterative optimization; step 5, automatic emotion segment segmentation and emotion recognition. The human-in-the-loop electroencephalogram emotion segment segmentation method provided by the application can effectively alleviate problems such as electroencephalogram noise interference, individual differences and fuzzy emotion boundaries, and significantly improves the accuracy and robustness of emotion segment segmentation and recognition while reducing the cost of manual annotation.

Description

Technical Field

[0001] This invention belongs to the field of intelligent processing of EEG signals and emotion computing technology, and particularly relates to a method for segmenting EEG emotion segments based on human-in-circuit, which is applicable to scenarios of emotion recognition and psychological state assessment based on EEG signals. Background Technology

[0002] Electroencephalography (EEG) is a non-invasive signal acquisition technology used to monitor and analyze brain neural activity. It features high temporal resolution and real-time performance, making it valuable for applications in emotion recognition and psychological state assessment. EEG-based emotion recognition technology analyzes brain activity patterns under different emotional states to automatically identify an individual's emotional state, and has been widely applied in scenarios such as mental health monitoring, human-computer emotional interaction, and assisted diagnosis and treatment.

[0003] Current EEG emotion recognition methods typically rely on extracting time-domain, frequency-domain, or time-frequency-domain features from EEG signals and combining them with traditional machine learning models such as support vector machines and random forests to classify emotions. These methods are highly dependent on manual feature design, making it difficult to fully characterize the complex nonlinear properties of EEG signals, and are also sensitive to noise interference, thus limiting overall recognition performance. With the development of deep learning technology, some studies have begun to use convolutional neural networks and recurrent neural networks for automatic feature learning from EEG signals. By jointly modeling spatial distribution features and temporal dependencies, they have improved the accuracy of emotion recognition to some extent.

[0004] However, the above-mentioned schemes based on fully automated deep models still have significant shortcomings: First, the segments in EEG signals that are truly related to emotions usually only account for a small proportion of the continuous signal. Existing methods mostly adopt fixed window or overall modeling strategies, which makes it difficult to achieve fine localization of emotional segments and easily introduces a large amount of irrelevant or noisy information. Second, there are significant individual differences in the EEG responses of different subjects under the same emotional conditions, resulting in limited generalization ability of the model. Third, existing methods mostly ignore the potential spatial topological relationships and long temporal dependent structures between brain regions, making it difficult to fully explore the high-order spatiotemporal features in EEG signals.

[0005] To alleviate the aforementioned problems, the "human-in-the-loop" modeling concept has been proposed in recent years. This approach introduces human experts to annotate or correct key emotional segments, guiding the model training process and thus improving segmentation and recognition accuracy. However, existing methods mostly remain at the stage of simple manual correction or post-processing, failing to form a closed-loop optimization mechanism tightly coupled with deep models and thus not fully leveraging the role of expert feedback in sample selection, model updates, and generalization improvement. Therefore, there is an urgent need for an EEG emotional segment segmentation method that integrates brain region spatial structure modeling, long-term feature learning, and active human-in-the-loop feedback mechanisms to achieve high-precision and robust automatic localization and recognition of emotional segments. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide a human-in-the-loop EEG emotion segmentation method that can effectively alleviate problems such as EEG noise interference, individual differences and blurred emotion boundaries, and significantly improve the accuracy and robustness of emotion segmentation and recognition while reducing manual annotation costs.

[0007] To address the aforementioned technical problems, this invention provides a human-in-the-loop (HIL) EEG emotion segmentation method, comprising the following steps: Step 1: EEG signal acquisition and preprocessing; Step 2: Initial manual annotation of emotion segments; Step 3: Training of an emotion segmentation model based on spatial-temporal joint modeling; Step 4: Active learning of HIL feedback and iterative optimization of the model; Step 5: Automated emotion segmentation and emotion recognition.

[0008] Further, step 1 specifically includes the following sub-steps: (1.1) Obtaining the subject's EEG signal under the emotional stimulation task through a multi-lead EEG acquisition device, and performing bandpass filtering, electrooculography and electromyography artifact removal, and amplitude normalization on the original signal in sequence to ensure that the data scale of each lead is consistent; (1.2) Using a multi-scale sliding window mechanism to segment the preprocessed EEG signal, setting windows of different time lengths to extract emotion-related features synchronously to adapt to emotional response patterns of different durations.

[0009] Furthermore, the time length in the sub-step (1.2) is 1S, 2S or 4S.

[0010] Further, step 2 specifically includes the following sub-steps: (2.1) Conduct a preliminary review of the EEG data, and combine emotional stimulus labels to annotate key emotion-related segments to construct an initial training sample set; (2.2) Convert the manual annotation results into a supervised learning label format for subsequent model training.

[0011] Further, step 3 specifically includes the following sub-steps: (3.1) Constructing multichannel EEG signals into graph structure data, where nodes correspond to electrode channels, edge weights are determined by the correlation coefficient between channels, and using a graph attention network to extract spatial topological features of brain regions, thus defining the EEG structure. ,in: Represents the set of EEG channel nodes; This represents the connection relationship between channels, and the node features are represented as follows: The graph attention weights are calculated as follows: ; in, This represents the feature vector of the i-th EEG channel. Let be a trainable linear transformation matrix. For attention weight vectors, Represents a node The neighborhood set, This represents vector concatenation operations; node updates. (3.2) Spatial features are input into a temporal Transformer network, and the EEG time series is modeled through a multi-head self-attention mechanism to capture long-range dependencies. Self-attention is represented as: Bullish Attention: ;in, These represent the query, key, and value matrices, respectively. For the corresponding linear mapping parameters, The dimension of the key vector is represented; (3.3) the spatial-temporal features extracted under different scale windows are fused, and the emotion boundary prediction head is introduced to simultaneously estimate the start and end times of the emotion segment. The emotion boundary probability is defined as: The start and end of an emotional segment are determined by a threshold: Used to automatically locate the start and end times of emotional segments, forming a preliminary emotional segment segmentation model; Indicates time The high-level characteristics indicate that and For trainable parameters, This represents the Sigmoid function. This represents the probability that the current moment is the emotional boundary.

[0012] Furthermore, in step 3.1, the feature dimension The value range is 32-128, and the neighborhood set is... The connections were selected by filtering through the Pearson correlation coefficient between channels, retaining only those with correlation coefficients exceeding a preset threshold.

[0013] Furthermore, in step 3.2, the number of attention heads... The value range is 4-8, and the time window length is... The value range is the number of sampling points corresponding to 1–4 seconds; the threshold in step 3.3 The value range is 0.4–0.6.

[0014] Further, step 4 specifically includes the following sub-steps: (4.1) Based on the prediction uncertainty of the model output, high-value samples are automatically selected and submitted to experts for review. The expert correction results are fed back to the model in the form of a weighted loss function to achieve active learning human-in-the-loop optimization. The weighted training loss is defined as follows: The weights are: ; Indicates the true label, This indicates the model's prediction results. Represents the basic loss function. Indicates the sample weight; (4.2) At the same time, an abnormal EEG automatic detection module is introduced to remove abnormal segments whose reconstruction error exceeds the statistical threshold, and the feature distribution differences between different subjects are reduced by cross-subject domain adaptation constraints. After multiple iterations, the model's generalization ability and segmentation accuracy are continuously improved.

[0015] Furthermore, in step 4.1, the expert corrects the weight coefficients of the samples. The value range is 1.5–3.0.

[0016] Further, step 5 specifically includes the following sub-steps: (5.1) applying the optimized model to the newly acquired EEG signals to achieve automated and fine segmentation of emotional segments; (5.2) performing emotional state discrimination on the segmented segments based on an end-to-end deep network, and outputting multi-category emotion recognition results including pleasure, calmness and anxiety; (5.3) further applying the recognition results to the emotion assessment and personalized mental health management system.

[0017] Compared with existing technologies, this invention offers the following advantages: The human-in-the-loop (HIL) EEG emotion segmentation method provided by this invention achieves automatic and accurate localization and recognition of emotion-related EEG segments by integrating brain region spatial topology modeling, time-series deep feature learning, and an expert feedback mechanism driven by active learning. This method utilizes human expert participation in the dynamic correction of high-uncertainty samples to guide continuous iterative optimization of the model. Combined with strategies such as multi-scale feature fusion, emotion boundary perception, and abnormal EEG rejection, it effectively reduces noise interference and the impact of individual differences, thereby improving the accuracy of emotion segmentation, recognition stability, and overall system robustness. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the process of human-in-circuit EEG emotion segmentation based on the present invention; Figure 2 This is an implementation diagram of the present invention's method for segmenting emotional segments from electroencephalograms based on human circuits. Detailed Implementation

[0019] The present invention will now be further described with reference to the accompanying drawings and embodiments.

[0020] Figure 1 This is a schematic diagram of the structure of the EEG emotion segmentation method based on human circuits according to the present invention.

[0021] Please see Figure 1 The present invention provides a method for segmenting emotional segments from electroencephalograms based on human circuits, comprising the following steps: (1) EEG signal acquisition and preprocessing, including the following sub-steps: (1.1) The EEG signals of the subjects under the emotional stimulation task were acquired by a multi-lead EEG acquisition device. The raw signals were then subjected to bandpass filtering, removal of artifacts from electrooculography and electromyography, and amplitude normalization in sequence to improve signal quality and ensure that the data scale of each lead was consistent. (1.2) In the multi-scale sliding window segmentation process, different window lengths correspond to different time scales of emotional dynamic changes. To avoid emotional segments being truncated or over-smoothed due to improper window length selection, this embodiment uses multiple windows of different lengths (e.g., 1S, 2S, 4S) to segment the EEG signal in parallel, maintaining a certain proportion of overlap between adjacent windows to ensure that information during the transition phase of emotional changes is fully captured. Simultaneously, to avoid excessive differences in feature distribution between different window scales, features at different scales are uniformly normalized during the feature fusion stage to ensure the stability of subsequent model training.

[0022] (2) Initial manual annotation of emotional segments, including the following sub-steps: (2.1) The EEG data were initially reviewed by experts with experience in EEG analysis, and key emotion-related segments were labeled with emotion stimulus tags to construct a high-quality initial training sample set. (2.2) Convert the manually labeled results into a supervised learning label format for subsequent model training.

[0023] (3) Training the emotion segmentation model based on space-time joint modeling includes the following sub-steps: (3.1) Multichannel EEG signals are constructed as graph-structured data, where nodes correspond to electrode channels, and edge weights are determined by the correlation coefficients between channels. Graph attention networks are used to extract spatial topological features of brain regions. EEG structure definition. ,in: Represents the set of EEG channel nodes; This represents the connection relationship between channels, and the node features are represented as follows: The graph attention weights are calculated as follows: ; in, This represents the feature vector of the i-th EEG channel. Let be a trainable linear transformation matrix. For attention weight vectors, Represents a node The neighborhood set, This indicates a vector concatenation operation.

[0024] In this embodiment, feature dimension The preferred value range is 32-128. When the dimension is too low, it may lead to insufficient expression of EEG spatial features; when the dimension is too high, it may introduce redundant information and increase computational complexity, affecting the stability of model training.

[0025] Neighborhood set The optimal selection is achieved through screening using the Pearson correlation coefficient between channels, retaining only connections with high correlation. When the neighborhood range is too large, it may introduce noise interference; when the neighborhood range is too small, effective brain region connectivity information may be lost, thus affecting the spatial modeling results.

[0026] Node update It is used for adaptive learning of spatial dependencies between different EEG channels; (3.2) Spatial features are input into a temporal Transformer network, and EEG time series are modeled through a multi-head self-attention mechanism to capture long-range dependencies. Self-attention is represented as: Bullish Attention: .in, These represent the query, key, and value matrices, respectively. For the corresponding linear mapping parameters, This represents the dimension of the key vector.

[0027] In this embodiment, the number of attention heads The optimal value range is 4-8. When the number of heads is too small, the model will have difficulty capturing diverse time dependencies; when the number of heads is too large, it may lead to an excessive number of model parameters, which may cause overfitting.

[0028] Time window length The optimal value range is the number of sampling points corresponding to 1–4 seconds. When the window is too short, it is difficult to capture the complete emotional change process; when the window is too long, irrelevant state information may be introduced, reducing the accuracy of emotion segmentation.

[0029] (3.3) Spatial-temporal features extracted under different scale windows are fused, and an emotion boundary prediction head is introduced to simultaneously estimate the start and end times of emotion segments. The probability of the emotion boundary is defined as: The start and end of an emotional segment are determined by a threshold: This is used to automatically locate the start and end times of emotional segments, forming a preliminary emotional segmentation model. Among these, Indicates time The high-level characteristics indicate that and For trainable parameters, This represents the Sigmoid function. This represents the probability that the current moment is the emotional boundary.

[0030] threshold The preferred value range is 0.4–0.6. When the threshold is too low, too many non-emotional segments may be misjudged as boundaries; when the threshold is too high, the start and end positions of real emotional segments may be missed, thereby reducing the integrity of the segmentation.

[0031] By setting the parameters mentioned above, a balance can be achieved between the accuracy and recall rate of boundary detection.

[0032] It should be noted that the spatial topological feature modeling and temporal series modeling in this embodiment are not independent, but rather synergistically coupled through joint feature representation. Specifically, the graph attention network first models the spatial dependencies between EEG channels to obtain spatial feature representations of brain regions; then, this spatial feature is embedded as input into a temporal Transformer network for sequence modeling, allowing the model to maintain brain region topological information while learning temporal dependencies. Through this joint spatial-temporal modeling approach, both functional connectivity relationships between brain regions and dynamic emotional changes can be captured simultaneously, thereby significantly improving the accuracy of emotional segmentation.

[0033] (4) In this embodiment, the human-in-the-loop mechanism is not only used for result correction but also participates in the dynamic selection process of model training data. By evaluating the uncertainty of model prediction, only samples with high uncertainty are submitted for expert review, thus forming a closed-loop optimization process of "model prediction - expert feedback - model update". This mechanism can continuously improve model performance while maintaining low manual annotation costs and avoid the time and cost overhead of traditional full manual annotation. Active learning human-in-the-loop feedback and model iterative optimization include the following sub-steps: (4.1) Based on the prediction uncertainty of the model output, high-value samples are automatically selected and submitted to experts for review. The expert correction results are fed back to the model in the form of a weighted loss function to achieve active learning human-in-the-loop optimization. The weighted training loss is defined as follows: The weights are: ; in, Indicates the true label, This indicates the model's prediction results. This represents the basic loss function (such as cross-entropy loss). This represents the sample weight.

[0034] In this embodiment, the expert corrects the weighting coefficients of the samples. The preferred value range is 1.5–3.0. When the weight is too low, experts report that it has little effect on model optimization; when the weight is too high, it may cause the model to overfit a small number of artificial samples, thereby reducing the overall generalization ability.

[0035] By assigning differentiated weights to different samples, the model's ability to learn key emotional segments can be effectively improved, achieving efficient closed-loop optimization of human-in-the-loop. (4.2) At the same time, an automatic abnormal EEG detection module is introduced to remove abnormal segments whose reconstruction error exceeds the statistical threshold. The differences in feature distribution among different subjects are reduced by cross-subject domain adaptation constraints. Through multiple iterations, the model's generalization ability and segmentation accuracy are continuously improved.

[0036] (5) Automated emotion segmentation and emotion recognition, including the following sub-steps: (5.1) The optimized model is applied to newly acquired EEG signals to achieve automated and precise segmentation of emotional segments; (5.2) Based on an end-to-end deep network, the segmented segments are used to determine the emotional state and output the recognition results of multiple categories of emotions such as pleasure, calmness, and anxiety. (5.3) The identification results will be further applied to the emotion assessment and personalized mental health management system.

[0037] Example Please continue reading. Figure 2 This embodiment uses multi-lead electroencephalogram (EEG) data as input, collecting continuous EEG signals during the subject's experience with different emotional stimuli. First, the raw EEG signals are sequentially processed with bandpass filtering, EEG and EMG artifact removal, and amplitude normalization to reduce noise interference and ensure consistent data scale across channels. Then, a multi-scale sliding window is used to segment the preprocessed EEG signals, simultaneously extracting emotion-related features at different time scales.

[0038] In the feature modeling stage, multi-channel EEG signals are constructed as graph-structured data, with electrode channels as nodes and inter-channel correlations as edge weights input to a graph attention network to extract spatial topological features of brain regions. Simultaneously, these spatial features are fed into a temporal Transformer network, where a multi-head self-attention mechanism is used to model the long-range dependencies of the EEG sequences, and multi-scale features are fused to form a joint spatial-temporal representation. Based on this, an emotion boundary prediction module is introduced to simultaneously estimate the start and end times of emotion segments, yielding preliminary emotion segment segmentation results.

[0039] During model training, an active learning human-in-the-loop mechanism is introduced. Based on the uncertainty of the model output, high-value samples are automatically selected and submitted to experts for review. The expert corrections are then fed back to the network parameters in the form of a weighted loss, achieving closed-loop iterative optimization of the model. Simultaneously, an autoencoder is used to assess the reconstruction error of EEG segments, automatically removing abnormal segments exceeding statistical thresholds. Furthermore, cross-subject domain adaptation constraints are used to reduce the differences in feature distribution among different subjects, thereby improving the model's robustness and generalization ability in complex scenarios.

[0040] After multiple rounds of human-loop optimization, the trained model was applied to newly collected EEG data to achieve automatic and precise segmentation of emotional segments, and the corresponding emotion category results were output through an end-to-end deep network. The final segmentation and recognition results can be further used in emotion assessment and personalized mental health management systems.

[0041] In actual experiments, it was found that by introducing a human-in-the-loop mechanism involving joint spatial-temporal modeling and active learning, not only can the segmentation accuracy of emotional segments be improved, but stable recognition performance can also be maintained with fewer manually labeled samples. Especially in cross-subject testing scenarios, the method of this invention can effectively mitigate the impact of differences in EEG patterns between different individuals, enabling the model to maintain good emotion recognition performance on newly encountered subject data. This performance is significantly better than traditional emotion recognition methods that rely solely on automatic learning or purely manual annotation.

[0042] In summary, the human-in-the-loop (HIL) EEG emotion segmentation method provided by this invention automatically extracts high-order spatiotemporal features of EEG signals by introducing a deep network structure combining brain region spatial topology modeling and temporal Transformer. Furthermore, it integrates multi-scale windows and emotion boundary perception mechanisms to collaboratively model emotional responses of varying durations, thereby improving the accuracy of starting and ending times of emotion segments. During model training, an active learning HIL feedback mechanism is further introduced, incorporating expert correction only for samples with high prediction uncertainty, and participating in model updates in the form of weighted loss, thus continuously improving model performance while reducing manual annotation costs. Simultaneously, an automatic abnormal EEG detection module removes noisy segments, and cross-subject domain adaptation constraints mitigate the impact of individual differences, enhancing the system's robustness and generalization ability in complex real-world scenarios. Based on these technical means, this invention constructs a collaborative framework of "spatial-temporal deep modeling—HIL closed-loop optimization—automatic emotion recognition," achieving integrated processing of emotion segment segmentation and recognition.

[0043] The human-in-the-loop mechanism proposed in this invention improves the accuracy of emotional segmentation while significantly enhancing system stability and intelligence, making it suitable for large-scale EEG data analysis and mental health-related applications.

[0044] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be defined by the claims.

Claims

1. A method for segmenting emotional segments from electroencephalograms based on human-circuit mapping, characterized in that, Includes the following steps: Step 1: EEG signal acquisition and preprocessing; Step 2: Initial manual annotation of emotional segments; Step 3: Training the emotion segmentation model based on joint spatial-temporal modeling; Step 4: Active learning, human-in-the-loop feedback, and iterative model optimization; Step 5: Automated emotional segmentation and emotion recognition.

2. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 1, characterized in that, Step 1 specifically includes the following sub-steps: (1.1) The EEG signals of the subjects under the emotional stimulation task were acquired by the multi-lead EEG acquisition device. The raw signals were then subjected to bandpass filtering, removal of artifacts from electrooculography and electromyography, and amplitude normalization to ensure that the data scale of each lead remained consistent. (1.2) A multi-scale sliding window mechanism was used to segment the preprocessed EEG signal and set windows of different durations to extract emotion-related features simultaneously, in order to adapt to emotional response patterns of different durations.

3. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 2, characterized in that, The time length in the sub-step (1.2) is 1S, 2S or 4S.

4. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 1, characterized in that, Step 2 specifically includes the following sub-steps: (2.1) Conduct a preliminary review of the EEG data, and combine emotional stimulus labels to annotate key emotion-related segments to construct an initial training sample set; (2.2) Convert the manually labeled results into a supervised learning label format for subsequent model training.

5. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 1, characterized in that, Step 3 specifically includes the following sub-steps: (3.1) Construct multichannel EEG signals into graph-structured data, where nodes correspond to electrode channels, and edge weights are determined by the correlation coefficients between channels. Use graph attention networks to extract spatial topological features of brain regions. EEG structure definition. ,in: Represents the set of EEG channel nodes; This represents the connection relationship between channels, and the node features are represented as follows: The graph attention weights are calculated as follows: ； in, This represents the feature vector of the i-th EEG channel. Let be a trainable linear transformation matrix. For attention weight vectors, Represents a node The neighborhood set, This represents a vector concatenation operation; Node update It is used for adaptive learning of spatial dependencies between different EEG channels; (3.2) Spatial features are input into a temporal Transformer network, and EEG time series are modeled through a multi-head self-attention mechanism to capture long-range dependencies. Self-attention is represented as: Bullish Attention: ;in, These represent the query, key, and value matrices, respectively. For the corresponding linear mapping parameters, Indicates the dimension of the key vector; (3.3) The spatial-temporal features extracted under different scale windows are fused, and an emotion boundary prediction head is introduced to simultaneously estimate the start and end times of emotion segments. The emotion boundary probability is defined as: The start and end of an emotional segment are determined by a threshold: Used to automatically locate the start and end times of emotional segments, forming a preliminary emotional segment segmentation model; Indicates time The high-level characteristics indicate that and For trainable parameters, This represents the Sigmoid function. This represents the probability that the current moment is the emotional boundary.

6. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 5, characterized in that, The feature dimension in step 3.1 The value range is 32-128, and the neighborhood set is... The connections were selected by filtering through the Pearson correlation coefficient between channels, retaining only those with correlation coefficients exceeding a preset threshold.

7. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 5, characterized in that, In step 3.2, the number of attention heads The value range is 4-8, and the time window length is... The value range is the number of sampling points corresponding to 1–4 seconds; the threshold in step 3.3 The value range is 0.4–0.

6.

8. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 1, characterized in that, Step 4 specifically includes the following sub-steps: (4.1) Based on the prediction uncertainty of the model output, high-value samples are automatically selected and submitted to experts for review. The expert correction results are fed back to the model in the form of a weighted loss function to achieve active learning human-in-the-loop optimization. The weighted training loss is defined as follows: The weights are: ； in, Indicates the true label, This indicates the model's prediction results. Represents the basic loss function. Indicates sample weights; (4.2) At the same time, an abnormal EEG automatic detection module is introduced to remove abnormal segments whose reconstruction error exceeds the statistical threshold. The feature distribution differences between different subjects are reduced by cross-subject domain adaptation constraints. Through multiple iterations, the generalization ability and segmentation accuracy of the model are continuously improved.

9. The method for segmenting EEG emotion segments based on human-in-the-loop as described in claim 8, characterized in that, In step 4.1, the expert corrects the weight coefficients of the sample. The value range is 1.5–3.

0.

10. The method for segmenting EEG emotion segments based on human-in-circuit as described in claim 1, characterized in that, Step 5 specifically includes the following sub-steps: (5.1) The optimized model is applied to newly acquired EEG signals to achieve automated and fine segmentation of emotional segments; (5.2) Based on an end-to-end deep network, the segmented segments are used to determine the emotional state and output multi-category emotion recognition results including pleasure, calmness and anxiety; (5.3) The identification results will be further applied to the emotion assessment and personalized mental health management system.

Images