Multi-mode depressive emotion feature recognition method based on brain imaging
By designing a multimodal emotion recognition task paradigm, combining semantics, speech and expression features, natural language processing and acoustic analysis are used to identify depressed emotions, solving the problem of limited recognition effects in the existing technology, and achieving efficient and accurate depression Emotion recognition.
Patent Information
- Application Number
- CN202410924779.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-15
AI Technical Summary
The existing technology lacks a comprehensive and efficient multimodal depression emotional feature recognition system. The traditional method is highly subjective, and single modal recognition cannot fully capture the complexity of emotions, resulting in limited recognition effects.
Design a multimodal emotion recognition task paradigm, combine semantics, speech and expression features, quantify emotional response intensity through natural language processing and acoustic analysis, extract main topics and keywords, evaluate language coherence, capture speech emotion information, and finally combine semantics and speech features to identify depression.
It improves the accuracy and objectivity of depression emotion recognition, provides strong support for the early diagnosis and intervention of depression, and overcomes the limitations of traditional methods.
Smart Images

Figure CN118969241A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of emotion recognition, and in particular relates to a multimodal depressive emotion feature recognition method based on brain imaging. Background Art
[0002] Depression is a common mood disorder that seriously affects an individual's mental health and social function. Traditional diagnosis relies on clinical evaluation, which is highly subjective and easily influenced by the experience of professionals and the patient's description. Currently, automatic identification of depression with the help of bioinformatics methods has become a research hotspot, especially multimodal data analysis, which provides a new perspective for emotion recognition. However, the existing technology lacks a comprehensive and efficient multimodal depression feature recognition system, making it difficult to objectively and accurately analyze and identify depression.
[0003] Traditional methods for identifying depression are highly subjective and rely on expert experience, which limits the accuracy and consistency of diagnosis. In addition, single-modality recognition methods cannot fully capture the complexity of emotions, resulting in limited recognition results. Summary of the invention
[0004] The purpose of the present invention is to provide a multimodal depressive emotion feature recognition method based on brain imaging, which can effectively recognize depressive emotions by combining semantic, speech and expression features, and has high efficiency and accuracy to solve the problems raised in the above background technology.
[0005] To achieve the above object, the present invention adopts the following technical solution: a multimodal depressive emotion feature recognition method based on brain imaging, comprising:
[0006] Design a multimodal emotion recognition task paradigm, which includes an emotional picture task, a language fluency task, an interview task, and a picture description task;
[0007] The data used for depression analysis were obtained through a multimodal emotion recognition task paradigm, including semantic, phonological and expression features;
[0008] Apply natural language processing techniques to extract semantic features from the data, including sentiment analysis, topic modeling, and semantic coherence analysis, to quantify the intensity and variation of emotional responses, extract main topics and key words, and assess the coherence and logic of language expression;
[0009] Through acoustic analysis methods, the speech emotion information in the data is captured and the speech features related to the emotional state are obtained;
[0010] Combining semantic features and phonetic features, the depressive emotional characteristics of the subjects are identified and analyzed.
[0011] Preferably, in the task paradigm, an emotional picture task is used to induce emotional responses of the subjects, and their immediate emotional changes are recorded; the emotional picture task comprises three groups of 12 emotional pictures, the picture contents are designed to trigger different emotional responses, and the display order is random.
[0012] Preferably, in the task paradigm, the language fluency and semantic generation ability of the subject are tested by a language fluency task while recording their speech output, wherein the language fluency task requires the subject to speak words related to a specific category within a limited time.
[0013] Preferably, in the task paradigm, an interview task is used to detect the differences in language and emotional expression of subjects in different emotional contexts, and the interview task requires the subjects to discuss different topics.
[0014] Preferably, in the task paradigm, a picture-speaking task is used to analyze the subject's emotional and language response patterns under visual stimulation, and the picture-speaking task requires the subject to describe the picture content.
[0015] Preferably, in the task paradigm, the speech data is converted into text form using an automatic speech recognition system, and preprocessed to remove stop words, punctuation marks and irrelevant characters, perform part-of-speech tagging and semantic standardization.
[0016] Preferably, the sentiment analysis is used to quantify the intensity and variation of the subject's emotional response.
[0017] Preferably, the topic modeling is used to extract main topics and key words as indicators for understanding the emotional state of the subject.
[0018] Preferably, the semantic coherence analysis is used to evaluate the coherence and logic of the subject's language expression.
[0019] Preferably, the acoustic analysis method is used to capture emotional information in speech and obtain sound indicators related to emotional states.
[0020] Technical effects and advantages of the present invention: Compared with the prior art, the multimodal depressive emotion feature recognition method based on brain imaging proposed by the present invention has the following advantages:
[0021] The present invention collects and analyzes semantic, speech and expression features by designing a multimodal emotion recognition task paradigm that includes emotional pictures, language fluency, interviews and picture descriptions. Natural language processing technology is used to quantify the intensity of emotional reactions, extract topics and evaluate semantic coherence, and acoustic analysis is combined to obtain speech emotion information. Finally, depressive emotions are identified by combining semantic and speech features. This method overcomes the limitations of traditional recognition methods, improves the accuracy and objectivity of depressive emotion recognition, and provides strong support for the early diagnosis and intervention of depression. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 The present invention is a flowchart of a multimodal depressive emotion feature recognition method based on brain imaging. DETAILED DESCRIPTION
[0023] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0024] The present invention provides Figure 1 The multimodal depressive emotion feature recognition method based on brain imaging is shown, which realizes effective recognition of depressive emotions by combining semantic, speech and expression features, and has high efficiency and accuracy. It mainly includes the following steps:
[0025] Design a multimodal emotion recognition task paradigm, which includes an emotional picture task, a language fluency task, an interview task, and a picture description task; obtain the data used for depression emotion analysis through the multimodal emotion recognition task paradigm, including semantic, speech, and expression features; apply natural language processing technology to extract semantic features from the data, including sentiment analysis, topic modeling, and semantic coherence analysis, to quantify the intensity and changes of emotional reactions, extract main topics and key words, and evaluate the coherence and logic of language expression; capture the speech emotion information in the data through acoustic analysis methods, and obtain speech features related to emotional states; combine semantic features and speech features to identify and analyze the depressive emotional characteristics of the subjects. The details are as follows:
[0026] 1. Task Stimulation Paradigm
[0027] In this embodiment, a detailed multimodal emotion recognition task paradigm is designed to collect and analyze various characteristics of depressive emotions. This task paradigm consists of four main tasks with a total duration of about 16 minutes and 30 seconds, covering the emotional picture task (EPT), the verbal fluency task (VFT), the interview task (I NT) and the picture description task (SPT). The following describes in detail the composition and purpose of each task to show how to identify and analyze depressive emotions through these task stimuli.
[0028] Emotional Picture Task (EPT): This task aims to induce emotional responses in subjects by viewing emotion-related pictures. There is a 25-second overall introduction before the task begins, followed by a 20-second task introduction and a 10-second example practice. After the practice, the subjects will watch three sets of 12 emotional pictures (each picture is displayed for 10 seconds). The picture content is designed to trigger different emotional responses, and the display order is random. This part aims to capture the subject's immediate emotional changes through visual stimulation and provide basic data for subsequent emotional state analysis.
[0029] Verbal Fluency Task (VFT): This task tests the subject's language fluency by asking them to say as many words as possible related to a specific category (such as four-legged animals, fruits, and cities) within a limited time. Each subtask includes 15 seconds of preparation time and 45 seconds of expression time. The whole process is recorded, and the three groups of tasks are performed in random order. This part tests the semantic generation ability and the fluency of language expression, as well as the emotional expression under task pressure, providing materials for semantic feature extraction.
[0030] Interview Task (I NT): This task uses interviews to obtain the subjects' verbal responses to different topics (negative, neutral, and positive). The discussion of each topic also includes 15 seconds of preparation and 45 seconds of presentation time, and the entire process is recorded. This task is designed to detect the differences in the subjects' language and emotional expressions in different emotional contexts, especially the natural expression of emotions when discussing personal experiences, which is crucial for a deep understanding of the semantic characteristics of depressive emotions.
[0031] SPT: Similar to EPT, SPT induces specific emotional responses through picture stimulation, but the difference is that the subjects need to describe the content of the picture. The discussion of each picture (negative, neutral, positive) is also 15 seconds of preparation and 45 seconds of presentation. This task combines visual and language information to analyze the emotional and language response patterns of the subjects under specific visual stimuli.
[0032] Through this series of carefully designed tasks, the present invention can not only collect data on depression from multiple dimensions and modalities, but also accurately identify the emotional state of the subject by analyzing his performance under different stimuli. The application of this method may greatly improve the diagnosis and treatment of depression and provide more objective and scientific data support.
[0033] 2. Obtaining semantic features
[0034] In the multimodal depressive emotion recognition method, semantic data is collected through verbal fluency tasks (VFT) and interview tasks (INT). In the verbal fluency task, the subject is asked to say as many words as possible related to a given category, such as animals, fruits, or cities, within a limited time. These tasks are designed to stimulate the subject's thinking and language expression, and the recording device captures all language output. The interview task focuses more on the subject's response and expression to a specific topic (negative, neutral, positive). In this way, rich emotional semantic information can be obtained from the subject's natural language expression.
[0035] The collected speech data first needs to be converted into text form, which is usually done through an automatic speech recognition (ASR) system. After transcription, the text data will go through a series of preprocessing steps, including removing stop words (such as common words such as "and" and "is"), punctuation and irrelevant characters, as well as part-of-speech tagging and semantic standardization. These preprocessing steps help reduce data noise and improve the quality and efficiency of subsequent analysis.
[0036] Natural language processing (NLP) technology will be used to extract key semantic features from the preprocessed text data. The main technologies include:
[0037] Sentiment Analysis: Sentiment analysis tools are used to assess the emotional tendencies expressed by subjects in different situations, such as positive, neutral, or negative. This analysis helps quantify the intensity and changes of the subjects' emotional reactions.
[0038] Topic Modeling: Apply topic modeling techniques such as Latent Dirichlet Allocation (LDA) model to extract main topics and key words from the subject’s language expression. These topics and words will serve as key indicators for understanding the subject’s emotional state.
[0039] Semantic coherence analysis: evaluates the coherence and logic of the subject's language expression. In a depressed state, the individual's language expression may appear scattered or jumpy. By analyzing the coherence of the language, the emotional state can be further judged.
[0040] 3. Obtaining voice features
[0041] Acoustic analysis methods are used, such as the extraction of parameters such as fundamental frequency, energy and timbre, to capture emotional information in speech. Key sound indicators related to emotional state are obtained through glottal features and spectral analysis of audio signals. In the multimodal depressive emotion recognition method of this patent, speech data is collected through verbal fluency tasks (VFT) and interview tasks (INT). These tasks are not only designed to induce the expression of different emotions, but also record the subject's voice response at the same time. These recordings capture the natural speech expressions of the subjects under various emotional stimulations, providing raw data for acoustic analysis.
[0042] The preprocessing of speech data includes several key steps: De-noising: Using acoustic filtering techniques, such as Wiener filters or spectral subtraction, remove background noise from the recording to ensure the accuracy of the analysis. Segmentation: Splitting continuous speech recordings into shorter speech segments, usually based on voice activity detection (VAD) technology to identify speech and non-speech parts. Normalization: Normalizing the amplitude of the speech signal to eliminate the effects caused by changes in the distance between the recording device or the speaker and the microphone.
[0043] The main process of acoustic feature extraction is achieved through acoustic analysis, focusing on the following types of features:
[0044] Fundamental frequency (F0): Fundamental frequency is a parameter that reflects the pitch of a speaker's voice. The speech of patients with depression usually shows reduced variations in fundamental frequency.
[0045] Energy density: The energy density feature reflects the intensity and dynamic range of the speech signal. The speech of depressed patients may show a decrease in energy density level and a reduction in dynamic range.
[0046] Glottal features: including glottal opening time and glottal closing time, these features can reveal the speaker's pronunciation habits and emotional state.
[0047] Spectral features: such as Mel-frequency cepstral coefficients (MFCC) and linear predictive coding (LPC) coefficients, these features can capture the unique texture and quality of speech and are particularly effective for recognizing emotions.
[0048] 4. Get facial expression features
[0049] Facial recognition technology is used, such as extracting key points of facial expressions through deep learning algorithms, and analyzing facial action units to identify different emotional expressions. These facial expression features will be used for subsequent emotion classification. In the multimodal depression emotion recognition method of this patent, expression data is collected by setting up high-resolution cameras to capture the facial expressions of subjects when performing tasks (such as language fluency tasks and interview tasks). These video data record the facial dynamics of the subjects under different emotional stimulations, providing rich original materials for expression analysis.
[0050] The preprocessing of expression video data includes several key steps: Frame extraction: Extract key frames from the video stream, usually selecting frames with significant expression changes or frames with fixed time intervals. Face detection and tracking: Use face detection algorithms (such as Haar cascade classifiers) to locate faces in video frames, and then apply facial tracking technology to ensure the continuity of faces in sequence frames. Face alignment: Geometrically standardize the detected faces to ensure that the positions of key features such as eyes and mouths are consistent to eliminate interference from posture changes. Normalization: Standardize lighting conditions to reduce the impact of lighting changes on expression recognition.
[0051] The feature extraction of facial expression features is achieved through a deep learning model, focusing on the following types of features: Facial Action Units (AUs): According to the Facial Action Coding System (FACS), action units representing different facial muscle movements are extracted, which can accurately describe the changes in facial expressions. Facial Key Points: Using a convolutional neural network (CNN) with a deep learning algorithm, key points (corners of the eyes, corners of the mouth, etc.) are extracted from facial images. The positions and movement trajectories of these key points provide detailed information on the changes in expression.
[0052] A typical deep neural network model used for facial expression feature recognition is the convolutional neural network (CNN), combined with the recurrent neural network (RNN), especially the long short-term memory network (LSTM), to process the temporal data of facial expressions. The CNN part is mainly responsible for extracting spatial features from each frame of the image, such as the location of facial key points and facial texture. The LSTM part: processes the temporal information in the sequence frames, captures the dynamic changes of expressions over time, and effectively recognizes emotional patterns from continuous expression changes.
[0053] 5. Build a classification model
[0054] Combining the above three types of features, a model is built using machine learning methods (such as support vector machines or deep neural networks) to accurately identify depression. The model will be optimized through training data and validated on independent datasets to ensure its generalization ability and recognition accuracy.
[0055] After collecting semantic, speech, and expression features through the task, the next step is to integrate these features and use machine learning methods to build a classification model to achieve the verification and identification of depressive emotions. The following is the whole process of building a model, including model selection, training, verification, and application. The first is the feature fusion process, which requires fusing the features obtained from semantic, speech, and expression analysis. The key to this step is to create a unified feature vector that combines all the individual feature sets to provide a comprehensive representation of the emotional state. The fusion method can be a simple feature concatenation, or use more complex feature fusion techniques such as feature transformation (such as principal component analysis PCA) or deep learning methods (such as multimodal deep learning networks) to effectively integrate information from different sources.
[0056] Regarding the selection of classification models, considering the complexity of depression recognition and the characteristics of multimodal data, the following models are considered: Support Vector Machine (SVM): suitable for processing high-dimensional data, especially performing well on small sample data sets.
[0057] Random forest: A powerful ensemble learning method suitable for handling large feature spaces and providing feature importance assessments. Deep learning models: Such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or their combinations, these models can handle complex data structures and large amounts of data.
[0058] A collected labeled dataset containing samples of known emotional states (depressed or non-depressed) is used to train the selected model. During the training process, the model parameters (such as learning rate, hidden layer size, regularization parameters, etc.) need to be adjusted to optimize the performance of the model. In addition, cross-validation methods are used to avoid overfitting and ensure that the model has good generalization ability.
[0059] After training, an independent test dataset is used to verify the accuracy of the model. This step is key to assessing whether the model can effectively identify depression from unseen data. Evaluation metrics can include accuracy, recall, F1 score, etc. After the validation phase, if the model performs well, it can be deployed in real-world applications such as clinical diagnosis support, remote health monitoring, etc. During the application process, new data is continuously collected and the model is retrained regularly to adapt to possible data drift or changes in emotion recognition requirements.
[0060] Through the above steps, an effective depression recognition system can be built. The system uses advanced machine learning technology to provide strong scientific and technological support for the early diagnosis and treatment of depression by integrating multimodal data of semantics, speech and expression. This method not only improves the objectivity and accuracy of depression diagnosis, but also provides possibilities for other applications in the field of mental health.
[0061] In summary, the present invention collects and analyzes semantic, speech and expression features by designing a multimodal emotion recognition task paradigm that includes emotional pictures, language fluency, interviews and picture descriptions. Natural language processing technology is used to quantify the intensity of emotional reactions, extract topics and evaluate semantic coherence, and acoustic analysis is combined to obtain speech emotion information, and finally semantic and speech features are combined to identify depression. This method overcomes the limitations of traditional recognition methods, improves the accuracy and objectivity of depression recognition, and provides strong support for the early diagnosis and intervention of depression.
[0062] In addition, the present invention also provides a multimodal depression emotion feature recognition system based on brain imaging, comprising:
[0063] A task design module, used to design a multimodal emotion recognition task paradigm, the task paradigm including an emotional picture task, a language fluency task, an interview task and a picture description task;
[0064] Multimodal data acquisition module, used to obtain data used for depression analysis through a multimodal emotion recognition task paradigm, including semantic, speech and expression features;
[0065] A semantic feature extraction and analysis module, for extracting semantic features from the data using natural language processing techniques, including sentiment analysis, topic modeling, and semantic coherence analysis, to quantify the intensity and variation of emotional responses, extract main topics and key words, and evaluate the coherence and logic of language expression;
[0066] The speech feature extraction and analysis module is used to capture the speech emotion information in the data through acoustic analysis methods and obtain the speech features related to the emotional state;
[0067] The depression feature recognition and analysis module is used to combine semantic features and speech features to recognize and analyze the depression features of the subject.
[0068] The above-mentioned task design module, multimodal data acquisition module, semantic feature extraction and analysis module, speech feature extraction and analysis module, and depression feature recognition and analysis module are also used to implement other steps of the above-mentioned brain imaging-based multimodal depression feature recognition method when executed.
[0069] In addition, the present invention also provides a terminal device. The multimodal depressive emotion feature recognition method based on brain imaging involved in this embodiment is mainly applied to the terminal device, which can be a PC, a portable computer, a mobile terminal, or other device with display and processing functions.
[0070] Specifically, the terminal device may include a processor (such as a CPU), a communication bus, a user interface, a network interface, and a memory. The communication bus is used to realize the connection and communication between these components; the user interface may include a display screen (Displ ay), an input unit such as a keyboard (Keyboard); the network interface may optionally include a standard wired interface, a wireless interface (such as a WIFI interface); the memory may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory, and the memory may optionally be a storage device independent of the aforementioned processor.
[0071] Among them, the memory stores a readable storage medium, and the readable storage medium stores a feature recognition program. The processor can call the feature recognition program stored in the memory and execute the multimodal depressive emotion feature recognition method based on brain imaging provided by an embodiment of the present invention.
[0072] It is understood that a readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disk read-only memories (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the above. The computer-readable storage medium used here is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0073] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0074] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, by utilizing state information of computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.
[0075] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A multimodal depressive emotion feature recognition method based on brain imaging, characterized in that: include: Design a multimodal emotion recognition task paradigm, which includes an emotional picture task, a language fluency task, an interview task, and a picture description task; The data used for depression analysis were obtained through a multimodal emotion recognition task paradigm, including semantic, phonological and expression features; Apply natural language processing techniques to extract semantic features from the data, including sentiment analysis, topic modeling, and semantic coherence analysis, to quantify the intensity and variation of emotional responses, extract main topics and key words, and assess the coherence and logic of language expression; Through acoustic analysis methods, the speech emotion information in the data is captured and the speech features related to the emotional state are obtained; Combining semantic features and phonetic features, the depressive emotional characteristics of the subjects are identified and analyzed.
2. The multimodal depressive emotion feature recognition method based on brain imaging according to claim 1, characterized in that: In the task paradigm, the emotional response of the subjects is induced through the emotional picture task, and their immediate emotional changes are recorded; the emotional picture task includes three groups of 12 emotional pictures, the picture content is designed to trigger different emotional responses, and the display order is random.
3. The method for multimodal depressive emotion feature recognition based on brain imaging according to claim 1, characterized in that: In the task paradigm, the language fluency and semantic production ability of the subjects are tested through a language fluency task while recording their speech output, wherein the language fluency task requires the subjects to speak words related to a specific category within a limited time.
4. The multimodal depressive emotion feature recognition method based on brain imaging according to claim 1, characterized in that: In the task paradigm, an interview task is used to detect the differences in language and emotional expression of subjects in different emotional contexts, and the interview task requires the subjects to discuss different topics.
5. The method for multimodal depressive emotion feature recognition based on brain imaging according to claim 1, characterized in that: In the task paradigm, a picture-speaking task is used to analyze the subjects' emotional and language response patterns under visual stimulation, and the picture-speaking task requires the subjects to describe the content of the picture.
6. The method for multimodal depressive emotion feature recognition based on brain imaging according to claim 1, characterized in that: In the task paradigm, the speech data is converted into text form using an automatic speech recognition system, and preprocessed to remove stop words, punctuation marks and irrelevant characters, perform part-of-speech tagging and semantic standardization.
7. The method for multimodal depressive emotion feature recognition based on brain imaging according to claim 1, characterized in that: The sentiment analysis is used to quantify the intensity and variation of the subjects' emotional responses.
8. The method for multimodal depressive emotion feature recognition based on brain imaging according to claim 1, characterized in that: The topic modeling is used to extract main topics and key words as indicators for understanding the emotional state of the subjects.
9. The multimodal depressive emotion feature recognition method based on brain imaging according to claim 1, characterized in that: The semantic coherence analysis is used to evaluate the coherence and logic of the subject's language expression.
10. The multimodal depressive emotion feature recognition method based on brain imaging according to claim 1, characterized in that: The acoustic analysis method is used to capture emotional information in speech and obtain sound indicators related to emotional states.
Citation Information
Cited By
Emotion recognition method and device, storage medium, system and program product
CN119293738A
Emotion Recognition Method, Device, Storage Medium, System and Program Product Using Adaptive Two-Stage Task
CN119293738B
Voice interaction method and device, equipment and storage medium
CN119673160A
