Emotion recognition method and device, storage medium, system and program product

Abstract

The invention provides an emotion recognition method and device, a storage medium, a system and a program product, and the method comprises the steps: outputting first task information, so as to prompt a target user to execute a first task corresponding to the first task information; in the process that the target user executes the first task, obtaining first physiological data of the target user; processing the first physiological data by using a first emotion recognition model to obtain coarsely screened emotion information; determining second task information from the multiple pieces of candidate task information according to the coarse screening emotion information; outputting the second task information to prompt the target user to execute a second task corresponding to the second task information; in the process that the target user executes the second task, obtaining second physiological data of the target user; and processing the second physiological data by using a second emotion recognition model to obtain emotion recognition information. According to the method, a self-adaptive task arrangement scheme is adopted, the emotional tendency stably existing in the user can be deeply stimulated and explored, and long-time emotions can be accurately recognized.

Description

Technical Field

[0001] The present disclosure generally relates to the field of data processing, and more specifically, to an emotion recognition method, device, storage medium, system and program product. Background Art

[0002] Existing emotion recognition methods usually focus on capturing users' instantaneous emotions. However, there is a lack of more accurate and reliable recognition methods for users' relatively stable long-term emotions, such as happiness, joy, pride, nostalgia, sadness, guilt, resentment, anxiety, and depression.

[0003] Specifically, although there are schemes in the related art that detect near-infrared brain imaging data of users in task states to understand the blood perfusion state of the user's prefrontal cortex (PFC), and compare it with standard data of blood perfusion states under several different types of long-term emotions, thereby identifying the user's long-term emotions, such schemes can only roughly distinguish the types of long-term emotions and have limited recognition capabilities. Summary of the invention

[0004] The present disclosure provides an emotion recognition method, device, storage medium, system and program product to solve at least one of the above problems.

[0005] According to one aspect of the present disclosure, an emotion recognition method is provided, which includes: outputting first task information to prompt a target user to perform a first task corresponding to the first task information; obtaining first physiological data of the target user during the process of the target user performing the first task; using a first emotion recognition model to process the first physiological data to obtain coarse-screened emotion information; determining second task information from multiple candidate task information based on the coarse-screened emotion information; outputting the second task information to prompt the target user to perform a second task corresponding to the second task information; obtaining second physiological data of the target user during the process of the target user performing the second task; and using a second emotion recognition model to process the second physiological data to obtain emotion recognition information.

[0006] Optionally, the first task includes an emotional image task lasting a first duration and a semantic fluency task lasting a second duration, and there is an interval lasting a preset interval duration between the emotional image task and the semantic fluency task.

[0007] Optionally, the first duration ranges from 65 seconds to 105 seconds; the second duration ranges from 190 seconds to 230 seconds; and the preset interval duration ranges from 20 seconds to 30 seconds.

[0008] Optionally, the coarse-screened emotion information includes emotion indexes of multiple preset emotions, each emotion index is used to indicate the possibility that the user has the corresponding preset emotion, and the multiple candidate task information includes general task information and emotion task information corresponding one-to-one to the multiple preset emotions, wherein determining the second task information from the multiple candidate task information based on the coarse-screened emotion information includes: determining an emotion index greater than an index threshold from the emotion indexes of the multiple preset emotions, and taking the preset emotion corresponding to the determined emotion index as the target emotion; when the number of target emotions is less than the preset number, determining the emotion task information corresponding to the target emotion from the multiple candidate task information as the second task information; when the number of target emotions is greater than or equal to the preset number, determining the general task information as the second task information.

[0009] Optionally, the emotional task corresponding to the emotional task information includes an interview task, and the general task corresponding to the general task information includes a picture description task.

[0010] Optionally, the second task information lasts for a third duration, and a value range of the third duration is 100 seconds to 140 seconds.

[0011] Optionally, the second physiological data includes voice data and near-infrared brain imaging data, the voice data is data obtained by recording the content spoken by the target user, and the second emotion recognition model includes a natural language processing model, a near-infrared emotion recognition model, and an information fusion model, wherein the use of the second emotion recognition model to process the second physiological data to obtain emotion recognition information includes: using the natural language processing model to perform natural language processing on the voice data to identify the emotions of the target user and obtain semantic emotion information; using the near-infrared emotion recognition model to process the near-infrared brain imaging data to identify the emotions of the target user and obtain near-infrared emotion information; using the information fusion model to fuse the semantic emotion information and the near-infrared emotion information to obtain the emotion recognition information.

[0012] Optionally, the emotion recognition method also includes: performing a consistency comparison based on the coarse-screened emotion information, the semantic emotion information and the near-infrared emotion information to obtain consistency information, wherein the consistency information is used to reflect the degree of consistency between the different identified emotion information; when the consistency information indicates that the degree of consistency between the different identified emotion information is lower than a preset degree, outputting a prompt message to indicate that the recognition result is inaccurate.

[0013] Optionally, the emotion recognition method further includes: receiving feedback information from the target user regarding the emotion recognition information; and updating the second emotion recognition model according to the feedback information.

[0014] According to another aspect of the present disclosure, an emotion recognition device is provided, which includes: a first output unit, configured to output first task information to prompt a target user to perform a first task corresponding to the first task information; a first acquisition unit, configured to acquire first physiological data of the target user during the process of the target user performing the first task; a coarse screening unit, configured to use a first emotion recognition model to process the first physiological data to obtain coarse-screened emotion information; a determination unit, configured to determine second task information from multiple candidate task information based on the coarse-screened emotion information; a second output unit, configured to output the second task information to prompt the target user to perform the second task corresponding to the second task information; a second acquisition unit, configured to acquire second physiological data of the target user during the process of the target user performing the second task; and a recognition unit, configured to use a second emotion recognition model to process the second physiological data to obtain emotion recognition information.

[0015] Optionally, the first task includes an emotional image task lasting a first duration and a semantic fluency task lasting a second duration, and there is an interval lasting a preset interval duration between the emotional image task and the semantic fluency task.

[0016] Optionally, the first duration ranges from 65 seconds to 105 seconds; the second duration ranges from 190 seconds to 230 seconds; and the preset interval duration ranges from 20 seconds to 30 seconds.

[0017] Optionally, the coarse-screened emotion information includes emotion indexes of multiple preset emotions, each emotion index is used to indicate the possibility that the user has the corresponding preset emotion, the multiple candidate task information includes general task information and emotion task information corresponding one-to-one to the multiple preset emotions, and the determination unit is further configured to: determine an emotion index greater than an index threshold from the emotion indexes of the multiple preset emotions, and use the preset emotion corresponding to the determined emotion index as the target emotion; when the number of target emotions is less than a preset number, determine the emotion task information corresponding to the target emotion from the multiple candidate task information as the second task information; when the number of target emotions is greater than or equal to the preset number, determine the general task information as the second task information.

[0018] Optionally, the emotional task corresponding to the emotional task information includes an interview task, and the general task corresponding to the general task information includes a picture description task.

[0019] Optionally, the second task information lasts for a third duration, and a value range of the third duration is 100 seconds to 140 seconds.

[0020] Optionally, the second physiological data includes voice data and near-infrared brain imaging data, the voice data is data obtained by recording the content spoken by the target user, the second emotion recognition model includes a natural language processing model, a near-infrared emotion recognition model, and an information fusion model, and the recognition unit is further configured to: use the natural language processing model to perform natural language processing on the voice data to identify the emotions of the target user and obtain semantic emotion information; use the near-infrared emotion recognition model to process the near-infrared brain imaging data to identify the emotions of the target user and obtain near-infrared emotion information; use the information fusion model to fuse the semantic emotion information and the near-infrared emotion information to obtain the emotion recognition information.

[0021] Optionally, the emotion recognition device also includes: a comparison unit, configured to perform consistency comparison based on the coarse-screened emotion information, the semantic emotion information and the near-infrared emotion information to obtain consistency information, wherein the consistency information is used to reflect the degree of consistency between the different identified emotion information; a prompt unit, configured to output prompt information to indicate that the recognition result is inaccurate when the consistency information indicates that the degree of consistency between the different identified emotion information is lower than a preset degree.

[0022] Optionally, the emotion recognition device further includes: a receiving unit configured to receive feedback information from the target user regarding the emotion recognition information; and an updating unit configured to update the second emotion recognition model according to the feedback information.

[0023] According to another aspect of the present disclosure, a computer-readable storage medium storing instructions is provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the emotion recognition method as described above.

[0024] According to another aspect of the present disclosure, a system is provided, including at least one computing device and at least one storage device storing instructions, wherein when the instructions are executed by the at least one computing device, the at least one computing device is prompted to perform the emotion recognition method as described above.

[0025] According to another aspect of the present disclosure, a computer program product is provided, comprising instructions, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the emotion recognition method as described above.

[0026] According to the emotion recognition method, device, storage medium, system and program product of the exemplary embodiment of the present disclosure, the conventional idea of ​​improving the ability to mine data rules is broken. By constructing an adaptive two-stage task, the first task is used to roughly identify the user's emotions to obtain coarse-screened emotional information, and then the corresponding second task is released according to the coarse-screened emotional information to stimulate the user's richer physiological data in the corresponding emotions, that is, the second physiological data, which can increase the effective basic data required for emotion recognition in a targeted manner, help to achieve more detailed emotion recognition from a data-driven perspective, and improve the recognition ability of the target user's long-term emotions. Specifically, since long-term emotions are relatively stable emotions that exist in the user in the recent period of time, they will affect the user's thinking tendency and the instantaneous emotional response when facing specific things. For example, when facing the same emotional image, users in two different long-term emotions of anxiety and happiness are likely to have completely different instantaneous emotions. Based on this, by inviting the target user to perform a specific task, the user's instantaneous emotions can be stimulated, and then the recognition of long-term emotions can be achieved by identifying the stable emotional tendencies in the instantaneous emotions. By adopting the adaptive task arrangement scheme disclosed in the present invention, it is possible to deeply stimulate and explore the stable emotional tendencies of the target users, and realize the accurate recognition of the long-term emotions of the target users.

[0027] Additional aspects and / or advantages of the present general inventive concept will be set forth in part in the following description and in part will be apparent from the description or may be learned through practice of the present general inventive concept. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] These and / or other aspects and advantages of the present disclosure will become clear and more easily understood from the following description of the embodiments in conjunction with the accompanying drawings.

[0029] Figure 1 is a flowchart illustrating an emotion recognition method according to an exemplary embodiment of the present disclosure.

[0030] Figure 2 FIG. 4 is a logical diagram showing a task release according to a specific embodiment of the present disclosure.

[0031] Figure 3 is a block diagram illustrating an emotion recognition apparatus according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the embodiments of the present invention as defined by the claims and their equivalents. Various specific details are included to assist in understanding, but these details are considered to be exemplary only. Therefore, one of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. In addition, descriptions of well-known functions and structures are omitted for clarity and brevity.

[0033] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three types of parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three types of parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0034] Refer to the following Figures 1 to 3 The emotion recognition method, apparatus, storage medium, system, and program product according to exemplary embodiments of the present disclosure are described in detail.

[0035] Figure 1 is a flow chart showing an emotion recognition method according to an exemplary embodiment of the present disclosure. The emotion recognition method according to an exemplary embodiment of the present disclosure can be implemented in a computing device with sufficient computing power.

[0036] Reference Figure 1 In step S101, first task information is output to prompt a target user to perform a first task corresponding to the first task information.

[0037] The target user is the object of emotion recognition, that is, the user whose emotion needs to be recognized. When outputting the first task information, as an example, the first task information can be output in the form of text, video or voice through a display screen, a loudspeaker or other device to implement the release of the first task, so that the target user can perform the first task according to the prompt.

[0038] Optionally, the first task includes an emotional image task lasting a first duration and a semantic fluency task lasting a second duration, and there is an interval between the emotional image task and the semantic fluency task that lasts a preset interval duration. By successively releasing different types of tasks, on the one hand, different tasks can be used to provide different stimuli for the target user, which is convenient for obtaining more information. On the other hand, the target user can gradually enter the state, which is helpful for obtaining effective physiological data reflecting the target user's emotions in the subsequent step S102. Specifically, it is found through research and analysis that the effect of activating emotions in the emotional image task is relatively weaker than that in the semantic fluency task. Based on this, by first releasing the emotional image task, the target user can be helped to establish the initial adaptability, so that the target user enters a state of concentration and establishes a reference baseline for physiological data. After the emotional image task is completed, after a short break, the target user can be fully activated by releasing the semantic fluency task, and physiological data that effectively reflects the target user's emotions can be obtained. In addition, the emotional image task can also capture the target user's immediate emotional changes through visual stimulation, providing basic data for subsequent emotional state analysis. The semantic fluency task can also test the target user's semantic generation ability and language expression fluency, as well as the target user's emotional response when facing task pressure.

[0039] Specifically, the goal of the Emotion Picture Task (EPT) is to induce emotional responses in target users by letting them observe emotion-related images. Before the task officially begins, a brief task description can be given. After the description, a number of randomly selected emotion images from the gallery are displayed to the target users in sequence. The contents of these images are designed to stimulate different emotional responses, and the order of display is also random. In other words, the emotion picture task includes two stages: task description and emotion image display. Each stage has a specific duration. The emotion image display stage displays a number of randomly selected emotion images in sequence. Accordingly, the information corresponding to the emotion image task in the first task information includes emotion task description information, a number of randomly selected emotion images, task description duration, and single image display duration. When outputting this part of information, the emotion task description information is first output within the task description duration, and then a number of randomly selected emotion images are output one by one, and the duration of each emotion image output is the single image display duration.

[0040] The goal of the semantic fluency task (SFT) is to assess the target user's language fluency by asking them to say as many words as possible related to a specific category (e.g., quadrupeds, vegetables, stationery) within a limited time. In addition to the semantic fluency task, the verbal fluency task (VFT) also includes the phonetic fluency task (FFT), whose stimulation mode focuses on saying words with voice as a clue, such as "please list the words that start with the letter 'T'" or "please list the words that start with 'white'". Research and analysis have found that the phonetic fluency task sometimes cannot effectively distinguish different emotions. Based on this, choosing a more effective semantic fluency task can improve the reliability of emotional screening. Before the task officially begins, a brief task description can also be given. After the description, the target user will first release several subtasks randomly selected from the task library in turn. Each subtask is completed within the same limited time, and there is a certain rest time after each subtask, so as to repeatedly activate the target user's emotional response. In other words, the semantic fluency task includes two stages: task description and category display. Each stage has a specific duration. The category display stage further includes several sub-stages, each of which includes category name display and rest between groups. In particular, the last sub-stage may not include rest between groups. Accordingly, the information corresponding to the semantic fluency task in the first task information includes semantic task description information, several randomly selected category names, task description duration, single category name display duration, and rest between groups. When outputting this part of information, the semantic task description information is first output within the task description duration, and then a randomly selected category name is output. After the single category name display duration, it stops. After the rest between groups, the next randomly selected category name is output, and this continues until the output of all randomly selected category names is completed.

[0041] Optionally, the first duration ranges from 65 seconds to 105 seconds; the second duration ranges from 190 seconds to 230 seconds. By limiting the duration ranges of different stages in the first task, the task duration can be reasonably arranged. The lower limit of the task duration can ensure that the task duration is long enough to obtain sufficient information. The upper limit of the task duration can avoid the target user from having obvious negative emotions and interfering with the target user's emotional response due to the task being too long, which helps to ensure the accuracy of the emotion recognition results.

[0042] Optionally, the preset interval duration ranges from 20 seconds to 30 seconds. The lower limit can ensure that the target user gets enough rest and reduces the target user's fatigue, and the upper limit can reduce the risk of distracting the user's attention due to too long a rest time. By reserving a reasonable rest time, the effectiveness of the physiological data collected during the task can be improved, and the emotion recognition effect can be improved.

[0043] In step S102, when the target user performs the first task, first physiological data of the target user is obtained.

[0044] Specifically, the first physiological data can be detected by a biosignal sensor. Optionally, the first physiological data can be physiological data collected during the execution of the entire first task, or physiological data collected during the execution of part of the first task (such as a semantic fluency task), and the present disclosure does not limit this. Through software control, the data collection of the biosignal sensor is synchronized with the output of the corresponding task information, and a timestamp can be added to the collected first biological data, so that the first physiological data can be corresponded to the corresponding task stage, which helps to achieve more accurate emotion recognition. Of course, it is also possible to start collecting physiological data before the corresponding task starts, and stop collecting physiological data after a period of time after the corresponding task is completed, and then intercept the physiological data during the execution of the corresponding task as the first physiological data. In other words, it is limited here only to obtain the target user's physiological data as the first physiological data during the target user's execution of the first task, but there is no restriction on how to collect and obtain the physiological data.

[0045] As an example, the first physiological data includes near-infrared brain imaging data. The near-infrared brain imaging data is obtained through near-infrared brain imaging technology (Functional near-infrared spectroscopy, referred to as fNIRS), which is a spectral imaging technology that continuously irradiates the human cerebral scalp with near-infrared light of a wavelength of 650nm-850nm, so that the near-infrared light passes through the scalp and skull, enters the cerebral cortex at a depth of 2cm-3cm, forms a propagation arc, and is detected by another detector on the scalp. During the propagation of near-infrared light in human tissue, the absorption of photons by tissue has a certain regularity, which is particularly special for oxygenated hemoglobin and deoxygenated hemoglobin in the capillaries in the cerebral cortex. Due to this particularity, the concentration change of oxygenated hemoglobin and deoxygenated hemoglobin in the irradiation interval can be inferred from the absorption rate obtained on the detector. When the human brain is working, nerve cells discharge continuously, which consumes a lot of energy. All of this energy is provided by the oxygen carried by the capillaries surrounding the nerve cells. Therefore, when the brain is activated, its supply of oxygenated hemoglobin will increase, thereby accelerating the oxygen supply and reducing the concentration of deoxyhemoglobin. After activation, the energy is converted, the concentration of oxygenated hemoglobin decreases, and the concentration of deoxyhemoglobin increases. The cerebral cortex is in an oxygen-demanding state again, waiting for the next oxygen supply. This cycle repeats itself, and this is the neurovascular coupling mechanism in neuroscience.

[0046] The prefrontal lobe of the human brain is responsible for various high-level emotional activities and regulation. Current studies have shown that emotions originate from the limbic system deep in the brain, and the regulation of emotions is a very complex process, which is organically regulated by various parts of the prefrontal lobe. Therefore, studying the hemodynamic feedback mechanism of the prefrontal lobe can provide insights into an individual's response and regulation ability to different emotions. Using near-infrared brain imaging technology to act on the prefrontal lobe, the cerebral cortex can be activated through task states (including emotional stimulation tasks, language fluency tasks, resting state tasks, etc., where the resting state task refers to the subject remaining still and relaxed without performing specific cognitive tasks). By observing the changes in oxygenated hemoglobin and deoxygenated hemoglobin, the individual's emotional characteristics can be obtained, thereby establishing an individual emotion regulation model.

[0047] It should be understood that near-infrared brain imaging data needs to be collected with corresponding equipment, such as a frontal headgear and a detector for collecting data. It should also be understood that in order to ensure that the near-infrared brain imaging data of the target user in the process of performing the first task is fully obtained, data collection can be started before the target user starts to perform the first task, and the collection can be stopped a period of time after the target user completes the first task, and then the data corresponding to the execution period of the first task can be intercepted. In other words, it is limited here to only obtain the first physiological data during the process of the target user performing the first task, but there is no restriction on how to collect and obtain the data.

[0048] In step S103, the first emotion recognition model is used to process the first physiological data to obtain coarse-screened emotion information.

[0049] As an example, the first emotion recognition model can be trained by a machine learning method, and the training samples used for training may include sample physiological data of sample users and sample emotion labels. Each sample emotion label is used to mark a long-term emotional state, which can be a specific emotion type or a collection of multiple emotion types. By configuring appropriate sample emotion labels, the first emotion recognition model can have different emotion coarse screening capabilities. Accordingly, the coarse screening emotion information can be a long-term emotional state that the target user is most likely to be in, or it can be the possibility of the target user being in each long-term emotional state. In addition, before using the first emotion recognition model to process the first physiological data, the first physiological data can also be preprocessed, such as including signal baseline correction, sliding window low-pass filtering, and artifact signal processing to reduce noise interference and improve data quality.

[0050] In step S104, second task information is determined from a plurality of candidate task information according to the roughly screened emotion information.

[0051] This step is used to determine one of the multiple candidate task information that best matches the coarse-screened emotional information identified in step S103 as the second task information, thereby achieving adaptive task release and targeted emotional stimulation.

[0052] Optionally, the rough screening emotional information includes emotional indexes of multiple preset emotions. The preset emotions may be long-term emotional types, such as happiness, joy, pride, nostalgia, sadness, guilt, resentment, anxiety, depression and other long-term emotional types, so as to achieve rich emotional recognition. For example, only negative long-term emotional types such as guilt, resentment, anxiety, depression and the like may be included to achieve monitoring and early warning of negative long-term emotions. Each emotional index is used to indicate the possibility that the user has the corresponding preset emotion. The multiple candidate task information includes general task information and emotional task information corresponding to the multiple preset emotions. Step S104 includes: determining an emotional index greater than an index threshold from the emotional indexes of the multiple preset emotions, and taking the preset emotion corresponding to the determined emotional index as the target emotion; when the number of target emotions is less than the preset number, determining the emotional task information corresponding to the target emotion from the multiple candidate task information as the second task information; when the number of target emotions is greater than or equal to the preset number, determining the general task information as the second task information. By configuring the rough screening emotional information as the emotional index of multiple specific preset emotions, and configuring multiple candidate tasks as general tasks and emotional tasks corresponding to these preset emotions one by one, and then selecting the matching task as the second task according to the emotional index of each preset emotion identified, the adaptive arrangement of the second task can be achieved. Specifically, by configuring the index threshold as a reference standard, the intensity of each preset emotion identified during the rough screening can be reliably judged, so as to find the target emotion with higher intensity. By configuring the preset number, the data processing amount during the task arrangement and accurate emotion identification in the second stage can be controlled, that is, when the number of target emotions with higher intensity is small, the corresponding second task is arranged for each target emotion to achieve in-depth stimulation and exploration of each target emotion, and when the number of target emotions with higher intensity is large, the general task is directly arranged, and the comprehensive emotion is deeply stimulated and explored on the basis of the first task to appropriately improve the recognition accuracy of long-term emotions. As an example, the preset number is 1 or 2, and the present disclosure is not limited to this. The value of the index threshold can be reasonably set according to actual needs, and the present disclosure is not limited to this.

[0053] As an example, in the case where the preset emotions include both positive and negative long-term emotion types, general tasks can be configured for positive emotions and negative emotions respectively. If the number of target emotions is greater than or equal to the preset number, the proportion of positive emotions and / or negative emotions in the target emotions is first determined, and then when the proportion of positive emotions is greater than or equal to the preset proportion (for example, 50%, or 80%), the general task corresponding to the positive emotion is selected, and when the proportion of negative emotions is greater than or equal to the preset proportion, the general task corresponding to the negative emotion is selected. Furthermore, if the preset proportion is set to a high level, there may be a situation where the proportion of both positive and negative emotions in the target emotions is less than the preset proportion. For this reason, neutral emotions can be further included in the preset emotions, and general tasks applicable to neutral emotions can be configured accordingly. In the case where the number of target emotions is greater than or equal to the preset number, and the proportion of both positive and negative emotions in the target emotions is less than the preset proportion, a general task applicable to neutral emotions is selected.

[0054] Optionally, the emotional task corresponding to the emotional task information includes an interview task (INT), which can arrange corresponding interview tasks for different preset emotions, and can effectively stimulate the corresponding preset emotions by arranging the interview content as needed, which helps to accurately quantify the preset emotions and improve the accuracy of emotion recognition. The general task corresponding to the general task information includes a storytelling picture task (SPT), which can be applied to a variety of preset emotions and further improve the accuracy of emotion recognition on the basis of the first task.

[0055] Optionally, the second task information lasts for a third duration, and the third duration ranges from 100 seconds to 140 seconds. The lower limit value can ensure that the task duration is long enough to obtain sufficient information, and the upper limit value can avoid the target user from developing obvious negative emotions and interfering with the target user's emotional response due to the task being too long, thereby helping to ensure the accuracy of the emotion recognition results.

[0056] In step S105, the second task information is output to prompt the target user to perform the second task corresponding to the second task information.

[0057] The output method of the second task information can refer to the output method of the first task information. For example, for an interview task, the interview questions can be displayed on a display screen, or the interview question audio can be played through a loudspeaker. A microphone or other sound pickup device can be further configured to collect the target user's answer audio to provide more reference data for emotion recognition. Of course, the answer audio can also be omitted, and emotion recognition can be achieved only by relying on the second physiological data of the target user obtained in the subsequent step S106. The present disclosure does not impose any restrictions on this.

[0058] In step S106, while the target user is performing the second task, second physiological data of the target user is acquired.

[0059] The content and acquisition method of the second physiological data may refer to the content and acquisition method of the first physiological data, and will not be repeated here.

[0060] In step S107, the second emotion recognition model is used to process the second physiological data to obtain emotion recognition information.

[0061] Optionally, the second physiological data includes voice data and near-infrared brain imaging data, and the voice data is data obtained by recording the content spoken by the target user, such as the recording data of the content stated when the target user performs the interview task and the picture description task described in the previous example, and the recording data of the content stated when the target user performs other types of second tasks, and the present disclosure does not limit this. Correspondingly, the second emotion recognition model includes a natural language processing model, a near-infrared emotion recognition model, and an information fusion model. Step S106 includes: using the natural language processing model, performing natural language processing on the voice data to identify the emotions of the target user and obtain semantic emotion information; using the near-infrared emotion recognition model, processing the near-infrared brain imaging data to identify the emotions of the target user and obtain near-infrared emotion information; using the information fusion model, fusing the semantic emotion information and the near-infrared emotion information to obtain emotion recognition information. By first using the natural language processing model and the near-infrared emotion recognition model to respectively process the voice data and the near-infrared brain imaging data collected during the execution of the second task, semantic emotion information and near-infrared emotion information are obtained, and then using the information fusion model to fuse the two emotion recognition results to achieve accurate emotion recognition.

[0062] As an example, when processing speech data, the natural language processing model can first perform speech recognition on the speech data to convert the speech data into text data, and then perform word segmentation and part-of-speech tagging on the text data, and determine the semantic emotional information based on the part-of-speech tagging results. When determining semantic emotional information, for example, the method of emotional dictionary can be used to record a series of words and their corresponding emotional polarity (positive or negative) in the emotional dictionary in advance, and the emotional tendency of the text can be judged by calculating the ratio of positive words and negative words in the text as semantic emotional information. For another example, machine learning algorithms can be used, such as but not limited to support vector machines (SVM), naive Bayes, decision trees, etc. These algorithms learn emotional features by training a large amount of labeled data, and are used for emotional classification of new texts. The classification results obtained are used as semantic emotional information.

[0063] It should be understood that the first emotion recognition model is used to achieve rough screening of emotions, and the data type of the first physiological data processed may be relatively small, for example, only near-infrared brain imaging data is processed; in order to improve the recognition accuracy, the second emotion recognition model may process more data types of the second physiological data, such as the voice data and near-infrared brain imaging data introduced here. In order to achieve the processing of near-infrared brain imaging data, the type of the near-infrared recognition model in the second emotion recognition model may be the same as or different from the type of the first emotion recognition model. In the case of the same type, the parameters of the two may also be the same (that is, the near-infrared recognition model in the first emotion recognition model and the near-infrared recognition model in the second emotion recognition model are the same model) or different. Of course, in the case of different types, the parameters of the two must also be different. In the case where the near-infrared recognition model in the second emotion recognition model is a different model from the first emotion recognition model, different emotion recognition models may be further configured for different candidate task information, so that the emotion recognition model corresponding to the second task information is used as the second emotion recognition model.

[0064] Regarding the recognizable emotion tags, that is, the content of the emotion tags that can be represented by the recognized emotion information, as an example, the content of the emotion tags that can be represented by the semantic emotion information and the near-infrared emotion information can be the same, so that the information fusion model can fuse the semantic emotion information and the near-infrared emotion information. Of course, they can also be different, and the present disclosure does not limit this. In addition, the content of the emotion tags that can be represented by the coarse-screened emotion information, the semantic emotion information, and the near-infrared emotion information can also be the same or different, and the present disclosure does not limit this. For specific emotion tags, please refer to the introduction of steps S103 and S104 above, which will not be repeated here.

[0065] Optionally, according to different detection needs, when training the second emotion recognition model, different label information can be configured for the training samples, such as emotion type labels and emotion intensity labels. The emotion intensity labels can correspond to several levels, such as strong, general, or some, or can be continuous quantities, such as expressed in the form of scores. The present disclosure does not impose any restrictions on this.

[0066] Optionally, the content of the emotion labels that can be represented by the rough screening emotion information, the semantic emotion information and the near infrared emotion information is the same. According to the emotion recognition method of the exemplary embodiment of the present disclosure, it can further include: performing consistency comparison according to the rough screening emotion information, the semantic emotion information and the near infrared emotion information to obtain consistency information, wherein the consistency information is used to reflect the consistency degree between the different emotion information identified; when the consistency information indicates that the consistency degree between the different emotion information identified is lower than the preset degree, outputting prompt information to prompt that the recognition result is inaccurate. By further adding the processing step of consistency comparison, it is helpful to automatically identify abnormal situations in which the target user has emotional instability, such as but not limited to deliberately concealing emotions, answering irrelevant questions, unstable mental state, and resisting answers, and outputting corresponding prompt information to provide an extra layer of protection for the recognition result. It should be understood that the consistency information is used to reflect the consistency degree between the different emotion information identified, which can be realized by the value of an indicator, and the value of the indicator can be positively correlated with the consistency degree, or negatively correlated with the consistency degree, which are all implementation methods of the present disclosure and fall within the protection scope of the present disclosure.

[0067] As an example, the rough screening emotion information, semantic emotion information and near infrared emotion information can be combined to calculate the cooperation index (CI) as the consistency information through the following XOR weighted formula: .

[0068] In this formula, N is the total number of identifiable emotion labels, Represents a sentiment label. , , Respectively represent the first emotion recognition model, the second emotion recognition model, and the near infrared recognition model in the second emotion recognition model The recognition result of this emotion label, all three are Boolean values, 0 means the recognition result is not , 1 means the recognition result is . is an operator representing the exclusive OR operation (XOR for short), which is a binary operation with the basic rule that if the two quantities being compared are the same, the result is 0; if the two quantities being compared are different, the result is 1. Therefore, for each emotion tag, the value of the consistency calculation result may be 0, 1, 2, or 3, which can reflect whether the recognition results of the three models for each emotion tag are consistent, and the smaller the value of the calculation result, the higher the degree of consistency. Then add the consistency calculation results for each emotion tag to get the degree of fit. CI, which indicates the consistency of the overall recognition result. It can be understood that the degree of fit is CI The value range of is [0,3N], and CI The larger the value, the more unstable the target user's emotions are, and the more likely the detection reliability is to be compromised. N =3, if CI >7, it indicates unreliable emotion recognition results or extremely unstable results; if CI <3, it indicates stable emotions and reliable recognition results. And if necessary, CI In most cases, manual intervention can be performed to determine the specific cause, which is a specific application of the output prompt information, and the present disclosure does not limit this. It should be understood that the XOR weighted formula given here is only an example, and other specific forms of calculation formulas can also be used based on the XOR operation, provided that the meaning of the consistency information is satisfied, which are not listed here one by one.

[0069] remove Figure 1 In addition to the steps shown, the emotion recognition method according to the exemplary embodiment of the present disclosure may further include: receiving feedback information from the target user regarding the emotion recognition information; and updating the second emotion recognition model according to the feedback information. The target user's feedback information on the emotion recognition information can represent the target user's evaluation of the accuracy of the emotion recognition information. By using the feedback information to update the second emotion recognition model, the model can be continuously optimized as the model is used, which helps to continue to improve the accuracy of emotion recognition and the emotion recognition ability during use.

[0070] As an example, the feedback information may include simple accurate and inaccurate evaluations, and may further include the target user's self-assessment of emotional information. The content items contained in the self-assessment emotional information may be consistent with the content items of the emotion recognition information. At this time, the second physiological data and the corresponding self-assessment emotional information can be used to form an updated sample for updating the second emotion recognition model.

[0071] In general, to achieve emotion recognition, an emotion recognition system can be constructed, which includes a task information output device (for example, including a display screen), a biosignal sensor (for example, including a frontal headgear and a data acquisition instrument), and a computing device that executes the emotion recognition method of the exemplary embodiment of the present disclosure.

[0072] Next, combine Figure 2 , introduces an emotion recognition method according to a specific embodiment of the present invention.

[0073] In this specific embodiment, a brain activation induced task paradigm for near-infrared brain imaging is designed, referring to Figure 2, the first stage (i.e., the first task) of the task paradigm includes: an emotional image task of 75 seconds (first duration), a rest of 25 seconds (preset interval duration), and a semantic fluency task of 215 seconds (second duration). The emotional image task is the initial adaptive task with the lowest effect of activating emotions. It can allow the target user to enter a state of concentration and establish a reference baseline for the blood oxygen signal. The role of the semantic fluency task is to repeatedly activate the prefrontal lobe through four subtasks so that the corresponding emotional index can be obtained later. After the semantic fluency task, the system will analyze the blood oxygen dynamic characteristics of the semantic fluency task through a neural network algorithm (i.e., the first emotion recognition model) to achieve preliminary emotion recognition, and obtain multiple preset emotional indexes, including anxiety emotional index, depression emotional index, and compulsive emotional index, as coarse-screened emotional information, and return it to the system.

[0074] After another 25-second break, a second task of 120 seconds (third duration) determined based on the emotion index will be given according to the result of emotion index recognition as an extended task of the second stage. The second task is a task determined from the interview task INT-A, the interview task INT-B, the interview task INT-C, and the picture description task, which can induce activation of the prefrontal cortex of the target user's brain through continuous tasks. If the anxiety emotion index is greater than the index threshold, the interview task INT-A for anxiety emotion will be pushed; if the depression emotion index is greater than the index threshold, the interview task INT-B for depression emotion will be pushed; if the compulsive emotion index is greater than the index threshold, the interview task INT-C for compulsive emotion will be pushed; if all three emotion indices exceed the index threshold, the picture description task will be pushed.

[0075] Specifically, the goal of the emotional image task is to elicit emotional responses from the target user by having them observe emotion-related images. Before the task begins, there is a 15-second task description. After the description, the target user will see three groups of 6 emotional images (randomly displayed from a gallery of 24 images divided into three groups with similar emotions in each group, and each image is displayed for 10 seconds). The content of these images is designed to elicit different emotional responses, and the order of display is random. The goal of this part is to capture the target user's immediate emotional changes through visual stimulation, and provide basic data for subsequent emotional state analysis.

[0076] The goal of the semantic fluency task is to assess the target users' language fluency by asking them to say as many words as possible related to a specific category (e.g., quadrupeds, vegetables, stationery) within a limited time. There is a 15-second task introduction at the beginning, a total of 4 subtasks (randomly assigned from 16 groups of tasks), each subtask is 35 seconds, and there is a 15-second break after each subtask. The goal of this part is to test the target users' semantic generation ability and language expression fluency, as well as their emotional response when facing task pressure, to provide data for the extraction of semantic features.

[0077] The picture description task requires the target user to verbally describe the content of the image. The task introduction is 15 seconds, and the discussion of each picture (negative, neutral, positive) includes 5 seconds of preparation time and 20 seconds of description time. This task combines visual and language information, and aims to analyze the target user's emotional and language response patterns under specific visual stimuli.

[0078] During the execution of the task, the region of interest (ROI) for near-infrared brain imaging data collection is concentrated in the prefrontal cortex. The functions of the prefrontal cortex are mainly related to cognition, emotion, and behavior management, especially the processing of various recently acquired information, and its function is called working memory. Basic research shows that under normal circumstances, neuronal activity in the brain region increases oxygen consumption in the region, while cerebral blood volume and blood flow also increase, carrying a large amount of oxygenated hemoglobin. This causes a significant increase in the local oxygenated hemoglobin concentration, while the deoxygenated hemoglobin concentration fluctuates accordingly. The total hemoglobin concentration and blood oxygen saturation also increase, placing the brain region in a state of blood perfusion with a high oxygen concentration. The absorption and scattering rates of oxyhemoglobin and deoxyhemoglobin in human tissues for infrared light in the band of 700~900nm are significantly different and overlap. Therefore, the changes in the relative concentrations of oxyhemoglobin and deoxyhemoglobin in the capillaries of the prefrontal cortex can be indirectly calculated through near-infrared light imaging technology and mathematical methods, thereby obtaining relevant data on the blood perfusion status of the prefrontal cortex.

[0079] In order to obtain near-infrared brain imaging data with the same time sequence as that when the brain activation task is induced, the task paradigm module of the software controls and starts the near-infrared data acquisition module, so that the timestamp contained in the collected near-infrared brain imaging data is equal to the timestamp of the task paradigm operation. At the same time, under the action of the timestamp, the near-infrared data in the brain activation state are divided into three sections: 75 seconds for the emotional image task, 215 seconds for the semantic fluency task, and 75 seconds for the second task.

[0080] In order to retain indicators that are highly correlated with the prefrontal working memory network, the near-infrared brain imaging data collected in the prefrontal area needs to be preprocessed. There are three main steps in this process: signal baseline correction, sliding window low-pass filtering, and artifact signal processing.

[0081] When performing signal baseline correction, the cutoff frequency is typically set low (e.g., 0.01 Hz) to ensure that all frequency changes associated with the cognitive task are preserved while filtering out slower physiological changes and other drifts.

[0082] Assuming there is a time series signal x(t), a high-pass filter H1(f) can be used to process this signal, where f represents the frequency. A simple implementation of a high-pass filter is a first-order Butterworth filter, whose transfer function can be expressed as: .

[0083] Among them, f c is the cutoff frequency. After applying this high-pass filter to the signal x(t), its output y(t) can be obtained by Fourier transform. , filtering and inverse Fourier transform To obtain: .

[0084] Sliding window low-pass filtering is an effective signal processing technique that can be used to smooth the noise in near-infrared brain imaging data, thereby restoring the true fluctuations of the blood oxygen signal. This method uses a time series Butterworth filter to process the signal through a progressive algorithm to achieve a low-pass filtering effect. The Butterworth filter is named for its flat amplitude response and can effectively remove high-frequency noise without introducing too much signal distortion. For a low-pass filter, its transfer function H2(f) is usually defined as: .

[0085] Where n represents the length of the sliding window.

[0086] Regarding artifact signal processing, artifacts cannot be removed when processing task-state near-infrared brain imaging data. The ASR (Artifact Subspace Reconstruction) algorithm is a subspace separation-based technique that can extract artifact components from complex signals. ASR learns the statistical properties of clean calibration data and compares these statistics with new data that may contain artifacts. The ASR algorithm consists of two parts: calibration and processing. During the calibration process, the data is filtered and then the covariance matrix is ​​estimated using the geometric median of subsequent samples of the input data segment.

[0087] Figure 3 is a block diagram illustrating an emotion recognition apparatus according to an exemplary embodiment of the present disclosure.

[0088] Reference Figure 3 The emotion recognition device 300 includes a first output unit 301 , a first acquisition unit 302 , a coarse screening unit 303 , a determination unit 304 , a second output unit 305 , a second acquisition unit 306 , and a recognition unit 307 .

[0089] The first output unit 301 is configured to output first task information to prompt a target user to perform a first task corresponding to the first task information.

[0090] Optionally, the first task includes an emotional image task lasting a first duration and a semantic fluency task lasting a second duration, and there is an interval lasting a preset interval duration between the emotional image task and the semantic fluency task.

[0091] Optionally, the first duration ranges from 65 seconds to 105 seconds; the second duration ranges from 190 seconds to 230 seconds; and the preset interval duration ranges from 20 seconds to 30 seconds.

[0092] The first acquiring unit 302 is configured to acquire first physiological data of the target user when the target user performs the first task.

[0093] The coarse screening unit 303 is configured to use the first emotion recognition model to process the first physiological data to obtain coarse screening emotion information.

[0094] The determining unit 304 is configured to determine the second task information from the plurality of candidate task information according to the roughly screened emotion information.

[0095] Optionally, the coarse-screened emotion information includes emotion indexes of multiple preset emotions, each emotion index is used to indicate the possibility that the user has the corresponding preset emotion, and the multiple candidate task information includes general task information and emotion task information corresponding to the multiple preset emotions one by one. The determination unit is further configured to: determine an emotion index greater than an index threshold from the emotion indexes of the multiple preset emotions, and use the preset emotion corresponding to the determined emotion index as the target emotion; when the number of target emotions is less than the preset number, determine the emotion task information corresponding to the target emotion from the multiple candidate task information as the second task information; when the number of target emotions is greater than or equal to the preset number, determine the general task information as the second task information.

[0096] Optionally, the emotional task corresponding to the emotional task information includes an interview task, and the general task corresponding to the general task information includes a picture description task.

[0097] Optionally, the second task information lasts for a third duration, and the third duration ranges from 100 seconds to 140 seconds.

[0098] The second output unit 305 is configured to output the second task information to prompt the target user to perform the second task corresponding to the second task information.

[0099] The second acquiring unit 306 is configured to acquire second physiological data of the target user when the target user performs the second task.

[0100] The recognition unit 307 is configured to process the second physiological data using the second emotion recognition model to obtain emotion recognition information.

[0101] Optionally, the second physiological data includes voice data and near-infrared brain imaging data, the voice data is data obtained by recording the content spoken by the target user, the second emotion recognition model includes a natural language processing model, a near-infrared emotion recognition model, and an information fusion model, and the recognition unit 307 is also configured to: use the natural language processing model to perform natural language processing on the voice data to identify the emotions of the target user and obtain semantic emotion information; use the near-infrared emotion recognition model to process the near-infrared brain imaging data to identify the emotions of the target user and obtain near-infrared emotion information; use the information fusion model to fuse the semantic emotion information and the near-infrared emotion information to obtain emotion recognition information.

[0102] Optionally, the emotion recognition device 300 further includes a comparison unit (not shown in the figure) and a prompt unit (not shown in the figure). The comparison unit is configured to perform consistency comparison based on the coarse-screened emotion information, the semantic emotion information and the near-infrared emotion information to obtain consistency information, wherein the consistency information is used to reflect the consistency degree between the different emotion information identified; the prompt unit is configured to output prompt information to indicate that the recognition result is inaccurate when the consistency information indicates that the consistency degree between the different emotion information identified is lower than a preset degree.

[0103] Optionally, the emotion recognition device 300 further includes a receiving unit (not shown in the figure) and an updating unit (not shown in the figure). The receiving unit is configured to receive feedback information from the target user regarding the emotion recognition information; the updating unit is configured to update the second emotion recognition model according to the feedback information.

[0104] The above has been referred to Figures 1 to 3 An emotion recognition method and apparatus according to exemplary embodiments of the present disclosure are described.

[0105] Figure 3Each unit in the illustrated emotion recognition device may be configured as software, hardware, firmware, or any combination of the above items to perform a specific function. For example, each unit may correspond to a dedicated integrated circuit, or may correspond to a pure software code, or may correspond to a module that combines software and hardware. In addition, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, a client, or a server, etc.).

[0106] In addition, refer to Figure 1 The described emotion recognition method can be implemented by a program (or instruction) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions can be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the emotion recognition method according to the present disclosure.

[0107] The computer program in the computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. It should be noted that the computer program can also be used to perform additional steps in addition to the above steps or perform more specific processing when performing the above steps. The contents of these additional steps and further processing have been described in reference to Figure 1 It is mentioned in the description of the related method, so it will not be repeated here to avoid repetition.

[0108] It should be noted that the various units in the emotion recognition device according to the exemplary embodiment of the present disclosure can completely rely on the operation of the computer program to realize the corresponding functions, that is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (for example, lib library) to realize the corresponding function.

[0109] on the other hand, Figure 3 The various units shown may also be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segment for performing the corresponding operation may be stored in a computer-readable medium such as a storage medium, so that the processor may perform the corresponding operation by reading and running the corresponding program code or code segment.

[0110] For example, an exemplary embodiment of the present disclosure may also be implemented as a computing device, which includes a storage component and a processor, wherein a set of computer executable instructions is stored in the storage component, and when the set of computer executable instructions is executed by the processor, the emotion recognition method according to the exemplary embodiment of the present disclosure is executed.

[0111] Specifically, the computing device can be deployed in a server or client, or can be deployed on a node device in a distributed network environment. In addition, the computing device can be a PC, a tablet device, a personal digital assistant, a smart phone, a web emotion recognition, or other device capable of executing the above instruction set.

[0112] Here, the computing device is not necessarily a single computing device, but may also be any device or circuit collection that can execute the above instructions (or instruction sets) individually or jointly. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0113] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0114] According to the exemplary embodiment of the present disclosure, some operations described in the emotion recognition method can be implemented by software, some operations can be implemented by hardware, and in addition, these operations can also be implemented by a combination of software and hardware.

[0115] The processor may execute instructions or codes stored in one of the storage components, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0116] The storage component may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage component may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage component and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the storage component.

[0117] In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the computing device may be connected to each other via a bus and / or a network.

[0118] The emotion recognition method according to the exemplary embodiment of the present disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logical device or operate according to non-precise boundaries.

[0119] Therefore, refer to Figure 1 The described emotion recognition method may be implemented by a system including at least one computing device and at least one storage device storing instructions.

[0120] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing an emotion recognition method according to an exemplary embodiment of the present disclosure, and a computer executable instruction set is stored in the storage device. When the computer executable instruction set is executed by the at least one computing device, the execution reference Figure 1 The described emotion recognition method.

[0121] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, comprising instructions, which, when executed by at least one computing device, prompt the at least one computing device to perform the emotion recognition method according to the present disclosure.

[0122] The above describes various exemplary embodiments of the present disclosure, and it should be understood that the above description is only exemplary and not exhaustive, and the present disclosure is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present disclosure, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present disclosure should be based on the scope of the claims.

Claims

1. An emotion recognition method, characterized in that: The emotion recognition method comprises: Outputting first task information to prompt a target user to perform a first task corresponding to the first task information; Acquiring first physiological data of the target user during the process of the target user performing the first task; Using a first emotion recognition model, processing the first physiological data to obtain coarse-screened emotion information; Determining second task information from a plurality of candidate task information according to the roughly screened emotional information; outputting the second task information to prompt the target user to perform the second task corresponding to the second task information; Acquiring second physiological data of the target user during the process of the target user performing the second task; The second physiological data is processed using a second emotion recognition model to obtain emotion recognition information.

2. The emotion recognition method according to claim 1, characterized in that: The first task includes an emotional image task lasting a first duration and a semantic fluency task lasting a second duration, and there is an interval lasting a preset interval duration between the emotional image task and the semantic fluency task; The first duration ranges from 65 seconds to 105 seconds; The second duration ranges from 190 seconds to 230 seconds; The preset interval duration ranges from 20 seconds to 30 seconds.

3. The emotion recognition method according to claim 1, characterized in that: The roughly screened emotional information includes emotional indexes of a plurality of preset emotions, each emotional index is used to indicate the possibility that the user has a corresponding preset emotion, the plurality of candidate task information includes general task information and emotional task information corresponding to the plurality of preset emotions one by one, wherein determining the second task information from the plurality of candidate task information according to the roughly screened emotional information includes: Determine an emotion index greater than an index threshold from the emotion indexes of the plurality of preset emotions, and use the preset emotion corresponding to the determined emotion index as the target emotion; When the number of the target emotions is less than a preset number, determining the emotion task information corresponding to the target emotion from the plurality of candidate task information as the second task information; When the number of the target emotions is greater than or equal to the preset number, the general task information is determined as the second task information.

4. The emotion recognition method according to claim 3, characterized in that: The emotional task corresponding to the emotional task information includes an interview task, and the general task corresponding to the general task information includes a picture description task; and / or The second task information lasts for a third duration, and a value range of the third duration is 100 seconds to 140 seconds.

5. The emotion recognition method according to any one of claims 1 to 4, characterized in that: The second physiological data includes voice data and near-infrared brain imaging data, the voice data is data obtained by recording the oral content of the target user, the second emotion recognition model includes a natural language processing model, a near-infrared emotion recognition model, and an information fusion model, wherein the second physiological data is processed using the second emotion recognition model to obtain emotion recognition information, including: Using the natural language processing model, performing natural language processing on the voice data to identify the emotion of the target user and obtain semantic emotion information; Using the near-infrared emotion recognition model, processing the near-infrared brain imaging data to identify the target user's emotions, and obtaining near-infrared emotion information; The information fusion model is used to fuse the semantic emotion information and the near-infrared emotion information to obtain the emotion recognition information.

6. The emotion recognition method according to claim 5, characterized in that: The emotion recognition method further comprises: Performing consistency comparison on the coarse-screened emotion information, the semantic emotion information and the near-infrared emotion information to obtain consistency information, wherein the consistency information is used to reflect the consistency degree between the identified different emotion information; When the consistency information indicates that the consistency between the identified different emotion information is lower than a preset degree, a prompt message is output to indicate that the recognition result is inaccurate.

7. An emotion recognition device, characterized in that: The emotion recognition device comprises: A first output unit is configured to output first task information to prompt a target user to perform a first task corresponding to the first task information; A first acquiring unit is configured to acquire first physiological data of the target user during the process in which the target user performs the first task; a coarse screening unit, configured to process the first physiological data using a first emotion recognition model to obtain coarse screening emotion information; a determining unit configured to determine second task information from a plurality of candidate task information according to the roughly screened emotion information; a second output unit, configured to output the second task information to prompt the target user to perform a second task corresponding to the second task information; A second acquiring unit is configured to acquire second physiological data of the target user during the process of the target user performing the second task; The recognition unit is configured to use a second emotion recognition model to process the second physiological data to obtain emotion recognition information.

8. A computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the emotion recognition method according to any one of claims 1 to 6.

9. A system comprising at least one computing device and at least one storage device storing instructions, characterized in that: When the instructions are executed by the at least one computing device, the at least one computing device is prompted to perform the emotion recognition method according to any one of claims 1 to 6.

10. A computer program product comprising instructions, characterized in that When the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the emotion recognition method according to any one of claims 1 to 6.

Images