Training device, estimation device, training method, estimation method, training program, and estimation program

Abstract

A training apparatus (10) comprises: acquisition units (11, 12) that acquire brain activity data pertaining to a subject under examination, and biological data pertaining to said subject and obtained when the brain activity data is measured, the brain activity data being acquired when said subject recognizes an emotional stimulus that consists of an image, video, or sound presenting an object said subject feels sympathetic to and is provided with an emotional label indicating a sympathetic emotion; a classification unit (13) that classifies the brain activity data into first brain activity data and second brain activity data, the first brain activity data corresponding to a period of time in which a variation in the numerical value of the biological data does not fall within a prescribed range, and the second brain activity data corresponding to a period of time in which the variation in the numerical value of the biological data falls within the prescribed range; and a training unit (16) that trains, with the first brain activity data as training data, a self-emotion estimator (141) for estimating the self-emotion of a subject for estimation on the basis of brain activity data pertaining to said subject, and trains, with the second brain activity data as training data, a cognitive emotion estimator for estimating, on the basis of the brain activity data pertaining to the subject for estimation, a cognitive emotion recognized from an object recognized by said subject.

Description

Technical Field

[0001] The present invention relates to a training device, an estimation device, a training method, an estimation method, a training program, and an estimation program.

Background Art

[0002] As a method for evaluating empathy for smooth communication, psychological empathy evaluation scales such as the Multidimensional Empathy Scale (MES), the Interpersonal Reactivity Index (IRI), the Questionnaire Measure of Emotional Empathy (QMEE), and the Theory of Mind Development Screening Test (ToM test) have been proposed.

[0003] These psychological empathy evaluation scales have problems in that it is difficult to evaluate empathy in real time because they are tests for subjectively and objectively evaluating each item, and there is uncertainty due to artificial evaluation.

[0004] In recent years, since the synchronization phenomenon of brain waves and electrocardiograms between two people during empathy has been discovered, research has been conducted on measuring the empathy state from biological information (Non-Patent Documents 1 and 2).

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

[0006] Human "empathy" can be broadly divided into two types: emotional empathy and cognitive empathy, both of which have advantages and disadvantages. Therefore, it is considered that an ideal empathy in communication is to balance the two types of empathy. By making these two empathy states measurable, it becomes possible to identify and intervene in extreme empathy states that cause communication stress.

[0007] However, the studies described in Non-Patent Documents 1 and 2 have a problem in that they regard the empathy state as "becoming the same state as the other party" and cannot distinguish between emotional empathy and cognitive empathy, which are the two states in empathy.

[0008] The present invention has been made in view of the above, and an object thereof is to provide a training device, an estimation device, a training method, an estimation method, a training program, and an estimation program that can quantitatively and in real time distinguish and measure emotional empathy and cognitive empathy.

Means for Solving the Problems

[0009] In order to solve the above-described problems and achieve the object, a training device according to the present invention includes a first estimation unit having a self-emotion estimator that estimates the self-emotion of the estimation target person based on the brain activity data of the estimation target person, and a second estimation unit having a cognitive-emotion estimator that estimates the cognitive emotion recognized from the object recognized by the estimation target person based on the brain activity data of the estimation target person. An acquisition unit that acquires brain activity data of a subject at the time of recognition of an emotional stimulus to which an emotional label indicating an emotion to be empathized is attached, which is an image, video, or audio in which the object to be empathized appears, and biological data of the subject at the time of measurement of the brain activity data; A classification unit that classifies the brain activity data into first brain activity data corresponding to a time period in which the numerical variation in the biological data is outside a predetermined range and second brain activity data corresponding to a time period in which the numerical variation in the biological data is within the predetermined range; And a training unit that trains the self-emotion estimator using the first brain activity data as training data and trains the cognitive-emotion estimator using the second brain activity data as training data.

[0010] In addition, the estimation device according to the present invention includes a first estimation unit having a self-emotion estimator that estimates the self-emotion of the person to be estimated based on brain activity data of the person to be estimated when recognizing an emotional stimulus that is an image, video, or audio in which the object of empathy appears and is given an emotion label indicating the emotion of empathy, a second estimation unit having a cognitive emotion estimator that estimates the cognitive emotion recognized from the object recognized by the person to be estimated based on the brain activity data of the person to be estimated, a first measurement unit that measures the degree of coincidence between the self-emotion estimated by the self-emotion estimator and the emotion label of the emotional stimulus and outputs the measurement result as the emotional empathy degree, and a second measurement unit that measures the degree of coincidence between the cognitive emotion estimated by the cognitive emotion estimator and the emotion label of the emotional stimulus and outputs the measurement result as the cognitive empathy degree.

Effect of the Invention

[0011] According to the present invention, it is possible to measure emotional empathy and cognitive empathy separately in real time, quantitatively, and in real time.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited by this embodiment. In the description of the drawings, the same parts are denoted by the same reference numerals.

[0014] [Embodiment] The present embodiment relates to a training device and an estimation device that enable quantitative measurement of two types of empathy, emotional empathy and cognitive empathy.

[0015] [Emotional Empathy, Cognitive Empathy] Human empathy is roughly divided into two types: emotional empathy and cognitive empathy, both of which have advantages and disadvantages. Therefore, it is considered that an ideal empathy in communication is to perform the two types of empathy in a well-balanced manner.

[0016] Emotional empathy is empathy that is accompanied by a change in one's own emotions and is understood by synchronizing with the emotions of the object. Correct emotional empathy is "one's own emotions (hereinafter referred to as self-emotions)" = "the emotions of the object". Emotional empathy has the merit of being able to feel the emotions of the other person as if they were one's own, but has the demerit of being centered on one's own feelings and not being able to consider the perspective of the other person.

[0017] Cognitive empathy is empathy that is not accompanied by a change in one's own emotions and is understood based on the state of the other person. Correct cognitive empathy is "the emotional state of the other person recognized (hereinafter referred to as cognitive emotions)" = "the emotions of the object". Cognitive empathy has the merit of being able to infer the state of the other person from the information obtained without being swayed by emotions, but has the demerit that there is a possibility that the emotional aspect is insufficiently inferred.

[0018] By making these two empathy states measurable, it becomes possible to identify and intervene in the "extreme empathy state" that causes communication stress. In this embodiment, two new quantitative measurements of empathy, namely emotional empathy and cognitive empathy, were established.

[0019] In order to estimate a person's detailed empathy state in real time, expression information (such as facial expressions and actions) and biological information (such as brain waves and heartbeats) are useful. However, expression information that can be faked or hidden has a high degree of uncertainty.

[0020] Here, brain activity data contains a vast amount of internal information of a person, including not only emotions but also cognitive information such as "what is being thought". Therefore, it can be said that brain activity data is the most suitable biological information for estimating the empathy state.

[0021] However, since brain activity data is a rich but noisy information source, it is very difficult to identify information related to the empathy state from the vast amount of information.

[0022] Therefore, in this embodiment, by combining two types of biological information, brain activity data and electro dermal activity (EDA), to train an estimator, an emotional empathy measurement device that can quantitatively measure emotional empathy and a cognitive empathy measurement device that can quantitatively measure cognitive empathy were established.

[0023] [Emotional Empathy Measurement Unit] The emotional empathy measurement unit (emotional empathy measurement device) in the embodiment will be described. FIG. 1 is a diagram for explaining the overview of the emotional empathy measurement unit.

[0024] As shown in Fig. 1, the emotional empathy measurement unit 24 has a self-emotion estimator 221 in the previous stage. The self-emotion estimator 221 is trained and estimates the emotional state of the person to be estimated from the input brain activity data of the person to be estimated, and outputs it as the self-emotion (emotion category or emotion dimension).

[0025] The brain activity data of the person to be estimated input to the self-emotion estimator 221 is the brain activity data when the person to be estimated recognizes an emotional stimulus. Also, the emotional stimulus is an image in which the object of empathy appears and to which an emotion label indicating the emotion to be empathized is attached, a video, or an audio. The emotional stimulus is, for example, the image "a face photo of a crying woman", and the emotion label "sad" is attached to it.

[0026] The emotional empathy measurement unit 24 measures the degree of consistency between the self-emotion output from the self-emotion estimator 221 and the emotion label of the emotional stimulus, and outputs the measured degree of consistency as the emotional empathy. Note that the emotional empathy measurement unit 24 calculates the correlation or distance between the self-emotion output from the self-emotion estimator 221 and the emotion label of the emotional stimulus as the degree of consistency.

[0027] [Cognitive Empathy Measurement Device] The cognitive empathy measurement unit (cognitive empathy measurement device) in the embodiment will be described. Fig. 2 is a diagram for explaining the outline of the cognitive empathy measurement unit.

[0028] As shown in Fig. 2, the cognitive empathy measurement unit 25 has a cognitive emotion estimator 231 in the previous stage. The cognitive emotion estimator 231 is trained and estimates the emotion recognized by the person to be estimated from the input brain activity data of the person to be estimated, and outputs it as the cognitive emotion (emotion category or emotion dimension).

[0029] The brain activity data of the person to be estimated input to the cognitive emotion estimator 231 is the brain activity data when the person to be estimated recognizes an emotional stimulus. The cognitive empathy measurement unit 25 measures the degree of coincidence between the cognitive emotion output from the cognitive emotion estimator 231 and the emotion label of the emotional stimulus, and outputs it as the cognitive empathy. The cognitive empathy measurement unit 25 calculates the correlation or distance between the cognitive emotion output from the cognitive emotion estimator 231 and the emotion label of the emotional stimulus as the degree of coincidence.

[0030] [Training data] In the present embodiment, two types of biological information, brain activity data and EDA value data, are combined to train the self-emotion estimator and the cognitive emotion estimator.

[0031] Here, it has been shown that there is a correlation between EDA value data and emotional changes (Reference 1). Reference 1: Lopez-Gil, Juan-Miguel et al., “Method for Improving EEG Based Emotion Recognition by Combining It with Synchronized Biometric and Eye Tracking Technologies in a Non-invasive and Low Cost Way”, rontiers in computational neuroscience vol. 10 85. 19 Aug. 2016, doi:10.3389 / fncom.2016.00085

[0032] The EDA value is known as biological information that reflects the activity of the autonomic nerves in the body. In Reference 1, it is described that the EDA value shows fluctuations when emotions change greatly. From this, in the embodiment, it is assumed that the EDA value fluctuates during emotional empathy accompanied by changes in self-emotion, and does not fluctuate greatly during cognitive empathy not accompanied by changes in self-emotion. A hypothesis shown in FIG. 3 was established, and training data for the self-emotion estimator and the cognitive emotion estimator was created.

[0033] FIG. 3 is a diagram showing the relationship between the EDA value in the state of empathy and each state of empathy. In the embodiment, as shown in FIG. 3, when there is a variation in the EDA value, it is set that the state of empathy is dominated by emotional empathy rather than cognitive empathy, and the brain activity data is dominated by self-emotion rather than cognitive emotion. Also, in the embodiment, when there is no variation in the EDA value, it is set that the state of empathy is dominated by cognitive empathy rather than emotional empathy, and the brain activity data is dominated by cognitive emotion rather than self-emotion.

[0034] Accordingly, in the embodiment, the brain activity data of the subject at the time of recognizing the emotional stimulus and the EDA value data of the subject at the time of measuring the brain activity data are acquired. And, in the embodiment, by capturing the change in emotion from the variation in the EDA value, the simultaneously measured brain activity data is classified into a state where emotional empathy is dominant and a state where cognitive empathy is dominant, and the self-emotion estimator and the cognitive emotion estimator are trained respectively.

[0035] Specifically, in the embodiment, the brain activity data is classified into first brain activity data corresponding to a time period in which the numerical variation in the EDA value data is outside a predetermined range and second brain activity data corresponding to a time period in which the numerical variation in the EDA value data is within the predetermined range. Note that the predetermined range may be set based on the accumulated data of the brain activity data and the EDA value data of a plurality of past subjects, or may be set for each subject based on the variation state of the EDA value data.

[0036] In the first brain activity data, self-emotion is dominant over cognitive emotion, corresponding to a state of empathy in which emotional empathy is dominant over cognitive empathy. In the second brain activity data, cognitive emotion is dominant over self-emotion, and the state of empathy is dominated by cognitive empathy rather than emotional empathy.

[0037] And, in the embodiment, the first brain activity data is used to train the self-emotion estimator as training data for the self-emotion estimator. Also, in the embodiment, the second brain activity data is used to train the self-emotion estimator as training data for cognitive emotion estimation.

[0038] [Training device] The training device according to the embodiment will be described. FIG. 4 is a diagram schematically showing an example of the configuration of the training device according to the embodiment.

[0039] The training device 10 according to the embodiment is realized, for example, by a computer including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., in which a predetermined program is loaded and the CPU executes the predetermined program. Further, the training device 10 has a communication interface for transmitting and receiving various information to and from other devices connected via a network or the like. As shown in FIG. 4, the training device 10 includes a brain activity data acquisition unit 11, an EDA value acquisition unit 12, a classification unit 13, a first estimation unit 14, a second estimation unit 15, and a training unit 16.

[0040] The brain activity data acquisition unit 11 acquires the brain activity data of the subject at the time of recognizing the emotional stimulus and inputs it to the classification unit 13. The brain activity data is, for example, time-series data obtained by measuring the brain activity of the subject, such as electroencephalogram, functional magnetic resonance imaging (fMRI), etc. The EDA value acquisition unit 12 acquires the EDA value data of the subject at the time of measuring the brain activity data acquired by the brain activity data acquisition unit 11 and inputs it to the classification unit 13.

[0041] The classification unit 13 classifies the input brain activity data into first brain activity data corresponding to a time period in which the numerical variation of the input EDA value data is outside a predetermined range, and second brain activity data corresponding to a time period in which the numerical variation of the EDA value data is within the predetermined range. The classification unit 13 classifies the brain activity data into first brain activity data and second brain activity data based on the hypothesis described with reference to FIG. 3. Then, the classification unit 13 inputs the first brain activity data to the first estimation unit 14 and inputs the second brain activity data to the second estimation unit.

[0042] The first estimation unit 14 includes a self - emotion estimator 141 that estimates the self - emotion of the person to be estimated based on the brain activity data of the person to be estimated. The self - emotion estimator 141 takes the first brain activity data as input and estimates the first self - emotion of the subject.

[0043] The second estimation unit 15 includes a cognitive - emotion estimator 151 that estimates the cognitive emotion recognized by the person to be estimated from the object recognized by the person to be estimated based on the brain activity data of the person to be estimated. The cognitive - emotion estimator 151 takes the second brain activity data as input and estimates the second cognitive emotion recognized by the subject from the object of the emotion stimulus.

[0044] The training unit 16 trains the self - emotion estimator 141 using the first brain activity data as training data, and trains the cognitive - emotion estimator 151 using the second brain activity data as training data.

[0045] [Overview of the training process] FIG. 5 is a diagram for explaining the overview of the training process according to the embodiment. As shown in FIG. 5, in the embodiment, when the subject executes the empathy task of recognizing an emotion stimulus (in (1) of FIG. 5), the brain activity data and the EDA value data of this subject are measured simultaneously (in (2) of FIG. 5).

[0046] In the training device 10, the classification unit 13 classifies this brain activity data based on the EDA value data (in (3) of FIG. 5). The classification unit 13 classifies the input brain activity data into first brain activity data corresponding to the time period when the numerical variation in the input EDA value data is outside the predetermined range (described as "with variation" in FIG. 5), and second brain activity data corresponding to the time period when the numerical variation in the EDA value data is within the predetermined range (described as "without variation" in FIG. 5).

[0047] After the empathy task, the subject answers a questionnaire (in (4) of FIG. 5). The questionnaire has items related to self - feeling and cognitive emotion.

[0048] The subject answers, according to the items of the questionnaire on self - emotions, about the emotional state of himself / herself (the subject) and the degree of emotional empathy. In the embodiment, the emotional state of the subject reported by the subject himself / herself (self - emotion) is used as the correct label for the training of the self - emotion estimator 141.

[0049] Also, the subject answers, according to the items of the questionnaire on cognitive emotions, about the emotional state recognized from the stimulus and the degree of cognitive empathy. In the embodiment, the emotional state recognized from the stimulus (cognitive emotion) reported by the subject himself / herself is used as the correct label for the training of the cognitive - emotion estimator 151.

[0050] As shown in the hypothesis shown in FIG. 3, the first brain - activity data contains a lot of information on self - emotions evoked by the stimulus. Therefore, the training unit 16 uses the first brain - activity data as the training data for the self - emotion estimator 141, and uses the self - emotion that the subject reported himself / herself in the questionnaire as the correct label to train the self - emotion estimator 141 (item (5) in FIG. 5).

[0051] Specifically, the training unit 16 inputs the first brain - activity data into the self - emotion estimator 141 as training data, and makes the self - emotion estimator 141 estimate the first self - emotion of the subject. Then, the training unit 16 updates the parameters of the self - emotion estimator 141 so that the first self - emotion estimated by the self - emotion estimator 141 matches the self - emotion of the correct label reported by the subject himself / herself.

[0052] As shown in the hypothesis of FIG. 3, the second brain - activity data contains a lot of information on emotions recognized from the stimulus. Therefore, the training unit 16 uses the second brain - activity data as the training data for the cognitive - emotion estimator 151, and uses the cognitive emotion that the subject reported himself / herself in the questionnaire as the correct label to train the cognitive - emotion estimator 151 (item (6) in FIG. 5).

[0053] Specifically, the training unit 16 inputs the second brain activity data into the cognitive emotion estimator 151 as training data, and causes the cognitive emotion estimator 151 to estimate the second cognitive emotion of the subject. Then, the training unit 16 updates the parameters of the cognitive emotion estimator 151 so that the second cognitive emotion estimated by the cognitive emotion estimator 151 matches the cognitive emotion of the correct label self-reported by the subject.

[0054] [Processing Procedure of Training Process] Next, the processing procedure of the training method executed by the training device 10 will be described. FIG. 6 is a flowchart showing the processing procedure of the training method according to the embodiment.

[0055] As shown in FIG. 6, the training device 10 acquires the brain activity data of the subject at the time of recognizing the emotional stimulus and the EDA value data of the subject at the time of measuring the brain activity data. (Steps S1, S2). The classification unit 13 classifies the input brain activity data into first brain activity data and second brain activity data. (Step S3).

[0056] The training unit 16 inputs the first brain activity data into the self-emotion estimator 141 as training data (Step S4), and causes the self-emotion estimator 141 to execute a first estimation process for estimating the first self-emotion of the subject. (Step S5).

[0057] The training unit 16 updates the parameters of the self-emotion estimator 141 so that the first self-emotion matches the self-emotion of the correct label self-reported by the subject. (Step S6). The training unit 16 determines whether a predetermined end condition is satisfied. (Step S7). The end condition is, for example, when the number of parameter updates reaches a predetermined number, when the parameter update amount becomes equal to or less than a predetermined threshold, and the like.

[0058] If the training device 10 does not satisfy the predetermined end condition (Step S7: No), it returns to Step S4. The training device 10 repeats Steps S4 to S6 until the predetermined end condition is satisfied.

[0059] Further, the training unit 16 inputs the second brain activity data into the cognitive emotion estimator 151 as training data (step S8), and causes the cognitive emotion estimator 151 to execute a second estimation process for estimating the second cognitive emotion of the subject (step S9).

[0060] The training unit 16 updates the parameters of the cognitive emotion estimator 151 so that the second self-emotion matches the cognitive emotion of the correct label self-reported by the subject (step S10). The training unit 16 determines whether or not a predetermined end condition is satisfied (step S11). The end condition is, for example, when the number of parameter updates reaches a predetermined number, when the parameter update amount becomes equal to or less than a predetermined threshold, and the like.

[0061] If the training device 10 does not satisfy the predetermined end condition (step S11: No), it returns to step S4. The training device 10 repeats steps S8 to S10 until the predetermined end condition is satisfied.

[0062] When the training device 10 satisfies the predetermined end condition (step S7: Yes), (step S11: Yes), it ends the training process.

[0063] [Estimation device] Next, an estimation device having a self-emotion estimator and a cognitive emotion estimator trained in the training device 10 will be described. FIG. 7 is a diagram schematically showing an example of the configuration of the estimation device according to the embodiment.

[0064] The estimation device 20 according to the embodiment is realized, for example, by a computer including a ROM, a RAM, a CPU, etc., in which a predetermined program is loaded and the CPU executes the predetermined program. Further, the estimation device 20 has a communication interface for transmitting and receiving various information to and from other devices connected via a network or the like. As shown in FIG. 7, the estimation device 20 includes a brain activity data reception unit 21, a first estimation unit 22, a second estimation unit 23, an emotional empathy measurement unit 24 (first measurement unit), and a cognitive empathy measurement unit 25 (second measurement unit).

[0065] The brain activity data reception unit 21 receives the input of the brain activity data of the person to be estimated at the time of recognizing an emotional stimulus.

[0066] The first estimation unit 22 has a self - emotion estimator 221 that estimates the self - emotion of the person to be estimated based on the brain activity data of the person to be estimated. The self - emotion estimator 221 has been trained by the training device 10. As described above, the self - emotion estimator 221 uses, as training data, the first brain activity data corresponding to the time zone in which the variation in the numerical value of the EDA value data of the subject at the time of measuring the brain activity data is outside the predetermined range among the brain activity data of the subject at the time of recognizing an emotional stimulus.

[0067] The second estimation unit 23 has a cognitive - emotion estimator 231 that estimates the cognitive emotion recognized from the object recognized by the person to be estimated based on the brain activity data of the person to be estimated. The cognitive - emotion estimator 231 has been trained by the training device 10. The cognitive - emotion estimator 231 uses, as training data, the second brain activity data corresponding to the time zone in which the variation in the numerical value of the EDA value data of the subject at the time of measuring the brain activity data is within the predetermined range among the brain activity data of the subject at the time of recognizing an emotional stimulus.

[0068] The emotional empathy measurement unit 24 measures the degree of coincidence between the self - emotion estimated by the self - emotion estimator 221 and the emotion label of the emotional stimulus, and outputs the measurement result as the emotional empathy.

[0069] The cognitive empathy measurement unit 25 measures the degree of coincidence between the cognitive emotion estimated by the cognitive - emotion estimator 231 and the emotion label of the emotional stimulus, and outputs the measurement result as the cognitive empathy.

[0070] [Outline of Estimation Processing] FIG. 8 is a diagram for explaining the outline of the estimation processing according to the embodiment. As shown in FIG. 8, in the embodiment, the brain activity data of the person to be estimated who recognizes an emotional stimulus is input to the self - emotion estimator 221 and the cognitive - emotion estimator 231 (FIG. 8(1)).

[0071] The self - emotion estimator 221 estimates the self - emotion of the person to be estimated based on the brain activity data of the person to be estimated (Fig. 8(2)). The emotional empathy measurement unit 24 measures the degree of coincidence (correlation or distance) between the self - emotion output from the self - emotion estimator 221 and the emotion label of the emotional stimulus, and outputs the measured degree of coincidence as the emotional empathy (Fig. 8(3)).

[0072] The cognitive - emotion estimator 231 estimates the cognitive emotion recognized from the object recognized by the person to be estimated based on the brain activity data of the person to be estimated (Fig. 8(4)). The cognitive empathy measurement unit 25 measures the degree of coincidence (correlation or distance) between the cognitive emotion output from the cognitive - emotion estimator 231 and the emotion label of the emotional stimulus, and outputs it as the cognitive empathy (Fig. 8(5)).

[0073] During the training of the self - emotion estimator 221 and the cognitive - emotion estimator 231, the brain activity data was classified using the EDA value. When performing the estimation, the measured brain activity data is input into the self - emotion estimator 221 and the cognitive - emotion estimator 231. If the brain activity data contains a lot of information about self - emotion, a high value of emotional empathy is output, and a low value of cognitive empathy is output. If the brain activity data contains a lot of information about cognitive emotion, a low value of emotional empathy is output, and a high value of cognitive empathy is output.

[0074] Also, during the training and experimental stages, the empathy for emotional stimuli with emotion labels is measured. However, during the estimation process, if the emotional state of the target is clear, it is applicable even for an actual person. In that case, the emotional state of the target is defined by using existing emotion recognition technologies or the self - report of emotions by the target person. Then, in the estimation device 20, the degree of coincidence between the defined emotional state and the self - emotion output from the self - emotion estimator 221, or the cognitive emotion output from the cognitive - emotion estimator 231, is measured as the emotional empathy and the cognitive empathy.

[0075] [Processing Procedure of Estimation Processing] Next, the processing procedure of the training method executed by the estimation device 20 will be described. Fig. 9 is a flowchart showing the processing procedure of the estimation method according to the embodiment.

[0076] As shown in FIG. 9, when brain activity data of a person to be estimated who recognizes an emotional stimulus is input (step S21), the estimation device 20 inputs this brain activity data to the self-emotion estimator 221 and the cognitive-emotion estimator 231.

[0077] In the first estimation unit 22, the self-emotion estimator 221 performs a first estimation process of estimating the self-emotion of the person to be estimated based on the brain activity data of the person to be estimated (step S22). The emotional empathy measurement unit 24 measures the degree of coincidence between the self-emotion estimated by the self-emotion estimator 221 and the emotion label of the emotional stimulus as the emotional empathy degree (step S23), and outputs the measured emotional empathy degree (step S24).

[0078] In the second estimation unit 23, the cognitive-emotion estimator 231 performs a second estimation process of estimating the cognitive emotion recognized from the object recognized by the person to be estimated based on the brain activity data of the person to be estimated (step S25). The cognitive empathy measurement unit 25 measures the degree of coincidence between the cognitive emotion estimated by the cognitive-emotion estimator 231 and the emotion label of the emotional stimulus as the cognitive empathy degree (step S26), and outputs the measured cognitive empathy degree (step S27).

[0079] [Effects of the Embodiment] As described above, the training device 10 according to the embodiment classifies the brain activity data of the subject into first brain activity data in which emotional empathy is dominant and second brain activity data in which cognitive empathy is dominant based on the presence or absence of fluctuations outside the predetermined range of the EDA value. Then, the training device 10 uses the first brain activity data in which emotional empathy is dominant as training data to train the self-emotion estimator 141, and uses the second brain activity data in which cognitive empathy is dominant as training data to train the cognitive-emotion estimator 151.

[0080] Therefore, according to the training device 10, it is possible to provide a self-emotion estimator that can accurately estimate the self-emotion of the person to be estimated and a cognitive-emotion estimator that can accurately estimate the cognitive emotion of the person to be estimated.

[0081] Then, the estimation device 20 estimates by distinguishing the self-emotion and cognitive emotion of the estimator using the self-emotion estimator 221 and the cognitive emotion estimator 231 trained by the training device 10.

[0082] Therefore, according to the estimation device 20, it is possible to quantitatively distinguish and measure emotional empathy and cognitive empathy. In addition, the estimation device 20 can measure emotional empathy and cognitive empathy only by inputting the brain activity data of the estimation target into the self-emotion estimator 221 and the cognitive emotion estimator 231, so emotional empathy and cognitive empathy can be acquired almost in real time.

[0083] In this way, according to the training device 10 and the estimation device 20, it is possible to distinguish and measure emotional empathy and cognitive empathy in real time, quantitatively, and in real time. As a result, based on the measurement result of the estimation device 20, the estimation target person can become aware of what tendency his / her empathy state has.

[0084] In addition, according to the estimation device 20, it is possible to visualize the causes of communication stress such as being unable to correctly understand the state or thoughts of the target due to extreme emotional empathy, or being mentally burdened by being overly influenced by the negative emotions of the target due to extreme cognitive empathy.

[0085] Moreover, grasping one's own empathy state using the estimation device 20 can not only improve the communication stress of healthy people, but also be applied to rehabilitation and intervention for people who are not good at empathy (for example, alexithymia, autism patients, etc.).

[0086] In the present embodiment, although the brain activity data is classified into first brain activity data and second brain activity data using EDA value data, the brain activity data may be classified using other biological data capable of capturing autonomic nerve activity, such as an electrocardiogram. In this case, the brain activity data may be classified into first brain activity data corresponding to a time period in which the numerical variation in the biological data is outside a predetermined range, and second brain activity data corresponding to a time period in which the numerical variation in the biological data is within the predetermined range.

[0087] [Regarding the System Configuration of the Embodiment] Each component of the training device 10 and the estimation device 20 is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific forms of the distribution and integration of the functions of the training device 10 and the estimation device 20 are not limited to those shown in the figure, and all or part of them can be functionally or physically distributed or integrated in any unit according to various loads, usage situations, etc.

[0088] In addition, each process performed in the training device 10 and the estimation device 20 may be realized in whole or in any part by a program analyzed and executed by a CPU, a GPU (Graphics Processing Unit), and a CPU, a GPU. Also, each process performed in the training device 10 and the estimation device 20 may be realized as hardware by wired logic.

[0089] In addition, among the processes described in the embodiment, all or part of the processes described as being automatically performed can also be performed manually. Or, all or part of the processes described as being performed manually can also be automatically performed by a known method. In addition, regarding the above-described and illustrated process procedures, control procedures, specific names, and information including various data and parameters, they can be appropriately changed unless otherwise specified.

[0090] [Program] FIG. 10 is a diagram showing an example of a computer in which a training device 10 and an estimation device 20 are realized by executing a program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0091] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System), for example. The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0092] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs defining the respective processes of the training device 10 and the estimation device 20 are implemented as a program module 1093 in which executable code by the computer 1000 is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing the same processes as the functional configurations in the training device 10 and the estimation device 20 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0093] In addition, the setting data used in the processes of the above-described embodiments is stored, for example, in the memory 1010 or the hard disk drive 1090 as program data 1094. Then, the CPU 1020 reads out the program modules 1093 and the program data 1094 stored in the memory 1010 or the hard disk drive 1090 to the RAM 1012 and executes them as necessary.

[0094] Note that the program modules 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, and may be stored, for example, in a removable storage medium and read out by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program modules 1093 and the program data 1094 may be stored in another computer connected via a network (such as a LAN (Local Area Network) or a WAN (Wide Area Network)). Then, the program modules 1093 and the program data 1094 may be read out by the CPU 1020 from the other computer via the network interface 1070.

[0095] As described above, the embodiments to which the invention made by the present inventor is applied have been described. However, the present invention is not limited by the description and the drawings that form a part of the disclosure of the present invention according to the present embodiment. That is, all other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiment are included in the scope of the present invention.

Description of Reference Numerals

[0096] 10 Training device 11 Brain activity data acquisition unit 12 EDA value acquisition unit 13 Classification unit 14, 22 First estimation unit 15, 23 Second estimation unit 16 Training unit 20 Estimation device 21 Brain activity data reception unit 24 Emotional empathy measurement unit 25 Cognitive empathy measurement unit 141,221 Self-emotion estimator 151,231 Cognitive emotion estimator

Claims

1. A first estimation unit having a self - emotion estimator that estimates the self - emotion of the target person based on the brain activity data of the target person; A second estimation unit having a cognitive - emotion estimator that estimates the cognitive emotion recognized by the target person from the object recognized by the target person based on the brain activity data of the target person; An acquisition unit that acquires the brain activity data of a subject at the time of recognizing an emotional stimulus to which an emotion label indicating the emotion to be empathized is attached, which is an image, video, or audio in which the object to be empathized appears, and the biological data of the subject at the time of measuring the brain activity data; A classification unit that classifies the brain activity data into first brain activity data corresponding to a time period in which the numerical variation in the biological data is outside a predetermined range and second brain activity data corresponding to a time period in which the numerical variation in the biological data is within the predetermined range; A training unit that trains the self - emotion estimator using the first brain activity data as training data and trains the cognitive - emotion estimator using the second brain activity data as training data; A training device, characterized by comprising the above.

2. The self - emotion estimator estimates the first self - emotion of the subject using the first brain activity data as an input; The cognitive - emotion estimator estimates the second cognitive emotion recognized by the subject from the object of the emotional stimulus using the second brain activity data as an input; The training unit updates the parameters of the self - emotion estimator so that the first self - emotion matches the self - emotion at the time of recognizing the emotional stimulus self - reported by the subject, and updates the parameters of the cognitive - emotion estimator so that the second cognitive emotion matches the cognitive emotion at the time of recognizing the emotional stimulus self - reported by the subject. The training device according to Claim 1, characterized by the above.

3. A first estimation unit having a self - emotion estimator that estimates the self - emotion of the target person based on the brain activity data of the target person at the time of recognizing an emotional stimulus to which an emotion label indicating the emotion to be empathized is attached, which is an image, video, or audio in which the object to be empathized appears; A second estimation unit having a cognitive - emotion estimator that estimates the cognitive emotion recognized by the target person from the object recognized by the target person based on the brain activity data of the target person; A first measurement unit that measures the degree of coincidence between the self - emotion estimated by the self - emotion estimator and the emotion label of the emotional stimulus, and outputs the measurement result as an emotional empathy degree; A second measuring unit that measures the degree of coincidence between the cognitive emotion estimated by the cognitive emotion estimator and the emotion label of the emotion stimulus, and outputs the measurement result as the cognitive empathy degree; An estimation device characterized by comprising the same. **Claim 4** The self-emotion estimator is trained using, as training data, first brain activity data corresponding to a time period in which the numerical variation among the biological data of the subject at the time of measurement of the brain activity data is outside a predetermined range, among the brain activity data of the subject at the time of recognition of the emotion stimulus; The estimation device according to claim 3, wherein the cognitive emotion estimator is trained using, as training data, second brain activity data corresponding to a time period in which the numerical variation among the biological data of the subject at the time of measurement of the brain activity data is within a predetermined range, among the brain activity data of the subject at the time of recognition of the emotion stimulus. **Claim 5** A training method executed by a training device, comprising: An acquisition step of acquiring brain activity data of a subject at the time of recognition of an emotion stimulus that is an image, video, or audio in which a target of empathy appears and to which an emotion label indicating the emotion to be empathized is attached, and biological data of the subject at the time of measurement of the brain activity data; A classification step of classifying the brain activity data into first brain activity data corresponding to a time period in which the numerical variation among the biological data is outside a predetermined range and second brain activity data corresponding to a time period in which the numerical variation among the biological data is within the predetermined range; A first training step of training a self-emotion estimator that estimates the self-emotion of the estimation target person based on the brain activity data of the estimation target person, using the first brain activity data as training data; A second training step of training a cognitive emotion estimator that estimates the cognitive emotion recognized from the target recognized by the estimation target person based on the brain activity data of the estimation target person, using the second brain activity data as training data; A training method characterized by including the above. **Claim 6** An estimation method executed by an estimation device, comprising: A first estimation step of estimating the self-emotion of an estimation target person based on the brain activity data of the estimation target person at the time of recognition of an emotion stimulus that is an image, video, or audio in which a target of empathy appears and to which an emotion label indicating the emotion to be empathized is attached, using a self-emotion estimator; A second estimation step of estimating the cognitive emotion recognized from the target recognized by the estimation target person based on the brain activity data of the estimation target person, using a cognitive emotion estimator; A first measurement step of measuring a degree of coincidence between the self-emotion estimated by the self-emotion estimator and the emotion label of the emotion stimulus, and outputting the measurement result as an emotional empathy degree; A second measurement step of measuring a degree of coincidence between the cognitive emotion estimated by the cognitive emotion estimator and the emotion label of the emotion stimulus, and outputting the measurement result as a cognitive empathy degree; An estimation method characterized by including the above.

7. A training program for causing a computer to function as the training device according to Claim 1 or 2.

8. An estimation program for causing a computer to function as the estimation device according to Claim 3 or 4.

Images