Emotion estimation device, emotion display method, and emotion display program

Abstract

To accurately estimate an emotion.SOLUTION: An emotion estimation device for estimating an emotion of an object person comprises a controller. The controller acquires a biological signal of the object person detected by a sensor, calculates a first index and a second index based on the biological signal, and arranges emotion type information corresponding to each quadrant with the first index and the second index as axes. The controller provides an emotion map in which a mark image is arranged at a coordinate position based on the first index and the second index.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an emotion estimation device, an emotion display method, and an emotion display program.

Background Art

[0002] There is known a technique for estimating a subject's emotion by applying information obtained from the waveform of the subject's heart (electrocardiogram waveform) to the Russell circle model (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional technology has a problem that it is difficult to accurately estimate emotions based on biological signals.

[0005] The Russell circle model is a model in which emotion types are arranged on a circle centered at the origin in a coordinate plane with arousal on the vertical axis (AROUSAL) and valence, an emotional value of pleasure - displeasure, on the horizontal axis (VALENCE).

[0006] In this Russell circle model, the emotion type of the subject is estimated by plotting the arousal and valence of the subject on the coordinate plane of the Russell circle model.

[0007] In Russell's annular model, arousal and valence are psychological construct concepts and there are many problems in realizing them as an actual emotion estimation device. For example, in Russell's annular model, it is necessary to estimate the arousal and valence of a subject. However, when realizing it as an emotion estimation device, some kind of biological signal of the subject needs to be measured, and the arousal and valence are estimated from the measured values. However, regarding what kind of biological signal of the subject is processed and how to estimate these arousal and valence, it has not been established, which is a major problem for realizing an emotion estimation device.

[0008] For this reason, although proposals for emotion estimation devices based on the Russell annular model have been made conventionally, it is difficult to accurately estimate the emotions of subjects.

[0009] The present invention has been made in view of the above, and an object thereof is to accurately estimate emotions.

Means for Solving the Problems

[0010] In order to solve the above-described problems and achieve the object, an emotion estimation device according to the present invention is an emotion estimation device that estimates the emotion of a target person and includes a controller. The controller acquires the biological signal of the target person detected by a sensor, calculates a first index and a second index based on the biological signal, and arranges emotion type information corresponding to each quadrant with the first index and the second index as axes. The controller provides an emotion map in which a mark image is arranged at a coordinate position based on the first index and the second index.

Effects of the Invention

[0011] According to the present invention, since emotions are estimated according to the state of the first index and the state of the second index based on the biological signal of the subject, the state of the first index and the state of the second index are set appropriately based on scientific (medical) evidence and experimental results, and also the emotion type for the combined state of the state of the first index and the state of the second index is set appropriately based on scientific (medical) evidence and experimental results. Thus, it becomes possible to accurately estimate emotions.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the emotion estimation device, emotion display method, and emotion display program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited by the embodiments shown below.

[0014] [First Embodiment] First, the estimation system according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing a configuration example of the estimation system according to the first embodiment.

[0015] As shown in FIG. 1, the estimation system 1 includes a server 10, a terminal device 20, a sensor 31, and a sensor 32. The estimation system 1 estimates the emotion of the subject U02.

[0016] The subject U02 is, for example, an e-sports player. The estimation system 1 estimates the emotion of the subject U02 who is playing a video game. In the description of this embodiment, for the sake of clarity and easy understanding, as an application example, the application scenario in the above e-sports is assumed, and the state transition and the like will also be described.

[0017] The estimation result of the emotion is used, for example, for the mental training of the subject U02 in e-sports. For example, during the play of a video game, for a scene where the subject U02 feels emotions (such as anxiety, anger, etc.) that are disadvantageous in the game, it is determined that intensive training corresponding to the emotional state is necessary.

[0018] In addition, as another application example, the subject U02 may be a patient in a medical institution. In this case, the emotion estimated by the estimation system 1 is used for examinations, treatments, and the like.

[0019] For example, when the subject U02, who is a patient, feels anxious, the staff in the medical institution can take countermeasures such as counseling.

[0020] Also, the subject U02 may be a student in an educational institution. In this case, the emotion estimated by the estimation system 1 is used to improve the content of the lessons.

[0021] For example, when the subject U02, who is a student, feels bored with the lesson, the teacher can improve the content of the lesson to be more interesting to the student.

[0022] Also, the subject U02 may be a driver of a vehicle. In this case, the emotion estimated by the estimation system 1 is used to promote safe driving.

[0023] For example, when the subject U02, who is a driver, does not feel appropriate tension during driving, the in-vehicle device outputs a message prompting the driver to concentrate on driving.

[0024] Also, the subject U02 may be a viewer of content such as video and music. In this case, the emotion estimated by the estimation system 1 is used for creating further content.

[0025] For example, a distributor of video content can collect scenes that the subject U02, who is a viewer, feels enjoyable and create a highlight video.

[0026] The server 10 and the terminal device 20 are connected via the network N. For example, the network N is the Internet or an intranet.

[0027] For example, the terminal device 20 is a personal computer, a smartphone, a tablet computer, or the like. The terminal device 20 is used by the analyst U01.

[0028] The sensors 31 and 32 transmit the detected sensor signals to the terminal device 20.

[0029] The sensor 31 is, for example, a headgear type electroencephalogram sensor. Also, the sensor 32 is, for example, a wristband type pulse sensor.

[0030] For example, the sensors 31 and 32 are communicatively connected to the terminal device 20 according to communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark), and transmit sensor signals to the terminal device 20.

[0031] The processing flow of the estimation system 1 will be described with reference to FIG. 1.

[0032] Server 10 extracts emotion types in advance from medical evidence (step S1). The medical evidence is, for example, papers and books. The method for extracting emotion types will be described later.

[0033] The terminal device 20 transmits the index values of a plurality of indicators based on the biological signal to the server 10 (step S2). For example, the terminal device 20 transmits the index values of two different indicators related to brain waves or heartbeats.

[0034] Here, the index value is the value of an indicator related to the biological signal. For example, "average of heart rate intervals" and "heart rate LF (Low Frequency) component" are indicators. Also, the specific value (for example, a numerical value) corresponding to each indicator is the index value. Note that the index value is the sensor value of each sensor or a value calculated from the sensor value.

[0035] Server 10 generates a model based on the extracted emotion types (step S3). At this time, server 10 generates a model that matches the index values received from terminal device 20. The method for generating the model will be described later.

[0036] Then, server 10 uses the generated model to identify the emotion type from the index values (step S4). Server 10 provides the identification result of the emotion type to terminal device 20 (step S5).

[0037] FIG. 2 is a diagram showing a configuration example of the server according to the first embodiment. Server 10 is an example of a computer that executes the generation method. Also, server 10 is an example of an estimation device.

[0038] As shown in FIG. 2, server 10 includes a communication unit 11, a storage unit 12, and a control unit 13, a so-called controller 13.

[0039] The communication unit 11 is an interface for data communication with other devices via the network N. The communication unit 11 is, for example, a NIC (Network Interface Card).

[0040] The storage unit 12 and the controller 13 of the server 10 are realized by, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, input / output ports, etc., and various circuits.

[0041] The CPU of the computer functions as the extraction unit 131, the generation unit 132, the specifying unit 133, and the providing unit 134 of the controller 13 by, for example, reading and executing a program stored in the ROM.

[0042] Also, the storage unit 12 corresponds to a RAM or a flash memory. The RAM and the flash memory store an emotion type information table 121 and the like.

[0043] Note that the server 10 may acquire the above-described program and various information via other computers (servers) connected by a wired or wireless network and a portable recording medium.

[0044] The emotion type information table 121 is information that associates information regarding an index with an emotion type. Here, an overview of each item of the emotion type information table 121 will be described. The creation method and the usage method of the emotion type information table 121 will be described later.

[0045] FIG. 3 is a diagram showing an example of a data table for storing emotion type information. As shown in FIG. 3, the items of the emotion type information table 121 include an index ID, an index name, a sensor, positive / negative, an explanation, an emotion axis, and positive and negative emotion types. Emotion type information is formed by these data. For the sake of clarity in the following description, when indicating the data (value) itself in each item, the word "data" is added for expression. For example, "index ID data" represents the values themselves such as "VS01", "VS02", etc.

[0046] The index ID of the emotion type information table 121 is a character string that functions as an ID for identifying an index (the character string is stored). Using this index ID data as the primary key data, that is, a record is configured for each index ID data, and the record stores the index name data, sensor data, explanation data, emotion axis data, positive emotion type data, and negative emotion type data associated with the index ID data.

[0047] The "index name" of the emotion type information table 121 is information (character string) representing the name of the index.

[0048] The "sensor" of the emotion type information table 121 is information for specifying the sensor necessary to obtain the index value of the corresponding index. For example, when the index ID is VS01 (record), the target sensor is an electroencephalogram sensor.

[0049] Here, in the process described later, the index is used as an axis constituting a coordinate plane. Also, in this process, whether the index value is in the positive direction (larger) or the negative direction (smaller) from the origin (judgment threshold value) has meaning.

[0050] The "positive emotion type data" and "negative emotion type data" of the emotion type information table 121 are information indicating whether the record stores data corresponding to the case where the index value is positive and the case where it is negative, respectively.

[0051] Therefore, it is assumed that the index value is standardized so that the average is 0. That is, when the biological signal is the average value, the above normalization is performed based on the presumption that no emotion associated with the biological signal has occurred (the subject is in a normal state). Note that the index value may be converted not only to the above normalization but also to a format that is easy to handle in the coordinate plane or a format that improves the accuracy of the estimation result (such as correction based on the evaluation result of the emotion estimation result). When the index value is standardized, the determination threshold for classifying the state of the index is 0.

[0052] In addition, the server 10 may change the normalization method or conversion method of the index value according to the purpose of using the estimated emotion. For example, in the scenarios of e-sports and automobile driving, the levels of emotions affected are different (in automobile driving, calmness is required more than in e-sports from the perspective of safe driving). Therefore, the server 10 may change the normalization method or conversion method of the index value so that, for example, the excitement emotion is judged with a lower determination threshold.

[0053] Alternatively, instead of converting the format of the index value, the position of the origin of the coordinate plane may be adjusted according to the index value.

[0054] The "explanation" in the emotion type information table 121 is an explanatory text regarding the record. For example, it explains the relationship between the data of each item of the data.

[0055] The "emotion axis" in the emotion type information table 121 is a character string used as a label when the index is used as an axis of the coordinate plane. In the example shown in FIG. 3, the index name data indicates the meaning of the emotion axis when the values on each side of the positive / negative data are obtained.

[0056] The "positive-side emotion type" and "negative-side emotion type" in the emotion type information table 121 are sets of keywords representing the emotion types corresponding to the record, and indicate the emotion types of the subject when the index name data becomes the values on each side of the positive / negative data.

[0057] For example, in the record on the first line of the emotion type information table 121 in FIG. 3 (the record with the index ID data "VS01"), the index name data is "β wave / α wave of brain waves", the sensor type data for obtaining the index value of the index is "brain wave sensor", the positive explanation data is "the β wave of brain waves increases relatively to the α wave" (positive side), the emotion axis data is "awake - not awake", and the positive emotion type data is "happy, joyful, angry, sad, depressed", and the negative emotion type data is "unpleasant, anxious, scared, relaxed, calm" are stored.

[0058] In other words, the index ID data "VS01" is an index for measuring brain waves using a "brain wave sensor". And if the measurement result (index value) that "the β wave of brain waves increases relatively to the α wave" is positive, it is presumed that there may be emotions such as "happy, joyful, angry, sad, depressed" in the "awake" state. Also, if the measurement result (index value) that "the β wave of brain waves increases relatively to the α wave" is negative (no), it is presumed that there may be emotions such as "unpleasant, anxious, scared, relaxed, calm" in the "not awake" state. That is what it means.

[0059] The processing content of each part of the controller 13 will be described. The main body of the processing by the extraction unit 131, generation unit 132, specification unit 133, and provision unit 134 in the following description can be paraphrased as the controller 13.

[0060] The extraction unit 131 extracts the emotion type in association with the index from medical evidence. FIG. 4 is a diagram for explaining the method of extracting the emotion type.

[0061] As shown in FIG. 4, the extraction unit 131 extracts information regarding the index and emotion type by performing natural language analysis on the text written in papers, books, etc. For example, the first emotion type information or the second emotion type information is generated by the language analysis of the document in which medical evidence is described.

[0062] The extraction unit 131 may perform natural language analysis using existing machine learning methods. Further, the process of extracting the emotion type by associating it with an index from medical evidence may be performed manually. In this case, the extraction unit 131 extracts information regarding the index and the emotion type based on the input information by the operator.

[0063] In the example of FIG. 4, based on the text “As the β wave increases, emotions such as joy, anger, and sadness are amplified.”, the extraction unit 131 extracts and associates the emotion types of “happy”, “anger”, and “sadness” with the increase in the β wave.

[0064] Also, in the example of FIG. 4, based on the text “When the α wave increased, there were significantly more people who felt anxiety or fear statistically.”, the extraction unit 131 extracts and associates the emotion types of “anxiety” and “fear” with the increase in the α wave.

[0065] Also, although the evidence examples are omitted, in the same manner as above, for example, based on the text “In the waking state, the β wave of the brain wave becomes larger than the α wave.”, “waking” is associated and extracted with respect to the emotion axis.

[0066] Note that which of the β wave and the α wave increases is an example of an index based on the brain wave which is one of the biological information. Further, based on such information extracted by the extraction unit 131, the emotion type information table 121 is generated. In the above example, in the record of the index ID data “VS01” (the index ID data is appropriately set so that the same value does not exist), as the index name data, “β wave / α wave of the brain wave”, as the sensor data, “brain wave sensor”, as the explanation data, “the β wave of the brain wave increases relatively to the α wave”, as the emotion axis data, “waking - non - waking”, as the positive - side emotion type data, “happy, joy, anger, sadness, depression”, and as the negative - side emotion type data, “unpleasant, anxiety, fear, relaxation, calmness” are stored. That is, these extracted emotion types become emotion type candidates for each index such as the first index based on the ratio of the β wave and the α wave of the brain wave set in the emotion estimation model.

[0067] The generation unit 132 obtains first emotion type information associated with a first index based on a biological signal and second emotion type information associated with a second index based on the biological signal from the information extracted by the extraction unit 131. Then, the generation unit 132 associates, according to each combination state composed of a combination of a first index state, which is each state in the first index, and a second index state, which is each state in the second index, that is, according to the combined first index state and second index state, an emotion type selected from the first emotion type information and the second emotion type information, and generates an emotion estimation model (an empty emotion estimation model with no emotion type set).

[0068] When imaging the emotion estimation model generated in this way, it becomes the emotion estimation model shown in the emotion map of FIG. 6. In FIG. 6, the first index is the vertical axis and the second index is the horizontal axis. The first index and the second index are each separated by a threshold value (0 when normalized), and each has two separated states. Then, the combination state, which is the combination of the separated states of the first index and the second index, becomes the first to fourth quadrants of the emotion map. This corresponds to the empty emotion estimation model. Then, by arranging (setting) emotion candidates corresponding to each of these quadrants, an emotion estimation model is formed. Hereinafter, the description will proceed based on the imaged emotion map.

[0069] The generation unit 132 obtains, from the emotion type information table 121 that associates an index related to a biological signal (for example, the first index and the second index) with an emotion type, the emotion types corresponding to each of two or more specified indexes, that is, the overlapping emotion types (for example, the first emotion type and the second emotion type).

[0070] Then, the generation unit 132 generates a model (emotion estimation model) in which the obtained emotion types are arranged in each quadrant of the space defined by the axes associated with each of the two or more indexes.

[0071] The space mentioned here means a Euclidean space. That is, the space mentioned here includes a two-dimensional plane and a space of three or more dimensions.

[0072] Using FIG. 5, the method for generating a model by the generation unit 132 will be specifically described. FIG. 5 is a diagram for explaining the method for generating a model.

[0073] Here, it is assumed that as indices, the β wave / α wave of the electroencephalogram (index “VS01”) and the standard deviation of the LF component of the heartbeat (index “VS02”) are specified.

[0074] The generation unit 132 refers to the emotion type information table 121 and acquires various data from each record with the index ID “VS01” and each record with the index ID “VS02”.

[0075] Then, as shown in FIG. 5, the generation unit 132 assigns the index “VS01” to the vertical axis and the index “VS02” to the horizontal axis. Note that the assignment of the axes may be the reverse of that shown in FIG. 5. Specifically, since the emotion axis data of the index “VS01” is “awake - non - awake”, the vertical axis becomes “awake - non - awake (positive side - negative side)”, and since the emotion axis data of the index “VS02” is “strong emotion - weak emotion”, the horizontal axis becomes “strong emotion - weak emotion (positive side - negative side)”.

[0076] Next, the generation unit 132 analyzes the relationship between the positive - side emotion type data and the negative - side emotion type data of the index “VS01” for each emotion type, and the positive - side emotion type data and the negative - side emotion type data of the index “VS02” for the same emotion type.

[0077] Then, based on the analyzed relationship, the generation unit 132 arranges each emotion type in each quadrant of the two - dimensional coordinate plane defined by the vertical axis and the horizontal axis.

[0078] Specifically, the generation unit 132 arranges the emotion type data that commonly exists between the positive - side emotion type data of the index “VS01” and the positive - side emotion type data of the index “VS02” in the first quadrant.

[0079] Further, the generation unit 132 arranges the emotion type data that commonly exists in the negative emotion type data in the index "VS01" and the positive emotion type data in the index "VS02" in the fourth quadrant.

[0080] Also, the generation unit 132 arranges the emotion type data that commonly exists in the negative emotion type data in the index "VS01" and the negative emotion type data in the index "VS02" in the third quadrant.

[0081] Also, the generation unit 132 arranges the emotion type data that commonly exists in the positive emotion type data in the index "VS01" and the negative emotion type data in the index "VS02" in the second quadrant.

[0082] For example, as shown in FIG. 3, the emotion type "happy" in the index "VS01" is included in the positive emotion type data, and the emotion axis data of the index "VS01" is "awake - non - awake (positive - negative)". Also, the emotion type "happy" in the index "VS02" is included in the positive emotion type data, and the emotion axis data of the index "VS02" is "strong emotion - weak emotion (positive - negative)".

[0083] Therefore, as shown in FIG. 6, the generation unit 132 arranges the emotion type "happy" in the first quadrant in a two - dimensional Euclidean space where the vertical axis is "awake - non - awake (positive - negative)" and the horizontal axis is "strong emotion - weak emotion (positive - negative)".

[0084] Note that, as shown in FIG. 5, the generation unit 132 treats the emotion type data that is not included as the emotion type data in at least one of the two selected indices, for example, the emotion type "boring" that is not included in the index "VS01" (but is included in "VS02") as not adopted and does not arrange it on the coordinate plane here. In other words, only the emotion type data that is included as emotion type data in both of the two selected indices is arranged on the coordinate plane. Note that there is also a method of arranging such non - adopted emotion types on the coordinate plane, but that method will be described later.

[0085] In this way, the generation unit 132 forms a two-dimensional coordinate plane defined by a first emotion axis associated with index data representing a first emotion (first emotion index data) and a second emotion axis associated with index data representing a second emotion (second emotion index data). Then, the generation unit 132 determines the position of each first emotion type data associated with the first emotion index data with respect to the first emotion axis (in the example of FIG. 3, the positive emotion type is positive / negative emotion type is positive). Also, the generation unit 132 determines the position of each second emotion type data associated with the second emotion index data with respect to the second emotion axis. Then, based on the positions of each first emotion type data and each second emotion type data with respect to each emotion axis, the generation unit 132 generates a model in which each emotion type data is arranged in each quadrant of the formed two-dimensional coordinate plane.

[0086] Thereby, a model capable of specifying emotion types based on different emotion axes such as a first emotion axis (for example, arousal level) and a second emotion axis (for example, intensity of emotion) can be generated.

[0087] The first index or the second index may be an index representing an arousal state other than the above (VS01). The first index or the second index may be an index representing the intensity of emotion other than the above (VS02).

[0088] (Example 1 of the model) It is known that the arousal state affects the autonomic nerves (sympathetic nerves and parasympathetic nerves) and changes the contractility of the heart, and thus its influence appears in the heart rate interval. And as emotion types that result in an arousal state, "happy", "joy", "anger", "anxiety", "moderate tension" are known. Also, as emotion types that result in a non-aroused state, "depressed", "bored", "relaxed", "calm", "unpleasant", "sadness" are known. Based on medical evidence showing these facts, the emotion type information table 121 is generated.

[0089] In this example, the record in the emotion type information table 121 where the index ID data is "VS03" stores the index name data as "inter-beat interval (RRI)", the sensor data as "heart rate sensor", the explanation data as "decrease in inter-beat interval", the emotion axis data as "arousal - non-arousal", the positive-side emotion type data as "happy, joy, anger, anxiety, moderate tension", and the negative-side emotion type data as "depression, boredom, relaxation, calmness, unpleasantness, sadness".

[0090] On the other hand, it is known that the standard deviation of the heart rate LF component represents the activity levels of the sympathetic and parasympathetic nerves.

[0091] It is known that the activity of the sympathetic nerve is correlated with strong emotions, and the activity of the parasympathetic nerve is correlated with weak emotions. The emotion types that are strong emotions include "happy", "joy", "anger", "anxiety", "fear", "unpleasantness", "fun". Also, the emotion types that are weak emotions include "depression", "boredom", "relaxation", "calmness". Based on the medical evidence showing these facts, the emotion type information table 121 is generated.

[0092] In this example, the record in the emotion type information table 121 where the index ID data is "VS02" stores the index name data as "standard deviation of the heart rate LF component", the sensor data as "heart rate sensor", the explanation data as "activation of the sympathetic nerve", the emotion axis data as "strong emotion - weak emotion", the positive-side emotion type data as "happy, joy, anger, anxiety, fear, unpleasantness, sadness", and the negative-side emotion type data as "depression, boredom, relaxation, calmness".

[0093] Therefore, the generation unit 132 assigns "arousal - non-arousal" to the vertical axis and "strong emotion - weak emotion" to the horizontal axis.

[0094] Then, based on analysis processing such as which side of the positive or negative emotional type each emotional type in the index ID data "VS03" and "VS02" is assigned to, the generation unit 132 arranges each emotional type in the corresponding quadrant of the two-dimensional Euclidean space formed with the vertical axis being "awake - not awake (positive side - negative side)" and the horizontal axis being "strong emotion - weak emotion (positive side - negative side)".

[0095] The generation unit 132 arranges emotional types such as "happy", "joy", "anger", "anxiety" in the first quadrant, "anxiety", "unpleasant" in the third quadrant, and "depression", "boredom", "relaxed", "calm" in the fourth quadrant.

[0096] (Example of the model 2) Since the heart rate interval is greatly affected by breathing, the accuracy of the model may decrease. Therefore, the generation unit 132 may generate a model that avoids the influence of breathing by assigning an index related to brain waves to the vertical axis.

[0097] As emotional types that become awake due to the influence of brain waves, "happy", "joy", "anger", "sadness", "depression" are known. Also, as emotional types that become not awake due to the influence of brain waves, "depression", "boredom", "relaxed", "calm" are known. Based on medical evidence showing these facts, the emotional type information table 121 is generated.

[0098] This example is a record in the emotional type information table 121 where the index ID data is "VS01", the index name data is "β wave / α wave of brain waves", the sensor data is "brain wave sensor", the explanation data is "relative increase in the β wave of brain waves", the emotional axis data is "awake - not awake", the positive side emotional type data is "happy, joy, anger, sadness, depression", and the negative side emotional type data is "unpleasant, anxiety, fear, relaxed, calm" are stored.

[0099] Therefore, the generation unit 132 assigns "awake - not awake" to the vertical axis and "strong emotion - weak emotion" to the horizontal axis.

[0100] Then, based on analysis processing such as which side of the positive or negative emotional type each emotional type in the index ID data "VS01" and "VS02" is assigned to, the generation unit 132 arranges each emotional type in the corresponding quadrant of the two-dimensional Euclidean space formed with the vertical axis being "awake - not awake (positive side - negative side)" and the horizontal axis being "strong emotion - weak emotion (positive side - negative side)".

[0101] Specifically, the generation unit 132 arranges emotional types such as "happy", "joy", "anger", "sadness" in the first quadrant, "depression" in the second quadrant, "relaxed", "calm" in the third quadrant, and "uneasy", "fear", "unpleasant" in the fourth quadrant.

[0102] In this way, the generation unit 132 acquires the emotional types corresponding to each of the index representing the arousal level based on the brain waves and the index representing the intensity of the emotion based on the heart rate from the emotional type information table 121.

[0103] Thereby, by using an electroencephalogram sensor capable of estimating the arousal level based on the β wave / α wave of the electroencephalogram from the active state of the neocortex, the influence of breathing is avoided.

[0104] Although the method for generating the model has been described above, specific data will be given and described with reference to the specific example shown in FIG. 5. FIG. 6 is a diagram showing an example of the coordinate plane of the specific model created according to this specific example.

[0105] As shown in FIG. 5, the vertical axis of the coordinate plane is the emotional axis "awake - not awake" of the index ID data "VS01". The emotional types located on the positive side of the vertical axis are the positive emotional types "happy", "joyful", "angry", "sad", "depressed", and the emotional types located on the negative side of the vertical axis are the negative emotional types "unpleasant", "uneasy", "fearful", "relaxed", "calm".

[0106] On the other hand, the horizontal axis of the coordinate plane is the emotional axis "strong emotion - weak emotion" of the index ID data "VS02". The emotional types located on the positive side of the vertical axis are the positive-side emotional types of "happy", "joyful", "angry", "anxious", "fearful", and "unpleasant". Also, the emotional types located on the negative side of the vertical axis are the negative-side emotional types of "depressed", "bored", "relaxed", and "calm".

[0107] These are determined based on the emotional type information shown in Figure 3.

[0108] Next, the generation unit 132 determines on which side (positive or negative) of the vertical axis and the horizontal axis the same emotional type is located, and determines in which quadrant of the coordinate plane of the model to place it based on the result. For example, since the emotional type "happy" is located on the positive side of the vertical axis ("arousal - non-arousal") and the positive side of the horizontal axis ("strong emotion - weak emotion"), it is in the "first quadrant".

[0109] By performing such processing for each emotional type, as shown in Figure 6, the emotional types of "happy", "joy", "anger", and "sadness" are in the region 210 of the first quadrant, the emotional type of "depressed" is in the region 220 of the second quadrant, the emotional types of "relaxed" and "calm" are in the region 230 of the third quadrant, and the emotional types of "anxious", "fearful", and "unpleasant" are placed in the region 240 of the fourth quadrant.

[0110] Furthermore, as shown in Figure 7, the generation unit 132 may place the emotional types that were not adopted by the method in Figure 5 in the region between two quadrants. Figure 7 is a diagram showing an example of the coordinate plane of the model.

[0111] The emotional types that are not adopted in the placement of the model on the coordinate plane occur when the emotional type data does not exist in at least one of the two emotional indicators selected as the targets of the two axes of the model's coordinate plane. In this case, although it was determined not to be adopted by the method shown in FIG. 5, as another way of thinking, when there is no correlation with a certain emotional indicator (neither positive nor negative emotional types exist), it can be considered that it can be judged as the emotional type at the 0 position. Therefore, in the coordinate plane of this model, in addition to the four quadrant regions, the generation unit 132 sets regions near the 0 values of the vertical and horizontal axes, and places the emotional types that were determined not to be adopted by the method shown in FIG. 5 in either the regions near the 0 values of the vertical and horizontal axes (based on the positive and negative positions of the emotional types in other indicators).

[0112] In the above example, as shown in FIG. 7, the generation unit 132 places the non - adopted emotional type "boredom" in the region 225 between the second quadrant and the third quadrant (treating the emotional type "boredom" as 0 (neutral) with respect to the "arousal - non - arousal" axis).

[0113] According to such a method, regarding the emotional types that are not adopted by the method shown in FIG. 5, the generation unit 132 can appropriately place the emotional types in the region 215 between the first quadrant and the second quadrant, the region 225 between the second quadrant and the third quadrant, the region 235 between the third quadrant and the fourth quadrant, and the region 245 between the fourth quadrant and the first quadrant.

[0114] In the above description, a method of placing emotional types in each quadrant by language analysis processing of medical evidence, etc. has been presented. However, it is also possible to place emotional types in each quadrant according to a manual by developers, etc. For example, while measuring the biological signals of the subjects in various situations, a questionnaire survey on the emotional types held is conducted. Then, in each situation, the quadrants of the emotion map are calculated based on the measured biological signals, and the emotional types of the questionnaire results are placed in the calculated quadrants. By such a method, emotional types can be set for each quadrant of the emotion map.

[0115] The specific unit 133 identifies the emotional type of the subject based on the values of the indicators obtained from the biological signals of the subject U02. That is, the biological signals from the sensors worn by the subject U02 are analyzed and processed into corresponding indicator values. Note that by analyzing and processing the biological signals, two types of indicator values will be obtained. Then, the indicator values are applied to the model (the coordinate plane of the model) that generated the indicator values, and the emotional type of the region corresponding to the indicator values is identified as the emotional type of the subject U02.

[0116] In this way, the specific unit 133 acquires a plurality of types of first biological signals and second biological signals, converts the first biological signals into first indicator values that are indicators of emotions, converts the second biological signals into second indicator values that are indicators of emotions, and applies the first indicator values and the second indicator values converted from the first biological signals and the second biological signals to an emotion model in which the emotion estimation value is determined by the combination of the first indicator value and the second indicator value, and determines the emotion estimation value as the estimated emotion.

[0117] The providing unit 134 provides the identified emotional type to the terminal device 20. And by providing the identified emotional type to the user such as the subject U02 through display or the like, the user such as the subject U02 can grasp and estimate the emotions of the subject U02, and can be used for training the subject U02 and the like.

[0118] For example, the providing unit 134 causes a result display screen to be displayed on the display (configured by a liquid crystal display or the like) of the terminal device 20. FIG. 8 is a diagram showing an example of the result display screen.

[0119] As shown in FIG. 8, on the result display screen 301, indicators assigned to each axis, the indicator values of the subject, the emotional types related to the estimated emotion results, text information such as messages, and related images suggesting their contents are displayed. Also, an emotion map is displayed on the result display screen 301. Note that these images are generated by the controller 13 based on the calculated indicator values and the estimated emotion results.

[0120] The emotion map shows the coordinates (the subject's emotion coordinates) obtained from the biological signals of the subject U02 plotted on the coordinate plane of the model, where the index values are plotted.

[0121] In the example of FIG. 8, since the emotion coordinates are in the first quadrant, it is estimated that the emotion of the subject U02 is one of "happy", "joy", "anger", or "sadness". Also, based on the position plotted on the emotion map, it is possible to estimate to some extent the strength of the emotion type (the farther from the origin, the stronger the emotion of the corresponding emotion type is estimated). Alternatively, based on the position plotted on the emotion map, it is also possible to estimate the accuracy of the emotion determination (the farther from the origin, the higher the determination accuracy for the corresponding emotion type is estimated).

[0122] Here, an example in the case where two indexes are specified has been described. On the other hand, three or more indexes may be specified. For example, when three indexes are specified, the generation unit 132 arranges the emotion types in one of eight quadrants (three-dimensional space).

[0123] For example, when the index ID data is "VS01", "VS02", and "VS03", the coordinate space is defined with the emotion axes of the vertical axis, the horizontal axis, and the depth axis, i.e., the three-dimensional axes of the so-called XYZ. Then, the specifying unit 133 plots using the three index values based on each biological signal in the coordinate space, and specifies the emotion type arranged in the space of the region where the plotted point is located as the emotion of the subject.

[0124] Also, in the above example, the determination threshold values for the two indexes were each one, but two or more determination threshold values may be used. For example, the first index based on the ratio of the beta wave and the alpha wave of the electroencephalogram may be stratified by two determination threshold values, and the first index may be stratified into three states (electroencephalogram states) based on the first index.

[0125] According to the method as described above, since the number of regions where the emotion types are arranged increases and the types of indexes used also increase, a more detailed emotion determination becomes possible.

[0126] Using FIGS. 9 and 10, the flow of the process executed by server 10 will be described. FIG. 9 is a flowchart showing the flow of the extraction process performed by controller 13 (extraction unit 131). This extraction process is performed, for example, based on a start command from the user, but the user needs to execute this process before performing emotion estimation. Further, FIG. 10 is a flowchart showing the flow of the estimation process. This estimation process is performed, for example, based on a start command from the user when the user desires the emotion estimation result.

[0127] As shown in FIG. 9, in step S101, controller 13 (extraction unit 131) substitutes 1 for n and proceeds to step S102. n is a number that identifies the index to be extracted. Also, X is the number of indices to be extracted. X is set by the user as needed, but the minimum number is 2.

[0128] In step S102, controller 13 (extraction unit 131) extracts the nth index based on the biological signal and proceeds to step S103. In step S103, controller 13 (extraction unit 131) extracts data such as the emotion type specified by the extracted index, associates it with the index ID as emotion type information as shown in FIG. 3, and stores it, and proceeds to step S104.

[0129] Note that controller 13 (extraction unit 131) extracts index data and emotion type data, for example, by performing natural language analysis on medical papers and books.

[0130] Further, controller 13 (extraction unit 131) may store the corresponding data input by a human using an input operation device such as a keyboard as emotion type information, or may take in the corresponding data from a social shared database such as a medical database and store it as emotion type information.

[0131] Then, in step S104, the controller 13 (extraction unit 131) determines whether the number n of indicators from which the emotion type information has been extracted has reached the set number X. If the set number has been reached, the process ends; if not, the process proceeds to step S105. In step S105, the controller 13 (extraction unit 131) adds 1 to the counter value n indicating the number of indicators from which the emotion type information has been extracted, and returns to step S102. That is, the processes of step S102 and step S103 are repeated until the number n of indicators from which the emotion type information has been extracted reaches the set number X.

[0132] Also, as shown in FIG. 10, in step S201, the controller 13 (generation unit 132) determines a plurality of index values to be used for emotion estimation, and proceeds to step S202. This determination is made, for example, by the controller 13 (provision unit 134) providing the user with the index information necessary to estimate the emotion the user desires to estimate, and the controller 13 (generation unit 132) taking in the indexes selected by the user through a selection operation.

[0133] In step S202, the controller 13 (generation unit 132) generates a model corresponding to the determined index values, and proceeds to step S203. Note that the controller 13 (generation unit 132) can generate a model by the method shown in FIG. 5 or the like.

[0134] In step S203, the controller 13 (generation unit 132) acquires a biological signal corresponding to the determined index values from a sensor or the like worn by the user, and proceeds to step S204. Note that the controller 13 (generation unit 132) processes the acquired biological signal as necessary and converts it into an index value.

[0135] Also, prior to that, the controller 13 (provision unit 134) provides the user with information such as the sensors that need to be worn by the user and guidance on the start of emotion estimation, to assist the user in preparation.

[0136] Then, the controller 13 (generation unit 132) applies it to the model that has generated the index value based on the acquired biological signal in step S204, specifies the emotion type corresponding to the index value based on the emotion estimation method described with reference to FIG. 6 and the like, and ends the process. For example, the controller 13 (generation unit 132) specifies the emotion type according to which quadrant the emotion coordinates obtained by plotting the index value on the coordinate plane of the model are located in.

[0137] When the indices of the biological signals to be used are the index based on the ratio of the β wave and the α wave of the electroencephalogram (first index: arousal level) and the index based on the low-frequency component of the heartbeat (second index: emotion intensity), these operations (operations of the emotion estimation device) are represented by the configuration of the emotion estimation model (emotion map) of the emotion estimation device and the processing performed by the controller as follows.

[0138] The emotion estimation model is formed by combining two electroencephalogram states (arousal - non - arousal) obtained by classifying the first index based on the ratio of the β wave and the α wave of the electroencephalogram with a first determination threshold value, and two heartbeat states (strong emotion - weak emotion) obtained by classifying the second index based on the low - frequency component of the heartbeat with a second determination threshold value into four combined states (from the first quadrant to the fourth quadrant). Emotion types (first quadrant: "happy", "joy", "anger", "sadness", second quadrant: "melancholy", third quadrant: "relaxed", "calm", fourth quadrant: "uneasy", "fear", "unpleasant") are set for each of the four combined states (from the first quadrant to the fourth quadrant).

[0139] Then, the controller classifies the first index calculated based on the electroencephalogram acquired from the subject with a first determination threshold value to determine the electroencephalogram state (arousal - non - arousal), classifies the second index calculated based on the heartbeat acquired from the subject with a second determination threshold value to determine the heartbeat state (strong emotion - weak emotion), determines the combined state (for example, the first quadrant) in the model corresponding to the determined electroencephalogram state and heartbeat state, and sets the emotion type (for example, "happy") corresponding to the determined combined state (for example, the first quadrant) as the estimated emotion (for example, "happy").

[0140] Server 10 may accept the designation of an emotion type instead of an index and generate a model from the designated emotion type. That is, for example, it is a method used when a user wants to know the state (presence or absence of occurrence and its intensity) of a certain emotion type.

[0141] At this time, the controller 13 (generation unit 132) acquires two or more indexes corresponding to the designated emotion type (including the emotion type designated as the positive-side emotion type or the negative-side emotion type) from the emotion type information table 121. Further, the controller 13 (generation unit 132) generates a model in which the emotion type is arranged in each quadrant of the space defined by the axes associated with each of the two or more indexes. Then, the subject wears the necessary sensors based on the emotion type information table 121, and the controller 13 acquires biological data from the sensors. Thereafter, based on the biological information, the emotion of the subject is estimated in the same manner as the above-described method.

[0142] Thereby, even when the analyst U01 does not have sufficient knowledge about the index, it is possible to obtain information estimating the state of the desired emotion type.

[0143] An example of the user interface of such an emotion estimation device will be described. FIG. 11 is a diagram showing an example of an analysis support screen. As shown in FIG. 11, on the analysis support screen 302, a check box group that can select a plurality of emotion types is displayed together with a message "Please select an emotion type and press the search button."

[0144] The emotion types displayed together with the check boxes are data of the emotion types included in the positive-side emotion type or the negative-side emotion type in the emotion type information table 121.

[0145] When the search button is pressed, the providing unit 134 identifies a combination of indexes for which the emotion type selected by the check box can be estimated, and displays information regarding the identified combination as a search result.

[0146] Furthermore, the providing unit 134 provides information regarding sensors necessary to obtain the indicators displayed as search results.

[0147] For example, as shown in FIG. 11, on the analysis support screen 302, it is assumed that the analyst U01 has selected the emotion types of "happy" and "boring". Then, based on each data in the emotion type information table 121 shown in FIG. 3, the indicators "standard deviation of the LF component of the heart rate of VS02" and "heart rate interval (RRI) of VS03", which are the indicators including the emotion types of "happy" and "boring" (search for the positive-side emotion type or negative-side emotion type in the emotion type information table 121), are searched.

[0148] Then, on the analysis support screen 302, as search results, the combination of "standard deviation of the LF component of the heart rate" and "heart rate interval (RRI)" is displayed. Also, on the analysis support screen 302, the "heart rate sensor", which is the sensor necessary for performing analysis in combination with the standard deviation of the LF component of the heart rate and the heart rate interval (RRI), is shown. The user checks this screen, prepares the "heart rate sensor", and performs emotion estimation.

[0149] FIG. 12 is a diagram for explaining a method of specifying an indicator. Note that the example in FIG. 12 is for the case where the analyst U01 has specified the analysis of the emotions of "happy" and "boring". As shown in FIG. 12, the providing unit 134 refers to the emotion type information table 121 and checks whether the "happy" and "boring" selected by the analyst U01 are included (yes or no) in the emotion type of each indicator.

[0150] Then, the providing unit 134 identifies (adopts) the combination of indicators for which both "happy" and "boring" are yes, and provides the information regarding these identified indicators and the information of the sensors for calculating the indicators to the analyst U01 or the subject.

[0151] In addition, when more than the adopted number of indicators meet the adoption conditions, the providing unit 134 automatically identifies a combination of indicators according to a predetermined criterion. For example, the providing unit 134 may set criteria such as identifying a combination of indicators with a large number of past selected achievements, identifying a combination of indicators estimated to be highly accurate, identifying a combination of indicators with a high penetration rate of necessary sensors, and identifying a combination of indicators with as many identifiable emotion types as possible.

[0152] Alternatively, the providing unit 134 may recommend a combination of combinable indicators to the analyst U01 and adopt the combination of indicators selected by the analyst U01 therefrom.

[0153] The biological signals that can be used in the first embodiment are not limited to those listed in the previous descriptions. For example, the emotion type information table 121 may include emotion types corresponding to indicators obtained from biological signals such as body surface temperature, face image, and pupil dilation. The arousal level may be, for example, body temperature, body surface temperature, pupil diameter, brain wave fluctuation, degree of sleepiness measured by image recognition, etc.

[0154] Also, a configuration for realizing functions equivalent to those of the extraction unit 131, generation unit 132, identification unit 133, and providing unit 134 of the server 10 (functions realized by the controller 13) may be provided in the terminal device 20 so that the terminal device 20 performs the emotion estimation operation performed by the above-described server 10. In that case, the terminal device 20 has the same configuration as the above-described server 10.

[0155] As described above, the controller 13 of the server 10 according to the first embodiment acquires first emotion type information associated with a first indicator related to a biological signal, acquires second emotion type information associated with a second indicator related to the biological signal, and for each combined indicator state composed of a combination of a first indicator state which is each state in the first indicator and a second indicator state which is each state in the second indicator, generates an emotion estimation model in which an emotion type selected from the first emotion type information and the second emotion type information according to the combined first indicator state and second indicator state is associated.

[0156] In this way, by generating a model according to the information (emotion type information table 121) that associates the index related to the biological signal with the emotion type, emotions can be estimated with high accuracy based on the biological signal.

[0157] In particular, if the emotion type information table 121 is based on medical evidence, the error (degree of deviation) in the association between the biological signal and the emotion can be reduced.

[0158] Also, the emotion type information table 121 can be updated at any time with information extracted from medical evidence. In this way, when the emotion type information table 121 is updated, the learning of the model progresses and the estimation accuracy is further improved.

[0159] [Second Embodiment] Although an example of the result display screen for emotion estimation has been described with reference to FIG. 8, it is preferable that the display form of the result display screen is appropriate according to its usage form. Therefore, other display forms on the result display screen will be described next.

[0160] FIG. 13 is a diagram showing an example of the result display screen of the second embodiment. Similar to FIG. 8, in FIG. 13, the index value of the index representing the arousal state is assigned to the vertical axis, and the index value of the index representing the intensity of the emotion is assigned to the horizontal axis.

[0161] The controller 13 (provision unit 134) indicates whether the emotion of the subject is stable (in a calm (normal) state) on the result display screen 301. Specifically, the controller 13 (provision unit 134) displays a rectangular frame figure 3011 surrounded by a broken line indicating the stable region of the emotion, and indicates whether the emotion of the subject is stable based on whether the coordinates of the estimated emotion of the subject are within the frame figure 3011 (the inside of the frame figure 3011 is the stable state determination region).

[0162] The emotional stability region is limited by a value obtained by integrating predetermined coefficients of the maximum and minimum values in the vertical axis index value and the horizontal axis index value (defining the maximum and minimum values for the vertical and horizontal axes), and the predetermined coefficient is set to an appropriate value based on experiments or the like.

[0163] For example, when the vertical axis index value and the horizontal axis index value are normalized with a maximum value of 1 and a minimum value of -1, the coefficient is set to an appropriate numerical value of 1 or less, such as 0.2. In this case, the coordinates of the four vertices of the frame shape 3011 are (0.2, 0.2), (-0.2, 0.2), (0.2, -0.2), and (-0.2, -0.2).

[0164] And, for example, if each index value is within the range of -0.2 to 0.2, the controller 13 (providing unit 134) displays a mark of the estimated emotion (a star mark in the example of FIG. 10) at the emotion coordinates indicating the estimated emotion within the frame shape 3011 of the emotional stability region. Further, the controller 13 (providing unit 134) displays a message saying, "It seems that the subject's emotion is stable." Thus, according to the second embodiment, it becomes possible to clearly show whether or not the subject's emotion is in a stable state.

[0165] In the above example, it is suggested that the subject's emotion is in a stable state. However, by appropriately setting the position and size of the frame shape 3011, it becomes possible to suggest whether or not the subject is in a specific emotional state. Regarding the appropriate position and size of the frame shape 3011, it can be defined by methods such as setting the range of emotion coordinates corresponding to the state to be confirmed based on experiments or the like. For example, when a worker in a factory is the subject, the range of the emotional state that can be regarded as the subject being concentrated on the work is set as the frame shape 3011. Also, for example, when the driver of a vehicle is the subject, the range of the emotional state that can be regarded as the subject driving calmly is set as the frame shape 3011.

[0166] Furthermore, the controller 13 (provider 134) may change the scale of the axis into an appropriate format. For example, the controller 13 (provider 134) may use a non-linear scale such that the smaller the index value, the smaller the scale, so as to suggest in detail a specific range of the emotion coordinates, and perform a display with good visibility according to the purpose of use, the characteristics of the estimated emotion type or the index type.

[0167] [Third Embodiment] Depending on the purpose of using the estimated emotion, it may be appropriate to change the criterion (threshold) for estimating the emotion. For example, when using the estimated emotion information for the evaluation of a horror movie, it is considered that an appropriate evaluation of the movie using the estimated emotion can be performed by methods such as changing the threshold of the horror emotion according to the targeted horror level or changing the threshold of the horror emotion according to the age of the viewers of the horror movie.

[0168] Therefore, in the third embodiment, as shown in FIG. 14, the position (origin) of the emotion axis (both or one of the vertical axis and the horizontal axis) is moved to change the range of each quadrant for emotion determination.

[0169] In normal emotion determination, the vertical axis 3012a and the horizontal axis 3013a use an emotion map that intersects at the position where each index value is 0, that is, the origin. Note that the vertical axis 3012a and the horizontal axis 3013a are for explanatory purposes and are not actually displayed on the result display screen 301 in the third embodiment.

[0170] However, in the above-mentioned horror movie, in the case of evaluating a movie targeting a higher sense of horror, a more appropriate evaluation can be performed by using a threshold corresponding to a stronger sense of horror than usual. That is, depending on the purpose of use, it may be appropriate to move the positions of the vertical axis and the horizontal axis in the emotion map for appropriate emotion estimation. Therefore, in the third embodiment, according to the purpose of using the estimated emotion, specifically, based on the type of the device that uses the estimated emotion, the input of the purpose of use by the user, or the operation of adjusting the position of the emotion axis by the user, the position of the emotion axis (the determination threshold for emotion determination) is changed.

[0171] Specifically, when the vertical axis index value and the horizontal axis index value are normalized with a maximum value of 1 and a minimum value of -1, the controller 13 (provider 134) moves, for example, 0.2 units to the negative side according to the purpose of use of the vertical axis position and the horizontal axis position, and displays them on the emotion map of the result display screen 301. That is, the horizontal axis is moved to the position of arousal -0.2 (the horizontal line passing through the coordinate (0, -0.2)), and the vertical axis is moved to the position of emotion intensity +0.2 (the vertical line passing through the coordinate (0.2, 0)) (the intersection coordinate becomes (0.2, -0.2)).

[0172] Then, the controller 13 (provider 134) displays a mark (for example, a star mark) at the coordinate position of each index value based on the biological signal of the subject. The emotion coordinates of the subject are located in the fourth quadrant when based on the vertical axis 3012a and the horizontal axis 3013a (before axis movement). Therefore, in this case, it is presumed that the subject is feeling fear.

[0173] On the other hand, the emotion coordinates of the subject are located in the second quadrant when based on the vertical axis 3012b and the horizontal axis 3013b (after axis movement). In this case, it is presumed that the subject is not feeling the targeted intensity of fear. In this case, for example, the controller 13 (provider 134) displays a message "It is presumed that the subject is not feeling sufficient fear." on the result display screen 301.

[0174] In this way, by moving the positions of the vertical axis and the horizontal axis according to the purpose of use of the estimated emotion, etc., the area of each quadrant can be adjusted. That is, the estimated result of the emotion can be adjusted according to the purpose of use of the estimated emotion, etc. For example, in the evaluation of content such as a horror movie, it is possible to evaluate the emotion (fear) according to the level of fear given to the target audience that the creation aims for.

[0175] [Regarding emotion type information] The emotion type information table 121 shown in FIG. 3 may be updated as appropriate. For example, in the emotion type information table 121 of FIG. 3, the keywords "boring" are not included in the "positive emotion type" and "negative emotion type" of the record with the index ID data "VS01".

[0176] In this case, regarding the relationship between the emotion of "boredom" and brain waves, if new medical evidence is discovered, or if it is proven by experiments or the like, based on the content of such medical evidence or experiments, the keyword "boredom" may be added to the corresponding side of the "positive emotion type" or "negative emotion type" in the record with the index ID data of "VS01".

[0177] Also, when a new index suitable for emotion estimation is discovered based on medical evidence or the like, a new data record may be generated for the index, and each data such as the sensor type, explanation, and emotion axis corresponding to the use may be stored.

[0178] Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the present invention are not limited to the specific details and representative embodiments presented and described as above. Accordingly, various changes can be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

Explanation of Reference Numerals

[0179] N Network U01 Analyst U02 Subject 1 Estimation System 10 Server 11 Communication Unit 12 Storage Unit 13 Controller 20 Terminal Device 31, 32 Sensors 121 Emotion Type Information Table 131 Extraction Unit 132 Generation Unit 133 Identification Unit 134 Provision Unit 210, 215, 220, 225, 230, 235, 240, 245 Regions 301 Result Display Screen 302 Analysis Support Screen

Claims

1. An emotion estimation device for estimating the emotion of a subject, comprising a controller The controller is configured to acquire the biological signal of the subject detected by a sensor, calculate a first index and a second index based on the biological signal, provide an emotion map in which emotion type information corresponding to each quadrant with the first index and the second index as axes is arranged, and a mark image is arranged at the coordinate position based on the first index and the second index Emotion estimation device.

2. The controller is configured to arrange text information of the first index and the second index on an emotion estimation result display screen The emotion estimation device according to Claim 1.

3. The controller is configured to arrange text information of the emotion type regarding the emotion estimation result The emotion estimation device according to Claim 1.

4. The controller is configured to arrange text information of a message regarding the emotion estimation result The emotion estimation device according to Claim 1.

5. The controller is configured to set the format of the first axis corresponding to the first index or the second axis corresponding to the second index according to the value of the first index or the second index The emotion estimation device according to Claim 1.

6. The controller is configured to perform a display on the coordinate space to suggest a specific emotion area in the coordinate space The emotion estimation device according to Claim 1.

7. The controller is configured to display the specific emotion area with a frame shape of a position and size corresponding to the emotion area The emotion estimation device according to Claim 6.

8. The controller is configured to move the intersection coordinates of the first axis corresponding to the first index and the second axis corresponding to the second index based on an input by a user The emotion estimation device according to Claim 1.

9. The controller is configured to move the intersection coordinates of the first axis and the second axis based on the type of utilization device of the estimated emotion, the utilization purpose input by the user, or the axis position adjustment operation by the user The emotion estimation device according to Claim 8.

10. The first index is arousal level, The second index is the intensity of emotion The emotion estimation device according to any one of Claims 1 to 9.

11. The controller is configured to calculate the arousal level based on the ratio of the β wave and the α wave of the brain wave acquired from the subject, calculate the intensity of the emotion based on the heartbeat acquired from the subject The emotion estimation device according to Claim 10.

12. An emotion estimation device for estimating the emotion of a subject, comprising a controller The controller acquires the biological signal of the subject detected by the sensor, calculates a first index and a second index based on the biological signal, provides an emotion map that shows the coordinate position based on the first index and the second index on a map with the first index and the second index as axes, and shows an emotion stable region and an unstable region emotion estimation device.

13. An emotion display method for displaying emotion information estimated based on a first index and a second index related to the emotion of a subject, generates an emotion map in which emotion type information corresponding to each quadrant with the first index and the second index as axes is arranged, and a mark image is arranged at the coordinate position based on the first index and the second index An emotion display method executed by a controller.

14. An emotion display program for displaying emotion information estimated based on a first index and a second index related to the emotion of a subject, including a process of generating an emotion map in which emotion type information corresponding to each quadrant with the first index and the second index as axes is arranged, and a mark image is arranged at the coordinate position based on the first index and the second index An emotion display program executed by a controller.

15. An emotion estimation device for estimating the emotion of a subject, comprising a controller and a display, wherein the controller acquires the biological signal of the subject detected by the sensor, calculates a first index and a second index based on the biological signal, generates an emotion map in which emotion type information corresponding to each quadrant with the first index and the second index as axes is arranged, and a mark image is arranged at the coordinate position based on the first index and the second index, and causes the emotion map to be displayed on the display emotion estimation device.

Images