Emotion estimation device and emotion estimation model generation method

Abstract

An emotion inference device according to the present invention is for inferring emotion, and comprises a controller and an emotion inference model. The emotion inference model has formed therein four combination states which are combinations of two brain wave states obtained by stratifying, by using a first determination threshold value, first indexes each based on the ratio of β-waves and α-waves in brain waves, and two heartbeat states obtained by stratifying, by using a second determination threshold value, second indexes each based on a low frequency component of heartbeats, and emotion types are set respectively for the four combination states. The controller determines the brain wave states by stratifying, by using the first determination threshold value, the first indexes each calculated on the basis of the brain waves acquired from a subject, determines the heartbeat state by stratifying, by using a second determination threshold value, the second indexes each calculated on the basis of the heartbeat acquired from the subject, determines the combination state in the model corresponding to the determined brain wave states and the determined heartbeat states, and determines, as an inferred emotion, an emotion type corresponding to the determined combination state.

Description

[Technical field]

[0001] The present invention relates to an emotion estimation device and an emotion estimation model generation method. [Background technology]

[0002] There is known a technique for estimating the emotion of a subject by applying information obtained from the waveform of the subject's heart (electrocardiogram waveform) to a Russell circle model (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-63324 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional techniques have a problem in that it is difficult to estimate emotions with high accuracy based on biosignals.

[0005] The Russell Circumplex model is a model in which emotion types are arranged on a circle centered at the origin on a coordinate plane with arousal on the vertical axis (AROUSAL) and emotional valence (pleasantness-unpleasantness) on the horizontal axis (VALENCE).

[0006] In this Russell Circumplex model, the type of emotion of the subject is estimated by plotting the subject's arousal level and emotional valence on the coordinate plane of the Russell Circumplex model.

[0007] In the Russell Circumplex model, arousal and valence are psychological constructs, and there are many challenges to realizing it as an emotion estimation device. For example, the Russell Circumplex model requires the estimation of the subject's arousal and valence, but when realizing it as an emotion estimation device, some kind of biological signal is measured from the subject, and the arousal and valence are estimated from the measured value. However, it has not been established what kind of biological signal from the subject should be processed and how to estimate these arousal and valence, which is a major challenge to realizing an emotion estimation device.

[0008] For this reason, emotion estimation devices based on the Russell Circle model have been proposed, but it is difficult to estimate the subject's emotion with high accuracy.

[0009] The present invention has been made in consideration of the above, and has an object to estimate emotions with high accuracy. [Means for solving the problem]

[0010] In order to solve the above-mentioned problems and achieve the object, the model generation method according to the present invention is a method for generating an emotion estimation model that estimates an emotion type based on a bio-signal, and the method acquires first emotion type information associated with a first indicator related to the bio-signal, acquires second emotion type information associated with a second indicator related to the bio-signal, and generates an emotion estimation model in which an emotion type selected from the first emotion type information and the second emotion type information is associated with each combined index state formed by a combination of a first index state which is each state in the first index and a second index state which is each state in the second index, according to the combined first index state and second index state. Effect of the Invention

[0011] According to the present invention, emotions are inferred according to the state of the first index and the state of the second index based on the subject's bio-signal, so emotions can be accurately inferred by appropriately setting the state of the first index and the state of the second index based on scientific (medical) evidence and experimental results, and by appropriately setting the emotion type for the combined state of the state of the first index and the state of the second index based on scientific (medical) evidence and experimental results. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an estimation system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a server according to the first embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of a data table that stores emotion type information. [Figure 4] FIG. 4 is a diagram for explaining a method for extracting an emotion type. [Diagram 5] FIG. 5 is a diagram for explaining a method for generating a model. [Figure 6] FIG. 6 is a diagram showing an example of a coordinate plane of the model. [Figure 7] FIG. 7 is a diagram showing an example of a coordinate plane of the model. [Figure 8] FIG. 8 is a diagram illustrating an example of a result display screen according to the first embodiment. [Figure 9] FIG. 9 is a flowchart showing the flow of the extraction process. [Figure 10] FIG. 10 is a flowchart showing the flow of the estimation process. [Figure 11] FIG. 11 is a diagram showing an example of the analysis support screen. [Figure 12] FIG. 12 is a diagram for explaining a method for identifying an index. [Figure 13] FIG. 13 is a diagram illustrating an example of a result display screen according to the second embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a result display screen according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of a model generation method and an estimation device disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0014] [First embodiment] First, an estimation system according to a first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the estimation system according to the first embodiment.

[0015] 1, the estimation system 1 includes a server 10, a terminal device 20, and a sensor 31 and a sensor 32. The estimation system 1 estimates the emotion of a subject U02.

[0016] The subject U02 is, for example, an e-sports player. The estimation system 1 estimates the emotion of the subject U02 who is playing a video game. In the description of this embodiment, in order to make the description more specific and easier to understand, the above-mentioned e-sports application scene is assumed as an application example, and the description will be given with state transitions and the like.

[0017] The emotion estimation result is used for mental training of the subject U02 in e-sports, for example. For example, when the subject U02 feels an unfavorable emotion (anxiety, anger, etc.) while playing a video game, it is determined that intensive training corresponding to the emotional state is necessary.

[0018] As another application example, the subject U02 may be a patient at a medical institution. In this case, the emotion estimated by the estimation system 1 is used for examination, treatment, and the like.

[0019] For example, if the patient subject U02 feels anxious, the staff of the medical institution can provide countermeasures such as counseling.

[0020] The subject U02 may be a student at an educational institution. In this case, the emotion estimated by the estimation system 1 is used to improve the content of lessons.

[0021] For example, if a student, a subject U02, feels that a lesson is boring, the teacher improves the content of the lesson to make it more interesting to the student.

[0022] The subject U02 may also be a vehicle driver. In this case, the emotion estimated by the estimation system 1 is used to promote safe driving.

[0023] For example, if the subject U02, who is the driver, does not feel an appropriate level of tension while driving, the in-vehicle device outputs a message encouraging the subject U02 to concentrate on driving.

[0024] The subject U02 may also be a viewer of content such as video and music. In this case, the emotion estimated by the estimation system 1 is used to create further content.

[0025] For example, a distributor of video content can create a highlight video by collecting scenes that the viewer, subject U02, found enjoyable.

[0026] The server 10 and the terminal device 20 are connected via a network N. For example, the network N is the Internet or an intranet.

[0027] For example, the terminal device 20 is a personal computer, a smartphone, a tablet computer, etc. The terminal device 20 is used by the analyst U01.

[0028] The sensor 31 and the sensor 32 transmit the detected sensor signals to the terminal device 20.

[0029] The sensor 31 is, for example, a headgear-type brain wave sensor, and the sensor 32 is, for example, a wristband-type pulse sensor.

[0030] For example, the sensor 31 and the sensor 32 are communicatively connected to the terminal device 20 in accordance with communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark), and transmit a sensor signal to the terminal device 20.

[0031] The process flow of the estimation system 1 will be described with reference to FIG.

[0032] The server 10 extracts emotion types from medical evidence in advance (step S1). The medical evidence is, for example, a paper or a book. The method of extracting emotion types will be described later.

[0033] The terminal device 20 transmits index values ​​of a plurality of indices based on the biological signal to the server 10 (step S2). For example, the terminal device 20 transmits index values ​​of two different indices related to brain waves or heartbeats.

[0034] Here, the index value is a value of an index related to a biosignal. For example, the "average heartbeat interval" and the "heartbeat LF (Low Frequency) component" are indexes. Also, a specific value (e.g., a numerical value) corresponding to each index is the index value. The index value is the sensor value of each sensor or a value calculated from the sensor value.

[0035] The server 10 generates a model based on the extracted emotion type (step S3). At this time, the server 10 generates a model that matches the index value received from the terminal device 20. The method of generating the model will be described later.

[0036] The server 10 then uses the generated model to identify an emotion type from the index value (step S4). The server 10 provides the identified emotion type to the terminal device 20 (step S5).

[0037] 2 is a diagram illustrating an example of the configuration of a server according to the first embodiment. The server 10 is an example of a computer that executes the generation method. The server 10 is also an example of an estimation device.

[0038] As shown in FIG. 2, the server 10 includes a communication unit 11, a storage unit 12, and a control unit 13, ie, a controller 13.

[0039] The communication unit 11 is an interface for communicating data with other devices via the network N. The communication unit 11 is, for example, a network interface card (NIC).

[0040] The memory unit 12 and controller 13 of the server 10 are realized by a computer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, an input / output port, etc., or various circuits.

[0041] The CPU of the computer functions as an extracting unit 131, a generating unit 132, a specifying unit 133, and a providing unit 134 of the controller 13, for example, by reading and executing a program stored in the ROM.

[0042] The storage unit 12 corresponds to a RAM or a flash memory. The RAM or the flash memory stores an emotion type information table 121 and the like.

[0043] The server 10 may obtain the above-mentioned programs and various information via other computers (servers) or portable recording media connected via a wired or wireless network.

[0044] The emotion type information table 121 is information that associates information related to an index with an emotion type. Here, an overview of each item in the emotion type information table 121 will be explained. How to create and use the emotion type information table 121 will be described later.

[0045] Fig. 3 is a diagram showing an example of a data table that stores emotion type information. As shown in Fig. 3, the items in emotion type information table 121 include index ID, index name, sensor, positive / negative, explanation, emotion axis, and positive and negative emotion types, and these pieces of data form emotion type information. For ease of explanation, hereinafter, when indicating the data (value) itself in each item, the word "data" is added. For example, "index ID data" represents the value itself, such as "VS01", "VS02", etc.

[0046] The index ID of emotion type information table 121 is a character string that functions as an ID for identifying an index (a character string is stored). This index ID data is used as primary key data, that is, a record is configured for each index ID data, and the index name data, sensor data, explanation data, emotion axis data, positive emotion type data, and negative emotion type data linked to the index ID data are stored in the record.

[0047] The "index name" in the emotion type information table 121 is information (character string) that indicates the name of the index.

[0048] "Sensor" in emotion type information table 121 is information for identifying a sensor required to obtain an index value of a corresponding index. For example, when the index ID is VS01 (record), the target sensor is an electroencephalogram sensor.

[0049] In the process described below, the index is used as an axis that constitutes a coordinate plane. In the process, it is significant whether the index value is in the positive direction (larger) or in the negative direction (smaller) than the origin (determination threshold).

[0050] The "positive emotion type data" and "negative emotion type data" in the emotion type information table 121 are information indicating whether data corresponding to positive and negative index values ​​of the record in question is stored.

[0051] For this reason, the index value is standardized so that the average is 0. In other words, when the biosignal is the average value, the above standardization is performed based on the estimation that the emotion associated with the biosignal is not occurring (is in a normal state). Note that the index value is not limited to the above standardization, and may be converted into a format that is easy to handle on a coordinate plane or a format that increases the accuracy of the estimation result (such as correction based on the evaluation result of the emotion estimation result). Note that when the index value is standardized, the judgment threshold for stratifying the state of the index is 0.

[0052] In addition, the server 10 may change the standardization method or conversion method of the index value depending on the purpose of using the estimated emotion, etc. For example, since the level of the emotion influenced by the scene of e-sports and car driving is different (car driving requires more calmness than e-sports from the viewpoint of safe driving), the server 10 may change the standardization method or conversion method of the index value so as to determine the emotion of excitement with a lower determination threshold, for example.

[0053] Also, instead of converting the format of the index value, the position of the origin of the coordinate plane may be adjusted to match the index value.

[0054] The "explanation" in the emotion type information table 121 is an explanatory text about the record. For example, it explains the relationship between the data items of the data.

[0055] The "emotion axis" in the emotion type information table 121 is a character string used as a label when the index is used as an axis of a coordinate plane. In the example shown in Fig. 3, the index name data indicates the meaning of the emotion axis when it becomes a value on each side of the positive / negative data.

[0056] The "Positive emotion type" and "Negative emotion type" in emotion type information table 121 are a set of keywords that indicate the emotion type corresponding to the record, and indicate the emotion type held by the subject when the index name data becomes a value on the respective side of the positive / negative data.

[0057] For example, the record in the first row of emotion type information table 121 in FIG. 3 (the record with index ID data “VS01”) stores “β waves / α waves of electroencephalogram” as index name data, “EEG sensor” as sensor type data for obtaining the index value of the index, “β waves of electroencephalogram are relatively increased compared to α waves” (positive side) as positive explanation data, “awake-unaroused” as emotion axis data, “joy, joy, anger, sadness, depression” as positive emotion type data, and “discomfort, anxiety, fear, relaxation, calm” as negative emotion type data.

[0058] In other words, the index ID data "VS01" is an index for measuring brain waves using an "EEG sensor." If the measurement result (index value) of "beta waves of brain waves are increased relatively to alpha waves" is positive, it is estimated that emotions such as "fun, joy, anger, sadness, depression" may be occurring in an "awake" state. If the measurement result (index value) of "beta waves of brain waves are increased relatively to alpha waves" is negative (negative), it is estimated that emotions such as "unpleasant, anxious, frightened, relaxed, calm" may be occurring in an "unawake" state.

[0059] The following describes the processing content of each unit of the controller 13. In the following description, the subject of processing by the extraction unit 131, the generation unit 132, the identification unit 133, and the provision unit 134 can be rephrased as the controller 13.

[0060] The extraction unit 131 extracts emotion types from medical evidence in association with the indices. Fig. 4 is a diagram for explaining a method of extracting emotion types.

[0061] 4, the extraction unit 131 extracts information related to an index and an emotion type by performing natural language analysis on text written in a paper, a book, etc. For example, the first emotion type information or the second emotion type information is generated by language analysis of a document describing medical evidence.

[0062] The extraction unit 131 may perform natural language analysis using an existing machine learning method. Also, the process of extracting emotion types from medical evidence in association with indicators may be performed manually. In this case, the extraction unit 131 extracts information related to indicators and emotion types based on information input by an operator.

[0063] In the example of Figure 4, the extraction unit 131 extracts emotion types "joy," "anger," and "sadness" in association with increases in beta waves based on the text "As beta waves increased, emotions such as joy, anger, and sadness increased."

[0064] In addition, in the example of FIG. 4, the extraction unit 131 extracts emotion types, "anxiety" and "fear," in association with an increase in alpha waves, based on the text, "When alpha waves increase, the number of people who feel anxiety or fear is statistically significantly higher."

[0065] Also, although examples of evidence are omitted, similarly to the above, for example, based on the text "In an awake state, beta waves in the brain are larger than alpha waves," "awakening" is associated with the emotion axis and extracted.

[0066] Incidentally, whether beta waves or alpha waves are increased is an example of an index based on electroencephalograms, which are one type of biological information. Based on such information extracted by the extraction unit 131, the emotion type information table 121 is generated. In the above example, in the record of the index ID data "VS01" (the index ID data is appropriately set so that there are no identical values), "beta waves / alpha waves of electroencephalograms" as index name data, "electroencephalogram sensor" as sensor data, "beta waves of electroencephalograms are relatively increased compared to alpha waves" as explanation data, "awakening-unarousal" as emotion axis data, "fun, joy, anger, sadness, melancholy" as positive emotion type data, and "unpleasant, anxious, fear, relaxed, calm" as negative emotion type data are stored. In other words, these extracted emotion types become emotion type candidates for each index, such as the first index based on the ratio of beta waves to alpha waves of electroencephalograms set in the emotion estimation model.

[0067] The generation unit 132 acquires first emotion type information associated with the first index based on the biosignal, and acquires second emotion type information associated with the second index based on the biosignal, from the information extracted by the extraction unit 131. The generation unit 132 then generates an emotion estimation model (a null emotion estimation model in which no emotion type is set) that can associate an emotion type selected from the first emotion type information and the second emotion type information according to each combined state formed by a combination of a first index state, which is each state in the first index, and a second index state, which is each state in the second index, that is, according to the combined first index state and second index state.

[0068] When the emotion estimation model generated in this way is visualized, it becomes the emotion estimation model shown in the emotion map in Figure 6. In Figure 6, the first index is the vertical axis, and the second index is the horizontal axis. The first index and the second index are each stratified by a threshold (0 if normalized), and each has two stratified states. The combined states, which are combinations of the stratified states of the first index and the second index, become the first to fourth quadrants of the emotion map. This corresponds to the empty emotion estimation model. The emotion estimation model is formed by arranging (setting) emotion candidates corresponding to each of these quadrants. Hereafter, the explanation will be given based on the visualized emotion map.

[0069] The generation unit 132 acquires emotion types (e.g., the first emotion type and the second emotion type) that correspond to each of the two or more designated indices, that is, overlapping emotion types, from the emotion type information table 121 that associates indices related to biological signals (e.g., the first index and the second index) with emotion types.

[0070] Then, the generation unit 132 generates a model (emotion estimation model) in which the acquired emotion types are arranged in each quadrant of a space defined by axes respectively associated with two or more indices.

[0071] The space referred to here means Euclidean space, which means that it includes two-dimensional planes and spaces with three or more dimensions.

[0072] A method for generating a model by the generation unit 132 will be specifically described with reference to Fig. 5. Fig. 5 is a diagram for explaining the method for generating a model.

[0073] Here, it is assumed that the indices designated are β / α brainwaves (indicator "VS01") and the standard deviation of the heart rate LF component (indicator "VS02").

[0074] The generation unit 132 refers to the emotion type information table 121 and acquires various data from the record with the index ID "VS01" and the record with the index ID "VS02".

[0075] 5, the generation unit 132 assigns the index "VS01" to the vertical axis and the index "VS02" to the horizontal axis. The axis assignment may be reversed from that shown in FIG. 5. Specifically, since the emotional axis data of the index "VS01" is "awake-unaroused", the vertical axis is "awake-unaroused (positive side-negative side)", and since the emotional axis data of the index "VS02" is "strong emotion-weak emotion", the horizontal axis is "strong emotion-weak emotion (positive side-negative side)".

[0076] Next, the generating unit 132 analyzes the relationship between the positive emotion type data and negative emotion type data in the index "VS01" of each emotion type and the positive emotion type data and negative emotion type data in the index "VS02" of the same emotion type.

[0077] Then, based on the analyzed relationship, generation unit 132 arranges each emotion type in each quadrant of a two-dimensional coordinate plane defined by the vertical axis and the horizontal axis.

[0078] Specifically, the generation unit 132 places the emotion type data that exists in common between the positive emotion type data in the index "VS01" and the positive emotion type data in the index "VS02" in the first quadrant.

[0079] Furthermore, the generation unit 132 places the emotion type data that is commonly present in the negative emotion type data in the index "VS01" and the positive emotion type data in the index "VS02" in the fourth quadrant.

[0080] Furthermore, the generating unit 132 places the emotion type data that is commonly present in the negative emotion type data in the index "VS01" and the negative emotion type data in the index "VS02" in the third quadrant.

[0081] Furthermore, the generation unit 132 places the emotion type data that exists in common between the positive emotion type data in the index "VS01" and the negative emotion type data in the index "VS02" in the second quadrant.

[0082] For example, as shown in Fig. 3, the emotion type "fun" in the index "VS01" is included in the positive emotion type data, and the emotion axis data of the index "VS01" is "awakening-unarousal (positive-negative)". Also, the emotion type "fun" in the index "VS02" is included in the positive emotion type data, and the emotion axis data of the index "VS02" is "strong emotion-weak emotion (positive-negative)".

[0083] For this reason, as shown in FIG. 6, the generation unit 132 places the emotion type “fun” in the first quadrant in a two-dimensional Euclidean space formed by the vertical axis being “awake-unaroused (positive side-negative side)” and the horizontal axis being “strong emotion-weak emotion (positive side-negative side).”

[0084] As shown in Fig. 5, the generation unit 132 treats emotion type data that is not included as emotion type data in at least one of the two selected indices, for example, the emotion type "boredom" that is not included in the index "VS01" (but is included in "VS02"), as unadopted and does not arrange it on the coordinate plane here. In other words, only emotion type data that is included as emotion type data in both of the two selected indices is arranged on the coordinate plane. Note that a method of arranging such unadopted emotion types on the coordinate plane is also possible, but this method will be described later.

[0085] In this way, the generation unit 132 forms a two-dimensional coordinate plane defined by a first emotion axis associated with index data (first emotion index data) representing a first emotion and a second emotion axis associated with index data (second emotion index data) representing a second emotion. Then, the generation unit 132 determines the position of each first emotion type data associated with the first emotion index data on the first emotion axis (in the example of FIG. 3, a positive emotion type is positive side / a negative emotion type is positive side). In addition, the generation unit 132 determines the position of each second emotion type data associated with the second emotion index data on the second emotion axis. Then, the generation unit 132 generates a model in which each emotion type data is arranged in each quadrant of the formed two-dimensional coordinate plane based on the position of each first emotion type data and each second emotion type data on each emotion axis.

[0086] This makes it possible to generate a model capable of identifying emotion types based on different emotion axes, namely, a first emotion axis (eg, arousal level) and a second emotion axis (eg, intensity of emotion).

[0087] The first index or the second index may be an index representing an arousal state other than the above-mentioned (VS01). The first index or the second index may be an index representing the intensity of an emotion other than the above-mentioned (VS02).

[0088] (Model Example 1) It is known that the state of arousal affects the autonomic nerves (sympathetic and parasympathetic nerves) and changes the contractile force of the heart, which affects the heartbeat interval. Emotion types that are known to result in an arousal state include "fun," "joy," "anger," "anxiety," and "moderate tension." Emotion types that are known to result in a state of non-arousal include "melancholy," "boredom," "relaxation," "calmness," "unpleasantness," and "sadness." The emotion type information table 121 is generated based on medical evidence showing these facts.

[0089] An example of this is a record whose index ID data in emotion type information table 121 is "VS03", which stores "heart rate interval (RRI)" as index name data, "heart rate sensor" as sensor data, "decreasing heart rate interval" as explanation data, "awakening-unarousal" as emotion axis data, "fun, joy, anger, anxiety, moderate tension" as positive emotion type data, and "melancholy, boredom, relaxation, calm, unpleasantness, sadness" as negative emotion type data.

[0090] On the other hand, it is known that the standard deviation of the heart rate LF component represents the activity of the sympathetic and parasympathetic nerves.

[0091] It is known that sympathetic nerve activity correlates with strong emotions, while parasympathetic nerve activity correlates with weak emotions. Emotion types that are known to be strong emotions include "fun," "joy," "anger," "anxiety," "fear," "unpleasantness," and "pleasure." Emotion types that are known to be weak emotions include "melancholy," "boredom," "relaxation," and "calmness." The emotion type information table 121 is generated based on medical evidence showing these facts.

[0092] This example is a record whose index ID data in emotion type information table 121 is "VS02", which stores "standard deviation of heartbeat LF component" as index name data, "heartbeat sensor" as sensor data, "sympathetic nerve activation" as explanation data, "strong emotion-weak emotion" as emotion axis data, "fun, joy, anger, anxiety, fear, unpleasantness, sadness" as positive emotion type data, and "melancholy, boredom, relaxation, calm" as negative emotion type data.

[0093] Therefore, the generation unit 132 assigns "awake-unaroused" to the vertical axis and "strong emotion-weak emotion" to the horizontal axis.

[0094] Then, based on an analysis process of determining whether each emotion type in the index ID data "VS03" and "VS02" is assigned to a positive emotion type or a negative emotion type, the generation unit 132 places each emotion type in a corresponding quadrant of a two-dimensional Euclidean space formed by the vertical axis being "arousal-unarousal (positive side-negative side)" and the horizontal axis being "strong emotion-weak emotion (positive side-negative side)."

[0095] The generation unit 132 places emotion types such as "fun," "joy," "anger," and "anxiety" in the first quadrant, "anxiety" and "unpleasantness" in the third quadrant, and "melancholy," "boredom," "relaxation," and "calmness" in the fourth quadrant.

[0096] (Model Example 2) Since the heartbeat interval is significantly affected by breathing, the accuracy of the model may decrease. Therefore, the generation unit 132 may generate a model that avoids the influence of breathing by allocating an index related to an electroencephalogram to the vertical axis.

[0097] Known types of emotions that are caused by the influence of brain waves to cause an arousal state include "fun," "joy," "anger," "sadness," and "melancholy." Also, known types of emotions that are caused by the influence of brain waves to cause a non-arousal state include "melancholy," "boredom," "relaxation," and "calmness." The emotion type information table 121 is generated based on medical evidence showing these facts.

[0098] This example is a record whose index ID data in emotion type information table 121 is "VS01", which stores "β waves / α waves of brain waves" as index name data, "brain wave sensor" as sensor data, "relative increase in β waves of brain waves" as explanation data, "awakening-unarousal" as emotion axis data, "fun, joy, anger, sadness, depression" as positive emotion type data, and "discomfort, anxiety, fear, relaxation, calm" as negative emotion type data.

[0099] Therefore, the generation unit 132 assigns "awake-unaroused" to the vertical axis and "strong emotion-weak emotion" to the horizontal axis.

[0100] Then, based on an analysis process of determining whether each emotion type in the index ID data "VS01" and "VS02" is assigned to a positive emotion type or a negative emotion type, the generation unit 132 places each emotion type in a corresponding quadrant of a two-dimensional Euclidean space formed by the vertical axis being "arousal-unarousal (positive side-negative side)" and the horizontal axis being "strong emotion-weak emotion (positive side-negative side)."

[0101] Specifically, the generation unit 132 arranges emotion types such as "fun," "joy," "anger," and "sadness" in the first quadrant, "melancholy" in the second quadrant, "relaxation" and "calmness" in the third quadrant, and "anxiety," "fear," and "displeasure" in the fourth quadrant.

[0102] In this way, the generation section 132 acquires, from the emotion type information table 121, emotion types corresponding to the index representing the arousal level based on the electroencephalogram and the index representing the intensity of the emotion based on the heart rate, respectively.

[0103] This avoids the effects of breathing by using an electroencephalogram sensor that can estimate the level of alertness based on the beta / alpha waves of the brainwaves from the activity state of the cerebral cortex.

[0104] The method of generating a model has been explained above, and will be explained by giving specific data using the specific example shown in Fig. 5. Fig. 6 is a diagram showing an example of the coordinate plane of a specific model created by this specific example.

[0105] As shown in FIG. 5, the vertical axis of the coordinate plane is the emotion axis "awakening-unarousal" of the index ID data "VS01", and the emotion types located on the positive side of the vertical axis are the positive emotion types of "joy", "happiness", "anger", "sadness", and "depression", while the emotion types located on the negative side of the vertical axis are the negative emotion types of "displeasure", "anxiety", "fear", "relaxation", and "calmness".

[0106] On the other hand, the horizontal axis of the coordinate plane is the emotion axis "strong emotion-weak emotion" of the index ID data "VS02", and the emotion types located on the positive side of the vertical axis are the positive emotion types of "fun", "happiness", "anger", "anxiety", "fear", and "displeasure", while the emotion types located on the negative side of the vertical axis are the negative emotion types of "depression", "boredom", "relaxation", and "calmness".

[0107] These are determined based on the emotion type information shown in FIG.

[0108] The generation unit 132 then determines whether the same emotion type is on the positive or negative side of the vertical and horizontal axes, and determines which quadrant of the model's coordinate plane to place it in based on the result. For example, the generation unit 132 places the emotion type "fun" in the "first quadrant" since it is on the positive side of the vertical axis ("awakening-restless") and the positive side of the horizontal axis ("strong emotion-weak emotion").

[0109] By performing this processing for each emotion type, as shown in FIG. 6, emotion types of “fun”, “joy”, “anger”, and “sadness” are arranged in area 210 of the first quadrant, emotion type of “melancholy” in area 220 of the second quadrant, emotion types of “relaxation” and “calmness” in area 230 of the third quadrant, and emotion types of “anxiety”, “fear”, and “displeasure” in area 240 of the fourth quadrant.

[0110] Furthermore, as shown in Fig. 7, the generation unit 132 may place the emotion types not adopted in the method of Fig. 5 in an area between two quadrants. Fig. 7 is a diagram showing an example of a coordinate plane of the model.

[0111] An emotion type that is not adopted for placement on the coordinate plane of the model occurs when the emotion type data does not exist in at least one of the two emotion indices selected as targets for the two axes of the coordinate plane of the model. In this case, it was decided to be not adopted using the method shown in Fig. 5, but another way of thinking is that when there is no correlation with a certain emotion index (no positive or negative emotion type exists), it can be determined to be an emotion type at the 0 position. Therefore, in the coordinate plane of this model, the generation unit 132 sets areas near the 0 value of the vertical axis and horizontal axis in addition to the areas of the four quadrants, and places the emotion type that was decided to be not adopted using the method shown in Fig. 5 in one of the areas near the 0 value of the vertical axis and horizontal axis (based on the positive and negative positions of the emotion types in the other indices).

[0112] In the above-mentioned example, as shown in FIG. 7, the generation unit 132 places the rejected emotion type “boredom” in the area 225 between the second and third quadrants (the emotion type “boredom” is treated as 0 (neutral) on the “awakening-unarousal” axis).

[0113] According to this method, even for emotion types that would not be adopted in the method shown in FIG. 5, the generation unit 132 can appropriately arrange the emotion types in the area 215 between the first and second quadrants, the area 225 between the second and third quadrants, the area 235 between the third and fourth quadrants, and the area 245 between the fourth and first quadrants.

[0114] In the above explanation, a method for arranging emotion types in each quadrant by language analysis processing of medical evidence, etc., has been presented, but emotion types can also be arranged in each quadrant manually by a developer, etc. For example, biosignals of subjects are measured under various circumstances, and a questionnaire survey on the emotion types they have is conducted. Then, in each situation, a quadrant of an emotion map is calculated based on the measured biosignals, and the emotion type of the questionnaire result is arranged in the calculated quadrant. With this method, emotion types can be set for each quadrant of an emotion map.

[0115] The determination unit 133 uses a model to determine the emotion type of the subject U02 based on the index value obtained from the biosignal of the subject U02. That is, the determination unit 133 analyzes and processes the biosignal from the sensor attached to the subject U02 into a corresponding index value. The determination unit 133 analyzes and processes the biosignal to obtain two types of index values. The determination unit 133 then fits the index value into the generated model (coordinate plane of the model) and determines the emotion type of the area to which the index value falls as the emotion type of the subject U02.

[0116] In this way, identification unit 133 acquires multiple types of first bio-signals and second bio-signals, converts the first bio-signals into a first index value that is an index of emotion, converts the second bio-signals into a second index value that is an index of emotion, and applies the first index value and second index value converted from the first bio-signal and the second bio-signal to an emotion model in which an emotion estimated value is determined by a combination of the first index value and the second index value, thereby determining an emotion estimated value as an estimated emotion.

[0117] The providing unit 134 provides the identified emotion type to the terminal device 20. Then, by providing the identified emotion type to a user such as the subject U02 by display or the like, the user such as the subject U02 can grasp and estimate the emotion of the subject U02, and can use it for training, etc. of the subject U02.

[0118] For example, the providing unit 134 displays the result display screen on a display (configured with a liquid crystal display or the like) of the terminal device 20. Fig. 8 is a diagram showing an example of the result display screen.

[0119] As shown in Fig. 8, the result display screen 301 displays the indices assigned to each axis, the subject's index value, the emotion type related to the estimated emotion result, text information such as messages, and related images suggesting the contents thereof. An emotion map is also displayed on the result display screen 301. These images are generated by the controller 13 based on the calculated index values ​​and the estimated emotion result.

[0120] The emotion map shows coordinates (emotion coordinates of the subject) obtained by plotting index values ​​obtained from the biosignals of the subject U02 on the coordinate plane of the model.

[0121] In the example of Fig. 8, since the emotion coordinates are in the first quadrant, the emotion of the subject U02 is estimated to be one of "fun", "joy", "anger", and "sadness". In addition, based on the position plotted on the emotion map, it is possible to estimate the strength of the emotion type to some extent (the farther from the origin, the stronger the emotion of the emotion type is estimated to be). Alternatively, based on the position plotted on the emotion map, it is possible to estimate the accuracy of emotion determination (the farther from the origin, the higher the accuracy of determination for the emotion type is estimated to be).

[0122] Here, an example in which two indexes are specified has been described. However, three or more indexes may be specified. For example, when three indexes are specified, the generation unit 132 arranges the emotion type in one of eight quadrants (three-dimensional space).

[0123] For example, a coordinate space is defined with emotion axes of index ID data "VS01", "VS02", and "VS03" as vertical, horizontal, and depth axes, that is, three-dimensional axes of XYZ. Then, the determination unit 133 plots three index values ​​based on each biosignal in the coordinate space, and determines the emotion type arranged in the space of the area where the plotted points are located as the emotion of the subject.

[0124] In the above example, the two indexes each have one judgment threshold, but two or more judgment thresholds may be used. For example, the first index based on the ratio of beta and alpha waves of the brainwaves may be stratified by two judgment thresholds, and the first index may be stratified into three states (brainwave states) based on the first index.

[0125] According to the above method, the number of regions in which emotion types are arranged is increased, and the number of types of indices used is also increased, making it possible to determine emotions in more detail.

[0126] The flow of the process executed by the server 10 will be described with reference to Figs. 9 and 10. Fig. 9 is a flowchart showing the flow of the extraction process executed by the controller 13 (extraction unit 131). This extraction process is executed based on, for example, a start command from the user, but the user needs to execute this process before estimating the emotion. Fig. 10 is a flowchart showing the flow of the estimation process. This estimation process is executed based on, for example, a start command from the user when the user desires an emotion estimation result.

[0127] 9, in step S101, the controller 13 (extraction unit 131) assigns 1 to n, and proceeds to step S102. n is a number that identifies the index to be extracted. Furthermore, X is the number of indexes to be extracted. X is set by the user as necessary, but the minimum number is 2.

[0128] In step S102, the controller 13 (extraction unit 131) extracts the n-th index based on the biological signal, and proceeds to step S103. In step S103, the controller 13 (extraction unit 131) extracts data such as emotion type specified by the extracted index, associates the data with the index ID as emotion type information as shown in Fig. 3, and stores the data, and proceeds to step S104.

[0129] The controller 13 (extraction unit 131) extracts the index data and the emotion type data by performing natural language analysis on, for example, medical papers and books.

[0130] In addition, the controller 13 (extraction unit 131) may store corresponding data input by a human being through an input operation device such as a keyboard as emotion type information, or may retrieve corresponding data from a socially shared database such as a medical database and store it as emotion type information.

[0131] Then, in step S104, the controller 13 (extraction unit 131) judges whether the number n of indicators from which emotion type information has been extracted has reached the set number X, and ends the process if it has reached the set number, and proceeds to step S105 if it has not. In step S105, the controller 13 (extraction unit 131) adds 1 to the counter value n indicating the number of indicators from which emotion type information has been extracted, and returns to step S102. That is, the processes of steps S102 and S103 are repeated until the number n of indicators from which emotion type information has been extracted reaches the set number X.

[0132] 10, the controller 13 (generation unit 132) determines a plurality of index values ​​to be used for emotion estimation in step S201, and proceeds to step S202. This determination is made, for example, by the controller 13 (provision unit 134) providing the user with index information required to estimate an emotion that the user wishes to estimate, and the controller 13 (generation unit 132) acquiring an index selected by the user through a selection operation.

[0133] In step S202, the controller 13 (the generation unit 132) generates a model corresponding to the determined index value, and proceeds to step S203. Note that the controller 13 (the generation unit 132) can generate the model by the method shown in FIG.

[0134] In step S203, the controller 13 (the generation unit 132) acquires a biological signal corresponding to the determined index value from a sensor or the like attached to the user, and proceeds to step S204. The controller 13 (the generation unit 132) processes the acquired biological signal as necessary to convert it into an index value.

[0135] Prior to that, the controller 13 (provision unit 134) also provides the user with information such as sensors that the user needs to wear and a guide for starting emotion estimation, thereby assisting the user in preparation.

[0136] Then, in step S204, the controller 13 (generation unit 132) applies the index value based on the acquired biological signal to the generated model, identifies the emotion type corresponding to the index value based on the emotion estimation method described with reference to Fig. 6 etc., and ends the process. For example, the controller 13 (generation unit 132) identifies the emotion type based on which quadrant the emotion coordinates obtained by plotting the index value on the coordinate plane of the model are located.

[0137] If the biosignal indices used are an index based on the ratio of beta waves and alpha waves in the brainwaves (first index: arousal level) and an index based on the low-frequency components of the heartbeat (second index: emotion intensity), these operations (operations of the emotion estimation device) can be expressed as follows using the configuration of the emotion estimation model (emotion map) of the emotion estimation device and the processing performed by the controller.

[0138] The emotion estimation model forms four combined states (quadrant 1 to quadrant 4) by combining two brain wave states (awake-relaxed) stratified by a first judgment threshold using a first index based on the ratio of beta waves to alpha waves in the brain wave, and two heart rate states (strong emotion-weak emotion) stratified by a second judgment threshold using a second index based on the low frequency components of the heart rate, and each of the four combined states (quadrant 1 to quadrant 4) is assigned an emotion type (quadrant 1: "joy", "happiness", "anger", "sadness"; quadrant 2: "melancholy"; quadrant 3: "relaxed" and "calm"; quadrant 4: "anxiety", "fear", and "unpleasant").

[0139] Then, the controller stratifies the first index calculated based on the brain waves acquired from the subject using a first judgment threshold to determine the brainwave state (awake-unaware), stratifies the second index calculated based on the heartrate acquired from the subject using a second judgment threshold to determine the heartrate state (strong emotion-weak emotion), determines the combined state (e.g., first quadrant) in the model corresponding to the determined brainwave state and heartrate state, and sets the emotion type (e.g., ``fun'') corresponding to the determined combined state (e.g., first quadrant) as an estimated emotion (e.g., ``fun'').

[0140] The server 10 may accept the specification of an emotion type instead of an index, and generate a model from the specified emotion type. That is, this method is used when the user wants to know the state of a certain emotion type (presence or absence of occurrence and its strength), for example.

[0141] At this time, the controller 13 (generation unit 132) acquires two or more indices corresponding to the designated emotion type (including the emotion type designated as the positive emotion type or the negative emotion type) from the emotion type information table 121. The controller 13 (generation unit 132) also generates a model in which the emotion types are arranged in each quadrant of a space defined by axes associated with each of the two or more indices. The subject then wears the necessary sensors based on the emotion type information table 121, and the controller 13 acquires biometric data from the sensors. Thereafter, the emotion of the subject is estimated based on the biometric information in a manner similar to that described above.

[0142] As a result, even if the analyst U01 does not have sufficient knowledge about the indices, he or she can obtain information that estimates the state of a desired emotion type.

[0143] An example of a user interface of such a feeling estimation device will be described. Fig. 11 is a diagram showing an example of an analysis support screen. As shown in Fig. 11, an analysis support screen 302 displays a message saying "Please select an emotion type and press a search button", along with a group of check boxes allowing multiple emotion types to be selected.

[0144] The emotion types displayed together with the check boxes are data of emotion types included in the positive emotion types or negative emotion types in the emotion type information table 121.

[0145] When the search button is pressed, the providing unit 134 identifies a combination of indices that can estimate the emotion type selected by the checkbox, and displays information about the identified combination as a search result.

[0146] Furthermore, the providing unit 134 provides information regarding sensors required to obtain the indices displayed as search results.

[0147] For example, as shown in Fig. 11, it is assumed that the analyst U01 selects emotion types "fun" and "boredom" on the analysis support screen 302. Then, based on each data of the emotion type information table 121 shown in Fig. 3, the "standard deviation of the heartbeat LF component of VS02" and the "heartbeat interval (RRI) of VS03" are searched for, which are indices including the emotion types "fun" and "boredom" (searching for positive emotion types or negative emotion types in the emotion type information table 121).

[0148] Then, the analysis support screen 302 displays a combination of "standard deviation of heartbeat LF component" and "heartbeat interval (RRI)" as a search result. The analysis support screen 302 also displays a "heartbeat sensor" which is a sensor required when performing analysis using a combination of the standard deviation of the heartbeat LF component and the heartbeat interval (RRI). The user checks this screen, prepares the "heartbeat sensor", and performs emotion estimation.

[0149] Fig. 12 is a diagram for explaining a method for identifying an index. The example in Fig. 12 is a case where analyst U01 specifies analysis of emotions of "fun" and "boredom." As shown in Fig. 12, the providing unit 134 refers to the emotion type information table 121 and checks whether or not the emotion types of each index include "fun" and "boredom" selected by the analyst U01 (yes or no).

[0150] Then, the providing unit 134 identifies (adopts) a combination of indices in which both "fun" and "boring" are acceptable, and provides information on these identified indices and information on the sensors used to calculate the indices to the analyst U01 or the subject.

[0151] In addition, when the number of indicators that meets the adoption condition is equal to or more than the number of adoptions, the providing unit 134 automatically identifies a combination of indicators according to a predetermined criterion. For example, the providing unit 134 may set a criterion such as identifying a combination of indicators that has a high track record of being selected in the past, identifying a combination of indicators that is estimated to have high accuracy, identifying a combination of indicators that has a high penetration rate of necessary sensors, or identifying a combination of indicators that has as many identifiable emotion types as possible.

[0152] Furthermore, the providing unit 134 may recommend combinations of indices that can be combined to the analyst U01, and the analyst U01 may adopt a combination of indices that he or she selects from the recommended combinations.

[0153] The biosignals that can be used in the first embodiment are not limited to those described above. For example, the emotion type information table 121 may include emotion types corresponding to indicators obtained from biosignals such as body surface temperature, face image, and pupil dilation. The level of arousal may be, for example, body temperature, body surface temperature, pupil diameter, electroencephalographic fluctuation, or the level of drowsiness measured by image recognition.

[0154] Also, it is possible to provide the terminal device 20 with a configuration that realizes functions equivalent to the extraction unit 131, the generation unit 132, the identification unit 133, and the provision unit 134 of the server 10 (functions realized by the controller 13), and to have the terminal device 20 perform the emotion estimation operation performed by the above-mentioned server 10. In that case, the terminal device 20 has a configuration equivalent to that of the above-mentioned server 10.

[0155] As described above, the controller 13 of the server 10 according to the first embodiment acquires first emotion type information associated with a first indicator related to a bio-signal, acquires second emotion type information associated with a second indicator related to a bio-signal, and generates an emotion estimation model in which an emotion type selected from the first emotion type information and the second emotion type information is associated with each combined index state constituted by a combination of a first index state which is each state in the first index and a second index state which is each state in the second index, according to the combined first index state and second index state.

[0156] In this way, by generating a model according to information (emotion type information table 121) in which indexes related to bio-signals are associated with emotion types, emotions can be estimated with high accuracy based on bio-signals.

[0157] In particular, if the emotion type information table 121 is based on medical evidence, it is possible to reduce errors (degree of deviation) in the association between the biological signal and the emotion.

[0158] Furthermore, the emotion type information table 121 can be updated at any time with information extracted from medical evidence. In this way, by updating the emotion type information table 121, model learning progresses and estimation accuracy is further improved.

[0159] [Second embodiment] Although an example of the emotion estimation result display screen has been described using Fig. 8, it is preferable that the display form of the result display screen be appropriate depending on the mode of use. Therefore, next, other display forms on the result display screen will be described.

[0160] Fig. 13 is a diagram showing an example of a result display screen of the second embodiment. As in Fig. 8, in Fig. 13, the vertical axis is assigned index values ​​of the index representing the arousal state, and the horizontal axis is assigned index values ​​of the index representing the intensity of the emotion.

[0161] The controller 13 (providing unit 134) indicates whether the emotion of the subject is stable (calm (normal) state) on the result display screen 301. Specifically, the controller 13 (providing unit 134) displays a rectangular frame graphic 3011 surrounded by a dashed line indicating a stable emotion region, and indicates whether the emotion of the subject is stable or not based on whether the coordinates of the estimated emotion of the subject are within the frame graphic 3011 (the area within the frame graphic 3011 is a stable state determination region).

[0162] The stable emotional region is, for example, a region limited by a value obtained by multiplying the maximum and minimum values ​​of the vertical axis index value and the horizontal axis index value by a predetermined coefficient (which specifies the maximum and minimum values ​​of the vertical axis and horizontal axis), and the predetermined coefficient is set to an appropriate value based on experiments, etc.

[0163] For example, if the vertical axis index value and horizontal axis index value are normalized with a maximum value of 1 and a minimum value of -1, the coefficient is set to an appropriate value less than 1, such as 0.2. In this case, the coordinates of the four vertices of frame graphic 3011 are (0.2, 0.2), (-0.2, 0.2), (0.2, -0.2), and (-0.2, -0.2).

[0164] Then, for example, if each index value is within the range of -0.2 to 0.2, the controller 13 (providing unit 134) displays a mark of the estimated emotion (a star in the example of FIG. 10) at the emotion coordinates indicating the estimated emotion within the frame graphic 3011 of the emotion stable area. Furthermore, the controller 13 (providing unit 134) displays a message saying "The subject's emotion seems to be stable." In this way, according to the second embodiment, it is possible to explicitly indicate whether the subject's emotion is in a stable state.

[0165] In the above example, the subject's emotions are suggested to be stable, but by appropriately setting the position and size of the frame graphic 3011, it is possible to suggest whether the subject is in a specific emotional state. The appropriate position and size of the frame graphic 3011 can be specified by a method such as setting the range of emotional coordinates corresponding to the state to be confirmed based on an experiment or the like. For example, when a factory worker is used as the subject, the range of emotional states in which the subject can be considered to be concentrating on work is set as the frame graphic 3011. Also, when a vehicle driver is used as the subject, the range of emotional states in which the subject can be considered to be driving calmly is set as the frame graphic 3011.

[0166] Furthermore, the controller 13 (providing unit 134) may change the scale of the axis to an appropriate format. For example, the controller 13 (providing unit 134) can provide a display with good visibility according to the purpose of use and the characteristics of the estimated emotion type or index type, such as suggesting a specific range of the emotion coordinates in detail, by making the scale nonlinear so that the scale is smaller as the index value is smaller.

[0167] [Third embodiment] Depending on the purpose of using the estimated emotion, it may be more appropriate to change the estimation standard (threshold) of the emotion. For example, when using estimated emotion information to evaluate a horror movie, it is possible to appropriately evaluate the movie using the estimated emotion by changing the threshold of the fear emotion according to the target fear level, or by changing the threshold of the fear emotion according to the age of the viewer of the horror movie.

[0168] Therefore, in the third embodiment, as shown in FIG. 14, the position (origin) of the emotion axis (vertical axis, horizontal axis, or either one) is moved, and the range of each quadrant for emotion determination is changed.

[0169] In normal emotion determination, an emotion map is used in which the vertical axis 3012a and the horizontal axis 3013a intersect at the position where each index value is 0, that is, at the origin. Note that the vertical axis 3012a and the horizontal axis 3013a are for explanation purposes only, and are not actually displayed on the result display screen 301 in the present embodiment 3.

[0170] However, in the case of a horror movie as described above, when evaluating a movie that aims for a higher sense of fear, a more appropriate evaluation can be performed by using a threshold value corresponding to a stronger sense of fear than usual. In other words, depending on the purpose of use, it may be possible to perform more appropriate emotion estimation by moving the positions of the vertical and horizontal axes in the emotion map. Therefore, in the present embodiment 3, the position of the emotion axis (the judgment threshold for emotion judgment) is changed according to the purpose of use of the estimated emotion, specifically, based on the type of device that uses the estimated emotion, the purpose of use input by the user, or the user's operation to adjust the emotion axis position.

[0171] Specifically, when the vertical axis index value and horizontal axis index value are normalized with a maximum value of 1 and a minimum value of -1, the controller 13 (providing unit 134) moves the vertical axis position and the horizontal axis position by, for example, 0.2 to the negative side depending on the purpose of use, and displays them on the emotion map of the result display screen 301. That is, the horizontal axis moves to the position of arousal level -0.2 (horizontal line passing through coordinates (0, -0.2)), and the vertical axis moves to the position of emotion intensity +0.2 (vertical line passing through coordinates (0.2, 0)) (the intersection coordinates become (0.2, -0.2)).

[0172] Then, the controller 13 (provider 134) displays a mark (e.g., a star) at the coordinate position of each index value based on the biological signal of the subject. The subject's emotion coordinates are located in the fourth quadrant when the vertical axis 3012a and the horizontal axis 3013a are used as references (before the axes are moved). Therefore, in this case, it is estimated that the subject is feeling fear.

[0173] On the other hand, the subject's emotion coordinates are located in the second quadrant when the vertical axis 3012b and the horizontal axis 3013b are used as references (after the axes are moved). In this case, it is estimated that the subject does not feel the targeted level of fear. In this case, for example, the controller 13 (provider 134) displays a message on the result display screen 301 saying, "It is estimated that the subject does not feel sufficient fear."

[0174] In this way, the area of ​​each quadrant can be adjusted by moving the positions of the vertical and horizontal axes according to the intended use of the estimated emotion, etc. In other words, the emotion estimation result can be adjusted according to the intended use of the estimated emotion, etc. For example, in evaluating content such as a horror movie, it is possible to evaluate the emotion (fear) according to the level of fear that the content is intended to inflict on viewers.

[0175] [About emotion type information] The emotion type information table 121 shown in Fig. 3 may be updated as appropriate. For example, in the emotion type information table 121 in Fig. 3, the keyword "boredom" is not included in the "positive emotion type" and "negative emotion type" of the record whose index ID data is "VS01".

[0176] In this case, if new medical evidence is discovered regarding the relationship between the emotion of "boredom" and electroencephalograms, or if proof is provided through experiments, etc., the keyword "boredom" may be added to the corresponding "positive emotion type" or "negative emotion type" of the record whose index ID data is "VS01" based on the content of the medical evidence or experiments, etc.

[0177] In addition, if a new indicator suitable for emotion estimation is discovered based on medical evidence, etc., a new data record may be generated for that indicator and each data item such as sensor type, description, emotion axis, etc. corresponding to that use may be stored.

[0178] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0179] N Network U01 Analyst U02 Subject 1. Estimation System 10 Server 11 Communications Department 12 Storage section 13 Controller 20 Terminal Equipment 31, 32 Sensor 121 Emotion type information table 131 Extraction part 132 Generation part 133 Specific part 134 Provision Department 210, 215, 220, 225, 230, 235, 240, 245 area 301 Results display screen 302 Analysis support screen

Claims

1. An emotion estimation device for estimating emotions, comprising: a controller; and an emotion estimation model; The emotion estimation model is four combined states are formed by combining two brain wave states obtained by stratifying a first index based on a ratio of beta waves and alpha waves of the brain wave using a first judgment threshold and two heart rate states obtained by stratifying a second index based on a low frequency component of the heart rate using a second judgment threshold, and an emotion type is set for each of the four combined states; The controller: The first index calculated based on the electroencephalogram obtained from the subject is classified by the first judgment threshold to determine the electroencephalogram state; The second index calculated based on the heart rate obtained from the subject is classified by the second judgment threshold to determine the heart rate state; The combination state in the emotion estimation model corresponding to the determined electroencephalogram state and heart rate state is determined, and the emotion type corresponding to the determined combination state is set as an estimated emotion. Emotion estimation device.

2. The emotion estimation model is An emotion type of “joy”, “happiness”, “anger”, or “sadness” is set to the combination state in which the first indicator is greater than the first judgment threshold and the second indicator is greater than the second judgment threshold; An emotion type of “melancholy” is set for the combination state in which the first indicator is greater than the first judgment threshold and the second indicator is less than the second judgment threshold; An emotion type of “relaxed” or “calm” is set to the combined state in which the first index is smaller than the first judgment threshold and the second index is smaller than the second judgment threshold; The combination state in which the first index is smaller than the first judgment threshold and the second index is larger than the second judgment threshold is set with an emotion type of "anxiety," "fear," or "unpleasantness." The emotion estimation device according to claim 1 .

3. The controller: The first judgment threshold or the second judgment threshold is adjusted based on an adjustment operation input. The emotion estimation device according to claim 1 .

4. The controller: Infer the purpose of use of the estimated emotion, The first determination threshold or the second determination threshold is adjusted based on the estimated purpose of use. The emotion estimation device according to claim 1 .

5. The controller: accepting an input of a selection operation by a user to select an index type to be used as the first index or the second index; A biosensor type corresponding to the indicator type selected by the selection operation is notified. The emotion estimation device according to claim 1 .

6. The controller displays on the display: displaying an emotion map with the first index as a vertical axis and the second index as a horizontal axis; A mark image is displayed at a coordinate position on the emotion map that is determined based on the first index and the second index, the coordinate position being determined based on the biological signal. Displaying text information about estimated emotions The emotion estimation device according to claim 1 .

7. An emotion estimation device for estimating emotions, comprising: a controller; and an emotion estimation model; The emotion estimation model is four combination states are formed by combining two heartbeat interval states obtained by stratifying a first index based on the heartbeat interval using a first judgment threshold and two low heartbeat frequency states obtained by stratifying a second index based on the low frequency component of the heartbeat using a second judgment threshold, and an emotion type is set for each of the four combination states; The controller: The first index calculated based on the heart rate obtained from the subject is classified by the first determination threshold to determine the heart rate interval state; The second index calculated based on the heart rate obtained from the subject is classified by the second determination threshold to determine the low heart rate state; The combination state in the emotion estimation model corresponding to the determined heartbeat interval state and the heartbeat low frequency state is determined, and the emotion type corresponding to the determined combination state is set as an estimated emotion. Emotion estimation device.

8. The emotion estimation model is An emotion type of “joy”, “happiness”, “anger”, or “anxiety” is set to the combination state in which the first indicator is greater than the first judgment threshold and the second indicator is greater than the second judgment threshold; An emotion type of “melancholy”, “boredom”, “relaxation”, or “calmness” is set to the combination state in which the first indicator is smaller than the first judgment threshold and the second indicator is smaller than the second judgment threshold; The emotion type of "unpleasant" or "sad" is set to the combination state in which the first index is smaller than the first judgment threshold and the second index is larger than the second judgment threshold. The emotion estimation device according to claim 7 .

9. A method for generating an emotion estimation model for estimating emotions, comprising: forming an empty model capable of setting an emotion type to be output as an estimation result for each of four combination states obtained by combining two electroencephalogram states obtained by stratifying a first index based on a ratio of beta waves and alpha waves of electroencephalograms using a first judgment threshold and two heart rate states obtained by stratifying a second index based on a low frequency component of heart rate using a second judgment threshold; extracting a first emotion candidate, which is a type of emotion felt in each of the two electroencephalogram states, from external information; extracting second emotion candidates, which are types of emotions felt in each of the two heart rate states, from external information; extracting an overlapping emotion type that overlaps between the first emotion candidate and the second emotion candidate, and the electroencephalogram state and the heartbeat state in which the overlapping emotion type is included as the first emotion candidate and the second emotion candidate; The corresponding overlapping emotion type is set for the combination state of the blank model corresponding to each of the extracted electroencephalogram states and the extracted heart rate state. A method for generating emotion estimation models.

10. the first index is determined based on information regarding a relationship between a ratio of beta waves to alpha waves of electroencephalograms extracted from external information and emotions; The second index is determined based on information regarding a relationship between a low-frequency component of a heart rate and emotions extracted from external information. The emotion estimation model generating method according to claim 9 .

Images