Emotion estimation device, emotion estimation method, emotion estimation program, and emotion estimation system

Abstract

To easily improve emotion estimation accuracy based on a biological signal.SOLUTION: An estimation device (a server) according to an embodiment estimates an emotion of a user based on variation with the lapse of time of a plurality of indexes based on a biological signal of the user.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention provides an emotion estimation device and an emotion estimation method. emotion estimation program And relating to emotion estimation systems. [Background technology]

[0002] A technique is known for estimating a subject's emotions by applying information obtained from the waveform of the subject's heart (electrocardiogram) to the Russell circumplex model (see, for example, Patent Document 1). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2019-63324 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] However, conventional technologies have a problem in that they cannot easily improve the accuracy of emotion estimation based on biosignals.

[0005] Biological information is greatly affected by individual and environmental differences (such as physical condition and emotions at the time). Therefore, when applying information obtained from biological signals to an emotion estimation model such as the Russell ring model to estimate emotions, it is necessary to correct (calibrate) the emotion estimation model to suit the subject in order to improve estimation accuracy.

[0006] On the other hand, calibrating emotion estimation models has several problems, including being time-consuming, burdensome to subjects (because it involves inducing subjects into specific emotional states by adding specific stimuli or having them perform specific postures and movements, and then performing calibration using measurements taken in that state, thus placing a physical burden on the subjects), and being affected by the external environment.

[0007] This invention has been made in view of the above, and aims to enable appropriate estimation of emotions based on biological signals while suppressing the influence of individual and environmental differences. [Means for solving the problem]

[0008] To solve the above-mentioned problems and achieve the objective, the estimation device according to the present invention is an emotion estimation device that estimates emotions, and has a controller, which estimates the user's emotions based on the changes over time of a plurality of indicators based on the user's biosignals. [Effects of the Invention]

[0009] According to the present invention, it is possible to appropriately estimate emotions based on biological signals while suppressing the influence of individual differences and environmental differences. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows an example of the configuration of the estimation system according to the embodiment. [Figure 2] Figure 2 shows an example of a server configuration according to this embodiment. [Figure 3] Figure 3 shows an example of an emotion estimation model (psychological plane). [Figure 4] Figure 4 shows an example of an emotion estimation model that uses indicators of physical condition as parameters. [Figure 5] Figure 5 shows an example of a sensor table and a psychological plane table. [Figure 6] Figure 6 shows an example of an emotion change table. [Figure 7] Figure 7 illustrates the method for extracting emotion types. [Figure 8] Figure 8 shows an example of an evaluation screen and a diagram illustrating the usage of information related to emotions. [Figure 9] Figure 9 shows an example of an evaluation screen and a diagram illustrating the usage of information related to emotions. [Figure 10]FIG. 10 is a diagram for explaining a calibration method of an emotion estimation model. [Figure 11] FIG. 11 is a diagram for explaining a learning method of a machine learning model. [Figure 12] FIG. 12 is a flowchart showing a model generation process. [Figure 13] FIG. 13 is a flowchart showing an emotion estimation process for estimating an emotional state.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of an emotion estimation device, an emotion estimation method, and an emotion estimation system will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited by the embodiments shown below.

[0012] First, the estimation system according to the embodiment will be described using FIG. 1. FIG. 1 is a diagram showing a configuration example of the estimation system according to the embodiment.

[0013] As shown in FIG. 1, the estimation system 1 includes a server 10, a terminal device 20, sensors 31a, 32a, 31b, and 32b. The estimation system 1 estimates the emotions of the subjects U02a and U02b. The server 10 is an example of an emotion estimation device.

[0014] The subjects U02a and U02b are, for example, e-sports players. The estimation system 1 estimates the emotions of the subjects U02a and U02b who are playing a video game. In the description of this embodiment, for the sake of making the explanation concrete and easy to understand, as an application example, the application scenario in the above e-sports is assumed, and the state transition and the like will be described.

[0015] The results of the emotion estimation can be used, for example, for the mental training of subjects U02a and U02b in esports. For instance, if subject U02 experiences unfavorable emotions (anxiety, anger, etc.) during video game play, it is determined that intensive training corresponding to that emotional state is necessary. In this training, the emotion information of subjects U02a and U02b estimated by estimation system 1 is used.

[0016] Furthermore, in various types of esports, subjects U02a and U02b participate in cooperative and competitive games. By displaying the emotional state of each player in these esports games, it becomes possible to perform advanced gameplay, such as changing game tactics according to the emotional state. Additionally, game spectators can understand the emotional state of each player while watching the game, resulting in an enhanced viewing experience.

[0017] In addition, another application example is that the subjects may be patients in a medical institution. In this case, the emotions estimated by estimation system 1 are used for examinations and treatments, etc.

[0018] For example, if a patient (subject) is feeling anxious, medical staff can provide support such as counseling.

[0019] Furthermore, the subjects may be students in an educational institution. In this case, the emotions estimated by estimation system 1 will be used to improve the content of the lessons.

[0020] For example, if a student (the subject) finds a lesson boring, the teacher can improve the lesson content to make it more engaging for the student.

[0021] Furthermore, the subjects may be vehicle drivers. In this case, the emotions estimated by estimation system 1 are used to promote safe driving.

[0022] For example, if a subject who is the driver does not feel adequately focused while driving, the in-vehicle device can display a message encouraging them to concentrate on driving.

[0023] Furthermore, the subjects may be viewers of content such as videos and music. In this case, the emotions estimated by estimation system 1 are used to create further content.

[0024] For example, a video content provider can create a highlight reel by compiling scenes that viewers (subjects) found enjoyable. In this way, estimated emotions can be used for a variety of purposes.

[0025] Returning to the system description, Server 10 and Terminal Device 20 are connected via Network N. For example, Network N is the Internet or an intranet.

[0026] For example, terminal device 20 consists of a personal computer, a smartphone, and a tablet computer, etc. Terminal device 20 is used by analyst U01.

[0027] Sensors 31a, 32a, 31b, and 32b are attached to subjects U02a and U02b, detect the biological signals of subjects U02a and U02b as sensor signals, and transmit the detected sensor signals to the terminal device 20.

[0028] Specifically, sensors 31a and 31b are, for example, headgear-type electroencephalogram (EEG) sensors. Sensors 32a and 32b are, for example, wristband-type pulse sensors.

[0029] Furthermore, sensors 31a, 32a, 31b, and 32b have a communication interface and communicate with the terminal device 20 according to communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark), and transmit sensor signals to the terminal device 20.

[0030] Figure 1 will be used to explain the processing flow of estimation system 1.

[0031] Server 10 is assumed to have registered models (calculation formulas and conversion data tables) for calculating biostate index values ​​for emotion estimation based on the subject's biosignals, and also has registered emotion estimation models for estimating the subject's emotions from multiple indicators based on the subject's biosignals. The emotion estimation models are created based on medical evidence (papers, etc.). The emotion estimation models will be described later.

[0032] Sensors 31a, 32a, 31b, and 32b measure the biological state (signals) of subjects U02a and U02b, and output the measured sensor signals to the terminal device 20 (step S1). The terminal device 20 then transmits the input sensor signals to the server 10 (step S2).

[0033] Server 10 generates a biological state index value, which is an index value indicating the biological state, based on the input sensor signal biological signal. For example, Server 10 generates two biological state index values ​​related to the sensor signal, either electroencephalogram or heart rate. For example, the indices based on the sensor signal of a heart rate sensor are "average heart rate interval" and "heart rate LF component," and Server 10 calculates the index value (e.g., a numerical value) which is the value of these "average heart rate interval" and "heart rate LF component" based on the sensor signal of the heart rate sensor.

[0034] Furthermore, it is also possible to equip sensors 31a, 32a, 31b, and 32b with calculation functions (built-in computers, etc.) and configure them to calculate biological state index values. It is also possible to configure the terminal device 20 to calculate biological state index values ​​based on the biological signals input from sensors 31a, 32a, 31b, and 32b.

[0035] Then, based on the calculated biological state index values, the server 10 estimates the emotional states of subjects U02a and U02b using its built-in emotion estimation model (step S3). In this embodiment, the server 10 estimates emotional state information, such as emotional change states, based on changes in the biological state index values. The detailed content and estimation method of emotional state information, such as emotional change states, will be described later.

[0036] The server 10 then provides the estimated results of the emotional state information to the terminal device 20 (step S4). The terminal device 20 then performs actions such as displaying the received emotional state information based on the operations of the analyst U01, providing it to external information users based on the operations of the operator, and providing it to an external game system, so that the emotional state information is used in various ways.

[0037] Furthermore, the server 10, terminal device 20, and user-side device (e.g., game device) will each have a controller that performs the functions and operations described above.

[0038] Figure 2 shows an example of a server configuration according to the embodiment. Server 10 is an example of a computer that performs the estimation of emotional state information. Server 10 is also an example of an estimation device.

[0039] As shown in Figure 2, the server 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0040] The communication unit 11 is an interface for communicating data with other devices via the network N. The communication unit 11 is, for example, a NIC (Network Interface Card).

[0041] The memory unit 12 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or by storage devices such as hard disks or optical discs. The memory unit 12 contains a sensor table 121, a psychological plane table 122 (referred to as a psychological plane table for simplicity as it is a map table using two types of indicators; however, if there are three or more types, it becomes a 3-dimensional (map) table), and a psychological judgment table 123 in its memory area. The memory unit 12 is an example of a memory device.

[0042] The control unit 13 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs (not shown) stored in the memory unit 12 using RAM as the working area. The control unit 13 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0043] The control unit 13 includes an extraction unit 131, a psychological analysis unit 132, a generation unit 133, a specific unit 134, a provision unit 135, an acquisition unit 136, and a model selection unit 137, and realizes or executes the information processing functions and operations described later.

[0044] Server 10 may also acquire the aforementioned programs and various information via other computers or portable storage media connected by wired or wireless networks.

[0045] Next, we will explain the data tables (databases) for various processes stored in the memory unit 12, but to make it easier to understand, we will first briefly explain the psychological estimation model (psychological estimation principle).

[0046] Figure 3 shows an example of an emotion estimation model (psychological plane). According to various medical evidence related to psychology, there is a technical concept that psychology can be estimated based on two types of indicators that represent physical states. The psychological plane shown in Figure 3, which uses two types of indicators of mind and body states as axes, is an example of this model, with the vertical axis representing "activity level of thinking and judgment" and the horizontal axis representing "activity level of the body". Psychological states are assigned to each quadrant region separated by the vertical and horizontal axes, with the origin of the psychological plane (for example, the midpoint between "activity level of thinking and judgment" and "activity level of the body") as the reference point. For example, in the example in Figure 3, the psychological states of "happy, joyful, angry, and sad" are assigned to the first quadrant. The distance from the origin indicates the intensity of the corresponding psychological state.

[0047] In other words, the psychological state and its intensity are estimated based on the coordinates of the two types of mental and physical state indicators measured in the subject, their position in the psychological plane, and their distance from the origin.

[0048] Furthermore, academic papers and other sources have shown that physical states such as "thinking and judgment activity" and "physical activity" can be estimated from the subject's biosignals (outputs from various sensors attached to the subject). Based on these, two types of indicators representing the subject's physical state can be estimated based on the subject's biosignals, and by applying these two estimated indicators to an emotion estimation model (a model using these two indicators) as shown in Figure 3, the subject's psychology can be estimated.

[0049] Furthermore, the relationship between biosignals and indicators of physical state, and the relationship between indicators of physical state and psychological state, will be identified and estimated, for example, through measurement experiments of these biosignals (measurement of various sensor signals), physical state (measurement of various sensor signals), and psychological state (measurement of various sensor signals, questionnaires, observation and analysis of subject facial expressions, etc.).

[0050] Furthermore, in this embodiment, the mental and physical state is estimated based on the coordinate change state (coordinate trajectory) of the physical state indicators (two types) on the psychological plane, rather than their coordinate positions. Next, the relationship between the coordinate change state and the psychological state will be explained.

[0051] Figure 4 shows an example of an emotion estimation model that uses indicators of physical state as parameters. Similar to the emotion estimation model shown in Figure 3, this model is represented by a psychological plane with two types of indicators of physical state as axes, where the vertical axis represents "activity level of thinking and judgment" and the horizontal axis represents "activity level of the body." Models using other types of physical states on the vertical and horizontal axes can be created using a similar approach.

[0052] The model shown in Figure 4 illustrates the emotional changes associated with each direction of change in the physical activity state and the cognitive / judgment activity state. This model is based on the coordinate changes of the physical state indicators, and the coordinate positions of the physical state indicators are basically meaningless (although it can be used in conjunction with models based on the coordinate positions of the physical state indicators). For example, if the physical activity state is "physical sedation and improved digestion" and the cognitive / judgment activity state is "active thinking, recognition, and judgment, high arousal," the emotional change will be "confused and depressed, physical system: low energy and fatigue, brain system: high cognitive confusion." This model shown in Figure 4 is generated based on, for example, how the coordinate positions of the two types of physical states move (in which direction they move (in which psychological state becomes stronger, or in which psychological state becomes weaker)) for the psychological state regions arranged in each quadrant in the emotion estimation model shown in Figure 3.

[0053] Then, by stratifying and analyzing the corresponding emotional changes in each direction of movement between the physical activity state and the thinking / judgment activity state in each direction of the vertical and horizontal axes in Figure 4, it is possible to generate the psychological judgment table 123 shown in Figure 6, which will be described later.

[0054] Figure 5 shows an example of the sensor table 121 (table (A) in Figure 5) and the psychological plane table 122 (table (B) in Figure 5). As shown in table (A) of Figure 3, the items (data storage cells (storage frames)) of the sensor table 121 include "sensor ID", "sensor type", "biological signal type", "corresponding index ID", "corresponding index type", and "index conversion information".

[0055] The "Sensor ID" field in sensor table 121 stores sensor ID data, which is identification information used to identify data records in sensor table 121. Sensor ID data is also the primary key for data records in sensor table 121. In other words, in sensor table 121, a data record is created for each sensor ID data, and the data for each item associated with the sensor ID data is stored in that data record.

[0056] The "Corresponding Indicator Type" item in sensor table 121 stores indicator type data representing the type of biological status indicator.

[0057] The "Sensor Type" item in sensor table 121 stores information to identify the sensor type; in this case, it stores the sensor name (model number or other data is also acceptable).

[0058] The "Corresponding Indicator ID" item in sensor table 121 stores identification information for identifying the emotion estimation index generated (calculated) based on the biosignals detected by the sensor. The "Corresponding Indicator Type" item in sensor table 121 stores the type of index (name, etc.).

[0059] The "Biological Signal Type" item in sensor table 121 stores the type of measurement value based on the biological signal detected by the sensor. This biological signal type data is correlated with the corresponding index type data (it is recognized academically that the corresponding index type data can be estimated (calculated) from the corresponding biological signal type data).

[0060] The "Index Conversion Information" item in Sensor Table 121 stores conversion information (such as calculation formulas and conversion data tables) for calculating index values ​​based on the biological signals detected by the sensor. In other words, by converting the biological signals detected by the sensor corresponding to the sensor ID data according to the index conversion information data, the index data (value) of the emotion estimation index identified by the corresponding index ID is estimated (calculated).

[0061] For example, the data record for sensor IDSN01 in sensor table 121 shown in Figure 5 contains information such as: the biological signals of "beta waves / alpha waves of the brainwave" are calculated from the output signal of "EEG sensor BA," and an index of "activation of thought recognition judgment" can be calculated using the index conversion information of "FX01" from these "beta waves / alpha waves of the brainwave."

[0062] The psychological plane table 122 is a two-dimensional matrix table in which the indicator type (specifically, indicator ID data) is used as the parameter on the vertical and horizontal axes. The data for the psychological plane types that can be used with the data of the two indicator types is stored in memory cells (memory frames) determined by the two indicator types. For example, if the indicator types used as indicators are indicator type VSm and indicator type VSn, the psychological plane used for psychological estimation will be psychological plane mn, and information for processing using psychological plane mn will be read out and used for psychological estimation processing.

[0063] Furthermore, the same data is used for the index IDs in sensor table 121 and the index IDs in psychological plane table 122. That is, based on sensor table 121 and psychological plane table 122, the psychological plane mn used for psychological estimation can be determined using the two types of sensors attached to the subject. For example, if the types of sensors attached to the subject are "EEG sensor BA (index ID VS01)" and "heart rate sensor HA (index ID VS02)", the index ID values ​​corresponding to "EEG sensor BA (index ID VS01)" and "heart rate sensor HA (index ID VS02)" are determined based on sensor table 121 (for example, VS01: index ID value corresponding to "activation of thought, recognition, and judgment", VS02: index ID value corresponding to "physical activity"). Then, based on psychological plane table 122, psychological planes 01-02, using "activation of thought, recognition, and judgment" (VS01) and "physical activity" (VS02) as indicators, are determined as the psychological planes used for psychological estimation.

[0064] Figure 6 shows an example of an emotion change table that displays the emotional state based on the coordinate changes of the index values ​​in a two-dimensional emotion plane, where emotions are plotted on a plane with the index values ​​of two types of indexes ("Activity of thinking and judgment" and "Sympathetic nervous system activity") on the vertical axis and "Activity of thinking and judgment" on the horizontal axis. In other words, it shows an example of a data table for forming an emotion estimation model that uses the physical state index shown in Figure 4 as a parameter.

[0065] As shown in Figure 6, the items in the psychological judgment table 123 (memory cells (memory frames) in the data table) include "emotional change ID," "change pattern," "display symbol," "meaning of the direction of change," and "emotional type."

[0066] The "Emotional Change ID" item in Psychological Judgment Table 123 stores emotional change ID data, which is identification information used to identify data records in Psychological Judgment Table 123. The emotional change ID data is also the primary key in the data records of Psychological Judgment Table 123. In other words, in Psychological Judgment Table 123, a data record is created for each emotional change ID data, and the emotional change ID data of each item associated with the emotional change ID data is stored in that data record.

[0067] The "Change Pattern" item in Psychological Judgment Table 123 stores change direction information indicating the change pattern (type of coordinate trajectory shape) of the coordinate positions of two types of indicator data on a two-dimensional emotion plane. Here, the change direction is stratified into eight directions in stages (the angle region is divided into 45-degree units, and the change direction is stratified according to which region it belongs to), and the data for the direction to which the target data belongs among the eight stratified directions is stored. In addition to the change direction, the change pattern of the coordinate position can also be applied to the shape of the coordinate trajectory (combination of curves, straight lines, etc.: in this case, the change pattern of the measured value to be judged will be determined by similarity), the speed of change, or combinations thereof.

[0068] The "Display Symbol" item in Psychological Judgment Table 123 stores a symbol (image) that indicates the direction of change in the coordinate positions of two types of indicator data on a two-dimensional emotional plane. This symbol (image) is used when displaying emotional changes on a display, etc. The symbol (image) indicating the direction of change is also layered into eight directions, similar to the change direction information mentioned above, and is stored in the corresponding data record. Furthermore, the data content of this "Display Symbol" should be set appropriately according to the data content of the "Change Pattern". For example, if the data content of the "Change Pattern" is the shape of a coordinate trajectory, the data of the "Display Symbol" should be an image that shows the shape of the coordinate trajectory.

[0069] The item "Meaning of the direction of change" in the psychological judgment table 123 stores information (text data) indicating the change in mental and physical state in relation to the direction of change in the coordinate positions of two types of indicator data on a two-dimensional emotional plane.

[0070] The "Emotion Type" item in Psychological Judgment Table 123 stores the emotion type corresponding to the data record, that is, the emotion type (the type of emotion whose intensity increases) corresponding to the change pattern of the coordinate positions of the two types of indicator data on the two-dimensional emotion plane stored in the "Change Pattern" of the data record.

[0071] For example, the information for a data record where the value of the item "Change ID" is "D02" indicates that the direction of change in the coordinate positions of the two types of indicator data on the 2D emotion plane is "upper right direction," the symbol (image) used for display is "arrow pointing up to the right," the physical state corresponding to this change in coordinate position is "body / brain activation," and the corresponding emotion type (type of emotion whose intensity is increasing) is "happy, joyful, angry, sad." In other words, when the direction of change in the coordinate positions of the two types of indicator data on the 2D emotion plane is "upper right direction," the physical state is estimated to be "body / brain activation," and the emotion being experienced (the emotion that is getting stronger) is estimated to be "happy, joyful, angry, sad." Then, using this information, the display will show, for example, an "arrow pointing up to the right" as a display of the change in physical state, and text information such as "happy, joyful, angry, sad" as a display of the change in the emotion being experienced (the emotion that is getting stronger).

[0072] The psychological judgment table 123 is created by the designer and developer based on academic information in psychology, such as shown in Figure 4, but it can also be generated by using AI or the like to perform language analysis processing on academic information in psychology, such as research papers. In this embodiment, the server 10 (control unit 13) generates the psychological judgment table 123 through language analysis processing or the like.

[0073] The example in Figure 4 illustrates a technique (psychological concept) that assigns the estimated emotional state to each increase or decrease in the parameters on the horizontal and vertical axes (degree of physical activity / inactivity (sedation) and degree of brain activity / inactivity (low arousal)) in a coordinate plane (for example, the emotion estimation model described later) where the horizontal axis represents physical activity / inactivity (sedation) and the vertical axis represents brain activity / inactivity (low arousal). For example, the example in Figure 4 shows that when indicators based on biosignals change in a direction that increases the tendency toward physical sedation and low arousal, the emotion shifts toward "rest / relaxation."

[0074] The psychological judgment table 123 shown in Figure 6 incorporates the technical concept shown in Figure 4 into a data table, enabling processing by controllers such as the control unit 13.

[0075] The processing details of each part of the control unit 13 will be explained below. In the following explanation, the main processing performed by the extraction unit 131, generation unit 133, identification unit 134, and provision unit 135 can be rephrased as the control unit 13. Furthermore, the control unit 13 is an example of a controller.

[0076] The extraction unit 131 extracts information regarding the relationship between indicator data of physical state and emotion types from medical evidence (papers, etc.) on psychology obtained from external academic literature databases, etc., via the communication unit 11. Figure 7 is a diagram illustrating the method for extracting emotion types.

[0077] As shown in Figure 7, the extraction unit 131 extracts information regarding the relationship between biosignals, indicators of physical state, and types of emotions by performing natural language analysis on texts written in papers and books.

[0078] The extraction unit 131 performs natural language analysis using, for example, existing machine learning methods or AI (artificial intelligence) to extract information about the relationship between biosignals, indicators of physical condition, and types of emotions.

[0079] Specifically, in the example shown in Figure 7, the extraction unit 131 associates "increased activity of thinking and judgment" with the indicator for emotion estimation, based on the text information in the medical evidence information, which states, "As the beta waves (of the brainwaves) increase, the activity of thinking and judgment increases." Furthermore, the extraction unit 131 associates "increased activity of thinking and judgment (increased activity of thinking and judgment)" with the emotions "happy," "angry," and "sad," based on the text information, which states, "As the activity of thinking and judgment increases, emotions such as happiness, anger, and sadness increase."

[0080] Furthermore, the extraction unit 131 associates "decreased activity of thinking and judgment" with the indicator for emotion estimation, based on the text information in the medical evidence information that states, "When alpha waves (in brain waves) increase, the activity of thinking and judgment decreases." Also, based on the text information that states, "As the activity of thinking and judgment weakens, emotions such as anxiety and fear intensify," the extraction unit 131 associates "decreased activity of thinking and judgment (weakening of thinking and judgment)" with the emotions "anxiety" and "fear."

[0081] The generation unit 133 generates an emotion estimation model based on the information regarding the relationship between the biosignals, indicators of physical state, and emotion types extracted by the extraction unit 131. The emotion estimation model generates indicator values ​​of physical state based on biosignals and estimates the emotional state from two types of indicator values ​​of physical state. Specifically, it is a model mainly consisting of a psychological judgment table 123 that determines (estimates) the emotional state from the changes in the indicator values ​​of two types of physical state, as shown in Figure 4. The generation unit 133 also generates data tables used when using the emotion estimation model, that is, it generates the sensor table 121, the psychological plane table 122, and the psychological judgment table 123 shown in Figures 5 and 6.

[0082] To illustrate with a specific example (using the data record value of sensor ID SN01 in Figure 3 as an example), the generation unit 133 stores newly input sensor type data (EEG sensor BA) in the "Sensor Type" column of the sensor table 121, either through operator operation or automatic search processing of internet information, and automatically assigns a "Sensor ID" (SN01) accordingly. Furthermore, when the generation unit 133 obtains newly input sensor type data, it also separately stores various information related to the sensor (for example, measurement items) in the storage unit 12. Then, based on the measurement item data related to the sensor and the information showing the relationship between the biosignal and the index for emotion estimation extracted by the extraction unit 131 (judgment based on commonality), the generation unit 133 generates data to be stored in the "Biosignal Type" and "Corresponding Index Type" columns of the sensor table 121, and stores it in the sensor table 121 (EEG β / α waves and activation of thought recognition judgment). In this process, the generation unit 133 stores the "corresponding index ID" for new indexes for emotion estimation, along with the identification data for that index, and stores the already assigned corresponding identification data for existing indexes for emotion estimation (VS01). Furthermore, based on the information showing the relationship between the biosignals extracted by the extraction unit 131 and the indexes for emotion estimation (the relationship formula and conversion table information between the two), it generates data to be stored in the "index conversion information" of the sensor table 121 and stores it in the sensor table 121 (FX01).

[0083] Furthermore, when generating an index for new emotion estimation (specific data is exemplified as VS03), the generation unit 133 sets the new "index" (VS03 on both the vertical and horizontal axes) for "index A" and "index B" in the psychological plane table 122. Then, based on the information regarding the relationship between the physical state index and emotion type extracted by the extraction unit 131, the generation unit 133 arranges the emotions corresponding to the index values ​​of the new index (VS03) and the emotions corresponding to the index values ​​of each existing index (VS01, VS02) in a two-dimensional space (using each index value as the vertical and horizontal axes, and placing the emotions in the coordinates (coordinate quadrants) indicated by the values ​​of each index for each emotion) to generate psychological plane data (psychological planes 01-03, psychological planes 02-03) and stores it in the corresponding memory cells (frames) of the psychological plane table 122.

[0084] The psychological plane is a conceptual representation, and its concept is similar to the planar space shown in Figure 3. Specifically, it consists of a set of data (data tables, etc.) that can be processed on the psychological plane, as shown in the psychological judgment table 123 in Figure 6.

[0085] In this embodiment, a psychological planar table 122 was used in an example where emotions were estimated using two types of physical state indicators. However, in an example where emotions were estimated using three types of physical state indicators, a psychological three-dimensional table was used. In short, a psychological table with dimensions corresponding to the number of types of physical state indicators used would be employed.

[0086] Next, using the example of the emotion estimation model (psychological plane) shown in Figure 7, we will explain a specific example of how the generation unit 133 generates an emotion estimation model.

[0087] Here, the vertical axis of the indicator axis in the psychological plane uses the index of "activation of thinking, recognition, and judgment" (biosignal: beta / alpha waves of electroencephalography (indicator ID: "VS01")), and the horizontal axis of the indicator axis uses the index of "physical activity" (biosignal: fluctuation of low-frequency components of cardiac waveform (indicator ID: "VS01")).

[0088] It should be noted that the extraction unit 131 has previously extracted information on the emotional content of the indicators "Activity of thought, recognition, and judgment" and "Physical activity" based on medical evidence (papers, etc.) as follows. "Activation of thinking, perception, and judgment" is strong (activity level): melancholy, enjoyment, joy, anger, sadness Weak (inactive) "Activation of thought, perception, and judgment": Relaxation, calmness, anxiety, fear, unpleasantness High "physical activity" (activity level): pleasure, joy, anger, sadness, anxiety, fear, unpleasantness Weak "physical activity" (inactive state): Depression, relaxation, calmness

[0089] The generation unit 133 assigns each emotion (content) to the corresponding quadrant on the psychological plane based on the relationship between the above-mentioned indicators extracted by the extraction unit 131 and the emotional content in relation to the strength of those indicators, thereby completing the psychological plane.

[0090] For example, the first quadrant of the psychological plane is a region where the index value for "activation of thinking, cognition, and judgment" is high, and the index value for "physical activity" is also high. Therefore, the first quadrant will be assigned emotional content that is common to both the emotional content corresponding to a high index value for "activation of thinking, cognition, and judgment" and the emotional content corresponding to a high index value for "physical activity."

[0091] Specifically, the generation unit 133 places the emotion content common to both the positive side (the side with strong "activation of thought, recognition, and judgment") corresponding to index ID "VS01" and the positive side (the side with strong "physical activity") corresponding to index ID "VS02," which is "happy, joyful, angry, and sad," into the first quadrant region 210.

[0092] Furthermore, the generation unit 133 places the emotion content "melancholy," which is common to the positive emotion content corresponding to index ID "VS01" and the negative emotion content corresponding to index ID "VS02," in the region 220 of the second quadrant.

[0093] Furthermore, the generation unit 133 places the emotion type "relaxation, calmness," which is common to the negative emotion content corresponding to index ID "VS01" and the negative emotion content corresponding to index ID "VS02," in the third quadrant region 230.

[0094] Furthermore, the generation unit 133 places the emotion content common to both the positive emotion content corresponding to index ID "VS01" and the negative emotion content corresponding to index ID "VS02," namely "anxiety, fear, and unpleasantness," into the fourth quadrant region 240.

[0095] The acquisition unit 136 receives various biometric signals and other data from the terminal device 20 via the communication unit, and stores the necessary data in its internal memory or storage unit 12 for subsequent processing.

[0096] The model selection unit 137 selects a model to be used for emotion estimation based on the types of various biosignals contained in the data acquired by the acquisition unit 136. Specifically, the model selection unit 137 matches the data in the "Sensor Type" column of the sensor table 121 with the type of biosignal and extracts the "Corresponding Indicator Type" indicator data corresponding to the biosignal. Then, the model selection unit 137 matches the extracted indicator type data (two types used for emotion estimation) with the psychological plane table 122 and selects the corresponding model as the emotion estimation model to be used (the psychological plane model of the data cell at the intersection of the two indicator types). For example, if the sensor types are electroencephalogram sensor BA (sensor IDSN01) and heart rate sensor HA (sensor IDSN02), the corresponding biostate indices will be "Activity of thought recognition and judgment" (indicator IDVS01) and "Physical activity" (indicator IDVS02), and the model selection unit 137 will select the psychological plane 01-02 model as the emotion estimation model to be used.

[0097] The identification unit 134 applies an index based on the biosignal acquired by the acquisition unit 136, that is, the biosignal actually measured, to the emotion estimation model selected by the model selection unit 137, that is, the emotion estimation model corresponding to the index of the physical state to be used (in other words, the type of sensor to be used), in order to identify the type of emotion (estimate the emotion).

[0098] Specifically, the identification unit 134 reads index conversion information corresponding to the two types of biological signals acquired by the acquisition unit 136 from the sensor table 121 (for example, if the sensor types are electroencephalogram sensor BA (sensor IDSN01) and heart rate sensor HA (sensor IDSN02), then index conversion information FX01 and index conversion information FX02), and uses this index conversion information to convert the biological signals into biological state index values.

[0099] Then, the identification unit 134 sequentially plots the coordinates of the two types of biological state index values ​​calculated by the conversion process on the psychological plane (on the psychological plane 01-02 when using the electroencephalogram sensor BA and heart rate sensor HA) of the emotion estimation model selected by the model selection unit 137, and estimates the emotion type based on the trajectory. Specifically, the identification unit 134 performs a matching process (recognition of the geometric pattern of the trajectory) for each data of the "change pattern" in the psychological judgment table 123 based on the change state of the two types of index values ​​based on biological signals, and uses the corresponding "emotion type" data as the emotion estimation result.

[0100] The identification unit 134 acquires multiple indicators based on the biosignals of the user (for example, subjects U02a and U02b), and estimates the user's emotions based on the changes in these indicators over time.

[0101] Furthermore, the emotion estimation model converts biological signals into emotion estimation indices based on index conversion information, and estimates the user's emotions based on psychological map information (for example, psychological plane table 122) that associates the change in coordinate state represented by multiple types of emotion estimation indices with the emotion information.

[0102] Since changes in biosignals (indicators based on biosignals) are differences between biosignals (indicators), errors due to individual differences and differences in the surrounding environment are canceled out, and the estimation of emotions based on changes in biosignals (indicators) can be made less affected by individual differences and differences in the surrounding environment.

[0103] Next, we will specifically explain the method for estimating emotions when using the psychological plane table 122 shown in Figure 3.

[0104] Here, the first index at time t ("activation of thought, recognition, and judgment") is x t (X coordinate value), the second indicator ("physical activity") at time t is y t Let (Y coordinate value) be used. In other words, the coordinate (x) on the psychological plane of the object at time t (the psychological plane corresponding to psychological plane table 122). t ,y t(Let it be the (XY plane coordinate value).

[0105] In this case, the coordinates at time t + 1 after time t are (x t+1 , y t+1 ). The specific part 134 represents the change amount from time t to time t + 1 in the form of displaying a vector with the end point coordinates of the vector in a state where the starting point of the vector is set to the origin (0, 0), and calculates the vector (x t+1 - x t , y t+1 -y t ).

[0106] Subsequently, the specific part 134 identifies which change direction (change pattern) of the "change pattern" in the mental plane table 122 of FIG. 3 is closest to the direction indicated by the vector. Specifically, the specific part 134 calculates the conversion formula from rectangular coordinates to polar coordinates for the direction from the origin of the coordinate position of the vector (x t+1 - x t , y t+1 [[ID=z8]]-y t ). Since the index value coordinate change direction is classified into 8 directions, the comparison angles for vector direction discrimination are 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° for the directions of right, upper right, up, upper left, left, lower left, down, and lower right, respectively.

[0107] The specific part 134 identifies the angle closest to the direction (angle) of the above-mentioned vector among the above angles, and sets the corresponding direction as the change direction of the index value coordinates. For example, if the vector is (1.2, 1.5), the angle in polar coordinates is about 51°, which is closest to 45° among the above angles. Therefore, the specific part 134 identifies the change direction of the index value coordinates as upper right.

[0108] Furthermore, the identification unit 134 matches the "change pattern" data of the psychological plane table 122 in the identified direction (for example, upper right) and estimates the "emotion type" data when the direction of change of the index value coordinate is upper right as the subject's emotion (for example, uplifting psyching direction (increase in joy)).

[0109] The provisioning unit 135 provides the emotional information estimated by the identification unit 134 to the terminal device 20.

[0110] The analyst U01 then uses the emotional information provided by the provision unit 135 by viewing it via the terminal device 20. Figure 8 shows an example of the evaluation screen and the usage status of the emotional information.

[0111] The information provider 135 provides a message in natural language along with an arrow indicating emotional change (using the corresponding "display symbol" data in the psychological judgment table 123), that is, information indicating the direction of coordinate change on the corresponding psychological plane. The evaluation screen 21 of the terminal device 20 displays the information provided by the information provider 135 for each subject. This information can be realized by adding message data to the psychological judgment table 123, or by installing a database in the terminal device 20 that associates emotional information and message information.

[0112] In the following explanation, "Person A" refers to subject U02a, and "Person B" refers to subject U02b.

[0113] In the example shown in Figure 8, the evaluation screen 21 displays an arrow pointing to the upper right, along with the message, "Person A is psyching up. (This is the correct direction.)" Analyst U01 can refer to the information displayed on the evaluation screen 21 and give appropriate advice to subject U02a, for example, "Person A, keep it up," depending on Person A's mental state (by sending advice information, etc., to Person A's game console or other terminal).

[0114] Furthermore, in the example shown in Figure 8, the evaluation screen 21 displays an arrow pointing to the upper left, along with the message, "Person B is heading towards depression (the parasympathetic nervous system is dominant)." Analyst U01 can refer to the information displayed on the evaluation screen 21 and give appropriate advice to subject U02b, for example, "Person B, please be mindful of activating your sympathetic nervous system," according to Person B's mental state (by sending the advice information to Person A's game console or other terminal).

[0115] Next, we will explain an example of using estimation system 1 in esports. Subjects U01a and U02b are esports players. Here, in esports, it is desirable for players to consciously induce emotional changes in the upper right and lower left directions on the relevant psychological plane, and to avoid causing emotional changes in the upper left and lower right directions as much as possible. The type of emotional change that is desirable will differ depending on the title of the game being played, the scene, etc. Furthermore, the type of emotional change that is desirable will also differ depending on the usage mode other than gaming. Therefore, it is preferable to prepare a database that associates emotional information and message information according to the usage mode of estimation system 1, and to select and use this database according to the usage mode so that messages appropriate to the usage mode are presented.

[0116] In the example shown in Figure 9, the evaluation screen 21 displays the message "Person A is in the direction of relaxation (the correct direction)" along with an arrow pointing in the lower left corner. Analyst U01 can refer to the information displayed on the evaluation screen 21 and give appropriate advice to subject U02a, such as "Person A is learning well," based on Person A's mental state.

[0117] In addition, in the example shown in Figure 9, the evaluation screen 21 displays an arrow pointing downwards to the right, along with the message, "Person B is experiencing anxiety. (The sympathetic nervous system is dominant.)" Analyst U01 can refer to the information displayed on the evaluation screen 21 and give appropriate advice to subject U02b, such as, "Person B, your brain is aroused well. Let's work on controlling your autonomic nervous system," according to Person B's mental state.

[0118] In this way, it becomes possible to provide appropriate advice to players based on the estimated emotional state.

[0119] Furthermore, the emotion estimation index can be said to consist of a first index representing the degree of brain arousal (for example, the index corresponding to the vertical axis in Figure 3) and a second index representing the degree of activity of the user's autonomic nervous system (for example, the index corresponding to the horizontal axis in Figure 3).

[0120] (Calibration of emotion estimation model) The emotion estimation method described above captures changes in emotion (for example, a shift towards depression, or an increase in the tendency towards depression), that is, it captures relative emotions. However, it can also be applied to calibration of absolute emotions (for example, the level of depression). In this case, when server 10 becomes capable of calibration using multiple indicators based on the collected user's biosignals, it estimates the emotion based on the values ​​of the multiple indicators themselves.

[0121] Figure 10 illustrates the calibration method for the emotion estimation model. The sequence of arrows in Figure 10 represents the changes in the coordinate positions of the biological state indicators identified by the identification unit 134 at each time point, indicated by the arrows (showing the start and end points of the vectors).

[0122] Furthermore, the vertical and horizontal axes shown by dashed lines on the left side of Figure 10 represent the provisional positions of each indicator axis in the psychological plane of the emotion estimation model.

[0123] The identification unit 134 calculates the average of the coordinate values ​​for each of the two types of biological state indices, that is, the starting point (or ending point) of each arrow (after collecting a predetermined number of coordinate values ​​for which accuracy can be expected). Then, it calculates index correction values ​​for the two types of biological states so that the average coordinate value becomes the origin on the psychological plane (the biological state indices are corrected and the vertical and horizontal axes, which are represented by dashed lines on the left side of Figure 10, are relatively shifted to the vertical and horizontal axes, which are represented by solid lines on the right side of Figure 10). Then, the identification unit 134 corrects the two types of biological state index values ​​using these correction values ​​and applies these corrected index values ​​to an emotion estimation model that estimates absolute emotions to estimate emotions. In other words, statistical processing is applied to many biological state index values, and calibration is performed using these statistically processed values.

[0124] This cancels out or suppresses errors in biosignals caused by individual differences and the surrounding environment, thereby improving the accuracy of emotion estimation in emotion estimation models that estimate absolute emotions.

[0125] In the example above, calibration was performed using the average value of each coordinate value in the two types of biological state indicators. However, a correction can also be applied in which the centroid (the centroid of the rectangle on the psychological plane containing each coordinate value) is the origin of the region determined by the maximum and minimum values ​​of each coordinate value in the two types of biological state indicators (a correction that adjusts the size of the rectangle is also effective).

[0126] Furthermore, since this calibration process takes time to collect the necessary data, it is effective in terms of responsiveness to initially estimate emotions based on changes in biological state indicators, and then, after a sufficient amount of data has been collected, perform emotion estimation using an emotion estimation model that estimates absolute emotions (a combination of both is also possible).

[0127] (A machine learning model that learns patterns of emotional change) The estimation of emotional state by the identification unit 134 can also be performed using artificial intelligence. In this case, the identification unit 134 controls the learning of its built-in artificial intelligence based on operations that instruct the operator to learn and learning data provided by the operator. The identification unit 134 applies the input (acquired) biosignal to a trained machine learning model to estimate the emotional state, and the estimation result is provided to the terminal device 20 by the providing unit 135. In this case, the emotion estimation model is a machine learning model that has learned the correspondence between patterns of change in multiple indicators over time and emotions.

[0128] Figure 11 illustrates the learning method for a machine learning model. As shown in Figure 11, the specific unit 134 inputs data on changes in measured biological signals (difference data of biological signals measured at predetermined time intervals) and data on the emotional state at the time of measurement (emotional change: determined by analysis of questionnaires, facial expressions, etc.) as learning data (LNIN), along with a score (which represents the likelihood of an emotional state occurring in relation to the biological signal change data, and is set by, for example, the data creator). Through this learning process, a machine learning model is generated in which the emotional state is estimated and output (FEOUT) based on the input of biological signal data (LDIN).

[0129] Thus, in this example, the service provider 135 estimates the user's emotions using a machine learning model that has learned the correspondence between patterns of change in multiple indicators over time and emotions.

[0130] Next, Figures 12 and 13 will be used to explain the processing flow performed by the server 10 (control unit 13). Figure 12 is a flowchart showing the process of generating an emotion estimation model based on external medical evidence (papers, etc.), that is, the model generation process that generates the sensor table 121, the psychological plane table 122, and the psychological judgment table 123. The control unit 13 executes this process when an operator initiates the model generation process.

[0131] In step S101, the control unit 13 (extraction unit 131) of the server 10 collects medical evidence (papers, etc.) from medical information databases, internet information, etc., and then proceeds to step S102.

[0132] In step S102, the control unit 13 (extraction unit 131) performs linguistic analysis of the collected medical evidence (papers, etc.) and extracts sensor-related information (sensor identification data, sensor type, output biosignal type, etc.) related to various sensors, data on the relationship between biosignals and biostatus indicators, and data on the relationship between biostatus indicators and emotions (especially the relationship between changes in biostatus indicator values ​​and changes in emotions). It then performs processing such as stratification to create table data and proceeds to step S103.

[0133] In step S103, the control unit 13 (generation unit 133) generates a sensor table 121 based on sensor-related information and data relating to biological signals and biological state indicators, and then proceeds to step S104.

[0134] In step S104, the control unit 13 (generation unit 133) generates an emotion estimation model (psychological plane) based on data relating to two types of biological state indicators and emotions, and similarly generates an emotion estimation model for each combination of biological state indicators. By arranging the emotion estimation models in a matrix table of biological state indicators, the control unit 13 (generation unit 133) generates a psychological plane table 122 and completes the process. The emotion estimation model (psychological plane) is converted into table data to become a psychological judgment table 123.

[0135] Figure 13 is a flowchart showing the emotion estimation process for estimating emotional states. The control unit 13 starts execution upon receiving an emotion estimation request signal from the terminal device 20 (due to a request operation by an operator, etc.) or upon receiving an activation signal for a system that utilizes estimated emotions, such as a game, and continues to execute until the system that utilizes estimated emotions stops (e.g., the end of the game).

[0136] In step S201, the control unit 13 (acquisition unit 136) of the server 10 receives data such as various biosignals from the terminal device 20 via the communication unit 11, stores the necessary data in its internal memory or storage unit 12 for subsequent processing, and then proceeds to step S202.

[0137] In step S202, the control unit 13 (model selection unit 137) selects a model to be used for emotion estimation based on the types of various biosignals (two types of biosignals in this example) included in the data acquired from the terminal device 20 in step S201, and then proceeds to step S203.

[0138] In step S203, the control unit 13 (acquisition unit 136) receives various biosignal data from the terminal device 20 via the communication unit 11, stores the necessary data in its internal memory or storage unit 12 for subsequent processing, and then proceeds to step S204.

[0139] In step S204, the control unit 13 (specification unit 134) converts the biosignals (two types of biosignals in this example) acquired from the terminal device 20 into biostate index values ​​using the emotion estimation model (sensor table 121) selected in step S202, and then proceeds to step S205.

[0140] In step S205, the control unit 13 (specification unit 134) determines whether the biological state index values ​​converted and generated in step S204 have reached a predetermined number (the number necessary for trajectory determination, for example, 10). If the predetermined number has been reached, the unit proceeds to step S206 (at which point the count of generated biological state index values ​​is reset). If the predetermined number has not been reached, the unit returns to step S203.

[0141] In step S205, the control unit 13 (specification unit 134) calculates the trajectory (change state) of coordinates based on a predetermined number of biological state index values ​​(in the case of two types of biological state index values, one axis of the psychological plane becomes the vertical axis value and the other axis becomes the horizontal axis value), and then proceeds to step S206.

[0142] In step S206, the control unit 13 (specification unit 134) performs a matching process (recognition of the geometric pattern of the trajectory) with the psychological judgment table 123 based on the trajectory of the biological state index calculated in step S205, and proceeds to step S207 with the selected and extracted emotional information based on the matching result as the estimation result. These processes in steps S204, S205, and S206 constitute the application process to the emotion estimation model of the biological signal.

[0143] In step S207, the control unit 13 (providing unit 135) transmits (provides) the emotion information estimated in step S206 to the terminal device 20 via the communication unit 11, and completes the processing.

[0144] In this embodiment, the server 10 performs many processes such as emotion estimation and emotion estimation model generation. However, it is also possible to perform these processes by providing the above-described configuration on the terminal device 20, or by providing the configuration on the game console used by the game player. Furthermore, it is possible to distribute each process as appropriate among the communication-connected server 10, terminal device 20, and game console, for example, by having the terminal device 20 perform the process of converting biosignals into biostate index values.

[0145] The emotion estimation system comprises a game device on which the user plays a game, a server 10 that estimates emotions, and a terminal device 20 that manages the game. For example, the game device is the device operated by subjects U02a and U02b in Figure 1, and may be a PC or a so-called consumer console. Subjects U02a and U02b are examples of users.

[0146] The game device includes a game controller (e.g., a CPU). The game controller measures the user's biosignals using a biosensor, transmits the measured biosignals to a terminal device 20, receives estimated emotion-related information transmitted from the terminal device 20, and provides the received estimated emotion-related information to the user.

[0147] The terminal device 20 has a terminal controller (e.g., a CPU). The terminal controller receives the user's biometric signals from the game device, transmits the received user's biometric signals to the server 10, receives estimated emotion-related information transmitted from the server 10, and transmits the received estimated emotion-related information to the game device.

[0148] Server 10 has an emotion estimation controller (e.g., a CPU). The emotion estimation controller receives the user's biosignals from the terminal device 20, estimates the user's emotions based on the changes over time of multiple indicators based on the received user's biosignals, transmits estimated emotion-related information about the estimated emotions to the terminal device 20, obtains multiple indicators based on the user's biosignals measured by a biosensor, and estimates the user's emotions based on the changes over time of multiple indicators.

[0149] Furthermore, the terminal device 20 outputs information indicating the user's emotions as estimated by the server 10.

[0150] Furthermore, in this embodiment, biological signals were converted into emotion estimation indices and applied to the emotion estimation model. However, it is also possible to apply the biological signals themselves as emotion estimation indices to the emotion estimation model (a model that operates using biological signals is generated in the same manner as the model generation method for emotion estimation indices described above).

[0151] As described above, according to this embodiment, since the subject's emotional information is generated based on changes and fluctuations in the subject's biosignals, calibration of the emotional estimation model can be omitted. In other words, since changes in biosignals are difference components between biosignals, errors based on individual differences and differences in the surrounding environment are canceled out, so the estimation of emotions based on changes in biosignals can be made less affected by individual differences and differences in the surrounding environment. As a result, according to this embodiment, the accuracy of emotion estimation based on biosignals can be easily improved.

[0152] Furthermore, in this embodiment, biological signals are converted into an emotion estimation index, and this emotion estimation index is applied to an emotion estimation model for the emotion estimation index to estimate emotions. There are many types of sensors that measure biological signals, and it is difficult to prepare a complex emotion estimation model for each of these sensors. However, converting biological signals into an emotion estimation index is relatively simple and easy to implement, mainly by matching the output characteristics of the sensors. Therefore, by inserting a conversion process that normalizes the biological signals of each sensor, as in this embodiment, the number of emotion estimation models can be reduced, which is advantageous in terms of system size.

[0153] Further effects and modifications can be readily derived by those skilled in the art. Therefore, broader aspects of the present invention are not limited to the specific details and representative embodiments expressed and described above. Accordingly, various modifications are possible without departing from the spirit or scope of the overall concept of the invention as defined by the appended claims and equivalents. [Explanation of symbols]

[0154] N Network U01 Analyst U02 Subject 1. Estimation System 10 servers 11 Communications Department 12 Storage section 13 Control Unit 20 Terminal devices 21. Evaluation screen 31a, 31b, 32a, 32b sensors 121 Sensor Table 122 Psychological Planar Table 123 Psychological Judgment Table 131 Extraction part 132 Psychological Analysis Department 133 Generation part 134 Specific part 135 Provision Department 136 Acquisition Department 137 Model Selection Section 210, 220, 230, 240 fields

Claims

1. An emotion estimation device that estimates emotions, having a controller, The aforementioned controller, The user's first and second biosignals are acquired. Based on the acquired first biosignal, the first index value is calculated. Based on the acquired second biosignal, the second index value is calculated. Until a predetermined number of coordinate values ​​composed of the first and second indicator values ​​at the same time are calculated, the user's emotions are estimated based on the pattern of change of the coordinate values. After a predetermined number of coordinate values ​​have been calculated, the first and second correction values ​​are calculated using the average values ​​of the first and second index values ​​at each calculated time point. The first corrected index value is calculated by subtracting the first corrected index value from the first index value. The second corrected index value is calculated by subtracting the second corrected index value from the second index value. The user's emotions are estimated based on the first and second correction index values. Emotion estimation device.

2. A method for estimating emotions, The user's first and second biosignals are acquired. Based on the acquired first biosignal, the first index value is calculated. Based on the acquired second biosignal, the second index value is calculated. Until a predetermined number of coordinate values ​​composed of the first and second indicator values ​​at the same time are calculated, the user's emotions are estimated based on the pattern of change of the coordinate values. After a predetermined number of coordinate values ​​have been calculated, the first and second correction values ​​are calculated using the average values ​​of the first and second index values ​​at each calculated time point. The first corrected index value is calculated by subtracting the first corrected index value from the first index value. The second corrected index value is calculated by subtracting the second corrected index value from the second index value. The user's emotions are estimated based on the first and second correction index values. The emotion estimation method that the controller uses.

3. The user's first and second biosignals are acquired. Based on the acquired first biosignal, a first index value is calculated. Based on the acquired second biosignal, the second index value is calculated. Until a predetermined number of coordinate values ​​composed of the first and second indicator values ​​at the same time are calculated, the user's emotions are estimated based on the pattern of change of the coordinate values. After a predetermined number of coordinate values ​​have been calculated, the first and second correction values ​​are calculated using the average values ​​of the first and second index values ​​at each calculated time point. The first corrected index value is calculated by subtracting the first corrected index value from the first index value. The second corrected index value is calculated by subtracting the second corrected index value from the second index value. The user's emotions are estimated based on the first and second correction index values. An emotion estimation program that uses a computer to perform processing.

4. An emotion estimation system comprising an emotion estimation device for estimating emotions and a terminal device, The terminal device has a terminal controller The aforementioned terminal controller is The biosensor receives the user's biosignals, The received user biosignals are transmitted to the emotion estimation device. The estimated emotion-related information transmitted from the emotion estimation device is received. The emotion estimation device has an emotion estimation controller, The aforementioned emotion estimation controller, The terminal device acquires the user's first biosignal and second biosignal. Based on the acquired first biosignal, a first index value is calculated. Based on the acquired second biosignal, the second index value is calculated. Until a predetermined number of coordinate values ​​composed of the first and second indicator values ​​at the same time are calculated, the user's emotions are estimated based on the pattern of change of the coordinate values. After a predetermined number of coordinate values ​​have been calculated, the first and second correction values ​​are calculated using the average values ​​of the first and second index values ​​at each calculated time point. The first corrected index value is calculated by subtracting the first corrected index value from the first index value. The second corrected index value is calculated by subtracting the second corrected index value from the second index value. Based on the first and second correction index values, the user's emotions are estimated. The estimated emotion-related information concerning the estimated emotion is transmitted to the terminal device. The aforementioned terminal device is The emotion estimation device outputs information indicating the user's emotions, which it has estimated. Emotion estimation system.

5. An emotion estimation system comprising a game device in which a user plays a game, an emotion estimation device for estimating emotions, and a terminal device for managing the game, The game device has a game machine controller. The aforementioned game console controller is By using a biosensor, the user's biosignals are measured. The measured user's biosignals are transmitted to the terminal device. The terminal device receives the estimated emotion-related information transmitted from the terminal device. The system provides the user with the received estimated emotion-related information. The terminal device has a terminal controller The aforementioned terminal controller is The game device receives the user's biosignals, The received user biosignals are transmitted to the emotion estimation device. The estimated emotion-related information transmitted from the emotion estimation device is received. The received estimated emotion-related information is transmitted to the game device. The emotion estimation device has an emotion estimation controller, The aforementioned emotion estimation controller, The terminal device acquires the user's first biosignal and second biosignal. Based on the acquired first biosignal, a first index value is calculated. Based on the acquired second biosignal, the second index value is calculated. Until a predetermined number of coordinate values ​​composed of the first and second indicator values ​​at the same time are calculated, the user's emotions are estimated based on the pattern of change of the coordinate values. After a predetermined number of coordinate values ​​have been calculated, the first and second correction values ​​are calculated using the average values ​​of the first and second index values ​​at each calculated time point. The first corrected index value is calculated by subtracting the first corrected index value from the first index value. The second corrected index value is calculated by subtracting the second corrected index value from the second index value. Based on the first and second correction index values, the user's emotions are estimated. The estimated emotion-related information concerning the estimated emotion is transmitted to the terminal device. The aforementioned terminal device is The emotion estimation device outputs information indicating the user's emotions, which it has estimated. Emotion estimation system.

Images