Emotion estimation device, method, and program

Abstract

A presetting processing unit of an emotion estimation device: acquires, in response to a report by a user himself / herself, first variation graph information that represents variations in emotion that the user subjectively feels about an event that causes an emotional change; acquires a plurality of pieces of second variation graph information respectively obtained by objectively measuring, by means of a plurality of sensing techniques, biological reactions of the user for the event; compares the features of the respective variations between the first variation graph information and the plurality of pieces of second variation graph information; selects sensing techniques corresponding to pieces of second variation graph information in which matching degrees between the features of the variations satisfy the preset condition; and associates the selected sensing techniques with at least information representing the event and stores the associated result in a storage medium.

Description

[Technical Field]

[0001] One aspect of the present invention relates to an emotion estimation device, a method, and a program used to estimate a person's emotion. [Background technology]

[0002] Emotion estimation methods using biological information have been proposed as a method for objectively estimating human emotions. For example, Non-Patent Document 1 describes a method for estimating human emotions using an electrocardiogram, an electromyogram, and an acceleration sensor. Also, Non-Patent Document 2 describes an emotion estimation method using electroencephalograms and pulse waves. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "Predicting Listener Emotions toward Speakers during Group Discussions Using Multimodal Sensing," Genki Sakai, 34th Annual Conference of the Japanese Society for Artificial Intelligence, Session ID: 4E3-OS-19b-04, 2020 [Non-patent document 2] "Study on emotion estimation method using biological information," Takahashi et al., Proceedings of the Embedded Systems Symposium 2018, pp.106-107, 2018 Summary of the Invention [Problem to be solved by the invention]

[0004] Both of the cited documents 1 and 2 use fixed sensors prepared in advance and estimate emotions based on the relationship between the sensing data and emotions. However, the way in which emotional characteristics appear in the sensing data varies depending on the type of emotion and on the individual. For this reason, simply applying the technologies described in the cited documents 1 and 2 does not always result in an appropriate estimation of emotions depending on the type of emotion to be estimated and the individual.

[0005] The present invention has been made in light of the above circumstances, and aims to provide a technique that enables emotions to be consistently estimated appropriately regardless of the type of emotion to be estimated or individual differences. [Means for solving the problem]

[0006] In order to solve the above problem, one aspect of the emotion estimation device or method according to the present invention is a preset processing unit or Pre-configuration In the processing step, first fluctuation graph information showing fluctuations in emotions subjectively felt by the user in response to an event that causes a change in emotions is provided to the user. The In response to the user's declaration, a plurality of pieces of second fluctuation graph information are obtained by objectively measuring the user's biological reactions to the same event using a plurality of sensing methods, and the first fluctuation graph information and the plurality of pieces of second fluctuation graph information are compared in terms of the shape of the curve representing the characteristics of the fluctuation, and the sensing method corresponding to the second fluctuation graph information whose degree of coincidence of the curve shape satisfies a preset condition is selected, and the selected sensing method is stored in a storage medium in association with at least information representing the event and identification information of the user. In addition, in the emotion estimation processing unit or emotion estimation processing step, When information representing the event and identification information of the user to be estimated are input, the sensing method corresponding to each input information is selected. Selecting the sensing technique from the storage medium and acquiring third variation graph information corresponding to the user's biological reaction measured using the sensing method, and comparing the shape of the curve of the acquired third variation graph information with the shape of the curve of the second variation graph information corresponding to the sensing method. and calculates the degree of match, and determines whether the calculated degree of match is equal to or greater than a preset threshold value. By doing so, the emotion to be estimated is Whether or not The above is a method for determining whether the

[0007] According to one aspect of the present invention, in the pre-setting process for emotion estimation, first fluctuation graph information representing the fluctuations in emotions subjectively felt by a user in response to a certain event is compared with second fluctuation graph information simultaneously measured objectively using multiple sensing methods for the event, and from among the multiple sensing methods, the sensing method that measured the fluctuation graph that has a high degree of match with the user's subjective emotional fluctuations is selected and stored in association with the event. Then, the occurrence of the emotion to be estimated is determined by comparing the shape of the curve of the third fluctuation graph information corresponding to the user's biological reaction measured using the above sensing method with the shape of the curve of the second fluctuation graph information corresponding to the sensing method.

[0008] Therefore, for example, for each user and each type of emotion, a sensing method capable of measuring fluctuation characteristics that are closest to the subjective emotional fluctuations of the user is selected and saved in advance, making it possible to estimate the user's emotion using a sensing method that is optimal for the user and the type of emotion to be estimated. [Effects of the Invention]

[0009] That is, according to one aspect of the present invention, it is possible to provide a technology that enables emotions to be consistently estimated appropriately regardless of the type of emotion to be estimated or individual differences. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a system including a feeling estimation device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of a hardware configuration of the emotion estimation device shown in FIG. [Figure 3] FIG. 3 is a block diagram showing an example of the software configuration of the emotion estimation device shown in FIG. [Figure 4] FIG. 4 is a flowchart showing an example of the processing procedure and processing content of the presetting processing executed by the control unit of the feeling estimation device shown in FIG. [Figure 5] FIG. 5 is a flowchart showing an example of the procedure and content of the estimation process executed by the control unit of the feeling estimation device shown in FIG. [Figure 6A]FIG. 6A is a diagram showing a first example of fluctuation graph information representing fluctuations in emotions felt by a user. [Figure 6B] FIG. 6B is a diagram showing a second example of fluctuation graph information representing fluctuations in emotions felt by the user. [Figure 6C] FIG. 6C is a diagram showing a third example of fluctuation graph information representing fluctuations in emotions felt by the user. [Figure 7] FIG. 7 is a diagram showing an example of sensing data measured by a heart rate sensor. [Figure 8] FIG. 8 is a diagram showing an example of sensing data measured by the pulse sensor. [Figure 9] FIG. 9 is a diagram showing an example of sensing data measured by a body temperature sensor. [Figure 10] FIG. 10 is a diagram illustrating an example of the setting list data. [Figure 11] FIG. 11 is a diagram for explaining the process of estimating an emotion from sensing data measured by a pulse sensor during emotion estimation. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] [One embodiment] (Configuration example) (1) System FIG. 1 is a diagram showing an example of the configuration of an emotion estimation system according to an embodiment of the present invention. A feeling deduction system according to one embodiment includes a feeling deduction device EM, and enables information transmission via a network NW between this feeling deduction device EM and user terminals UT1 to UTn used by multiple users to be estimated.

[0013] The user terminals UT1 to UTn are configured by, for example, tablet terminals, personal computers, or smartphones, and transmit and receive various information related to the emotion estimation processing to and from the emotion estimation device EM using communication tools such as a browser or mailer.

[0014] According to an embodiment of the present invention, the user terminals UT1 to UTn have, when a user draws and inputs fluctuation graph information representing fluctuations in the user's own subjective emotions, a function to transmit the input fluctuation graph information to the emotion deduction device EM, and a function to receive and display information representing emotion estimation results transmitted from the emotion deduction device EM. The user terminals UT1 to UTn also have a function to receive biological sensing data transmitted from the wearable terminals WT1 to WTn worn by the users and transmit the received sensing data to the emotion deduction device EM.

[0015] The wearable devices WT1 to WTn each include a plurality of biosensors for measuring the user's bioreactions. Examples of biosensors include sensors that measure heart rate, pulse waves, brain waves, sweating, and gaze, but any type of sensor may be used as long as it can continuously or discretely measure changes in bioreactions.

[0016] The wearable terminals WT1 to WTn transmit sensing data of biological reactions measured by the plurality of biological sensors via a wireless interface for short-distance data transmission such as Bluetooth (registered trademark). The sensed data is transmitted to the user UT using the interface. If the wearable terminals WT1 to WTn themselves have a function to access the Internet using WiFi (registered trademark), a mobile communication network, or the like, the sensed data may be transmitted directly from the wearable terminals WT1 to WTn to the feeling estimation device EM.

[0017] The network NW comprises, for example, a wide area network with the Internet at its core, and an access network for accessing this wide area network. Examples of the access network include a public communication network using wired or wireless connections, a local area network (LAN) using wired or wireless connections, and a cable television (CATV) network. When the business support system is operated within a company or office, for example, the network NW is configured as an in-house network such as a LAN or wireless LAN.

[0018] (2) Emotion estimation device EM 2 and 3 are block diagrams showing an example of a hardware configuration and a software configuration, respectively, of the emotion estimation device EM.

[0019] The emotion estimation device EM is a hardware component such as a central processing unit (CPU). The system is equipped with a control unit 1 that uses a hardware processor. A storage unit having a program storage unit 2 and a data storage unit 3, and a communication interface (hereinafter, the interface will be abbreviated as I / F) unit 4 are connected to the control unit 1 via a bus 5.

[0020] Under the control of the control unit 1, the communication I / F unit 4 transmits information to and receives information from the user terminals UT1 to UTn using a communication protocol defined by the network NW.

[0021] The program storage unit 2 may be, for example, a storage medium such as an SSD (Solid State Drive). It is configured by combining a non-volatile memory that can be written and read at any time with a non-volatile memory such as a ROM (Read Only Memory), and stores middleware such as an OS (Operating System), as well as application programs required to execute various control processes according to one embodiment. Hereinafter, the OS and each application program will be collectively referred to as the "program."

[0022] The data storage unit 3 is, for example, a combination of a non-volatile memory such as an SSD that can be written to and read from at any time as a storage medium, and a volatile memory such as a RAM (Random Access Memory), and is equipped with a sensing data storage unit 31, a subjective emotion data storage unit 32, a setting list data storage unit 33, and a sensing data storage unit 34 as main storage units required to implement one embodiment of the present invention.

[0023] In the preset mode, the sensing data storage unit 31 stores sensing data of a plurality of biosensors transmitted from the wearable terminals WT1 to WTn or the user terminals UT1 to UTn.

[0024] In the preset mode, when fluctuation graph information drawn and input by the user himself is transmitted from the user terminals UT1 to UTn, the subjective emotion data storage section 32 stores this as subjective emotion data.

[0025] In the pre-setting mode, the setting list data storage unit 33 stores the setting list data generated by the control unit 1. The setting list data defines the sensing method used for the user emotion estimation process in the emotion estimation mode, and an example of this will be described in the operation example.

[0026] In the emotion estimation mode, the sensing data storage unit 34 stores sensing data measured by a biosensor, among the multiple biosensors of the wearable devices WT1 to WTn, that corresponds to the sensing method specified based on the setting list data.

[0027] The control unit 1 includes, as processing functions used to implement one embodiment of the present invention, a preset processing unit 10 that handles processing in a preset mode and an emotion estimation processing unit 20 that handles processing in an actual emotion estimation mode.

[0028] First, the presetting processing unit 10 includes a sensing data acquisition processing unit 11, a subjective emotion data acquisition processing unit 12, a formulation processing unit 13, and a sensing method selection and setting processing unit 14.

[0029] In the preset mode, the sensing data acquisition processing unit 11 causes the multiple biosensors included in the wearable terminals WT1 to WTn to measure the user's biological reactions to the presentation of an event (also called a trigger) that induces the generation of emotions.The sensing data acquisition processing unit 11 then receives multiple pieces of sensing data obtained by this measurement from the user terminals UT1 to UTn or the wearable terminals WT1 to WTn via the communication I / F unit 4.The sensing data acquisition processing unit 11 then stores the received multiple pieces of sensing data in the sensing data storage unit 31 in a state where each piece is associated with the sensor ID of the measurement source and the user ID of the measurement target.

[0030] In the preset mode, when a user draws and inputs graph information showing fluctuations in the emotions felt by the user in response to the presentation of the same trigger using the user terminal UT1-UTn, the subjective emotion data acquisition processing unit 12 acquires this fluctuation graph information from the user terminal UT1-UTn via the communication I / F 4. The subjective emotion data acquisition processing unit 12 then associates the acquired fluctuation graph information with the user ID and event ID of the sender, and stores this in the subjective emotion data storage unit 32 as subjective emotion data.

[0031] The formulation processing unit 13 graphs and then formulates each of the plurality of sensing data stored in the sensing data storage unit 31, to generate fluctuation graph characteristics corresponding to each of the sensing data. The formulation processing unit 13 also formulates fluctuation graph information of the subjective emotion data stored in the subjective emotion data storage unit 32, to generate fluctuation graph characteristics corresponding to the fluctuation graph information.

[0032] The sensing method selection and setting processing unit 14 compares the fluctuation graph characteristics corresponding to the formulated subjective emotion data with a plurality of fluctuation graph characteristics corresponding to each of the formulated sensing data, selects the sensing method most suitable for emotion estimation based on the comparison result, and sets the selected sensing method in the setting list data.

[0033] An example of the process for formulating the fluctuation graph information and the process for selecting and setting the sensing method will be explained in the operation example.

[0034] On the other hand, the emotion estimation processing unit 20 includes an estimation request acquisition processing unit 21, a sensing data acquisition processing unit 22, an emotion estimation processing unit 23, and an estimated information output processing unit 24.

[0035] In the emotion estimation mode, when estimating the actual occurrence of a user's emotion, the estimation request acquisition processing unit 21 acquires an estimation request including an emotion label indicating the type of emotion to be estimated, an event ID, and a user ID from the user terminals UT1 to UTn via the communication I / F unit 4.

[0036] Based on the emotion label, event ID, and user ID included in the estimation request, the sensing data acquisition processing unit 22 reads a corresponding sensing method from the setting list data storage unit 33. Then, the sensing data acquisition processing unit 22 selectively receives sensing data from a biosensor corresponding to the sensing method, and stores the received sensing data in the sensing data storage unit 34 in association with the sensor ID of the measurement source and the user ID.

[0037] The emotion estimation processing unit 23 reads the received sensing data from the sensing data storage unit 34, graphs the sensing data, and then formulates it to generate fluctuation graph characteristics.Then, the generated fluctuation graph characteristics are compared with fluctuation graph characteristics at the time of emotion occurrence that are separately set as sensing data equations in the setting list data, thereby determining whether or not an emotion has occurred. Note that an example of the emotion determination process will also be described in the operation example.

[0038] The estimated information output processing unit 24 generates estimated result information including information representing the determination result of the occurrence of the emotion, and transmits the generated estimated result information from the communication I / F unit 4 to the corresponding user terminals UT1 to UTn.

[0039] Each of the processing units 11 to 14 and 21 to 24 of the control unit 1 is realized by causing a hardware processor of the control unit 1 to execute an application program stored in the program storage unit 2. In this case, some or all of the processing units 11 to 14 and 21 to 24 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0040] (Example of operation) Next, an example of the operation of the device configured as above will be described.

[0041] (1) Pre-set mode FIG. 4 is a flowchart showing an example of the processing procedure and processing content of the presetting processing executed by the control unit 1 of the feeling estimation device EM.

[0042] (1-1) Acquisition of sensing data When performing pre-setting, an event that induces the occurrence of the emotion to be estimated is presented to the user as a trigger, such as content such as a movie, physical media such as a picture book or a toy, or experiential content such as a game.

[0043] When the presentation of the trigger starts, a wearable device (e.g., WT1) worn by the user continuously measures the user's biological reactions using multiple built-in biological sensors. For example, heart rate, pulse wave, and body temperature are measured. Then, sensing data indicating the biological reactions measured by each of the biological sensors is transmitted from the wearable device WT1 to the feeling estimation device EM via the user device UT1 or directly from the wearable device WT.

[0044] In addition, in measuring the above-mentioned biological reactions, in addition to operating multiple biological sensors simultaneously as described above to obtain multiple types of sensing data, it is also possible to obtain multiple types of sensing data by repeatedly using one biological sensor multiple times.

[0045] In response to this, the control unit 1 of the feeling estimation device EM monitors a pre-setting request transmitted from the user terminal UT1 when starting pre-setting in Step S10. Note that the pre-setting request includes an emotion label indicating the type of emotion to be pre-set, an event ID, and a user ID.

[0046] When the presetting request is received, in step S11, the control unit 1 of the feeling estimation device EM, under the control of the sensing data acquisition processing unit 11, receives each piece of sensing data transmitted from the user terminal UT1 or the wearable terminal WT1 via the communication I / F unit 4. Then, the control unit 1 stores each piece of received sensing data in the sensing data storage unit 31 together with the sensor ID of the measurement source and the user ID of the measurement target.

[0047] (1-2) Acquisition of subjective emotion data When an emotion occurs during the measurement of sensing data in response to the presentation of the trigger, the user draws and inputs the fluctuation of the emotion as a graph, for example, using the graphic input function of the user terminal UT1. When drawing this fluctuation graph, the user expresses the degree of the emotion in relation to the maximum value (max) of the emotion that the user can feel, which is 1, in terms of the height of the peak of the graph. do.

[0048] In addition, the user inputs, into the user terminal UT1, information indicating the trigger, such as information representing a scene of the content being viewed (e.g., an image capture), and further inputs the type of emotion that occurred in the scene, as well as the start time (start) and end time (end) of the emotion.

[0049] 6A to 6C are graphs showing an example of changes in the emotion (for example, "tension") felt by a user when a certain scene of movie content is presented to the user as a trigger.

[0050] For example, Figure 6A shows a pattern in which tension gradually increases and then subsides when the climax scene of a movie is presented as a trigger. Figure 6B shows a pattern in which tension continues for a long time in scenes in which the user's favorite actor plays a main role when a scene featuring the user's favorite actor is presented as a trigger. Figure 6C shows a pattern in which tension peaks momentarily in a scene in which the user is suddenly attacked when a shocking scene from a horror movie is presented.

[0051] When inputting the fluctuation graphs and the like, for example, a manual describing input procedures and the like may be downloaded from the feeling estimation device EM to the user terminal UT1 so that the user can input the data by referring to the manual, or an explainer may use an online service to explain the input procedures and the like to the user by voice, chat, etc. Also, the user may draw and input a graph using a paint function provided on a personal computer, or the user may handwrite something on paper and input it into the user terminal UT1 by reading it using OCR.

[0052] (1-3) Formulation of fluctuation graph When the process of acquiring the sensing data and subjective emotion data is completed, control unit 1 of emotion estimation device EM then performs a formulation process on the subjective emotion data and sensing data under the control of formulation processing unit 13 as follows.

[0053] That is, first, in step S13, the formulation processing unit 13 reads out a fluctuation graph from the subjective emotion data storage unit 32. Then, the read fluctuation graph is formulated. For example, in the example of FIG. 6A, the degree of tension peaks from the time "start" when the emotion (tension) starts to appear. The curve of the above fluctuation graph is formulated by applying the equation f(x) to the parameters t, the time it takes to reach the peak, h, the duration of tension, and P, the peak value of tension.

[0054] f(x) is, for example,

number

[0055] Next, in step S14, the formulation processing unit 13 sequentially reads out the plurality of pieces of sensing data stored in the sensing data storage unit 31, and first graphs each piece of the read sensing data. For example, a graphing method may be used in which recorded values ​​per second are plotted using a spreadsheet application, and then polynomial approximation is performed. This graphing method is described in detail in, for example, the following reference: [References] https: / / www.officepro.jp / excelgraph / scatter_plot / index5.html.

[0056] When graphing, the vertical scales of the multiple sensing data are different, so the vertical scales are unified in advance. For example, the maximum value of the user's past observations of the sensing data is set to "10" and the minimum value is set to "0."

[0057] The formulation processing unit 13 then performs formulation processing on each of the graphed fluctuation graphs of the plurality of pieces of sensing data. In this case, the formulation processing unit 13 extracts, for each piece of sensing data, a target time range from the start time to the end time of the emotion occurrence from the entire period of the fluctuation graph, and performs formulation processing on the extracted time range.

[0058] For example, assume that the heart rate data shown in Figure 7, pulse rate data shown in Figure 8, and body temperature data shown in Figure 9 have been acquired as sensing data. In this case, the formulation processing unit 13 uses the emotion onset start time (start) and end time (end) specified by the user during the subjective emotion data input process to extract time ranges corresponding to the onset start time (start) and end time (end) from the graphs of each piece of data. The extracted graphs are then formulated. The formulation method is the same as that used for formulating the fluctuation graph of subjective emotion data.

[0059] (1-4) Selection of sensing method Next, the control unit 1 of the feeling estimation device EM performs sensing method selection processing as follows under the control of the sensing method selection / setting processing unit 14.

[0060] That is, in step S15, the sensing method selection and setting processing unit 14 first compares the fluctuation graph of the subjective emotion data with each of the fluctuation graphs of the plurality of sensing data, and calculates the degree of match of the mountain shapes of the graphs. Then, in step S16, the sensing method selection and setting processing unit 14 selects, from the plurality of sensing data, the sensing data whose mountain shapes match at a level equal to or greater than a preset threshold and have the highest degree of match, and selects the biosensor corresponding to this sensing data as the optimal sensing method.

[0061] As a method for calculating the degree of similarity of the shape of the peaks of the graph, for example, a method is used in which the curvature is calculated as the degree of curvature of the peak of the formulated curve, and the absolute value of the difference in the calculated curvatures is used as the similarity of the shapes, i.e., the degree of similarity.

[0062] The formula for calculating the curvature is, for example,

number

[0063] Note that all or some of the sensing methods whose matching degree is equal to or greater than a threshold may be selected. In this case, the selected sensing methods may be prioritized in descending order of matching degree.

[0064] (1-5) Creating and storing setting list data When the sensing method selection and setting processing unit 14 finishes the above-mentioned sensing method selection processing, it subsequently generates setting list data as follows in step S17 and stores it in the setting list data storage unit 33.

[0065] An example of the setting list data is shown in Figure 10. As shown in the figure, the setting list data associates each "trigger information" with a "user ID," "emotion," "sensing method," and "sensing data equation."

[0066] In this example, the "trigger information" is divided into major categories and minor categories, with the major categories storing content categories such as movies, live music, comedy, etc., and the minor categories storing information representing specific scenes in the content. For example, image capture is used as information for identifying a scene. Note that the classification of trigger information is not limited to two categories, major and minor, and may be further divided into three or more categories.

[0067] It is also possible to obtain each user's hobby and preference characteristics from search history and life log data (for example, location information such as visited stores and purchase history), and prepare triggers that are more likely to cause emotional fluctuations tailored to the user based on the obtained life log data. For example, if user "0001" has a history of often watching movies starring actor "A," content related to actor "A" can be prioritized for measurement, making it possible to select and configure sensing methods based on data that more accurately reflects the user's emotions.

[0068] The "user ID" is linked to each individual user, and a clustered ID may be created as the user ID based on attributes, etc. In this case, for example, the most frequent item of the optimal sensing method for a group of users corresponding to an attribute such as "Japanese teenage females" may be used, or a new optimal sensing method may be set by comparing the average value of the drawing data of the subjective emotion data with the average value of the sensing data.

[0069] "Emotion" is stored as any emotional expression entered by each user or a label that defines it. In this case, all entered emotional expressions may be recorded as they are, or variations in expression may be absorbed by unifying "tension" and "excitement" into "tension."

[0070] In the "sensing method", information indicating the sensing method selected in the sensing method selection process, such as the name of the biosensor or the sensor ID, is stored as is.

[0071] The "sensing data equation" stores the formulation equation of the fluctuation graph used in the formulation process as is.

[0072] (2) Emotion estimation mode (2-1) Specifying and obtaining estimation requirements When trying to estimate the emotion occurrence state of a certain user in a certain scene of a movie, the user or an administrator inputs an estimation request to the emotion estimation device EM on their terminal. The estimation request includes, for example, trigger information, an emotion label indicating the type of emotion to be estimated, and a user ID.

[0073] Prior to inputting the estimation request, setting list data may be downloaded from the emotion estimation device EM to the terminal, and the user or administrator may input the estimation request by referring to the downloaded setting list data. The estimation request may be input by manual input of text data or by voice input. Furthermore, the emotion estimation device EM compares the input words with words in the setting list data, presents candidates that are a perfect match or a partial match to the user or administrator, and selects the content of the estimation request from the presented candidates. The administrator may specify this.

[0074] In response to this, when the control unit 1 of the feeling estimation device EM receives the estimation request in Step S20, under the control of the estimation request acquisition processing unit 21, in Step S21, it acquires each piece of information included in the estimation request.

[0075] (2-2) Specifying the sensing method and acquiring sensing data When the estimation request is acquired, the control unit 1 of the feeling estimation device EM, under the control of the sensing data acquisition processing unit 22, first searches the setting list data based on the trigger information, emotion label, and user ID included in the estimation request in step S22, and selects an appropriate sensing method.

[0076] Next, in step S23, the sensing data acquisition processing unit 22 specifies the appropriate sensing method for, for example, the corresponding user terminal UT1 or the wearable terminal WT1. In response, the wearable terminal WT1 drives a biosensor corresponding to the specified sensing method, measures the user's bioreaction with the biosensor, and transmits the sensing data to the feeling estimation device EM via the user terminal UT1 or directly from the wearable terminal WT1.

[0077] In step S23, under the control of the sensing data acquisition processing unit 22, the control unit 1 of the feeling estimation device EM receives the sensing data via the communication I / F unit 4 and stores the received sensing data in the sensing data storage unit 34 in association with the sensor ID and user ID of the sender.

[0078] If multiple appropriate sensing methods are registered in the setting list data, all of these sensing methods may be specified for the wearable device WT1, or if priorities are assigned, the sensing methods may be specified sequentially in accordance with the priorities. On the other hand, if an optimal sensing method is not registered, the emotion estimation process may be terminated, or an alternative sensing method may be prepared in advance and the sensing operation may be performed by specifying this alternative sensing method. As an alternative sensing method, for example, a sensing method that is often considered optimal for the average tendency of all users or a sensing method that is often considered optimal for the average tendency of the user in question may be used.

[0079] When acquiring sensing data, the feeling estimation device EM may selectively receive and store only the sensing data corresponding to the selected sensing method from among the multiple pieces of sensing data transmitted from the wearable device WT1, without specifying the sensing method to be measured in the wearable device WT1.

[0080] (2-3) Graphing and formulating sensing data When reception of sensing data starts, the control unit 1 of the feeling estimation device EM, under the control of the feeling estimation processing unit 23, first extracts fluctuation data in each interval of a predetermined extraction window (e.g., 60 seconds) from the received sensing data in step S24, while moving the extraction window in steps, for example, at 1-second intervals. An example is shown in Fig. 11.

[0081] The emotion estimation processor 23 then graphs the extracted data for each of the above sections and formulates the graphed data. The formulation method used by the formulation processor 13 in the preset mode is applied as is.

[0082] (2-4) Determining the occurrence of emotions Next, in step S25, the emotion estimation processing unit 23 calculates a curvature indicating the degree of curvature of the mountain at the peak of the rise of the formulated curve of the sensing data for each of the above sections, in the same way as the method previously performed by the sensing method selection and setting processing unit 14 in the pre-setting mode.

[0083] In the example of FIG. 11, for each of section 1 (0 to 60 seconds), section 2 (1 to 61 seconds), and section 3 (2 to 62 seconds), f1(x) = a1x 2 +b1x+c1→curvature=R1 f2(x) = a2x 2 +b2x+c2→curvature=R2 f3(x) = a3x 2 +b3x+c3→curvature=R3 Calculate.

[0084] Then, for each of the above sections 1, 2, and 3, the obtained curvatures R1, R2, and R3 are stored as sensing data. The curvature R of the curve defined by the equation

number

[0085] In step S26, the feeling estimation processor 23 compares the calculated degree of match with a preset threshold. If the degree of match is determined to be equal to or greater than the threshold, the feeling estimation processor 23 proceeds to step S27 and determines that the user experienced a feeling (e.g., tension) in the above section.

[0086] On the other hand, if the degree of coincidence is less than the threshold, it is assumed that the user has not expressed any emotion in that section, and the process proceeds to step S28, where it is determined whether or not the determination of the degree of coincidence for all sections of the sensing data has been completed. If there are still sections that have not been determined, the process returns to step S23, and the series of processes from acquiring the sensing data in steps S23 to S28, to determining the degree of coincidence of the graph curve of that sensing data, and determining whether estimation has ended are repeatedly executed.

[0087] (2-5) Generation and output of estimated information When the control unit 1 of the feeling estimation device EM completes the process of determining the occurrence of emotions for all sections of the sensing data, the control unit 1 proceeds to step S29. In step S29, under the control of the estimated information output processing unit 24, estimation result information is generated for reporting the determination result of the occurrence of emotions to the user or administrator who has requested the estimation. The estimated information output processing unit 24 then transmits the generated estimation result information from the communication I / F unit 4 to the terminal of the user or administrator who is the report destination.

[0088] Possible methods for the estimation result information include, for example, a method of displaying in real time using text data or the like to indicate that an emotion has occurred, such as "I feel nervous," or a method of listing whether or not an emotion has occurred and the time at which it occurred for each section over the entire period from the start to the end of estimation, and transmitting this list information after the estimation is completed.

[0089] Furthermore, in the process of comparing the curvatures in step S25, it may be determined which curve has a higher value indicating the peak height between the curve of the fluctuation graph of the sensing data and the curve represented by the equation, and based on the determination result, information indicating the degree of tension, such as "strong tension" or "weak tension," may be included in the estimation result information.

[0090] (Actions and Effects) As described above, in one embodiment, in the emotion estimation preset mode, fluctuation graph information representing fluctuations in emotions subjectively felt by a user in response to the presentation of a preset event is compared with fluctuation graph information objectively measured by a plurality of biosensors simultaneously for the event, and the biosensor that measured the fluctuation graph that most closely matches the subjective emotional fluctuation felt by the user is selected from the plurality of biosensors and stored in the setting list data in association with the event. Then, in the actual emotion estimation mode, when estimating the occurrence of an emotion of the user in response to the event, the preset biosensor is selected, and the occurrence of an emotion is estimated based on the sensing data measured by this biosensor.

[0091] Therefore, for example, a user's emotions are estimated using a biosensor that can measure fluctuation characteristics that are closest to the subjective emotional fluctuations of the user for each user and each type of emotion, so that appropriate emotion estimation can always be performed using the most appropriate biosensor regardless of the user and the type of emotion to be estimated.

[0092] [Other embodiments] In the embodiment, the functions of the emotion deduction device EM are provided in a server device on the Web or the cloud. However, the functions of the emotion deduction device EM may be provided in a server device located on a local area network such as a workplace or community, or in a personal computer shared by multiple users. Moreover, the functions of the emotion deduction device may be distributed among multiple server devices and terminals such as personal computers.

[0093] In addition, the type of emotion to be estimated, the type of event that induces the emotion, the type of sensing method, the functional configuration, processing procedure, processing content, etc. of the emotion estimation device can be modified and implemented in various ways without departing from the gist of the present invention.

[0094] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0095] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0096] 1...Control unit 2...Program memory section 3...Data storage unit 4...Communication I / F section 5...Bus 10...Presetting processing unit 11...Sensing data acquisition processing unit 12...Subjective emotion data acquisition processing unit 13...Formulation processing section 14...Sensing method selection and setting processing section 20...Emotion estimation processing unit 21...Estimated request acquisition processing unit 22...Sensing data acquisition processing unit 23...Emotion estimation processing unit 24...Estimated information output processing unit 31...Sensing data storage unit 32...Subjective emotion data storage unit 33...Setting list data storage section 34...Sensing data storage unit

Claims

1. An emotion estimation device that estimates an emotion of a user, a pre-setting processing unit; Emotion estimation processing unit and Equipped with The presetting processing unit a first processing unit that acquires, in response to a declaration by a user, first fluctuation graph information that represents fluctuations in emotions subjectively felt by the user in response to an event that causes a change in the emotions; a second processing unit that acquires a plurality of second fluctuation graph information obtained by objectively measuring the user's biological reaction to the event using a plurality of sensing methods; a third processing unit that compares the shapes of the curves of the graphs representing the characteristics of the fluctuations between the first fluctuation graph information and the plurality of second fluctuation graph information, selects the sensing method corresponding to the second fluctuation graph information whose degree of coincidence of the curve shapes satisfies a preset condition, and stores the selected sensing method in a storage medium in association with at least the information representing the event and the user's identification information; Equipped with The emotion estimation processing unit a fourth processing unit that, when information representing the event and identification information of the user to be estimated are input, selects from the storage medium the sensing method corresponding to each of the input information, and acquires third fluctuation graph information corresponding to the biological reaction of the user measured using the selected sensing method; a fifth processing unit that compares the shape of the curve of the acquired third variation graph information with the shape of the curve of second variation graph information corresponding to the sensing method to calculate a degree of agreement between them, and determines whether the calculated degree of agreement is equal to or greater than a predetermined threshold value or less, thereby determining whether the emotion to be estimated has occurred; An emotion estimation device comprising:

2. 2. The feeling estimation device according to claim 1, wherein the third processing unit formulates the first fluctuation graph information and the second fluctuation graph information using predetermined feature parameters, respectively, to obtain first fluctuation graph characteristics and second fluctuation graph characteristics representing characteristics of the fluctuations, and compares the first fluctuation graph characteristic with the second fluctuation graph characteristic.

3. The emotion estimation device according to claim 2 , wherein the third processing unit performs the formulation using a time until an emotion peak is reached, a duration of the emotion, and a magnitude of the emotion peak as the feature parameters.

4. 3. The feeling estimation device according to claim 2, wherein the third processing unit generates, for each of the events, setting list information in which identification information of the user, label information indicating a type of the emotion subjectively felt by the user, information indicating the selected sensing method, and the second variation graph characteristic corresponding to the sensing method are associated with each other, and stores the generated setting list information in the storage medium.

5. 2. The feeling estimation device according to claim 1, wherein the third processing unit calculates a curvature of change from each of the first fluctuation graph characteristic and the second fluctuation graph characteristic, and determines an absolute value of a difference between the calculated curvatures as a degree of agreement between features of the fluctuations.

6. An emotion estimation method executed by an emotion estimation device that estimates an emotion of a user, comprising: During the pre-configuration process, a first step of acquiring first fluctuation graph information representing fluctuations in emotions subjectively felt by the user in response to an event that causes a change in the emotions, in response to a declaration by the user; a second step of acquiring a plurality of second fluctuation graph information pieces obtained by objectively measuring the user's biological reactions to the event using a plurality of sensing methods; comparing the shapes of the curves of the graphs representing the characteristics of the fluctuations between the first fluctuation graph information and the plurality of second fluctuation graph information, selecting the sensing technique corresponding to the second fluctuation graph information whose degree of coincidence of the curve shapes satisfies a preset condition, and storing the selected sensing technique in a storage medium in association with at least the information representing the event and the user's identification information; Run In the emotion estimation process, a fourth step of, when information representing the event and identification information of the user to be estimated are input, selecting from the storage medium the sensing method corresponding to each of the input information, and acquiring third fluctuation graph information corresponding to the biological reaction of the user measured using the selected sensing method; a fifth step of comparing the shape of the curve of the acquired third variation graph information with the shape of the curve of second variation graph information corresponding to the sensing technique to calculate a degree of match, and determining whether the calculated degree of match is equal to or greater than a predetermined threshold value or less, thereby determining whether the emotion to be estimated has occurred; A sentiment estimation method that performs

7. A program that causes a processor provided in the emotion estimation device to execute at least one of the processes performed by each of the processing units provided in the emotion estimation device described in any one of claims 1 to 5.

Images