Wearable device, information processing method and program

Abstract

To solve the problem that in a conventional method, a learning time and an estimation time are separated; in such a method, estimation is not possible during learning and a certain lead time is required until performance durable for practical use is obtained.SOLUTION: A wearable device provided according to one aspect of the current invention acquires biological data of a user wearing the wearable device. The wearable device estimates the user's emotion on the basis of the biological data and a standard emotion estimation algorithm installed in advance.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a wearable device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses the following techniques 1 and 2. 1. Learning data representing the relationship between the acquired information representing the subject's emotions and the information representing the subject's activity state is generated and stored in memory. 2. In this state, the subject's current emotion is estimated based on the acquired information representing the subject's current activity state and the learning data stored in memory. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-208576 Summary of the Invention [Problem to be solved by the invention]

[0004] The method of Patent Document 1 is divided into a learning period and an estimation period. With this method, estimation cannot be performed during learning, and a certain lead time is required to achieve performance that is suitable for practical use. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided a wearable device that acquires biometric data of a user wearing the wearable device and estimates the user's emotion based on the biometric data and a pre-installed standard emotion estimation algorithm. [Brief explanation of the drawings]

[0006] [Figure 1]FIG. 1 is a diagram illustrating an example of a wearable device. [Figure 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of the smartwatch 100. [Figure 3] FIG. 3 is a flowchart showing an example of information processing in the smartwatch 100. [Figure 4] FIG. 4 is a diagram showing an example of a screen displayed on the input / output unit 250. As shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0008] In this specification, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and communication and calculation can be performed on a circuit in the broad sense, regardless of whether this information is represented by a high or low signal value as a binary bit collection consisting of 0 or 1, a physical numerical value of a signal value, or a quantum superposition.

[0009] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0010] In addition, the program for realizing the software appearing in the embodiments may be implemented in a manner that allows it to be downloaded from a server, the program may be executed on a cloud computer, or it may be stored on a non-volatile or volatile non-transitory storage medium and distributed.

[0011] <Embodiment 1> 1. System Configuration FIG. 1 is a diagram illustrating an example of a wearable device. As shown in FIG. 1, this specification describes a smartwatch 100 as an example of a wearable device. However, a smartwatch is just one example of a wearable device. Other examples of wearable devices include fitness trackers, smart glasses, vital sign monitors, smart rings, and badge-type wearable devices. As will be described later, any wearable device may be used as long as it can output an emotion estimation result to the user and, if the estimation result is incorrect, accept the correct emotion.

[0012] 2. Hardware Configuration 2 is a diagram showing an example of the hardware configuration of the smartwatch 100. The smartwatch 100 includes, as its hardware configuration, a control unit 210, a storage unit 220, a biological data measurement unit 230, an input / output unit 250, and a communication unit 260.

[0013] The control unit 210 is a CPU (Central Processing Unit) or the like, and controls the smartwatch 100 as a whole.

[0014] The storage unit 220 is any one of a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), or any combination thereof, and stores programs and data used when the control unit 210 executes processing based on the programs. The control unit 210 executes processing based on the programs stored in the storage unit 220, thereby realizing the functions of the smartwatch 100 and the processing of the flowchart shown in FIG. 3 (described later). The storage unit 220 is an example of a storage medium. Note that, in this embodiment, the data used when the control unit 210 executes processing based on the programs is described as being stored in the storage unit 220, but the data may also be stored in a storage unit of another device that can communicate with the smartwatch 100. In other words, the data may be stored in a storage unit of any device as long as it can be referenced by the control unit 210.

[0015] The biological data measurement unit 230 measures biological data of the user of the smart watch 100. Examples of biological data measured by the biological data measurement unit 230 include heart rate, pulse, blood pressure, oxygen concentration, skin temperature, and skin potential. The acceleration measurement unit 240 measures the acceleration of the smartwatch 100.

[0016] The input / output unit 250 is a touch panel display or the like, which displays information, accepts user operations, and selects or inputs information in response to the user operations. The communication unit 260 connects the smartwatch 100 to a network or the like and controls communication with other devices or the like.

[0017] 3. Information Processing The information processing according to this embodiment will be described below.

[0018] FIG. 3 is a flowchart showing an example of information processing in the smartwatch 100. In step S301, the control unit 210 acquires biometric data of the user wearing the smart watch 100. The processing of step S301 is assumed to be constantly executed while the smart watch 100 is running. The acquired biometric data is stored, for example, in the storage unit 220. The storage unit 220 may store biometric data for a predetermined period, and the biometric data after the predetermined period has elapsed may be stored in a server device or the like that can communicate with the smart watch 100.

[0019] In step S302, the control unit 210 determines whether or not a predetermined condition is satisfied. For example, the control unit 210 determines whether or not the biological data satisfies the predetermined condition. For example, the control unit 210 determines whether or not the heart rate is 60 to 100 beats per minute. If the heart rate is 60 to 100 beats per minute, the control unit 210 determines that the biological data satisfies the predetermined condition. If the heart rate is not 60 to 100 beats per minute, the control unit 210 determines that the biological data does not satisfy the predetermined condition. If the control unit 210 determines that the biological data satisfies the predetermined condition, the control unit 210 proceeds to step S303. If the control unit 210 determines that the biological data does not satisfy the predetermined condition, the control unit 210 returns to step S301. The heart rate is one example, and whether or not the predetermined condition is satisfied may also be determined based on the pulse, blood pressure, oxygen concentration, skin temperature, skin potential, etc.

[0020] In step S303, the control unit 210 infers the user's emotion based on the biometric data and a reference emotion inference algorithm. The reference emotion inference algorithm is stored in advance in the storage unit 220 or the like. In other words, the reference emotion inference algorithm is pre-installed in the smartwatch 100.

[0021] In step S304, the control unit 210 outputs information related to the estimated user emotion. The control unit 210 also prompts the input of feedback related to the estimated user emotion. More specifically, the control unit 210 prompts the input of feedback related to the user emotion by displaying a screen (FIG. 4(b) and / or FIG. 4(c) described later) that can accept the input of feedback related to the estimated user emotion.

[0022] That is, the control unit 210 displays the screen based on a predetermined condition. More specifically, the control unit 210 displays the screen based on the fact that the biometric data satisfies a predetermined condition.

[0023] Fig. 4 is a diagram showing an example of a screen displayed on the input / output unit 250. Fig. 4(a) is a diagram showing an example of a screen displayed on the input / output unit 250 when the smart watch 100 is operating in watch mode. The screen shown in Fig. 4(a) is displayed on the input / output unit 250 as the main screen. For most of the time that the smart watch 100 is running, the main screen is displayed on the input / output unit 250. In steps S301 to S303, the screen shown in Fig. 4(a) is displayed on the input / output unit 250.

[0024] FIG. 4(b) is a diagram showing an example of an emotion confirmation screen. The screen of FIG. 4(b) is a screen displayed on the input / output unit 250 in step S304. The emotion confirmation screen displays the user's emotion estimated in step S303. In the example of embodiment 1, one of the four emotions of joy, anger, sadness, and happiness is estimated as the user's emotion. In the example of FIG. 4(b), joy is estimated and displayed as the user's emotion. The emotion confirmation screen of FIG. 4(b) includes a Y button 410 that the user selects if the estimated emotion is correct, and an N button 420 that the user selects if the estimated emotion is incorrect. Note that the types of emotions are merely an example and are not limited to four. Furthermore, the classification of emotions is not limited to joy, anger, sadness, and happiness.

[0025] When the N button 420 is selected on the screen of FIG. 4(b), the control unit 210 transitions the screen displayed on the input / output unit 250 from FIG. 4(b) to FIG. 4(c). FIG. 4(c) is a diagram illustrating an example of an emotion type selection screen. The emotion type selection screen allows the user to select one of joy, anger, sadness, and happiness. For example, in the example of FIG. 4(c), if the user selects the anger button 450, the control unit 210 records that the estimation using the standard emotion estimation algorithm was incorrect and that the correct emotion of the user was anger. For example, the control unit 210 adds corrected emotion information to the biometric data used for estimation as the correct emotion. Furthermore, for example, in the example of FIG. 4(c), if the user selects the sadness button 460, the control unit 210 records that the estimation using the standard emotion estimation algorithm was incorrect and that the correct emotion of the user was sadness. If no emotion information is selected for a certain period of time (e.g., 30 seconds) after the screen of FIG. 4(c) is displayed, the control unit 210 may automatically transition the screen to the screen of FIG. 4(a), which is in clock mode.

[0026] In step S305, the control unit 210 determines whether the emotion information has been corrected on the screens shown in FIGS. 4(b) and 4(c). If the control unit 210 determines that the emotion information has been corrected, it proceeds to step S306. If the control unit 210 determines that the emotion information has not been corrected, it returns to step S310. If the control unit 210 determines that the emotion information has not been corrected, it transitions the screen displayed on the input / output unit 250 from FIG. 4(b) to FIG. 4(a). If the N button 420 is selected on the screen of FIG. 4(b) and the user's correct emotion information is input on the screen of FIG. 4(c), the control unit 210 determines that the emotion information has been corrected. If the Y button 410 is selected on the screen of FIG. 4(b), the control unit 210 determines that the emotion information has not been corrected. Note that when the Y button 410 is selected on the screen of FIG. 4(b), the control unit 210 may record that the estimation using the standard emotion estimation algorithm is correct. For example, the control unit 210 may add a label of the correct answer to the biometric data used for estimation together with information on the estimated emotion.

[0027] In step S306, the control unit 210 modifies the reference emotion estimation algorithm based on the input feedback. Here, feedback refers to information indicating the correct emotion after modification. More specifically, the control unit 210 modifies the reference emotion estimation algorithm by training the reference emotion estimation algorithm based on the biometric data and the input feedback.

[0028] In step S307, control unit 210 determines whether or not to end the processing of the flowchart shown in Fig. 3. When control unit 210 determines to end the processing of the flowchart shown in Fig. 3, it ends the processing of the flowchart shown in Fig. 3. When control unit 210 determines not to end the processing of the flowchart shown in Fig. 3, it returns the processing to step S301.

[0029] According to the processing of this embodiment, the smartwatch 100 can estimate and output the user's emotion using a standard emotion estimation algorithm pre-installed in the smartwatch 100. Therefore, it can be used without any lead time until it achieves practical performance. Furthermore, if the estimation result of the user's emotion using the standard emotion estimation algorithm is incorrect, the standard emotion estimation algorithm can be retrained by feedback such as user input or selection, thereby improving the estimation accuracy.

[0030] (Variation 1) A first modification of the first embodiment will now be described. In the first embodiment, an example of the predetermined condition in step S302 is whether the biometric data of the user satisfies the predetermined condition. In the first modification, as another example of the predetermined condition, whether or not biometric data arranged in chronological order satisfies the predetermined condition will be described.

[0031] The control unit 210 of Modification 1 (hereinafter simply referred to as the control unit 210) estimates the user's emotions based on whether first and second biometric data of the user acquired at different times satisfy a predetermined condition, and displays the estimation result on the screen. For example, if the blood pressure measured at the first time is 115 / 75 mmHg or less and the blood pressure measured at the second time a predetermined number of seconds (e.g., 15 seconds) after the first time is also 115 / 75 mmHg or less, the control unit 210 determines that the biometric data satisfies the predetermined condition. If the blood pressure measured at the first time is not 115 / 75 mmHg or less or the blood pressure measured at the second time is not 115 / 75 mmHg or less, the control unit 210 determines that the biometric data does not satisfy the predetermined condition.

[0032] That is, in the first modification, the control unit 210 estimates the user's emotions based on whether the biological data arranged in chronological order satisfies a predetermined condition, and outputs the estimation result.

[0033] According to the first modification, if the biometric data satisfies a predetermined condition for a certain period of time, the user's emotions can be estimated and the estimation result can be output.

[0034] (Variation 2) A second modification of the first embodiment will now be described. In the first embodiment, an example of the predetermined condition in step S302 is whether the biometric data of the user satisfies the predetermined condition. In the second modification, whether the acceleration of the smartwatch 100 measured by the acceleration measuring unit 240 satisfies a predetermined condition will be described as another example of the predetermined condition.

[0035] The control unit 210 of Modification 2 (hereinafter simply referred to as the control unit 210) acquires the acceleration of the smartwatch 100 measured by the acceleration measurement unit 240. The control unit 210 then determines whether the acceleration satisfies a predetermined condition. The control unit 210 estimates the user's emotion based on whether the acceleration satisfies the predetermined condition (for example, the acceleration value is within a set range), and displays the estimation result on the screen.

[0036] According to the second modification, if the acceleration satisfies a predetermined condition, the user's emotion can be estimated and the estimation result can be output.

[0037] (Variation 3) A third modification of the first embodiment will now be described. In the first embodiment, an example of the predetermined condition in step S302 is whether the biometric data of the user satisfies the predetermined condition. In the third modification, as another example of the predetermined condition, an explanation will be given using as an example whether the accelerations of the smart watch 100 measured by the acceleration measuring unit 240 and arranged in order of time satisfy the predetermined condition.

[0038] The control unit 210 of Modification 3 (hereinafter simply referred to as the control unit 210) estimates the user's emotions based on whether first acceleration data and second acceleration data related to the smart watch 100 acquired at different times satisfy a predetermined condition, and displays the estimation result on the screen. For example, if the acceleration value measured at the first time is within a set range, and the acceleration value measured at a second time after a predetermined number of seconds (e.g., 10 seconds) has elapsed since the first time, is also within the set range, the control unit 210 determines that the acceleration satisfies the predetermined condition.

[0039] According to the third modification, if the acceleration satisfies a predetermined condition for a certain period of time, the user's emotion can be estimated and the estimation result can be output.

[0040] (Variation 4) A fourth modification of the first embodiment will now be described. In the first embodiment, the processing is described as being performed by the smartwatch 100. However, the processing of the smartwatch 100 may be performed by an information processing system. As described above, the information processing system may be composed of multiple devices, or may be composed of a single device. When the information processing system is composed of a single device, an example of that device is the smartwatch 100. In such a configuration, Modification 4 is the same as Embodiment 1. When the information processing system is composed of multiple devices, an example of the multiple devices is the smartwatch 100 and a server device capable of communicating with the smartwatch 100, or the smartwatch 100 and a cloud system capable of communicating with the smartwatch 100.

[0041] The fourth modification can achieve the same effects as the first embodiment described above.

[0042] (Variation 5) A fifth modification of the first embodiment will now be described. The control unit 210 of Modification 5 (hereinafter simply referred to as the control unit 210) may output a graph of the transition of the user's daily emotion ratio to the input / output unit 250, etc., upon request. Note that a graph is a diagram illustrating two or more quantities that are related to each other. The control unit 210 may also be configured to output a graph of transition of emotions with respect to the measured biometric data based on a request to the input / output unit 250, etc. The control unit 210 may also be configured to output a graph of the ratio of the user's arousal with respect to the biometric data, where joy and anger among joy, anger, sadness, and happiness are defined as arousal and sadness and happiness are defined as sleepiness, based on a request to the input / output unit 250, etc. The control unit 210 may also be configured to output a graph of the ratio of the user's pleasure and discomfort with respect to the biometric data, where joy and happiness among joy, anger, sadness, and happiness are defined as pleasure and anger and happiness are discomfort, based on a request to the input / output unit 250, etc.

[0043] Furthermore, the control unit 210 may be configured to output a graph of the user's emotional transition for each predetermined time period (for example, every hour) of one day based on a request to the input / output unit 250, etc. Furthermore, the control unit 210 may be configured to output a graph of a transition pattern indicating the transition from each of joy, anger, sadness, and pleasure to which emotion the user has transitioned to, to the input / output unit 250, etc.

[0044] According to the fifth modification, various pieces of information relating to the user's emotions can be output as graphs.

[0045] Furthermore, it may be provided in the following aspects.

[0046] (1) A wearable device that acquires biometric data of a user wearing the wearable device and estimates the user's emotions based on the biometric data and a pre-installed standard emotion estimation algorithm.

[0047] (2) A wearable device according to (1) above, which outputs information relating to the estimated emotion of the user.

[0048] (3) A wearable device according to (2) above, which prompts the user to input feedback regarding the estimated emotion of the user.

[0049] (4) A wearable device according to (3) above, which prompts the user to input feedback regarding the user's emotions by displaying a screen that can accept input of feedback regarding the estimated emotions of the user.

[0050] (5) The wearable device according to (3) or (4) above, wherein the reference emotion estimation algorithm is modified based on the input feedback.

[0051] (6) The wearable device according to any one of (3) to (5) above, wherein the reference emotion estimation algorithm is corrected by learning the reference emotion estimation algorithm based on the biometric data and the input feedback.

[0052] (7) The wearable device according to (4) above, wherein the screen is displayed based on a predetermined condition.

[0053] (8) The wearable device according to (7) above, wherein the screen is displayed based on the biometric data satisfying the predetermined condition.

[0054] (9) A wearable device according to (8) above, which displays the screen based on the first biometric data and second biometric data of the user acquired at different times satisfying the specified condition.

[0055] (10) A wearable device according to (7) above, which acquires acceleration data relating to the wearable device and displays the screen based on whether the acceleration data satisfies the predetermined condition.

[0056] (11) A wearable device according to (10) above, which displays the screen based on first acceleration data and second acceleration data relating to the wearable device acquired at different times satisfying the specified condition.

[0057] (12) An information processing method executed by a wearable device, which acquires biometric data of a user wearing the wearable device and estimates the user's emotions based on the biometric data and a pre-installed standard emotion estimation algorithm.

[0058] (13) A program for causing a computer to function as the wearable device described in any one of (1) to (11) above. Of course, this is not the case.

[0059] For example, the above-described embodiments and modifications may be combined in any manner. In the first embodiment and the modifications described above, the device for acquiring biometric data and the device for outputting the emotion estimation result may be separate devices.

[0060] Finally, while various embodiments of the present invention have been described, these are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. The embodiments and their modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the appended claims. [Explanation of symbols]

[0061] 100: Smartwatch 210: Control unit 220: Storage section 230: Biological data measurement unit 240: Acceleration measurement unit 250: Input / output section 260: Communications Department

Claims

1. A wearable device, Acquire biometric data of a user wearing the wearable device; Estimating the user's emotion based on the biometric data and a pre-installed standard emotion estimation algorithm; displaying a screen including emotion information about the estimated emotion of the user and capable of receiving input of feedback about the emotion information, thereby prompting the user to input feedback about the emotion information; when it is determined that the emotion information of the user has been modified based on the feedback, modifying the emotion estimation algorithm based on the feedback; Wearable device.

2. The wearable device according to claim 1 , correcting the reference emotion estimation algorithm by training the reference emotion estimation algorithm based on the biometric data used for the estimation and the input feedback; Wearable device.

3. The wearable device according to claim 1 , displaying the screen based on a predetermined condition; Wearable device.

4. The wearable device according to claim 3, displaying the screen based on whether the biometric data satisfies the predetermined condition; Wearable device.

5. The wearable device according to claim 4, displaying the screen based on whether the first biometric data and the second biometric data of the user acquired at different times satisfy the predetermined condition; Wearable device.

6. The wearable device according to claim 3, acquiring acceleration data relating to the wearable device; displaying the screen based on the acceleration data satisfying the predetermined condition; Wearable device.

7. The wearable device according to claim 6, displaying the screen based on whether first acceleration data and second acceleration data related to the wearable device, which are acquired at different times, satisfy the predetermined condition; Wearable device.

8. An information processing method executed by a wearable device, comprising: Acquire biometric data of a user wearing the wearable device; Estimating the user's emotion based on the biometric data and a pre-installed standard emotion estimation algorithm; displaying a screen including emotion information about the estimated emotion of the user and capable of receiving input of feedback about the emotion information, thereby prompting the user to input feedback about the emotion information; when it is determined that the emotion information of the user has been modified based on the feedback, modifying the emotion estimation algorithm based on the feedback; Information processing methods.

9. A program, Computer, A program for causing a wearable device to function as the wearable device according to any one of claims 1 to 7.

Images