Wearable device, information processing method and program

Abstract

To solve the problem that in a conventional method, a learning time and an estimation time are separated; in such a method, estimation is not possible during learning and a certain lead time is required until performance durable for practical use is obtained.SOLUTION: A wearable device provided according to one aspect of the current invention acquires biological data of a user wearing the wearable device. The wearable device estimates the user's emotion on the basis of the biological data and a standard emotion estimation algorithm installed in advance.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a wearable device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses the following techniques 1 and 2. 1. Learning data is generated that represents the association between the acquired information representing the subject's emotions and the information representing the subject's activity state, and is stored in memory. 2. In this state, the subject's current emotion is estimated based on the acquired information representing the subject's current activity state and the learning data stored in the memory. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-208576 A Summary of the Invention [Problem to be solved by the invention]

[0004] The method of Patent Document 1 is divided into a learning time and an estimation time. In such a method, estimation cannot be performed during learning, and a certain lead time is required to obtain performance that can withstand practical use. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided a wearable device, which acquires biometric data of a user wearing the wearable device, and estimates the user's emotion based on the biometric data and a pre-installed standard emotion estimation algorithm. [Brief description of the drawings]

[0006] [Figure 1]FIG. 1 is a diagram illustrating an example of a wearable device. [Diagram 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of the smart watch 100. [Diagram 3] FIG. 3 is a flowchart showing an example of information processing in the smart watch 100. [Figure 4] FIG. 4 is a diagram showing an example of a screen displayed on the input / output unit 250. As shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0008] In this specification, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various information is handled in this embodiment, and communication and calculation can be performed on a circuit in the broad sense regardless of whether this information is represented by a high or low signal value as a binary bit collection consisting of 0 or 1, represented by a physical numerical value of a signal value, or represented by quantum superposition.

[0009] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0010] In addition, the program for realizing the software appearing in the embodiments may be implemented in a form that is downloadable from a server, the program may be executed on a cloud computer, or the program may be stored in a non-volatile or volatile non-transitory storage medium and distributed.

[0011] <Embodiment 1> 1. System Configuration FIG. 1 is a diagram showing an example of a wearable device. As shown in FIG. 1, in this specification, a smart watch 100 is taken as an example of a wearable device. However, a smart watch is one example of a wearable device. Other examples of wearable devices include fitness trackers, smart glasses, vital sign monitors, smart rings, and badge-type wearable devices. As described later, any wearable device may be used as long as it is capable of outputting an emotion estimation result to a user and accepting a correct emotion if the estimation result is incorrect.

[0012] 2. Hardware Configuration 2 is a diagram showing an example of the hardware configuration of the smartwatch 100. The smartwatch 100 includes a control unit 210, a storage unit 220, a biological data measurement unit 230, an input / output unit 250, and a communication unit 260 as the hardware configuration.

[0013] The control unit 210 is a CPU (Central Processing Unit) or the like, and controls the smart watch 100 as a whole.

[0014] The storage unit 220 is any one of a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive), or any combination thereof, and stores a program and data used when the control unit 210 executes processing based on the program. The control unit 210 executes processing based on the program stored in the storage unit 220, thereby realizing the functions of the smartwatch 100 and the processing of the flowchart shown in FIG. 3 described later. The storage unit 220 is an example of a storage medium. Note that in this embodiment, the data used when the control unit 210 executes processing based on the program is described as being stored in the storage unit 220, but the data may be stored in a storage unit of another device that can communicate with the smartwatch 100. In other words, the data may be stored in a storage unit of any device as long as the control unit 210 can refer to the data.

[0015] The biological data measurement unit 230 measures biological data of the user of the smart watch 100. Examples of biological data measured by the biological data measurement unit 230 include heart rate, pulse, blood pressure, oxygen concentration, skin temperature, and skin potential. The acceleration measurement unit 240 measures the acceleration of the smart watch 100.

[0016] The input / output unit 250 is a touch panel display or the like, which displays information, accepts user operations, and selects or inputs information in response to the user operations. The communication unit 260 connects the smartwatch 100 to a network or the like, and is responsible for communication with other devices and the like.

[0017] 3. Information Processing The information processing according to this embodiment will be described below.

[0018] FIG. 3 is a flowchart showing an example of information processing in the smart watch 100. In step S301, the control unit 210 acquires biometric data of the user wearing the smart watch 100. The process of step S301 is assumed to be constantly executed while the smart watch 100 is activated. The acquired biometric data is stored, for example, in the storage unit 220. The storage unit 220 stores biometric data for a predetermined period, and the biometric data after the predetermined period may be stored in a server device or the like capable of communicating with the smart watch 100.

[0019] In step S302, the control unit 210 determines whether or not a predetermined condition is satisfied. For example, the control unit 210 determines whether or not the biometric data satisfies a predetermined condition. For example, the control unit 210 determines whether or not the heart rate is 60 to 100 beats / minute. If the heart rate is 60 to 100 beats / minute, the control unit 210 determines that the biometric data satisfies the predetermined condition. If the heart rate is not 60 to 100 beats / minute, the control unit 210 determines that the biometric data does not satisfy the predetermined condition. If the control unit 210 determines that the biometric data satisfies the predetermined condition, the control unit 210 advances the process to step S303. If the control unit 210 determines that the biometric data does not satisfy the predetermined condition, the control unit 210 returns the process to step S301. The heart rate is an example, and it may be determined whether or not the predetermined condition is satisfied based on the pulse, blood pressure, oxygen concentration, skin temperature, skin potential, etc.

[0020] In step S303, the control unit 210 infers the user's emotion based on the biometric data and a reference emotion estimation algorithm. The reference emotion estimation algorithm is stored in advance in the storage unit 220 or the like. That is, the reference emotion estimation algorithm is pre-installed in the smartwatch 100.

[0021] In step S304, the control unit 210 outputs information related to the estimated user's emotion. The control unit 210 also prompts the input of feedback related to the estimated user's emotion. More specifically, the control unit 210 prompts the input of feedback related to the user's emotion by displaying a screen (FIG. 4(b) and / or FIG. 4(c) described later) that can accept the input of feedback related to the estimated user's emotion.

[0022] That is, the control unit 210 displays the screen based on a predetermined condition. More specifically, the control unit 210 displays the screen based on the fact that the biometric data satisfies a predetermined condition.

[0023] Fig. 4 is a diagram showing an example of a screen displayed on the input / output unit 250. Fig. 4(a) is a diagram showing an example of a screen displayed on the input / output unit 250 when the smart watch 100 is operating in watch mode. The screen shown in Fig. 4(a) is displayed on the input / output unit 250 as a main screen. Most of the time when the smart watch 100 is running, the main screen is displayed on the input / output unit 250. In steps S301 to S303, the screen shown in Fig. 4(a) is displayed on the input / output unit 250.

[0024] FIG. 4(b) is a diagram showing an example of an emotion confirmation screen. The screen of FIG. 4(b) is a screen displayed on the input / output unit 250 in step S304. The emotion confirmation screen displays the user's emotion estimated in step S303. In the example of the first embodiment, any one of four emotions, joy, anger, sadness, and pleasure, is estimated as the user's emotion. In the example of FIG. 4(b), joy is estimated and displayed as the user's emotion. The emotion confirmation screen of FIG. 4(b) includes a Y button 410 that the user selects when the estimated emotion is correct, and an N button 420 that the user selects when the estimated emotion is incorrect. Note that the types of emotions are merely an example and are not limited to four. Furthermore, the classification of emotions is not limited to joy, anger, sadness, and pleasure.

[0025] When the N button 420 is selected on the screen of FIG. 4(b), the control unit 210 transitions the screen displayed on the input / output unit 250 from FIG. 4(b) to FIG. 4(c). FIG. 4(c) is a diagram showing an example of an emotion type selection screen. On the emotion type selection screen, the user can select one of joy, anger, sadness, and pleasure. For example, in the example of FIG. 4(c), when the user selects the anger button 450, the control unit 210 records that the estimation using the standard emotion estimation algorithm was incorrect and that the correct emotion of the user was anger. For example, the control unit 210 adds corrected emotion information to the biometric data used for the estimation as the correct emotion. Also, for example, in the example of FIG. 4(c), when the user selects the sadness button 460, the control unit 210 records that the estimation using the standard emotion estimation algorithm was incorrect and that the correct emotion of the user was sadness. If no emotion information is selected for a certain period of time (e.g., 30 seconds) after the screen of FIG. 4(c) is displayed, control unit 210 may automatically transition the screen to the screen of FIG. 4(a), which is in clock mode.

[0026] In step S305, the control unit 210 determines whether the emotion information has been corrected on the screens shown in FIG. 4(b) and FIG. 4(c). If the control unit 210 determines that the emotion information has been corrected, it advances the process to step S306. If the control unit 210 determines that the emotion information has not been corrected, it returns the process to step S310. If the control unit 210 determines that the emotion information has not been corrected, it transitions the screen displayed on the input / output unit 250 from FIG. 4(b) to FIG. 4(a). If the N button 420 is selected on the screen of FIG. 4(b) and the user's correct emotion information is input on the screen of FIG. 4(c), the control unit 210 determines that the emotion information has been corrected. If the Y button 410 is selected on the screen of FIG. 4(b), the control unit 210 determines that the emotion information has not been corrected. Note that, when the Y button 410 is selected on the screen of FIG. 4(b), the control unit 210 may record that the estimation using the standard emotion estimation algorithm is correct. For example, the control unit 210 may add a label of the correct answer to the biometric data used for the estimation together with information on the estimated emotion.

[0027] In step S306, the control unit 210 corrects the reference emotion estimation algorithm based on the input feedback. Here, feedback refers to information indicating the correct emotion after correction. More specifically, the control unit 210 corrects the reference emotion estimation algorithm by having the reference emotion estimation algorithm learn based on the biometric data and the input feedback.

[0028] In step S307, control unit 210 determines whether or not to end the process of the flowchart shown in Fig. 3. When control unit 210 determines to end the process of the flowchart shown in Fig. 3, it ends the process of the flowchart shown in Fig. 3. When control unit 210 determines not to end the process of the flowchart shown in Fig. 3, it returns the process to step S301.

[0029] According to the process of this embodiment, the smartwatch 100 can estimate and output the user's emotion using a standard emotion estimation algorithm pre-installed in the smartwatch 100. Therefore, it can be used without a certain lead time until it achieves practical performance. In addition, if the estimation result of the user's emotion using the standard emotion estimation algorithm is incorrect, the standard emotion estimation algorithm can be re-trained by feedback such as the user's input or selection, thereby improving the estimation accuracy.

[0030] (Variation 1) A first modification of the first embodiment will be described. In the first embodiment, an example of the predetermined condition in step S302 is described in terms of whether the biometric data of the user satisfies the predetermined condition. In the first modification, as another example of the predetermined condition, a description will be given of whether or not biometric data arranged in chronological order satisfies a predetermined condition.

[0031] The control unit 210 of the first modification (hereinafter, simply referred to as the control unit 210) estimates the user's feelings based on the fact that the first biometric data and the second biometric data of the user acquired at different times satisfy a predetermined condition, and displays the estimation result on the screen. For example, if the blood pressure measured at the first time is 115 / 75 mmHg or less, and the blood pressure measured at the second time after the first time has elapsed a predetermined second (e.g., 15 seconds) is also 115 / 75 mmHg or less, the control unit 210 determines that the biometric data satisfies the predetermined condition. If the blood pressure measured at the first time is not 115 / 75 mmHg or less, or if the blood pressure measured at the second time is not 115 / 75 mmHg or less, the control unit 210 determines that the biometric data does not satisfy the predetermined condition.

[0032] That is, in the first modification, the control unit 210 infers the user's emotion based on whether the biometric data arranged in chronological order satisfies a predetermined condition, and outputs the inference result.

[0033] According to the first modification, when the biometric data satisfies a predetermined condition for a certain period of time, the user's emotion can be estimated and the estimation result can be output.

[0034] (Variation 2) A second modification of the first embodiment will now be described. In the first embodiment, an example of the predetermined condition in step S302 is described in terms of whether the biometric data of the user satisfies the predetermined condition. In the second modification, as another example of the predetermined condition, a description will be given of whether the acceleration of the smart watch 100 measured by the acceleration measuring unit 240 satisfies the predetermined condition.

[0035] The control unit 210 of the second modification (hereinafter, simply referred to as the control unit 210) acquires the acceleration of the smart watch 100 measured by the acceleration measuring unit 240. Then, the control unit 210 determines whether or not the acceleration satisfies a predetermined condition. The control unit 210 estimates the user's emotion based on whether the acceleration satisfies the predetermined condition (for example, the acceleration value is within a set range), and displays the estimation result on the screen.

[0036] According to the second modification, when the acceleration satisfies a predetermined condition, the user's emotion can be estimated and the estimation result can be output.

[0037] (Variation 3) A third modification of the first embodiment will now be described. In the first embodiment, an example of the predetermined condition in step S302 is described in terms of whether the biometric data of the user satisfies the predetermined condition. In the third modification, as another example of the predetermined condition, a description will be given of whether the acceleration of the smart watch 100 measured by the acceleration measuring unit 240 and arranged in order of time satisfies the predetermined condition.

[0038] The control unit 210 of the third modification (hereinafter, simply referred to as the control unit 210) estimates the user's emotions based on the fact that the first acceleration data and the second acceleration data related to the smart watch 100 acquired at different times satisfy a predetermined condition, and displays the estimation result on the screen. For example, if the acceleration value measured at the first time is within a set range, and the acceleration value measured at the second time after a predetermined number of seconds (e.g., 10 seconds) have elapsed since the first time, is also within the set range, the control unit 210 determines that the acceleration satisfies the predetermined condition.

[0039] According to the third modification, when the acceleration satisfies a predetermined condition for a certain period of time, the user's emotion can be estimated and the estimation result can be output.

[0040] (Variation 4) A fourth modification of the first embodiment will now be described. In the first embodiment, the processing is described as being performed by the smart watch 100. However, the processing of the smart watch 100 may be performed by an information processing system. As described above, the information processing system may be composed of multiple devices or may be composed of one device. When the information processing system is composed of one device, an example of the device is the smart watch 100. In such a configuration, variant example 4 is the same as embodiment 1. When the information processing system is composed of multiple devices, an example of the multiple devices is the smart watch 100 and a server device capable of communicating with the smart watch 100, or the smart watch 100 and a cloud system capable of communicating with the smart watch 100, etc.

[0041] The fourth modification can achieve the same effects as the first embodiment described above.

[0042] (Variation 5) A fifth modification of the first embodiment will now be described. The control unit 210 of the fifth modified example (hereinafter simply referred to as the control unit 210) may output a graph of the transition of the user's daily emotion ratio to the input / output unit 250, etc., upon request. Note that a graph is a diagram illustrating two or more quantities that are related to each other. The control unit 210 may also be configured to output a graph of transition of emotions for the measured biometric data based on a request to the input / output unit 250, etc. The control unit 210 may also be configured to output a graph of the ratio of the user's awakening for the biometric data to the input / output unit 250, etc., with happiness and anger among happiness, anger, sadness, and happiness as awakening and sadness and happiness as sleepiness, based on a request. The control unit 210 may also be configured to output a graph of the ratio of the user's pleasantness and unpleasantness for the biometric data to the input / output unit 250, etc., based on a request, with happiness and happiness among happiness, anger, sadness, and happiness as pleasantness and anger and happiness as unpleasantness.

[0043] Furthermore, the control unit 210 may output a graph of the user's emotion transition for each predetermined time period (for example, every hour) of a day to the input / output unit 250, etc., based on a request. The control unit 210 may also output a graph of a transition pattern indicating the transition from each emotion of joy, anger, sadness, and pleasure to another emotion to the input / output unit 250, etc.

[0044] According to the fifth modification, various information regarding the user's emotions can be output as graphs.

[0045] Furthermore, it may be provided in the following aspects:

[0046] (1) A wearable device that acquires biometric data of a user wearing the wearable device and estimates the user's emotions based on the biometric data and a pre-installed standard emotion estimation algorithm.

[0047] (2) A wearable device as described in (1) above, which outputs information regarding the estimated emotion of the user.

[0048] (3) A wearable device according to (2) above, which prompts the user to input feedback regarding the estimated emotion of the user.

[0049] (4) A wearable device according to (3) above, which prompts the user to input feedback regarding the user's emotions by displaying a screen capable of accepting input of feedback regarding the estimated emotions of the user.

[0050] (5) A wearable device according to (3) or (4) above, wherein the standard emotion estimation algorithm is modified based on the input feedback.

[0051] (6) The wearable device according to any one of (3) to (5) above, wherein the reference emotion estimation algorithm is corrected by learning the reference emotion estimation algorithm based on the biometric data and the input feedback.

[0052] (7) A wearable device according to the above (4), which displays the screen based on predetermined conditions.

[0053] (8) The wearable device according to (7) above, wherein the screen is displayed based on the biometric data satisfying the specified condition.

[0054] (9) A wearable device according to (8) above, which displays the screen based on the first biometric data and second biometric data of the user acquired at different times satisfying the specified condition.

[0055] (10) The wearable device according to (7) above, further comprising: acquiring acceleration data relating to the wearable device; and displaying the screen based on the acceleration data satisfying the specified condition.

[0056] (11) In the wearable device described in (10) above, the screen is displayed based on first acceleration data and second acceleration data regarding the wearable device acquired at different times satisfying the specified condition.

[0057] (12) An information processing method executed by a wearable device, comprising: acquiring biometric data of a user wearing the wearable device; and inferring emotions of the user based on the biometric data and a pre-installed standard emotion inference algorithm.

[0058] (13) A program for causing a computer to function as a wearable device according to any one of (1) to (11) above. Of course, this is not the case.

[0059] For example, the above-described embodiments and modifications may be implemented in any combination. In the first embodiment and the modifications described above, the device for acquiring biometric data and the device for outputting the emotion estimation result may be separate devices.

[0060] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The new embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0061] 100: Smartwatch 210: Control section 220: Storage section 230: Biological data measurement section 240: Acceleration measurement unit 250: Input / output section 260: Communications Department

Claims

1. A wearable device, Collect biometric data from the user wearing the wearable device, estimating the user's emotion based on the biometric data and a pre-installed standard emotion estimation algorithm; Wearable devices.

2. The wearable device according to claim 1 , Outputting information about the estimated emotion of the user. Wearable devices.

3. The wearable device according to claim 2, prompting for feedback regarding the estimated user's emotion; Wearable devices.

4. The wearable device according to claim 3, prompting the user to input feedback regarding the estimated emotion of the user by displaying a screen capable of receiving the input of feedback regarding the estimated emotion of the user; Wearable devices.

5. The wearable device according to claim 3, modifying the baseline emotion estimation algorithm based on the input feedback; Wearable devices.

6. The wearable device according to claim 3, correcting the reference emotion estimation algorithm by learning the reference emotion estimation algorithm based on the biometric data and the input feedback; Wearable devices.

7. The wearable device according to claim 4, displaying the screen based on a predetermined condition; Wearable devices.

8. The wearable device according to claim 7, displaying the screen based on the biometric data satisfying the predetermined condition; Wearable devices.

9. The wearable device according to claim 8, displaying the screen based on the first biometric data and the second biometric data of the user acquired at different times satisfying the predetermined condition; Wearable devices.

10. The wearable device according to claim 7, acquiring acceleration data relating to the wearable device; displaying the screen based on the acceleration data satisfying the predetermined condition; Wearable devices.

11. The wearable device according to claim 10, displaying the screen based on first acceleration data and second acceleration data regarding the wearable device acquired at different times satisfying the predetermined condition; Wearable devices.

12. An information processing method executed by a wearable device, comprising: Collect biometric data from the user wearing the wearable device, estimating the user's emotion based on the biometric data and a pre-installed standard emotion estimation algorithm; Information processing methods.

13. A program, Computer, A program for causing a wearable device to function as the wearable device according to any one of claims 1 to 11.

Images