Internal state estimation device and internal state estimation method

Abstract

To provide an internal state estimation device for improving estimation accuracy of internal states of test subjects by biological measurement data by a sensor.SOLUTION: In an internal state estimation device for measuring a test subject during work to acquire sensor data from the test subject, extracting a feature amount from the acquired sensor data, and estimating an internal state of the test subject based on the feature amount extracted from the sensor data and an internal state estimation model, the internal state estimation device includes: retaining statistical data relating to a neutral state of a difficulty level and a skill level; setting measurement tasks, with difficulty levels defined, to a test subject to measure a skill level for every of the difficulty levels; and adjusting a UI when the test subject deals with the measurement tasks such that the test subject approaches the neutral state of the difficulty level and the skill level defined in the statistical data and defining the adjusted UI as a UI of an environment in work. Here, the internal state of the test subject is represented by a psychological state represented by each quadrant of a two-dimensional coordinate, with the difficulty level on one axis and the skill level on the other axis.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an inner state estimating device and an inner state estimating method, and in particular to an inner state estimating device and an inner state estimating method that improve the accuracy of estimating the inner state of a subject employee in a challenged state using biomeasurement data obtained by a sensor, and that are suitable for use in contributing to the improvement of business operations at a company. [Background technology]

[0002] In recent years, in various fields, there has been much discussion about what well-being is, the ideal state of human beings. Well-being can be defined as a state in which humans, as social beings, are happy and fulfilled in all aspects, physically, mentally, and socially.

[0003] To balance individual well-being and corporate performance, individual challenges are essential, and to that end, efforts are being made to attach sensors that measure biometric data to workers while they are working, extract features from the sensor data, and use these to estimate their internal state while working, which can then be used to improve work and educate employees.

[0004] A technology for estimating the subject's emotion using sensor data in this way is disclosed in, for example, Patent Document 1. In the estimation device described in Patent Document 1, an accelerometer and a heart rate monitor are attached to the subject, and acceleration data and heart rate data are acquired at regular time intervals. Here, both data are acquired at regular time intervals (for example, 5 minutes). The heart rate data is recorded with an emotion label, and machine learning is performed to estimate the emotion from the heart rate data. When estimating the emotion, the estimation device adjusts the heart rate data using the acceleration data, estimates the subject's emotion based on the heart rate data, and presents the estimation result (Figure 4, paragraph

[0035] ).

[0005] In addition, non-patent document 1 discloses a behavioral motivation estimation system that collects biometric data such as acceleration, heart rate, and skin electrodermal activity from office workers and uses machine learning techniques to estimate and improve the individual behavioral motivation.

[0006] In addition, non-patent document 2 discloses a technology that monitors daily heart rate data from a wearable device attached to an office worker and uses machine learning to grasp work engagement (WE) (an index for grasping a positive and fulfilling psychological state related to work) from the daily biological and activity status.

[0007] Meanwhile, in the study of psychological states, Mihaly Csikszentmihalyi has proposed the flow theory. The psychological state of "flow" refers to a "mental state in which you are so completely focused and engrossed in a subject that you lose track of time," and he discusses the relationship between the conditions for reaching this psychological state and motivation for work.

[0008] Non-Patent Document 3 discloses technology related to flow exercises that lead to a flow state for non-preference tasks such as work. In this paper, subjects doing desk work are asked to wear glasses-type devices equipped with acceleration sensors and tilt sensors that detect blinking and head movement, and the relationship between the subject's work and the flow state is considered (Page 41-45). [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2016-165373 A [Non-patent literature]

[0010] [Non-Patent Document 1] Yuki Chiaki and 1 other, "Development and Evaluation of a Motivational Support System Using Short-Term Biometric Data," [online], March 17, 2020, Information Processing Society of Japan Research Report, Vol. 2020-GN-110 No. 19, [Retrieved August 13, 2023], Internet <URL:https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=repository_action_common_download&item_id=204186&item_no=1&attribute_id=1&file_no=1> [Non-Patent Document 2] Haruki Harashima and 3 others, "Estimation of Work Engagement Using Wearable Heart Rate Sensors," October 2021, [online], Proceedings of the 29th Multimedia Communication and Distributed Processing Workshop, [Retrieved August 13, 2023], Internet<URL: https: / / arakawa-lab.com / wp-content / uploads / 2022 / 01 / IPSJ-DPSWS2021016.pdf> [Non-Patent Document 3] Yumi Sera and 1 other, "Proposal of "Flow Exercise": To enhance the sense of fulfillment in non-preference tasks," 2017, [online], Master's thesis. 2017 System Design and Management, No. 263, [Retrieved August 13, 2023], Internet <URL:https: / / koara.lib.keio.ac.jp / xoonips / modules / xoonips / download.php / K040002001-00002017-0014.pdf?file_id=138036> Summary of the Invention [Problem to be solved by the invention]

[0011] As mentioned above, various studies are being conducted to improve work content and build desirable working relationships by attaching wearable sensors to subjects and estimating the subject's internal state (psychological state) from the features of the sensor data.

[0012] In general, it is quite difficult to estimate the quantitative feature amount of sensor data by associating it with the qualitative inner state of a subject. This is because, in addition to individual differences such as the subject's personality, there are also influences from the work content and inner state before measurement, and unless the inner state of the subject is made neutral, there is a problem that the accuracy of estimating the inner state will not improve.

[0013] The above-mentioned Patent Document 1 and Non-Patent Documents 1-3 do not take into consideration the neutral induction of the subject's internal state.

[0014] Furthermore, although the subject's inner state is essentially subjective, the techniques described in the above patent documents and non-patent documents attempt to estimate the subject's inner state from objective indices obtained from the subject's biometric data. Therefore, there is a risk of a discrepancy between the estimation result based on the objective indices obtained from the subject's biometric data and the subject's subjective evaluation result. The techniques described in the above patent document 1 and non-patent documents 1 to 3 do not take into consideration reducing the discrepancy between the estimation result based on the objective indices obtained from the subject's biometric data and the subject's subjective evaluation result, and obtaining a result that is easy for the subject to accept.

[0015] Moreover, no consideration is given to visually displaying in an easily understandable manner the degree to which such objective evaluation coincides with the user's own subjective evaluation.

[0016] An object of the present invention is to provide an inner state estimation device and an inner state estimation method that improve the accuracy of estimating a subject's inner state using biomeasurement data from a sensor, reduce the discrepancy between the estimation results based on objective indicators obtained from the subject's biomeasurement data and the subject's subjective evaluation results, and obtain results that are easy for the subject to understand. [Means for solving the problem]

[0017] The internal state estimation device of the present invention is configured to measure a subject while he or she is working, acquire sensor data from the subject, extract features from the acquired sensor data, and estimate the subject's internal state based on the features extracted from the sensor data and an internal state estimation model, retaining statistical data relating to a neutral state of difficulty and skill level, presenting measurement tasks with a specified difficulty level to the subject, measuring the skill level for each level of difficulty, adjusting the UI (User Interface) when the subject works on the measurement tasks so as to approach the neutral state of difficulty and skill level specified by the statistical data, and making the adjusted UI the UI of the environment during the work.

[0018] Another configuration of the inner state estimation device of the present invention is an inner state estimation device that measures a subject while he or she is working, acquires sensor data from the subject, extracts features from the acquired sensor data, and estimates the inner state of the subject based on the features extracted from the sensor data and an inner state estimation model, estimates the subject's inner state in relation to a plurality of tasks, quantifies and displays each of the estimated inner states, has the subject input a positive / negative questionnaire evaluation of the estimated results, increases the numerical value of the estimated inner state of the inner state when a positive evaluation is made, and decreases the numerical value of the estimated inner state of the inner state when a negative evaluation is made, and outputs a numerical value of the inner state based on the subject's positive / negative questionnaire evaluation of the estimated results. Effect of the Invention

[0019] According to the present invention, it is possible to provide an inner state estimation device and an inner state estimation method that improve the accuracy of estimating a subject's inner state using biomeasurement data from a sensor, reduce the discrepancy between the estimation result based on objective indices obtained from the subject's biomeasurement data and the subject's subjective evaluation result, and obtain results that are easy for the subject to understand. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a configuration diagram of an inner surface state estimation system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating the hardware and software configuration of the inner surface state estimation device. [Diagram 3] FIG. 10 is a diagram illustrating an expression model of an inner state according to an embodiment. [Figure 4] FIG. 13 is a diagram explaining the difficulty and UI adjustment of the measurement tasks for subjects. [Diagram 5] FIG. 13 is a diagram showing an example of a subject test result table. [Figure 6] FIG. 13 is a diagram illustrating an example of a UI profile table. [Figure 7] 11 is a flowchart showing a series of processes performed by the inner state estimating device, from UI profile generation to inner state estimation and associated display. [Figure 8] 13 is a flowchart showing a UI profile generation process. [Figure 9] 13 is a flowchart showing difficulty level / UI adjustment processing. [Figure 10] 13 is a flowchart showing a process for displaying the degree of agreement between the difficulty / skill level questionnaire evaluation and the inner state result. [Figure 11] FIG. 13 is a diagram showing an inner state result questionnaire screen. [Figure 12] FIG. 13 is a diagram showing an inner surface state estimation result screen incorporating a questionnaire evaluation. [Figure 13] FIG. 13 shows a difficulty / skill level questionnaire screen for the most recent task. [Figure 14] FIG. 13 shows the screen displaying the degree of agreement between the difficulty / skill level questionnaire evaluation and the inner state results. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, an embodiment of the present invention will be described with reference to FIGS.

[0022] First, the configuration of an inner surface state estimating system according to an embodiment will be described with reference to Figs. The inner state estimating system is a system that estimates the inner state from biomeasurement data of a subject from a sensor while the subject is working, and as shown in FIG. 1, is configured such that a data collection terminal 20 and an inner state estimating device 100 are connected via a network 5.

[0023] The network 5 may be either wired or wireless, and may be a global network such as the Internet, or a LAN (Local Area Network).

[0024] The sensor 10 is a device for measuring the state of the subject, such as a wearable sensor, such as a smart watch, a heart rate sensor, or a glasses-type sensor, or a camera for capturing an image of the subject 1. The smart watch can measure the heart rate and acceleration for each axis in three-dimensional space, the heart rate sensor can measure the heart rate, the glasses-type sensor can measure blinking, eye movement, and head movement, and the camera can measure the posture of the subject for each axis in three-dimensional space.

[0025] The sensor 10 and the data collection terminal 20 are connected by, for example, Bluetooth (registered trademark), and data measured by the sensor 10 is transmitted to the data collection terminal 20. The data collection terminal 20 is a terminal that collects biomeasurement data obtained from a sensor 10 that measures the state of the subject 1 . The data collection terminal 20 has, as its functional components, a sensor data acquisition unit 21 and a sensor data transmission unit 22 .

[0026] The sensor data acquiring unit 21 is a functional unit that acquires sensor data from the sensor 10. The sensor data transmitting unit 22 is a functional unit that transmits the sensor data to the inner surface state estimating device 100 via the network 5.

[0027] The inner state estimating device 100 is a device that extracts feature amounts based on sensor data sent from the data collection terminal 20, estimates the inner state of the subject according to an estimation model based on the feature amounts, and displays the results.

[0028] The inner state estimation device 100 has, as its functional configuration, a sensor data receiving unit 101, a feature extraction unit 110, an inner state estimation unit 111, an estimation result display unit 112, a measurement task setting unit 120, a difficulty / level matching unit 121, a UI (User Interface) profile generation unit 122, a difficulty / UI adjustment unit 123, an inner state numerical value calculation unit 130 reflecting questionnaire evaluation, an inner state result display unit 131 reflecting questionnaire evaluation, a difficulty / skill level questionnaire evaluation and inner state result agreement calculation unit 140, a difficulty / skill level questionnaire evaluation and inner state result agreement display unit 141, a questionnaire input / output unit 150, and a memory unit 160.

[0029] The sensor data receiving unit 101 is a functional unit that receives sensor data from the data collection terminal 20. The feature extraction unit 110 is a functional unit that extracts feature amounts from the sensor data received from the data collection terminal 20. The inner state estimation unit 111 is a functional unit that estimates the inner state of the subject based on the feature amounts extracted from the sensor data and the inner state estimation model. The estimation result display unit 112 is a functional unit that displays the result of the estimated inner state of the subject. The measurement task setting unit 120 is a functional unit that generates and outputs a measurement task for UI change. The difficulty level / level matching unit 121 is a functional unit that matches the difficulty level and skill level in the measurement task. The UI profile generation unit 122 is a functional unit that generates a UI profile for each person based on the result of the measurement task for the subject. The difficulty / UI adjustment unit 123 is a functional unit that changes the difficulty level of the measurement task and the UI when working on the measurement task based on the result of the measurement task for the subject. The questionnaire evaluation reflected inner state numerical value calculation unit 130 is a functional unit that calculates a numerical value representing the inner state of the subject based on a questionnaire evaluation of the subject on the estimation of the inner state based on the feature amount. The questionnaire evaluation reflected inner state result display unit 131 is a functional unit that displays a numerical value representing the inner state by the questionnaire evaluation reflected inner state numerical value calculation unit 130. The difficulty / skill level questionnaire evaluation and inner state result agreement calculation unit 140 is a functional unit that calculates the agreement between a questionnaire regarding the difficulty and skill level of the work of the subject and a numerical value representing the inner state calculated by the questionnaire evaluation reflected inner state numerical value calculation unit 130. The difficulty / skill level questionnaire evaluation and inner state result agreement display unit 141 is a functional unit that displays the agreement calculated by the difficulty / skill level questionnaire evaluation and inner state result agreement calculation unit 140. The questionnaire input / output unit 150 is a functional unit that displays a questionnaire regarding the estimation result of the inner state and a questionnaire regarding the difficulty and skill level of the work to the subject and allows the subject to input the answer. The storage unit 160 is a functional unit that stores data used in the inner state estimation device.

[0030] Memory unit 160 holds a subject test result table 200, a UI profile table 210, difficulty / skill level statistical data 220, subject sensor data 230, feature amount data 240, inner state estimation model data 250, inner state estimation result data 260, questionnaire result data 270, inner state estimation data reflected in questionnaire evaluation 280, and difficulty / skill level questionnaire evaluation and inner state result agreement data 290. A specific example of the table will be described later.

[0031] The subject test result table 200 is a table that stores the test results of the measurement task for the subject. The UI profile table 210 is a table that stores the results when the UI for the measurement task for the subject is changed. The difficulty level / skill level statistical data 220 is data for setting a measurement task for the subject and defining a general neutral state of the difficulty level / skill level. The subject sensor data 230 is data acquired from a sensor for the subject. The feature amount data 240 is feature data extracted from the subject sensor data 230. The inner state estimation model data 250 is data expressing a model for estimating the inner state from the feature amount of the subject. The inner state estimation model data 250 is generated, for example, by a known machine learning method. The inner state estimation result data 260 is data of the cone result of the inner state of the subject estimated based on the feature amount of the subject and a model for estimating the inner state. The questionnaire result data 270 is result data of a questionnaire given to the subject. The questionnaire evaluation reflected inner state estimation data 280 is inner state estimation data reflecting the result of the questionnaire. The difficulty / skill level questionnaire evaluation and inner state result agreement data 290 is data that indicates the agreement between the difficulty / skill level questionnaire evaluation calculated as a result of the questionnaire and the inner state estimation data 280 reflected with the questionnaire evaluation.

[0032] Although FIG. 1 shows a configuration in which the data collection terminal 20 and the inner state estimating device 100 are separate devices, the inner state estimating device 100 may receive sensor data directly from the sensor 10 . Furthermore, the inner surface state estimating device 100 may realize its functions in a cloud environment.

[0033] Next, the hardware and software configuration of the inner surface state estimation device will be described with reference to FIG. The hardware configuration of the inner surface state estimating device 100 is realized by, for example, a general information processing device such as a personal computer shown in FIG.

[0034] The inner surface state estimating device 100 includes a CPU (Central Processing Unit) 302, a main memory device 304, a network I / F (Interface) 306, a display I / F 308, an input / output I / F 310, and an auxiliary memory I / F 312, all of which are connected via a bus.

[0035] The CPU 302 controls each unit of the inner surface state estimating device 100, and loads necessary programs into the main storage device 304 and executes them.

[0036] The main storage device 304 is typically configured as a volatile memory such as a RAM, and stores the programs executed by the CPU 302 and data referenced by the CPU 302.

[0037] The network I / F 306 is an interface for connecting to the network 5 .

[0038] The display I / F 308 is an interface for connecting a display device 320 such as an LCD (Liquid Crystal Display).

[0039] The input / output I / F 310 is an interface for connecting input / output devices. In the example of Fig. 2, a keyboard 330 and a mouse 332 which is a pointing device are connected.

[0040] The auxiliary storage I / F 312 is an interface for connecting an auxiliary storage device such as a hard disk drive (HDD) 350 or a solid state drive (SSD).

[0041] The HDD 350 has a large storage capacity and stores the programs for executing the present embodiment.

[0042] Installed in the HDD 350 of the inner state estimation device 100 are a sensor data receiving program 361, a feature extraction program 362, an inner state estimation program 363, an estimation result display program 364, a measurement task setting program 365, a difficulty / level matching program 366, a UI profile generation program 367, a difficulty / UI adjustment program 368, an inner state numerical value calculation program 369 reflecting questionnaire evaluation, an inner state result display program 370 reflecting questionnaire evaluation, a difficulty / skill level questionnaire evaluation and inner state result agreement calculation program 371, a difficulty / skill level questionnaire evaluation and inner state result agreement display program 372, and a questionnaire input / output program 373.

[0043] A sensor data receiving program 361, a feature extraction program 362, an inner state estimation program 363, an estimation result display program 364, a measurement task setting program 365, a difficulty level / level matching program 366, a UI profile generation program 367, a difficulty / UI adjustment program 368, a questionnaire evaluation reflected inner state numerical value calculation program 369, a questionnaire evaluation reflected inner state result display program 370, a difficulty / skill level questionnaire evaluation and inner state result agreement calculation program 371, a difficulty / skill level questionnaire evaluation and inner state result agreement display program 372, and a questionnaire input / output program 373. , respectively, are programs that realize the functions of a sensor data receiving unit 101, a feature extraction unit 110, an inner state estimation unit 111, an estimation result display unit 112, a measurement task setting unit 120, a difficulty / level matching unit 121, a UI profile generation unit 122, a difficulty / UI adjustment unit 123, an inner state numerical value calculation unit 130 reflecting questionnaire evaluation, an inner state result display unit 131 reflecting questionnaire evaluation, a difficulty / skill level questionnaire evaluation and inner state result calculation unit 131, a difficulty / skill level questionnaire evaluation and inner state result agreement calculation unit 140, a difficulty / skill level questionnaire evaluation and inner state result agreement display unit 141, and a questionnaire input / output unit 150.

[0044] Although not shown in FIG. 2 , HDD 350 stores subject test result table 200, UI profile table 210, difficulty / skill level statistical data 220, subject sensor data 230, feature amount data 240, inner state estimation model data 250, inner state estimation result data 260, questionnaire result data 270, inner state estimation data reflected in questionnaire evaluation 280, and difficulty / skill level questionnaire evaluation and inner state result agreement data 290.

[0045] Next, the expression model of the inner state in this embodiment will be described with reference to FIG. The model for expressing the inner state in this embodiment is a model based on the flow theory by Mihaly Csikszentmihalyi, described in Non-Patent Document 3.

[0046] As shown in Fig. 3, in a two-dimensional space, the X-axis represents skill level and the Y-axis represents difficulty. Skill level is a quantity that represents the level of the subject's skill in the task, and difficulty is a quantity that represents the difficulty of the task the subject is working on. Then, as shown in Fig. 3, each quadrant corresponds to one of the four states: Flow, Anxious, Lathargic, and Relaxed. In other words, this model states that when (the subject's skill level is X, and the difficulty of the task the subject is working on is Y), the subject's internal state is in each state (see also Fig. 12, page 23 of Non-Patent Document 3).

[0047] Next, the difficulty and UI adjustment of the measurement tasks for the subjects will be explained with reference to FIG. The subject's internal state is thought to be influenced by the mood of the day and the work performed. Therefore, we consider the possibility of giving measurement tasks before extracting the feature quantities due to the work, and inducing the subject's internal state to be in a neutral state.

[0048] The measurement task may be, for example, a task that requires the subject to answer a calculation problem (written or mental calculation). The difficulty level is expressed by the calculation problem, and the skill level can be interpreted as the percentage of correct answers.

[0049] Naturally, the abilities of subjects differ, so as shown in Figure 4, for the same level of difficulty, the measurement results of subjects with high skill levels are expected to be plotted on L1, users with low skill levels on L3, and subjects in a neutral state on L2. Here, when X is the skill level and Y is the difficulty, L2 is expressed as Y=αX. Here, α is the slope, and it is assumed that α<0. In general, difficulty and skill level are negatively correlated (i.e., as the problem becomes more difficult, the rate of correct answers also decreases), so this assumption is considered to be valid.

[0050] A relationship like L2 can be found by presenting measurement tasks of varying difficulty to many subjects, calculating the percentage of correct answers, plotting the most common correct answer rate or the average correct answer rate, and applying a statistical method such as the least squares method. Data representing this neutral state between difficulty and skill level is stored in difficulty / skill level statistical data 220.

[0051] Therefore, we consider adjusting the UI to raise the motivation of subjects with high skill levels, and conversely, adjusting the UI to lower the motivation of subjects with low skill levels, in order to induce them into a neutral state. Here, UI refers to the environmental factors when subjects perform the tasks.

[0052] For example, the following are examples of motivating UI: -Use warm colors for the background of the measurement task display. -The increase in correct answer rate is displayed to the subjects. Play up-tempo background music. -Turn off notifications from systems that are not necessary for the subjects.

[0053] On the other hand, examples of UI that lowers motivation include the following: -Use cool colors for the background of the measurement task display. -Display the elapsed time to the subject. - Noise while working on a measurement task -Turn on notifications for subjects and display unnecessary information.

[0054] At this time, for example, subject A with a high skill level (subject with Y>αX, where X is the skill level and Y is the difficulty level) will be in the state of point a of L2. Also, subject B with a low skill level (subject with Y<αX) will be in the state of point b of L2.

[0055] For subject A, who has a high skill level, the difficulty of the measurement task is lowered from point a, so that the rate of correct answers increases, and the subject is induced into the neutral state O. For subject B, who has a low skill level, the difficulty of the measurement task is increased from point b, so that the rate of correct answers decreases, and the subject is induced into the neutral state O.

[0056] In this way, by adjusting the UI when performing the measurement task and the difficulty of the measurement task, the subject is induced into a neutral internal state, as shown in Figure 3.

[0057] Next, a data structure used by the inner surface state estimating device will be described with reference to FIGS.

[0058] The subject test result table 200 is a table for storing the results of the measurement tasks for the subjects, and as shown in FIG. 5, has fields for subject ID 200a, test date and time 200b, question difficulty 200c, and test result 200d.

[0059] The subject ID 200a stores an ID that uniquely identifies the subject. The test date and time 200b stores the date and time when the test was administered in the format yyyy / mm / dd hh:mm. The question difficulty 200c stores the difficulty of the test, for example, the number of digits in a calculation question. The test result 200d stores the test result of the subject, for example, the percentage of correct answers ((number of correct answers / number of questions)×100%).

[0060] The UI profile table 210 is a table that stores the relationship between the results of measurement tasks when the UI is changed for each subject, and as shown in Figure 6, it has fields for subject ID 210a, subject name 210b, test date and time 210c, UI content 210d, and skill level increase / decrease 210e.

[0061] The subject ID 210a stores an ID that uniquely identifies the subject. The subject name 210b stores a character string representing the subject's name. The test date and time 210c stores the date and time when the test was conducted in the format of yyyy / mm / dd hh:mm. The UI content 210d stores the UI content when the subject worked on the measurement task as a character string or ID. The skill level increase / decrease 210e stores the increase / decrease in the skill level in the UI described in the UI content 210d (e.g., correct answer rate) as a signed integer of + when there is an increase, and as a signed integer of - when there is a decrease.

[0062] Next, the process performed by the inner surface state estimating device will be described with reference to FIGS.

[0063] First, a series of processes performed by the inner state estimating device, from UI profile generation to inner state estimation and associated display, will be described with reference to FIG. First, the inner surface state estimating device 100 generates a UI profile (S01). The UI profile generation process will be described in detail later.

[0064] Next, the inner surface state estimating device 100 adjusts the inner surface state (S02). The details of the inner surface condition adjustment process will be described later.

[0065] Next, the inner surface state estimating device 100 initializes the number of evaluations n (n←1) ​​(S03). Next, the inner state estimating device 100 acquires sensor data of the biological measurement of the subject taken by the sensor 10 (S04).

[0066] Next, the inner state estimating device 100 extracts feature quantities from the sensor data (S05). The sensor data is time-series data, and the feature quantities include the mean, variance, frequency components, and the like.

[0067] Next, the inner state estimation device 100 estimates the inner state of the subject according to the extracted feature amounts and an inner state estimation model (S06). The inner state estimation model is assumed to have undergone machine learning to associate the feature amounts with the inner state in advance.

[0068] In this embodiment, the inner state is represented by four states, Flow, Anxious, Lathargic, and Relaxed, expressed as percentages according to the inner state model based on the flow theory by Mihaly Csikszentmihalyi shown in Fig. 3. Here, the result of estimating the inner state of the subject according to the feature quantity extracted the nth time and the inner state estimation model is that the values ​​for each of the four states, Flow, Anxious, Lathargic, and Relaxed, are f n , a n , l n , r n Assume that f n +a n +l n +r n =1.

[0069] Next, the inner state estimation device 100 notifies the subject of the result of the estimation of the inner state of the subject in the n-th evaluation (f n , a n , l n , r n ) is multiplied by 100 to display a percentage, and a questionnaire screen for conducting a questionnaire regarding them is displayed (S07). Note that the user interface screen at this time will be described later with reference to FIG.

[0070] Next, the inner state estimating device 100 performs an estimation that incorporates the n-th inner state questionnaire evaluation, and calculates a numerical value that indicates the possibility of each inner state, as follows (S08).

[0071] The possibility of each inner surface state in the n-th evaluation is calculated using the following (Equation 1) to (Equation 4).

[0072]

Number

[0073]

Number

[0074] Here, F n , A n , L n , R n are parameters representing the n-th four states of Flow, Anxious, Lathargy, and Relax respectively. PF n , PA n , PL n , PR n are evaluations that are normalized to sum to 1 as the estimated results incorporating the subjectivity of the inner state at the n-th time.

[0075] (In Equation (1), when n ≥ 2, it is the sum of the respective parameters up to i < n in the past. However, when n = 1, it follows Equation (2) as the initial value. The 1 / 2 of the coefficient in Equation (1) is the decay coefficient. That is, the measurement results closer to the present are emphasized more than the past measurement results for evaluation. D i is a coefficient for incorporating the evaluation of the subject's questionnaire as shown in Equation (3). When the subjective evaluation is positive (in agreement), D i > 1, and when it is negative, D i < 1.

[0076] PF n , PA n , PL n , PR n are obtained from F n , A n , L n , R n in Equation (1) according to Equation (4).

[0077] Next, the inner state estimating device 100 displays the estimation result of the inner state incorporating the questionnaire evaluation of the nth inner state (S09). The user interface screen for displaying the estimation result of the inner state incorporating the subjective evaluation of the nth inner state will be described later with reference to FIG.

[0078] Next, the inner surface state estimating device 100 increments the number of evaluations n (n←n+1) (S10).

[0079] Next, the inner state estimating device 100 determines whether the number of evaluations n is greater than the upper limit N of the number of evaluations (S11). If the number of evaluations n is equal to or less than the upper limit N of the number of evaluations (S11: NO), the process returns to S04, and if the number of evaluations n is greater than the upper limit N of the number of evaluations (S11: YES), the process ends. Here, the upper limit N of the number of evaluations is a positive integer value that is predetermined as the upper limit number of evaluations of the inner state.

[0080] Next, the UI profile generation process will be described with reference to FIG. This process corresponds to S01 in FIG. First, the inner state estimating device 100 issues a measurement task for generating a UI profile (S100).

[0081] Next, the inner state estimating device 100 measures the skill level (S101). In this embodiment, measuring the skill level means finding the percentage of correct answers to calculation problems.

[0082] Next, the inner state estimating device 100 determines whether or not measurements have been performed for the necessary UI (S102), and if measurements have been performed for the necessary UI (S102: YES), ends the processing, and if measurements have not been performed for the necessary UI (S102: NO), proceeds to S103.

[0083] Next, the inner state estimating device 100 changes the UI (S103), and measures the skill level again (S104).

[0084] Then, compare the results of the previous skill level measurement and the current skill level measurement (S105). When there is no change in the skill level (S105: NO), return to S103. When there is a change in the skill level (S105: YES), store the result (UP or DOWN) in the UI profile table 210 (S106), and return to S102.

[0085] Next, the inner surface state adjustment process will be described with reference to FIG. 9. This is the process corresponding to S02 in FIG. 7. First, the inner surface state estimation device 100 presents a measurement task (S200). In this embodiment, it was a calculation problem characterized by the number of digits in terms of difficulty.

[0086] Next, the inner surface state estimation device 100 measures the skill level (correct answer rate) of the subject for the presented measurement task (S201).

[0087] Next, the inner surface state estimation device 100 stores the measurement result in the subject test result table 200 shown in FIG. 5 (S202).

[0088] Next, the inner surface state estimation device 100 determines whether the measurement result of the subject is on the neutral state line (S203). When it is on the neutral state line (S203: YES), the process ends. When it is not on the neutral state line (S203: NO), proceed to S204.

[0089] Here, being on the neutral state line means that in the example of FIG. 4, when L2: Y = αX, with a certain width ε, when the measurement result of the subject is the skill level X and the difficulty Y, it is determined that αX - ε < Y < αX + ε.

[0090] Next, the inner surface state estimation device 100 adjusts the UI according to the content of the subject test result table 200 and the UI profile table 210 shown in FIG. 6 so that the skill level X for the difficulty Y is on the neutral state line.

[0091] Next, the inner surface state estimation device 100 determines whether or not it has reached the neutral state as a result of the adjustment (S205). When it has reached the neutral state (S205: YES), the process ends. When it is not in the neutral state (S205: NO), the difficulty level of the measurement task is adjusted so as to approach the neutral state (S206).

[0092] Here, reaching the neutral state is defined as -δ < X < +δ and -δ < Y < +δ when the skill level is X and the difficulty level is Y with respect to the width δ around the origin in the example of FIG. 4. The adjustment of the difficulty level of the measurement task in S206 is performed until -δ < Y < +δ is reached.

[0093] Next, the difficulty level / skill level questionnaire evaluation and the inner surface state result consistency display process will be described with reference to FIG. 10. The activation timing of this process is assumed to be when, after the inner surface state estimation device 100 has estimated the inner surface state for the nth time, a display request is received from the subject. The specific activation interface will be described later with reference to FIG. 12.

[0094] First, the inner surface state estimation device 100 conducts a questionnaire on the difficulty level / skill level in the most recent operation for the subject (S300). The user interface for the questionnaire on the difficulty level / skill level in the most recent operation will be described later with reference to FIG. 13.

[0095] Next, the inner surface state estimation device 100 calculates the degree of consistency between the estimated result of the current inner surface state and the subjective evaluation based on the questionnaire on the difficulty level / skill level in the most recent operation (S301).

[0096] At this time, the degree of consistency is obtained as follows.

[0097] Here, when the estimated result of the current inner surface state is (PF n , PA n , PL n , PR n ), mapping this to the coordinate system shown in FIG. 3 and setting the coordinates as (X ob , Y ob ), when (Xob ,Y ob )=(PF n + PR n -PA n -PL n ,PF n + PA n -PR n -PL n ) where -1≦X ob ,Y ob ≦+1.

[0098] The subjects' responses to the questionnaire were also mapped onto the coordinate system shown in Figure 3, and the coordinates were expressed as (X sub ,Y sub ) and when the subject answered "yes" (positive answer) to the question "Was the most recent task difficult?" in a questionnaire, D n = 1, and if the answer is "No" (negative answer), D n =-1, for neutral answers, D n = 0, and similarly, when the question "Was the most recent skill work high?" is answered "Yes" (positive answer), S n = 1, and if the answer is "No" (negative answer), S n =-1, and for a neutral answer, S n = 0, (X sub ,Y sub )=(S n ,D n )

[0099] And, (X ob ,Y ob ) and (X sub ,Y sub ) is taken as L, and is calculated using the following (Equation 5).

[0100]

number

[0101] From this, the degree of agreement c between the current estimation result of the inner state and the most recent questionnaire evaluation of difficulty and skill level is calculated using the following (Equation 6).

[0102]

number

[0103] For example, (PF n , P.A. n , P.L. n , P.R. n )=(0.48, 0.2, 0.07, 0.25), and a questionnaire given to the subjects shows that they answer "yes" to the question of whether their most recent work was difficult, and answer "yes" to the question of whether their most recent skill level was high.

[0104] In this case, (X ob ,Y ob )=(0.46,0.36), (X sub ,Y sub )=(1,1), L2Norm: L=0.831, and agreement c=0.408.

[0105] Next, the degree of agreement between the current estimation result of the inner state and the most recent questionnaire evaluation of the difficulty level and skill level is displayed (S303).

[0106] The user interface screen for displaying the current estimation result of the inner state and the degree of agreement with the most recent difficulty / skill level questionnaire evaluation will be described later with reference to FIG.

[0107] Next, a user interface screen provided to a subject by the inner state estimating device will be described with reference to Figs.

[0108] As shown in FIG. 11, the inner state result questionnaire screen 400 comprises the objects of an inner state estimation numerical value display field 401, an inner state estimation summary display field 402, an inner state estimation result questionnaire statement display field 403, a yes / no input button 404, and an OK button 405.

[0109] The estimated inner state value display field 401 is a field that indicates the estimated ratio of the inner state, and shows the value of the inner state estimated from the feature quantities extracted n times for Flow, Anxious, Lathargic, and Relaxed. n , a n , l n , r n Each of these values ​​is multiplied by 100 and displayed as a percentage.

[0110] The inner state estimation summary display field 402 is a field that shows the state with the largest value in the inner state estimation numerical value display field 401. In FIG. 11, the numerical value for the flow state is the largest, so "inner state estimated from measurement: flow" is displayed. The inner state estimation result questionnaire display field 403 is a field that displays a message to be posed to the subject, and in FIG. 11, it displays "Does the estimation result on the left match your subjective evaluation?". The yes / no input button 404 is a toggle button that allows the subject to input their answer to the questionnaire, and they can select either yes or no. The OK button 405 is a button that is pressed when the subject has finished inputting and this screen is closed.

[0111] Screen 410 showing the result of estimation of the inner state incorporating the questionnaire evaluation is a screen that displays an estimate of the inner state estimated from features extracted from the sensor data, in other words, the subject's answers to the questionnaire to the objective evaluation, in other words, an estimate of the inner state incorporating a subjective evaluation, and is the screen displayed in S09 of FIG. 7.

[0112] As shown in FIG. 12 , the inner state estimation result screen 410 incorporating the questionnaire evaluation is made up of the following objects: an inner state estimation numerical value display field 411 incorporating the questionnaire evaluation, an inner state estimation summary display field 412 incorporating the questionnaire evaluation, a result graph display field 413, a degree of agreement display button 414, and an OK button 415.

[0113] The estimated inner state numerical value display field 411 incorporating the questionnaire evaluation is displayed as a PF representing Flow, Anxious, Lathargic, and Relaxation, which are calculated by (Formula 4). n , P.A. n , P.L. n , P.R. n Each of these values ​​is multiplied by 100 and displayed as a percentage.

[0114] The inner state estimation summary display field 412 incorporating the questionnaire evaluation is a field that shows the state with the largest value in the inner state estimation numerical value display field 411 incorporating the questionnaire evaluation.

[0115] The result graph display section 413 is a section that displays the numerical values ​​in the inner state estimated numerical value display section 411 incorporating the questionnaire evaluation as a pie chart.

[0116] The degree of agreement display button 414 is a button for starting the process of displaying the degree of agreement between the difficulty / skill level questionnaire evaluation and the inner state result, which has been described with reference to the flowchart of FIG.

[0117] An OK button 415 is pressed to close this screen.

[0118] The most recent difficulty / skill level questionnaire screen 420 is a screen for conducting a questionnaire regarding the difficulty and skill level in a task during the n-th evaluation of inner state estimation, and is a screen displayed in S300 of FIG.

[0119] As shown in FIG. 13, the most recent difficulty / skill level questionnaire screen 420 comprises the following objects: a difficulty level questionnaire display field 421, a difficulty: yes / no input button 422, a skill level questionnaire display field 423, and a skill level: yes / no input button 424.

[0120] The difficulty level questionnaire message display field 421 is a field for displaying a message of a questionnaire about the difficulty level of the most recent task. In the example of FIG. 13, "Was the difficulty level of the most recent task high?" is displayed.

[0121] The difficulty: yes / no input button 422 is a toggle button that allows the test subject to input an answer to the questionnaire in the message in the difficulty questionnaire display field 421, and allows the test subject to select either yes or no.

[0122] The skill level questionnaire message display field 423 is a field for displaying a message of a questionnaire about the skill level for the most recent task. In the example of FIG. 13, it displays "Was your skill level for the most recent task high?".

[0123] The object of the skill level: yes / no input button 424 is a toggle button that allows the test subject to input an answer to the questionnaire in the message in the skill level questionnaire display field 423, and allows the test subject to select either yes or no.

[0124] An OK button 425 is a button that is pressed when the subject has finished inputting information and this screen is to be closed.

[0125] Difficulty / skill level questionnaire evaluation and inner state result agreement display screen 430 is a screen that displays information regarding the agreement between the difficulty / skill level questionnaire evaluation and the inner state estimation result, and is a screen that is displayed in S302 of FIG. 10.

[0126] As shown in FIG. 14 , the difficulty / skill level questionnaire evaluation and inner state result agreement display screen comprises the following objects: an inner state estimation numerical value display field 431 incorporating the questionnaire evaluation; an inner state estimation summary display field 432 incorporating the questionnaire evaluation; an agreement coordinate display field 433; an agreement information display field 434; and an OK button 435.

[0127] Inner state estimated numerical value display field 431 incorporating the questionnaire evaluation is similar to inner state estimated numerical value display field 411 incorporating the questionnaire evaluation of FIG.

[0128] Inner state estimation summary display field 432 incorporating questionnaire evaluation is similar to inner state estimation numerical value display field 412 incorporating questionnaire evaluation in FIG.

[0129] The degree of match coordinate display field 433 is a coordinate system in which the horizontal axis represents the skill level and the vertical axis represents the difficulty level (X ob ,Y ob ), (X sub ,Y sub ) is plotted and displayed.

[0130] The matching degree information display field 434 displays the information about the matching degree (X ob ,Y ob ), (X sub ,Y sub ) and edit and display the degree of match c.

[0131] As described above, the inner state estimating device of this embodiment creates a profile according to the individual characteristics of the subject, and takes into account the difficulty level and skill level to guide the inner state toward a neutral state. By modifying the UI to suit each individual, it is expected that the inner state can be more effectively guided and the estimation accuracy can be improved when estimating the inner state using biometric data. Furthermore, by surveying the subject about the validity of the estimation results of their inner state, which is to say by incorporating the subject's subjective evaluation of their inner state, the discrepancy between the features of the sensor data and the objective evaluation based on the inner state estimation model becomes smaller, making it easier to obtain results that are satisfactory to the user.

[0132] Furthermore, by conducting a questionnaire survey about the difficulty and skill level of the most recent task, the degree of agreement between the subject's subjective evaluation of their inner state and the objective evaluation based on the feature quantities of the sensor data and the inner state estimation model can be calculated and visually displayed, making it easier for the subject to look back on their own subjective evaluation and clarifying the extent to which the objective evaluation is useful. [Explanation of symbols]

[0133] 1...Subject, 5...Network, 10...Sensor, 20: data collection terminal, 21: sensor data acquisition unit, 22: sensor data transmission unit, 100... inner state estimation device, 101... sensor data receiving unit, 110... feature extraction unit, 111... inner state estimation unit, 112... estimation result display unit, 120... measurement task setting unit, 121... difficulty level matching unit, 122... UI profile generation unit, 123... difficulty / UI adjustment unit, 130... questionnaire evaluation reflected inner state numerical value calculation unit, 131... questionnaire evaluation reflected inner state result display unit, 140... difficulty / skill level questionnaire evaluation and inner state result agreement calculation unit, 141... difficulty / skill level questionnaire evaluation and inner state result agreement calculation unit, 150... questionnaire input / output unit, 160... memory unit, 200...subject test result table, 210...UI profile table, 220...difficulty level / skill level statistical data, 230...subject sensor data, 240...feature amount data, 250...inner state estimation model data, 260...inner state estimation result data, 270...questionnaire result data, 280...inner state estimation data reflecting questionnaire evaluation, 290...difficulty level / skill level questionnaire evaluation and inner state result agreement data

Claims

1. An inner state estimation device that measures a subject while he or she is working, acquires sensor data from the subject, extracts feature amounts from the acquired sensor data, and estimates an inner state of the subject based on the feature amounts extracted from the sensor data and an inner state estimation model, comprising: Keeping statistical data on the neutral state of difficulty and skill level, The test subjects are given measurement tasks with a set level of difficulty and their skill levels are measured for each level of difficulty. an inner state estimating device that adjusts a UI (User Interface) when the subject works on a measurement task so as to approach a neutral state of difficulty and skill level defined in the statistical data, and sets the adjusted UI as a UI of the work environment.

2. 2. The inner state estimating device according to claim 1, wherein the difficulty of the measurement task is adjusted so as to approach a neutral state between the difficulty level and the skill level defined by the statistical data.

3. A UI profile table is maintained for each subject, which defines the relationship between a UI and a skill level when the subject performs a measurement task using the UI; 2. The inner surface state estimating device according to claim 1, wherein the UI of the subject is changed based on the UI profile table.

4. 2. The inner state estimating device according to claim 1, wherein the inner state is expressed as a psychological state represented in each quadrant of a two-dimensional coordinate system having difficulty on one axis and skill level on the other axis.

5. An inner state estimation device that measures a subject while he or she is working, acquires sensor data from the subject, extracts feature amounts from the acquired sensor data, and estimates an inner state of the subject based on the feature amounts extracted from the sensor data and an inner state estimation model, comprising: The subject's internal state is estimated for multiple tasks, an inner state estimation device which converts each of the estimated inner states into a numerical value and displays it, has the subject input a positive / negative questionnaire evaluation of the estimated results, increases the numerical value of the estimated inner state of the inner state when a positive evaluation is received, and decreases the numerical value of the estimated inner state of the inner state when a negative evaluation is received, thereby outputting a numerical value of the inner state based on the subject's positive / negative questionnaire evaluation of the estimated results.

6. 6. An inner state estimating device as claimed in claim 5, further comprising: an inner state estimation device for estimating the inner state of the subject in relation to a plurality of tasks; a numerical value for each estimated inner state for each task is converted into a value and displayed; a questionnaire response from the subject as to whether the subject is positive or negative about the estimated results is input; the numerical value of the estimated inner state for a task which was rated positive is increased, and the numerical value of the estimated inner state for a task which was rated negative is decreased, and a numerical value of each estimated inner state of the subject is output according to a sum of the numerical values ​​of each estimated inner state of the subject in relation to a plurality of tasks.

7. 7. The inner state estimating device according to claim 6, wherein, in a time series in which the subject's inner state is estimated for a plurality of tasks, the numerical values ​​of the inner state estimated as a result of estimating the past inner state are made small, and as time passes, the numerical values ​​of the inner state estimated as a result of estimating the inner state are comparatively made large.

8. 6. The inner state estimating device according to claim 5, wherein the inner state is expressed as a psychological state represented in each quadrant of a two-dimensional coordinate system having difficulty on one axis and skill level on the other axis.

9. 9. The inner state estimating device according to claim 8, wherein the subject is asked to enter the results of a questionnaire about the difficulty of the task and the skill level of the task, and the degree of agreement between the subject's inner state and numerical values ​​of each estimated inner state on the two-dimensional coordinate system is displayed.

10. An inner state estimation method in which an inner state estimation device estimates an inner state of a subject by measuring biometric data of the subject, the method comprising: a step in which the inner surface state estimation device measures a subject while the subject is performing an activity and acquires sensor data from the subject; a step of the inner surface state estimating device extracting feature amounts from acquired sensor data; a step of said inner state estimating device estimating an inner state of a subject based on features extracted from said sensor data and an inner state estimation model; a step in which the inner state estimating device estimates the inner state of the subject in relation to a plurality of tasks, and quantifies and displays each inner state as a result of estimating the inner state for each task; a step of having the subject input a questionnaire result indicating whether the subject agrees or disagrees with the result of the estimation by the inner state estimating device; a step of said inner state estimating device calculating a sum of the numerical values ​​of each inner state of the result of estimating the inner state of the subject for a plurality of times of the task, by increasing the numerical value of the inner state of the result of estimating the inner state for an instance in which a positive evaluation was given and decreasing the numerical value of the inner state of the result of estimating the inner state for an instance in which a negative evaluation was given; and outputting a numerical value of each inner state estimated from the inner state of the subject according to the sum of the numerical values ​​of each inner state resulting from estimating the inner state of the subject for a plurality of tasks.

11. 11. The inner state estimating method according to claim 10, wherein in a time series in which the subject's inner state is estimated for a plurality of tasks, the numerical values ​​of the inner state estimated in the past are made small, and as time passes, the numerical values ​​of the inner state estimated in the past are comparatively made large.

Images