Information processing device

Abstract

An information processing device according to the present disclosure comprises an estimation unit that estimates an emotional state from a biological signal at a predetermined time interval, and an emotion control unit that determines whether to perform intervention for changing the emotional state, on the basis of first data indicating the emotional state estimated by the estimation unit and second data obtained by accumulating the first data over a period longer than a prescribed time interval.

Description

Information processing device

[0001] The present disclosure relates to an information processing device.

[0002] When a person's emotions change, they are expressed on the body surface as physiological responses such as brain waves, heart rate, and sweating. To estimate a person's emotions, these physiological responses are read as biosignals by a sensor device, and feature quantities such as physiological indices that contribute to the emotional response are extracted through signal processing. The user's emotions are then estimated from the feature quantities using a model formula obtained through machine learning.

[0003] International Publication No. 2021 / 199271 International Publication No. 2022 / 124085

[0004] Although there are technologies for providing feedback to users on emotion estimation results (see, for example, Patent Documents 1 and 2), there is a need for the development of technologies that further expand the scope of use of emotion estimation.

[0005] Therefore, it is desirable to provide an information processing device that can expand the range of use of emotion estimation.

[0006] An information processing device according to one embodiment of the present disclosure includes an estimation unit that estimates an emotional state from a biological signal at a predetermined time interval, and an emotional control unit that determines whether to intervene to change the emotional state based on first data indicating the emotional state estimated by the estimation unit and second data obtained by accumulating the first data over a period longer than the predetermined time interval.

[0007] In an information processing device according to one embodiment of the present disclosure, a determination is made as to whether to intervene to change the emotional state based on first data indicating the emotional state estimated at a predetermined time interval and second data obtained by accumulating the first data over a period longer than the predetermined time interval.

[0008] FIG. 1 is an explanatory diagram illustrating an overview of Kahneman's attention allocation model. FIG. 2 is an explanatory diagram illustrating an overview of the vagal Tank theory. FIG. 3 is an explanatory diagram illustrating an overview of the vagal Tank theory. FIG. 4 is a block diagram schematically illustrating an example configuration of an information processing device according to an embodiment of the present disclosure. FIG. 5 is a flowchart illustrating an example operation of intervention timing by the information processing device according to an embodiment. FIG. 6 is an explanatory diagram illustrating a specific example of dynamic change of integral weights by an integral weight selection unit of the information processing device according to an embodiment. FIG. 7 is a flowchart illustrating an example operation of attenuation and reset of an integral value of arousal level change by the integral weight selection unit of the information processing device according to an embodiment. FIG. 8 is an explanatory diagram illustrating a specific example of a table for calculating an integral value adjustment gain g by the integral weight selection unit of the information processing device according to an embodiment. FIG. 9 is an explanatory diagram illustrating a specific example of a UI image displayed by a visualization unit in the information processing device according to an embodiment. FIG. 10 is an explanatory diagram illustrating a specific example of a UI image displayed by a visualization unit in the information processing device according to an embodiment. FIG. 11 is an explanatory diagram illustrating a specific example of a UI image displayed by a visualization unit in the information processing device according to an embodiment. FIG. 12 is an explanatory diagram showing a specific example of a UI image displayed by a visualization unit in an information processing device according to an embodiment. FIG. 13 is a flowchart showing a specific example of context state determination performed by a context state determination unit in an information processing device according to an embodiment. FIG. 14 is a block diagram schematically showing a configuration example of an information processing device according to a modified example. FIG. 15 is an explanatory diagram showing an image of calculation of a time integral associated with accumulation of long-term data in a fatigue state and a recovery state. FIG. 16 is a flowchart showing an example of an operation of intervention timing performed by an information processing device according to a modified example. FIG. 17 is a flowchart showing an example of an operation of attenuating and resetting an integral value of a wakefulness change performed by an integral weight selection unit in an information processing device according to a modified example. FIG. 18 is an explanatory diagram showing a specific example of a table for calculating an integral value adjustment gain g performed by an integral weight selection unit in an information processing device according to a modified example. FIG. 19 is an explanatory diagram showing an image of a method for calculating the consumption and recovery amounts of cognitive resources based on the ratio between the fatigue state and the recovery state.Fig. 20 is a conceptual diagram showing the estimation result of the instantaneous arousal level used for the intervention trigger and the cognitive resource that is the accumulation of the instantaneous arousal level. Fig. 21 is a conceptual diagram showing an example of calculation of the cognitive resource by time series analysis that takes into account information on the past instantaneous arousal levels.

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be made in the following order: 0. Comparative Example 1. One Embodiment 1.1 Configuration 1.2 Operation 1.3 Effects 2. Modification 2.0 Overview 2.1 Configuration 2.2 Operation 2.3 Effects 3. Other Embodiments

[0010] <0. Comparative Example> (Outline and Problems of an Information Processing Apparatus According to a Comparative Example) FIG. 1 shows an outline of an information processing apparatus according to a comparative example.

[0011] Changes in a person's emotions are expressed on the body surface as physiological responses such as brain waves, heart rate, and sweating. To estimate a person's emotions, these physiological responses are read as biosignals using a sensor device, and feature quantities such as physiological indicators that contribute to the emotional response are extracted through signal processing. The user's emotions are then estimated from these feature quantities using a model formula derived through machine learning. Emotions are classified along two axes: pleasant / unpleasant and arousal, as described in the following academic literature 1, for example. [Academic literature 1] Posner, Jonathan, James A. Russell, and Bradley S. Peterson. "The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology." Development and psychopathology 17.3 (2005): 715-734.

[0012] In application areas such as mental health, there is a high need to utilize the balance of arousal determined by the activity of the autonomic and central nervous systems to visualize cognitive resources, arousal resources, or fatigue state, or to use it as an intervention trigger to improve user performance and manage stress. However, existing methods mainly focus on estimating these states as final outputs, and mechanism-based visualization methods and intervention detection methods have not yet been established.

[0013] The relationship between arousal resources and performance has long been studied in neurobiology and psychology. One example is Kahneman's attention allocation model, described in the following academic literature 2: [Academic literature 2] Kahneman: Attention and effort. Prentice Hall (1973)

[0014] Figure 1 provides an overview of Kahneman's attention allocation model. This model is an approach that moves researchers away from filter theory by postulating that attention has a limited overall capacity and that a person's ability to perform multiple tasks simultaneously is determined by how much "capacity" each task requires. Kahneman's attention allocation model suggests that arousal resources have a "capacity" and suggests a relationship between short-term and long-term changes in arousal.

[0015] A second example is the vagal tank theory, described in the following academic literature 3, which aims to better understand the complexity and mechanisms of autoregulatory interactions. [Academic literature 3] Laborde, Sylvain, et al., "Vagal tank theory: the three Rs of cardiac vagal control functioning—resting, reactivity, and recovery." Frontiers in neuroscience 12 (2018): 458.

[0016] An outline of the vagus nerve tank theory is shown in Figures 2 and 3. Figure 2 shows the case where a stressful event occurs as an event, and Figure 3 shows the case where a relaxing event occurs as an event.

[0017] The vagus nerve tank theory represents the vagus nerve as a tank and models cognitive resources and fatigue states depending on the type of stressor. The vagus nerve tank theory classifies emotional changes resulting from the physiological response of a person's cardiac vagus nerve into instantaneous changes (Phasic) and gradual changes (Tonic). A person's resting state serves as the baseline, and the subsequent state is determined by external stimuli (Events). The vagus nerve tank theory uses the metaphor of a tank to model the balance between stress and relaxation, depending on whether the external stimulus is a stressful event or a relaxing event.

[0018] The vagus nerve tank theory models how the state of the vagus nerve tank changes after an event (post-event). As shown in Figure 2, when a stressful event occurs as an event from a resting state, the tank's resources decrease, and after the event (post-event), the tank's resources recover (increase). Also, as shown in Figure 3, when a relaxing event occurs as an event from a resting state, the tank's resources recover, and after the event (post-event), the tank's resources are maintained or increased or decreased.

[0019] These models are merely theoretical frameworks studied in the field of psychology, and have not been adequately applied to state estimation models such as arousal levels. When constructing a physiologically plausible emotion estimation model in a laboratory environment and applying it to real-world problems, the context dependency of human emotional physiological responses becomes an issue.

[0020] In addition, the following academic literature 4 is available regarding the context dependency of emotional physiological responses. Academic literature 4 evaluates the exercise context dependency of the stress reactivity effect based on the Vagal Tank theory. Academic literature 4 compares heart rate variability (HRV) over four days between resting in a laboratory and in a real environment. The results suggest that the resting and reactivity sensitivity in a real environment is lower than in a laboratory. It also suggests that the lower the exercise intensity, the higher the reactivity to stress events. In other words, this indicates that a person's physiological response is dependent on the behavioral context in the real environment. [Academic literature 4] Melanie Bamert, et al., "Stress and recovery measured with heart rate variability: Do the findings of laboratory research translate to daily life?", Society for Ambulatory Assessment 2021

[0021] In particular, when applying this technology to everyday life, it is necessary to consider the signal quality of biological signals, distinguish between mental and physical arousal, and optimize processing according to the context, but prior art did not take these factors into account.

[0022] For example, Patent Document 1 (International Publication No. 2021 / 199271) proposes a technology for providing stress relief intervention based on changes in a user's short-term stress and long-term stress. Short-term stress and long-term stress are calculated separately, and a flag for stress relief intervention is set based on a combination of short-term stress information and long-term stress information. However, the technology described in Patent Document 1 does not fully consider the direct relationship between short-term stress and long-term stress, and is limited in its ability to address the issue of context-dependence of human physiological responses in real environments.

[0023] Patent Document 2 (WO 2022 / 124085) proposes a technology that divides a user's state of arousal into three stages, determines which of the three stages the user's state of arousal is in within a certain time window, and generates feedback information for the user based on the determination result. The technology described in Patent Document 2 does not propose using both instantaneous arousal levels and accumulated long-term changes in arousal levels, or generating feedback information according to context. The technology described in Patent Document 2 has limitations in addressing the issue of context-dependence of emotional physiological responses in everyday and real environments.

[0024] If it were possible to accurately visualize the accumulation of temporal changes in arousal and relaxation as an indicator of cognitive resources and fatigue state and detect the timing of intervention, it would lead to an expansion of the scope of use of emotion estimation (arousal estimation), and it is expected that the UX (User Experience) when using applications would be improved.

[0025] 1. One Embodiment> [1.1 Configuration] FIG. 4 schematically illustrates an example configuration of an information processing device according to one embodiment of the present disclosure.

[0026] In response to the technical challenges described above, an information processing device according to one embodiment utilizes the results of instantaneous estimation of alertness, realizing functions such as flagging for intervention when performance declines and visualizing long-term stress balance for life log data.

[0027] The information processing device according to one embodiment includes an emotion estimation unit 10 and an intervention determination and visualization unit 20 .

[0028] An information processing device according to an embodiment may be configured as a computer including, for example, one or more central processing units (CPUs), one or more read-only memories (ROMs), and one or more random access memories (RAMs). In this case, the processing of each block in the information processing device according to an embodiment may be realized by one or more CPUs executing processing based on a program stored in one or more ROMs or RAMs. Furthermore, the processing of each block in the information processing device according to an embodiment may be realized by one or more CPUs executing processing based on a program supplied from an external source via, for example, a wired or wireless network.

[0029] The emotion estimation unit 10 is an estimation unit that estimates an emotional state at short time intervals (predetermined time intervals) from a vital signal (biological signal) of a user whose emotional state is to be estimated.

[0030] The emotion estimation unit 10 includes a sensor data acquisition unit 11 , a filter pre-processing unit 12 , a feature extraction unit 13 , an emotional state determination unit 14 , and a context state determination unit 15 .

[0031] The sensor device 30 includes a vital sensor (biometric sensor) and a physical sensor such as a motion sensor. The sensor device 30 may be one or more devices. The biometric sensor and the physical sensor may be mounted on a single device or on separate devices. The sensor device 30 may be a mobile device such as a smartphone equipped with a biometric sensor or a physical sensor, or various wearable devices. The physical sensor may include a global positioning system (GPS), an inertial measurement unit (IMU), a microphone, etc. The biometric sensor detects biosignals for estimating the user's biological activity, such as sweating, pulse wave, electromyography, blood pressure, blood flow, or body temperature. The emotion estimation unit 10 estimates the user's activity based on the biometric signal detected by the biometric sensor. This activity can be used to determine the user's concentration state, wakefulness, etc.

[0032] The sensor data acquiring unit 11 acquires sensor information from the biosensor and sensor information from the physical sensor of the sensor device 30. The sensor data acquiring unit 11 outputs the sensor information from the biosensor as vital information (biological information) to the filter pre-processing unit 12. The sensor data acquiring unit 11 also outputs the sensor information from the physical sensor as physical information to the context state determining unit 15. When a smartphone is included as the sensor device 30, the sensor data acquiring unit 11 may also output information such as the smartphone's application usage history to the context state determining unit 15 as physical information.

[0033] The filter pre-processing unit 12 performs pre-processing of the biological signal to remove unnecessary components. The filter pre-processing unit 12 performs pre-processing such as band-pass filtering and noise removal on the biological signal measured by the biological sensor of the sensor device 30. The filter pre-processing unit 12 outputs the pre-processed biological signal to the feature extraction unit 13.

[0034] For example, when the biological signal is an electroencephalogram (EEG), the EEG is measured by attaching electrodes to the scalp and measuring the brain's action potentials leaking through the scalp, skull, etc. One of the characteristics of EEG measured in this way is that it is known to have a very low signal-to-noise ratio. Therefore, it is necessary to remove signals of unnecessary frequency components from the EEG, which is a time-series signal, and a band-pass filter (e.g., a passband of 0.1 Hz to 40 Hz) is applied as the filter pre-processing unit 12.

[0035] Furthermore, body movement components due to human body movement are superimposed on the biosignal as artifacts (noise other than the target signal, etc.). To address this, signal processing techniques such as adaptive filtering and independent component analysis are applied to the filter pre-processing unit 12. Similarly, pre-processing is performed in the filter pre-processing unit 12 when the biosignal is emotional sweating (EDA), pulse wave (PPG), blood flow (LDF), etc.

[0036] The feature extraction unit 13 extracts, as a feature, a physiological response that contributes to an emotion from the preprocessed biosignal. The feature extraction unit 13 extracts a feature x=(x１ , x ２ , ..., x Ｎ The feature extraction unit 13 outputs the extracted feature x to the emotional state determination unit 14.

[0037] Specifically, the feature extraction unit 13 observes biological signals from vital sensors (biosensors) such as electroencephalogram (EEG), psychoacoustic dehydration (EDA), pulse wave (PPG), and blood flow (LDF) as time-series data, and extracts physiological indices that contribute to changes in emotions as feature x. Generally, feature x is extracted using an analysis window (sliding window) of about 5 minutes.

[0038] The feature x is not limited to a physiologically known feature. The feature extraction unit 13 can also perform signal processing to extract the feature x that contributes to emotion in a data-driven manner, for example, by deep learning or an autoencoder.

[0039] The emotional state determination unit 14 receives the feature x calculated by the feature extraction unit 13 as an input and determines the emotional state using a machine learning model. The emotional state determination unit 14 analyzes the pattern of the time-series data of the feature x calculated by the feature extraction unit 13 and recognizes the emotional state of the user. The emotional state determination unit 14 identifies a predicted label of the time-series emotional state using a discrimination model, which is a machine learning model constructed in advance, and calculates Y ｋ = (y １ , y ２ , ..., y ｋ ) and label it as

[0040] The context state determination unit 15 determines the situation (context) of a user whose emotional state is to be estimated based on physical information acquired from a physical sensor of the sensor device 30. The context state determination unit 15 recognizes what kind of behavior the user holding the sensor device 30 is engaged in based on physical information (movement information) acquired as sensor information from the physical sensor of the sensor device 30. The function of the context state determination unit 15 can be realized, for example, by pattern recognition of waveform data from a motion sensor, which is a physical sensor, to determine the type of movement or the type of vehicle, but the realization method is not limited thereto. When the sensor device 30 includes a smartphone, the context state determination unit 15 may also output information such as the smartphone's application usage history as context information.

[0041] The intervention assessment and visualization unit 20 corresponds to an emotion control unit in an information processing device according to one embodiment. The intervention assessment and visualization unit 20 determines whether to perform an intervention to change the emotional state based on first data indicating the emotional state estimated by the emotion estimation unit 10 and long-term data (second data) obtained by accumulating the first data over a period longer than a predetermined time interval. Here, the first data includes instantaneous arousal levels. The intervention assessment and visualization unit 20 accumulates the long-term data of the emotion estimation results and performs intervention assessment and visualization.

[0042] The intervention determination and visualization unit 20 has a long-term data accumulation unit 21, a processing block 22, an integral weight selection unit 23, and an integral parameter storage unit 24. The processing block 22 has an intervention flag determination unit 25 and a visualization unit 26.

[0043] The long-term data accumulation unit 21 accumulates long-term data of the emotion estimation determination results and performs weighted time integration of the long-term data. The long-term data accumulation unit 21 calculates the consumption of cognitive resources or the remaining amount of cognitive resources by performing time integration of instantaneous changes in arousal. The long-term data accumulation unit 21 buffers samples of the emotional state determined by the emotional state determination unit 14 at each time and analyzes them as long-term data changes in arousal. Specifically, the long-term data accumulation unit 21 performs weighted time integration as shown in the following equation (A), where a high arousal state is defined as stress and a low arousal state is defined as recovery. The following equation (A) formulates arousal resources based on the psychological model shown in Figures 1 to 3 above.

[0044]

[0045] where MF(t) represents the consumption of cognitive resources. ｓｔｒｅｓｓ , w ｒｅｃｏｖｅｒｙ indicates the integral weight. ｓｔｒｅｓｓ (t) is a case where stress is judged at time t and takes a positive value. ｒｅｃｏｖｅｒｙ (t) indicates a case where recovery is judged at time t and takes a negative value. Specific examples of cases where recovery is judged include listening to relaxing music and stress relief activities (meditation, deep breathing, etc.). B(t=0) indicates the initial value of cognitive resource consumption (amount of cognitive resource consumption at time t=0).

[0046] Here, MF(t) is used as an example of the consumption of cognitive resources, but the remaining amount of cognitive resources may also be formulated using the above concept. ｓｔｒｅｓｓ (t) and P ｒｅｃｏｖｅｒｙ The sign of (t) is reversed.

[0047] The integral parameter storage unit 24 stores a table for calculating the integral value adjustment gain g, as shown in FIG. 8, which will be described later.

[0048] The integral weight selection unit 23 is a weight selection unit that dynamically changes the weight of the time integral by the long-term data accumulation unit 21 in accordance with the determination result of the context state determination unit 15. As a result, the long-term data accumulation unit 21 dynamically changes the integral weight in accordance with the determination result of the context state determined by the context state determination unit 15. By dynamically changing the weight of the time integral by the integral weight selection unit 23 (for example, by reducing the weight during exercise), it is possible to distinguish between motor arousal and mental arousal.

[0049] In addition, the integral weight selection unit 23 calculates an integral value adjustment gain g for dynamically attenuating or resetting the integral value of the time integral by the long-term data accumulation unit 21 based on the table stored in the integral parameter storage unit 24, depending on the judgment result of the context state judgment unit 15.

[0050] The intervention flag determination unit 25 detects the timing of intervention with the user. The intervention flag determination unit 25 predicts a decline in the user's performance based on the instantaneous emotional state calculated by the emotional state determination unit 14 and the consumption of arousal resources taking into account long-term changes in arousal calculated by the long-term data accumulation unit 21, and determines a stress release intervention flag. Here, intervention refers to encouraging the user to take stress release action. Stress release action refers to, for example, playing music suitable for stress release.

[0051] 5 , the intervention determination and visualization unit 20 performs stress release intervention when the context state determination unit 15 determines that a situation in which concentration is preferable and determines that the instantaneous arousal level is less than a threshold value. Also, when a situation in which concentration is preferable is determined that the instantaneous arousal level is equal to or greater than a threshold value and the consumption of cognitive resources is equal to or greater than a certain level, the intervention determination and visualization unit 20 performs stress release intervention.

[0052] 9 to 12, the visualization unit 26 visualizes the instantaneous arousal level as the first data and the long-term data as the second data. The visualization unit 26 feeds back the determination result of the intervention flag determination unit 25 to the user as recommendations for stress relief application experiences and a visualized log of the emotional state and the state of the arousal level resources.

[0053] [1.2 Operation] FIG. 5 is a flowchart showing an example of an operation of the information processing device according to one embodiment regarding intervention timing.

[0054] The intervention assessment and visualization unit 20 determines whether the situation is suitable for the user to release stress, based on the determination result of the context state. First, the intervention assessment and visualization unit 20 determines, for example, whether the situation is suitable for concentration, based on the context (step S101). For example, in the example of an office environment, a situation suitable for concentration would be a meeting. If it is determined that the situation is not suitable for concentration (step S101; N), the situation is not suitable for stress release, so the intervention assessment and visualization unit 20 does not set a flag (step S105).

[0055] On the other hand, if it is determined that the situation is favorable for concentration (step S101; Y), the intervention determination and visualization unit 20 then determines whether the instantaneous arousal level is equal to or greater than a threshold (step S102). If it is determined that the instantaneous arousal level calculated by the emotional state determination unit 14 is less than the threshold (step S102; N), it determines that the situation is favorable for concentration but the subject is not aroused (is unable to concentrate), and the intervention determination and visualization unit 20 sets an intervention flag indicating that it is time to release stress (step S106).

[0056] Furthermore, if the instantaneous level of arousal is determined to be equal to or greater than a threshold (step S102; Y), the intervention assessment and visualization unit 20 then determines whether the consumed cognitive resources are equal to or greater than a certain level (step S103). If the consumed cognitive resources are determined to be less than the certain level (step S103; N), the intervention assessment and visualization unit 20 determines that the level of arousal is high and that sufficient cognitive resources are secured, and does not set a flag (step S107). If the consumed cognitive resources are determined to be equal to or greater than a certain level (step S103; Y), the intervention assessment and visualization unit 20 determines that the level of arousal is high but that sufficient cognitive resources are not secured (i.e., maintaining concentration is difficult), and sets an intervention flag indicating that it is time to release stress (step S104).

[0057] FIG. 6 shows the integral weight (w in equation (A)) selected by the integral weight selection unit 23 of the information processing device according to the embodiment. ｓｔｒｅｓｓ , w ｒｅｃｏｖｅｒｙ ) is an explanatory diagram showing a specific example of dynamic change.

[0058] The integral weight selection unit 23 may dynamically change the integral weight depending on the context. The integral weight selection unit 23 may dynamically change the weight of the time integral to distinguish between motor arousal and mental arousal. The integral weight selection unit 23 may dynamically change the weight of the time integral to correct the experience value in mental arousal according to the situation and external stimuli. FIG. 6 shows an example in which the context states include office work, exercise, rest, and sleep. In an example of distinguishing between motor arousal and mental arousal, the integral weight selection unit 23 may deal with this by reducing the integral weight during exercise. Furthermore, in states such as rest and sleep, the user's state gradually recovers, but the instantaneous state may be determined to be stress. In such cases, since the arousal resource recovers, the integral weight selection unit 23 may select the weight w ｓｔｒｅｓｓ The increase or decrease of the integrated value of the change in alertness is controlled by setting a negative value as . Similarly, when the instantaneous alertness state is determined to be recovered, the positive or negative relationship of the integrated value of the change in alertness is determined.

[0059] Fig. 7 is a flowchart showing an example of the operation of attenuating and resetting the integral value of the wakefulness change by the integral weight selection unit 23 of the information processing device according to one embodiment. Fig. 8 is an explanatory diagram showing a specific example of a table for calculating the integral value adjustment gain g by the integral weight selection unit 23. Fig. 8 shows an example in which the context states include office work, exercise, rest, and sleep.

[0060] The integral weight selection unit 23 may dynamically reset the integral value of the arousal level change depending on the context. This allows the intervention flag determination unit 25 to set an intervention flag by focusing on a specific time period or context state. The integral weight selection unit 23 dynamically considers the influence of past arousal resource values ​​when a person's context state significantly changes. Specifically, the integral weight selection unit 23 first determines whether there is a transition in the context state (step S201). If it is determined that there is no transition in the context state (step S201; N), the integral weight selection unit 23 continues the integration without attenuating or resetting the integral value of the arousal level change (step S202). On the other hand, if it is determined that there is a transition in the context state (step S201; Y), the integral weight selection unit 23 attenuates or resets the integral value of the arousal level change depending on the state transition (step S203). When a context state transition occurs, the integral weight selection unit 23 performs a process of attenuating or resetting the integral value of the change in wakefulness by referring to a table previously stored in the integral parameter storage unit 24 for the integral value adjustment gain g based on the immediately preceding specific context state and the current context state. This enables the intervention flag determination unit 25 at the subsequent stage to set a flag focusing on a specific time period or context state. MF(t) in the above formula (A) is expressed as the following formula (B) using the adjusted integral value adjustment gain g: MF ｕｐｄａｔｅｄ (t)=g*MF(t)...(B)

[0061] 9 to 12 show specific examples of UI (User Interface) images by the visualization unit 26 in the information processing device according to one embodiment.

[0062] In the example of FIG. 9 , the emotional state for one day is visualized in a pie chart. In the example of FIG. 9 , the emotional state is divided into three stages (low, mid, high)—stress state (high arousal state, fatigue state) and recovery state (low arousal state)—and the ratio of the stress state to the recovery state for one day is visualized in a pie chart. The example of FIG. 9 also visualizes the total time spent in a stress state for one day (18 hours in the example of FIG. 9 ). The stress state and the recovery state may be displayed in different colors for easy identification. For example, the stress state may be displayed in reddish colors, and the recovery state in blued colors. Furthermore, the color intensity may be changed depending on the stage (low, mid, high) of the stress state and the recovery state.

[0063] 10 and 11 , the remaining amount of cognitive resources at any given time (e.g., the present) is visualized as a meter. FIG. 10 shows an example in which the remaining amount of cognitive resources is low, and FIG. 11 shows an example in which the remaining amount of cognitive resources is high. Note that the meter display color may be different depending on whether the remaining amount is low or high. For example, a low remaining amount may be displayed in red, and a high remaining amount may be displayed in blue.

[0064] In the example of Figure 12, emotional states are visualized in a time series. In the example of Figure 12, time series data for the stress state (high arousal state) and the recovered state (low arousal state) as emotional states are visualized in a time series bar graph. Also, time series data for the remaining amount of cognitive resources is visualized in a time series line graph. Note that the stress state and the recovered state may be displayed in different colors for easy understanding. For example, the stress state may be displayed in reddish colors, and the recovered state in blued colors.

[0065] FIG. 13 is a flowchart showing a specific example of a context state determination performed by the context state determination unit 15 of the information processing device according to an embodiment.

[0066] The context state determination unit 15 determines the activity state of the user from physical information (movement information) acquired by the sensor device 30 (a mobile device such as a smartphone, various wearable devices, etc.). The context state determination unit 15 recognizes what kind of activity the user holding the sensor device 30 is engaged in from sensor information from a physical sensor mounted on the sensor device 30. The context state determination unit 15 performs pattern recognition on waveform data from a motion sensor when the sensor device 30 is, for example, a mobile device carried in a pocket or bag, and determines the type of movement and the type of vehicle. The context state determination unit 15 may determine various modes of transportation such as bicycle, train, car, and elevator, in addition to movements such as walking and running, without using positioning information.

[0067] First, the context state determination unit 15 acquires sensor data from the sensor device 30 (step S301). Next, the context state determination unit 15 extracts features from the sensor data from the sensor device 30 (step S302). Next, the context state determination unit 15 performs pattern recognition using machine learning (step S303). Next, the context state determination unit 15 determines movement and vehicle as behavior recognition results (step S304).

[0068] [1.3 Effects] As described above, an information processing device according to one embodiment determines whether to intervene to change an emotional state based on first data indicating an emotional state estimated at a predetermined time interval and second data obtained by accumulating the first data for a period longer than the predetermined time interval. This makes it possible to provide an information processing device that can expand the scope of use of emotion estimation.

[0069] Furthermore, according to an information processing device according to an embodiment, an intervention flag is detected by separating the instantaneous state of the arousal level estimation from the weighted time integral, thereby taking into account the context dependency of the emotional physiological response and determining whether to intervene in consideration of the arousal resources in addition to the instantaneous arousal level of the user, thereby improving the user experience and performance.

[0070] Furthermore, according to an information processing device according to an embodiment, by dynamically changing the weight of the integrated value of the arousal level change according to the context, it is possible to distinguish between physical arousal and mental arousal, and it is also possible to correct the experience value for mental arousal according to the situation and external stimuli, thereby improving the user experience.

[0071] In addition, according to one embodiment of the information processing device, the integrated value of the change in alertness can be dynamically attenuated or reset depending on the context, thereby improving the user experience by focusing on a specific time period or context state and setting an intervention flag.

[0072] The effects described in this specification are merely examples and are not limiting, and other effects may also be achieved. The same applies to the effects of the following modifications and other embodiments.

[0073] 2. Modifications An information processing device according to a modification will be described below. Note that, in the following, parts that are substantially the same as the components of the information processing device according to the above embodiment will be given the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0074] [2.0 Overview] The emotion estimation results can be used in a variety of ways, one of which is to provide feedback to the user. The technology according to the above-described embodiment proposes, as an example of using the emotion estimation results for feedback, a technology for visualizing the fatigue state in past daily life based on physiological mechanisms and using the visualization as an intervention trigger.

[0075] The technology according to the embodiment described above estimates a person's emotional state by dividing their arousal level into an instantaneous state and a weighted time integral, and then detects an intervention flag when they are in a fatigued state. This approach visualizes the fatigue state and triggers intervention when performance declines. However, the technology according to the embodiment described above does not define an accurate degree of recovery (recovery state) when fatigue is recovered from an action involving relaxation (recovery action). Therefore, no estimation is performed to reflect the recovery state in cognitive resources (time integral) indicating the fatigue state. When a state transition involving a recovery action occurs, the time integral is simply decayed or reset based on context information, and the accurate internal state at the time of fatigue recovery is not reflected in cognitive resources (time integral).

[0076] Therefore, if it were possible to accurately define and visualize the degree of recovery based on a person's internal state when fatigue recovery occurs due to a person's recovery behavior and estimate how this recovery is reflected in cognitive resources (time integral), it would be possible to more accurately visualize the fatigue state when performance declines even after fatigue recovery and detect the timing of intervention. This would be expected to further improve the UX experience and expand applications. Furthermore, while the technology according to the above embodiment proposes the time integral of instantaneous arousal as a method for calculating cognitive resources, it is expected that the estimation of emotional states would be more accurate if various other calculation methods could be used.

[0077] This modification includes the following technical differences from the technology according to the above embodiment. Difference Point 1: While the technology according to the above embodiment only defined and visualized a fatigue state due to sustained wakefulness (a state in which a decline in the user's performance is expected and a state in which it has become difficult to maintain concentration), this modification also defines and visualizes a recovery state (a state in which a recovery in the user's performance is expected), which has a different time constant from a fatigue state. For example, the level of wakefulness during a fatigue state due to sustained wakefulness is used as a reference, and a decrease in wakefulness from that reference is defined as a recovery index, and the recovery state is visualized.

[0078] Difference point 2: For cognitive resources in fatigue and recovery states, it is now possible to use a calculation method that uses the ratio of the time integral of the fatigue and recovery states to a set standard, rather than just the time integral of instantaneous alertness. It also makes it possible to use a calculation method based on time series analysis that takes into account past information on the time series of instantaneous alertness in fatigue and recovery states.

[0079] -Difference point 3: By utilizing the newly defined and calculated recovery amount of cognitive resources in the recovery state, dynamic correction (decrease and reset of fatigue state due to recovery) of the cognitive resource consumption amount in the previous fatigue state can be achieved with high precision.

[0080] - Difference point 4: Based on the newly defined and calculated amount of cognitive resource recovery in the recovery state, a decision is made as to whether or not to intervene to change the state when fully recovered (for example, to encourage the timing of returning to work from rest).

[0081] 2.1 Configuration

[0082] FIG. 14 shows a schematic configuration example of an information processing device according to a modified example.

[0083] The information processing device according to the embodiment ( FIG. 4 ) utilizes the results of instantaneous estimation of wakefulness to implement functions for flagging intervention when performance declines and visualizing long-term stress balance for life log data. In this modified example, the results of instantaneous estimation of wakefulness are utilized to implement functions for flagging intervention not only when performance declines but also when performance recovers. Furthermore, in this modified example, a function for visualizing not only fatigue state but also recovery state is implemented.

[0084] In this modification, the emotion estimation unit 10 may have a configuration substantially similar to that of the information processing device according to the above embodiment ( FIG. 4 ). On the other hand, the components of the intervention assessment and visualization unit 20 are substantially similar to those of the information processing device according to the above embodiment ( FIG. 4 ), but the functions of some of the components are different.

[0085] In this modification, the long-term data accumulation unit 21 calculates the consumption or remaining amount of cognitive resources by integrating instantaneous changes in alertness over time, as in the information processing device according to the above embodiment. Furthermore, in this modification, the long-term data accumulation unit 21 calculates the recovery amount of cognitive resources. For example, the long-term data accumulation unit 21 defines a recovery index as a decrease in alertness from a reference level of alertness during a fatigued state due to sustained alertness. For example, the long-term data accumulation unit 21 calculates the recovery amount of cognitive resources by integrating the decrease in alertness from a reference level (a reference level of alertness) determined as a reference level (a reference level of alertness) when fatigue accumulates and performance declines. Specifically, the long-term data accumulation unit 21 performs time integration of instantaneous changes in alertness in stressful states, and time integration of the decrease in alertness in relaxed states. The context state determination unit 15 is used to determine whether a state is stressful or relaxed.

[0086] FIG. 15 shows an image of calculation of the time integral accompanying the accumulation of long-term data in the fatigue state and the recovery state.

[0087] In this modification, the long-term data storage unit 21 calculates the time integral of instantaneous arousal during stressful tasks as the cognitive resource consumption, indicating a state of fatigue, and the time integral of the instantaneous decrease in arousal during relaxation-related activities as the cognitive resource recovery, indicating a state of recovery. The intervention flag determination unit 25 detects the timing of intervention for the user. The intervention flag determination unit 25 predicts the user's performance decline and performance recovery based on the instantaneous emotional state calculated by the emotional state determination unit 14 and the arousal resources (cognitive resource consumption and recovery) calculated by the long-term data storage unit 21, taking into account the long-term arousal change, and determines the stress release intervention flag and the task restart intervention flag. Here, the two interventions refer to encouraging the user to engage in stress release behavior and encouraging them to engage in behavior that requires refocusing. Examples of stress release behavior include playing music suitable for stress release. Examples of behavior that requires concentration include desk work, meetings, and other tasks that require concentration.

[0088] In this modified example, as shown in the operation example of Figure 16 described later, when the context state determination unit 15 determines that a change in behavior from behavior involving concentration to behavior involving relaxation is being implemented, and when it determines that the amount of cognitive resources recovered is equal to or greater than a certain level, the intervention determination / visualization unit 20 performs an intervention to encourage behavior involving concentration (task restart intervention).

[0089] In this modification, the visualization unit 26 visualizes instantaneous alertness as the first data and long-term data as the second data, similarly to the information processing device according to the above embodiment. Furthermore, the visualization unit 26 also visualizes the fatigue state and recovery state determined based on the first data and the second data. The visualization unit 26 provides feedback to the user based on the determination result of the intervention flag determination unit 25, not only in the form of a recommendation for a stress relief application experience, but also in the form of a recommendation for returning to work after sufficient relaxation, and a visualization log of the emotional state and the state of alertness resources.

[0090] [2.2 Operation] FIG. 16 is a flowchart showing an example of an operation of the information processing device according to this modification regarding intervention timing.

[0091] The intervention assessment and visualization unit 20 determines, for example, whether the situation is suitable for the user to restart a task, based on the determination result of the context state. First, the intervention assessment and visualization unit 20 determines, for example, whether the situation is one in which a relaxing activity is being performed, based on the context (step S301). For example, in an office environment, a situation in which a relaxing activity is being performed would be a situation in which the user is sitting and resting, listening to music or relaxing sounds, etc. If it is determined that the situation is not one in which a relaxing activity is being performed (step S301; N), the situation is not one in which a task should be restarted, and the intervention assessment and visualization unit 20 does not set a flag (step S304).

[0092] On the other hand, if it is determined that a situation is being carried out in which behavior involving relaxation is being performed (step S301; Y), the intervention assessment / visualization unit 20 then determines whether the amount of cognitive resource recovery is above a certain level (step S302).

[0093] If it is determined that the recovery amount of cognitive resources is less than a certain amount (step S302; N), the intervention assessment and visualization unit 20 determines that the recovery is insufficient and does not set a flag (step S305). On the other hand, if it is determined that the recovery amount of cognitive resources is equal to or greater than a certain amount (step S302; Y), the intervention assessment and visualization unit 20 determines that the recovery is sufficient and that performance recovery is expected, and sets an intervention flag indicating that it is time to restart the task (step S303).

[0094] The integral parameter storage unit 24 stores a table or the like for calculating the integral adjustment gain g. The integral here means the consumption (time integral) of cognitive resources in a fatigued state, and the integral can be adjusted by g.

[0095] The integral weight selection unit 23 calculates an integral value adjustment gain g for dynamically attenuating or resetting the integral value of the consumption (time integral) of cognitive resources by the long-term data accumulation unit 21 based on the table stored in the integral parameter storage unit 24, in response to feedback of not only the judgment result of the context state judgment unit 15 but also the judgment result of the intervention flag judgment unit 25.

[0096] Fig. 17 is a flowchart showing an example of the operation of attenuating and resetting the integral value of the wakefulness change by the integral weight selection unit 23 of the information processing device according to this modification. Fig. 18 is an explanatory diagram showing a specific example of a table for calculating the integral value adjustment gain g by the integral weight selection unit 23 of the information processing device according to this modification. Fig. 18 shows an example in which there are multiple states associated with recovery, including recovery 1, recovery 2, recovery 3, and recovery 4. Recovery 4 indicates a state in which the degree of recovery (amount of recovery) is greater than that of recovery 1.

[0097] When a person's context state changes, especially when they perform a behavior involving relaxation (a recovery behavior), the integral weight selection unit 23 can consider the influence of not only the context information but also the actual internal state. Specifically, the integral weight selection unit 23 first determines whether or not a behavior involving relaxation is being performed (step S401). If it is determined that a behavior involving relaxation is not being performed (step S401; N), the long-term data accumulation unit 21 does not calculate the recovery amount of cognitive resources associated with recovery (step S405).

[0098] On the other hand, if it is determined that an action involving relaxation is being performed (step S401; Y), the long-term data accumulation unit 21 then calculates the amount of cognitive resource recovery associated with recovery (step S402). Next, the integral weight selection unit 23 determines whether there is a state transition from relaxation to task (step S403). If the integral weight selection unit 23 determines that relaxation has ended and that there has not been a transition to task (step S403; N), the long-term data accumulation unit 21 continues calculating the amount of cognitive resource recovery associated with recovery (step S406).

[0099] On the other hand, if the integral weight selection unit 23 determines that relaxation has ended and the transition to a task has occurred (step S403; Y), the integral weight selection unit 23 attenuates or resets the amount of cognitive resource consumption (the integral value of the change in alertness in the fatigued state) previously consumed in accordance with the amount of cognitive resource recovery associated with recovery calculated up to that point (step S404). When there is a state transition from a relaxed state to a task state, the integral weight selection unit 23 references the table stored in the integral parameter storage unit 24 and selects an appropriate integral value adjustment gain g based on the amount of cognitive resource recovery associated with recovery from the relaxed state. This allows the integral weight selection unit 23 to perform a process of attenuating or resetting the amount of cognitive resource consumption (integral value). This enables the intervention flag determination unit 25, which is provided later, to set a flag with higher accuracy, taking into account the internal state associated with recovery.

[0100] (Variations of Cognitive Resource Calculation) In the above explanation, a method for calculating cognitive resources in the fatigue state and recovery state by time integration has been proposed, but it is expected that state estimation will be more accurate by using various calculation methods other than this. For example, a calculation method using the ratio of the time integral of the fatigue state and recovery state to a set standard, or a calculation method using time series analysis that takes into account past information on the time series of the fatigue state and recovery state, can be used.

[0101] FIG. 19 shows an image of a method for calculating the consumption and recovery of cognitive resources based on the ratio between the fatigue state and the recovery state.

[0102] For example, the long-term data accumulation unit 21 may calculate the cognitive resource consumption as a ratio to the cognitive resource consumption when the instantaneous alertness is a first standard indicating a fatigued state. Regarding the fatigued state, for example, if the instantaneous alertness at time t0, when fatigue accumulates and performance declines, is defined as the alertness standard yha (first standard), and the alertness standard yha is always maintained during the task, the maximum total cognitive resource consumption is expressed by Equation (1). On the other hand, the cognitive resource consumption according to the transition of the actual instantaneous alertness f(t) is expressed by Equation (2). By using the ratio between Equation (1) and Equation (2), the cognitive resource consumption can be calculated as a ratio, as shown in Equation (3). Similarly, the remaining cognitive resource can be calculated as a ratio.

[0103]

[0104] Furthermore, the long-term data accumulation unit 21 may calculate the recovery amount of cognitive resources as a ratio to the recovery amount of cognitive resources when the instantaneous alertness is a second standard indicating a recovered state. Regarding the recovered state, for example, if the instantaneous alertness at time t0, when fatigue accumulates and performance declines, is defined as the alertness state standard yha, and the state when the subject is most relaxed is defined as the relaxed state standard yla (second standard), then if the relaxed state standard yla is always maintained, the maximum total recovery amount of cognitive resources will be expressed as Equation (4). On the other hand, the recovery amount of cognitive resources according to the change in actual instantaneous alertness f(t) will be expressed as Equation (5). By using the ratio between Equation (4) and Equation (5), the recovery amount of cognitive resources can be calculated as a ratio, as shown in Equation (6).

[0105]

[0106] Fig. 20 is a conceptual diagram showing the estimation result of the instantaneous arousal level (short-term arousal) used for the intervention trigger and the cognitive resource (long-term arousal) which is the accumulation of the instantaneous arousal level. Fig. 21 is a conceptual diagram showing an example of calculating the cognitive resource by time series analysis taking into account information on past instantaneous arousal levels. The long-term arousal (cognitive resource) is an estimation result obtained by accumulating the estimation result of the instantaneous arousal state over a longer period of time.

[0107] The long-term data accumulation unit 21 may accumulate time-series data on instantaneous arousal levels and calculate the consumption or remaining amount of cognitive resources and the recovery amount of cognitive resources through time-series analysis that takes into account past information on the time-series data on instantaneous arousal levels. Examples of time-series analysis that takes into account past information on the time-series data include deep neural networks (DNNs), recurrent neural networks (RNNs), long short-term memories (LSTMs), and transformers. Specifically, for example, cognitive resources that reflect past information on instantaneous arousal levels are calculated as shown in FIG. 21 .

[0108] [2.3 Effects] As described above, the information processing device according to this modification can visualize a recovery state having a different time constant from a fatigue state. Furthermore, it is possible to realize intervention of behavioral changes due to recovery, thereby improving the accuracy of intervention of behavioral changes due to fatigue. Furthermore, it is possible to calculate the fatigue state and recovery state not only by a calculation method using time integration, but also by a multifaceted method, thereby making it possible to more accurately estimate the state.

[0109] 3. Other Embodiments The technology according to the present disclosure is not limited to the description of the above embodiment, and various modifications are possible.

[0110] For example, the present technology may be configured as follows. According to the present technology configured as follows, a determination is made as to whether to intervene to change the emotional state based on first data indicating an emotional state estimated at a predetermined time interval and second data obtained by accumulating the first data for a period longer than the predetermined time interval. This makes it possible to provide an information processing device that can expand the scope of use of emotion estimation.

[0111] (1) An information processing device comprising: an estimation unit that estimates an emotional state from a biosignal at predetermined time intervals; and an emotion control unit that determines whether to intervene to change the emotional state based on first data indicating the emotional state estimated by the estimation unit and second data obtained by accumulating the first data for a period longer than the predetermined time interval. (2) The information processing device according to (1), wherein the emotion control unit has a visualization unit that visualizes the first data and the second data. (3) The information processing device according to (1) or (2), wherein the emotion control unit has a storage unit that accumulates the second data and performs weighted time integration of the second data. (4) The information processing device according to (3), wherein the estimation unit has a context state determination unit that determines a situation of a user whose emotional state is to be estimated based on physical information, and the emotion control unit has a weight selection unit that dynamically changes a weight for the time integration by the storage unit depending on a determination result of the context state determination unit. (5) The information processing device according to (4), wherein the weight selection unit calculates an integral value adjustment gain for dynamically attenuating or resetting the integral value of the time integration by the accumulation unit according to the determination result of the context state determination unit. (6) The information processing device according to (4), wherein the emotion control unit further has an intervention flag determination unit that determines whether to intervene to change the emotional state, and the weight selection unit calculates an integral value adjustment gain for dynamically attenuating or resetting the integral value of the time integration by the accumulation unit according to the determination result of the context state determination unit and the determination result of the intervention flag determination unit. (7) The information processing device according to any one of (4) to (6), wherein the first data includes instantaneous arousal, and the accumulation unit calculates the consumption of cognitive resources or the remaining amount of the cognitive resources by performing time integration of the change in the instantaneous arousal.(8) The information processing device according to (7), wherein the emotional control unit performs the intervention when the context state determination unit determines that the situation is favorable for concentration and determines that the instantaneous level of arousal is less than a threshold, or when the instantaneous level of arousal is equal to or greater than a threshold and the amount of cognitive resource consumption is equal to or greater than a certain level. (9) The information processing device according to (7), wherein the storage unit further calculates a recovery amount of the cognitive resource. (10) The information processing device according to (9), wherein the storage unit dynamically corrects the amount of cognitive resource consumption based on the recovery amount of the cognitive resource. (11) The information processing device according to (9) or (10), wherein the emotional control unit performs the intervention to encourage behavior involving concentration when the context state determination unit determines that the situation is one in which a behavior change from behavior involving concentration to behavior involving relaxation is being implemented and determines that the amount of cognitive resource recovery is equal to or greater than a certain level. (12) The information processing device according to any one of (9) to (11), wherein the accumulation unit calculates the amount of recovery of the cognitive resources by using an instantaneous level of alertness during continuous wakefulness as a reference and integrating the decrease from the reference level over time. (13) The information processing device according to any one of (9) to (12), wherein the accumulation unit calculates the amount of consumption of the cognitive resources as a ratio to the amount of consumption of the cognitive resources when the instantaneous level of alertness is a first reference level indicating a fatigued state, and calculates the amount of recovery of the cognitive resources as a ratio to the amount of recovery of the cognitive resources when the instantaneous level of alertness is a second reference level indicating a recovered state. (14) The information processing device according to (1) or (2), wherein the first data includes instantaneous level of alertness, and the emotion control unit has a accumulation unit that accumulates time-series data of the instantaneous level of alertness and calculates the amount of consumption of cognitive resources or the remaining amount of cognitive resources and the amount of recovery of the cognitive resources by time-series analysis taking into account past information on the time series of the instantaneous level of alertness.(15) The information processing device according to any one of (2) to (14), wherein the visualization unit visualizes a fatigue state and a recovery state determined based on the first data and the second data. (16) The information processing device according to any one of (1) to (15), wherein the estimation unit has: a preprocessing unit that performs preprocessing on the biological signal to remove unnecessary components, an extraction unit that extracts, from the preprocessed biological signal, a physiological response that contributes to emotion as a feature amount, and an emotional state determination unit that determines the emotional state based on the feature amount.

[0112] This application claims priority based on Japanese Patent Application No. 2023-158518 filed on September 22, 2023, and Japanese Patent Application No. 2024-58249 filed on March 29, 2024, in the Japan Patent Office, the entire contents of which are incorporated herein by reference.

[0113] Those skilled in the art will recognize that various modifications, combinations, subcombinations, and variations may occur depending on design requirements and other factors, and are intended to be within the scope of the appended claims and their equivalents.

Claims

1. An information processing device comprising: an estimation unit that estimates an emotional state from a biological signal at a predetermined time interval; and an emotion control unit that determines whether or not to intervene to change the emotional state based on first data indicating the emotional state estimated by the estimation unit and second data obtained by accumulating the first data for a period longer than the predetermined time interval.

2. The information processing device according to claim 1, wherein the emotion control unit has a visualization unit that visualizes the first data and the second data.

3. The information processing device according to claim 1, wherein the emotion control section has a storage section that stores the second data and performs weighted time integration of the second data.

4. The information processing device of claim 3, wherein the estimation unit has a context state determination unit that determines the situation of a user whose emotional state is to be estimated based on physical information, and the emotion control unit has a weight selection unit that dynamically changes the weight of the time integration by the accumulation unit depending on the determination result of the context state determination unit.

5. The information processing device according to claim 4, wherein the weight selection section calculates an integral value adjustment gain for dynamically attenuating or resetting the integral value of the time integration by the accumulation section according to the determination result of the context state determination section.

6. The information processing device of claim 4, wherein the emotion control unit further has an intervention flag determination unit that determines whether or not to intervene to change the emotional state, and the weight selection unit calculates an integral value adjustment gain for dynamically attenuating or resetting the integral value of the time integration by the accumulation unit depending on the determination result of the context state determination unit and the determination result of the intervention flag determination unit.

7. The information processing device according to claim 4, wherein the first data includes an instantaneous level of alertness, and the storage unit calculates the amount of cognitive resource consumption or the remaining amount of cognitive resource by performing a time integration of the change in the instantaneous level of alertness.

8. The information processing device of claim 7, wherein the emotional control unit performs the intervention when the context state determination unit determines that a situation is favorable for concentration, when the instantaneous level of alertness is determined to be below a threshold, or when the instantaneous level of alertness is determined to be equal to or greater than a threshold and the amount of cognitive resource consumption is determined to be equal to or greater than a certain level.

9. The information processing device according to claim 7, wherein the accumulation unit further calculates the recovery amount of the cognitive resources.

10. The information processing device according to claim 9, wherein the accumulation unit dynamically corrects the amount of cognitive resource consumption based on the amount of cognitive resource recovery.

11. The information processing device of claim 9, wherein the emotional control unit performs an intervention to encourage behavior involving concentration as the intervention when the context state determination unit determines that a change in behavior is being implemented from behavior involving concentration to behavior involving relaxation and when the amount of recovery of the cognitive resources is determined to be equal to or greater than a certain level.

12. The information processing device according to claim 9, wherein the accumulation unit calculates the recovery amount of the cognitive resources by using the instantaneous level of alertness during sustained alertness as a standard and integrating the decrease from the standard over time.

13. The information processing device of claim 9, wherein the accumulation unit calculates the amount of cognitive resource consumption as a ratio to the amount of cognitive resource consumption when the instantaneous level of alertness is a first criterion indicating a fatigued state, and calculates the amount of cognitive resource recovery as a ratio to the amount of cognitive resource recovery when the instantaneous level of alertness is a second criterion indicating a recovered state.

14. The information processing device of claim 1, wherein the first data includes an instantaneous level of alertness, and the emotional control unit has a storage unit that accumulates time series data of the instantaneous level of alertness and calculates the amount of cognitive resource consumption, or the remaining amount of cognitive resource, and the amount of cognitive resource recovery by time series analysis taking into account past information on the time series of the instantaneous level of alertness.

15. The information processing device according to claim 2, wherein the visualization unit visualizes a fatigue state and a recovery state determined based on the first data and the second data.

16. The information processing device according to claim 1, wherein the estimation unit has: a pre-processing unit that performs pre-processing on the bio-signal to remove unnecessary components; an extraction unit that extracts physiological responses that contribute to emotions from the bio-signal after the pre-processing as features; and an emotional state determination unit that determines the emotional state based on the features.