Information processing device and method

Abstract

The present disclosure relates to an information processing device and method that make it possible to suppress an increase in measurement time of a biometric signal while suppressing a reduction in estimation accuracy of an affective state. According to the present disclosure: a feature amount of a biometric signal of a user is derived; a baseline feature amount in a reference state of the user is estimated by using the derived feature amount and a derivation function that reflects the circadian rhythm pertaining to the feature amount; the derived feature amount is normalized by using the estimated baseline feature amount; and the affective state of the user is determined by using the normalized feature amount. The present disclosure can be applied to, for example, an information processing device, an electronic apparatus, an information processing method, a program, or the like.

Description

Information processing device and method

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that can suppress an increase in measurement time for a biological signal while suppressing a decrease in estimation accuracy of an emotional state.

[0002] Conventionally, a method has been used to detect a user's physiological responses as biosignals using sensor devices and estimate the user's affect from the resulting features. To achieve a highly accurate emotion estimation model that takes into account individual differences in emotional physiological responses, it is important to reduce the individual differences in the baseline state of human emotional physiological responses. Human physiological responses have a circadian rhythm with a cycle of approximately one day. Understanding these physiological and neurological characteristics can address individual differences in human physiological responses and improve the accuracy of emotion estimation.

[0003] Therefore, a method has been proposed in which pulse waves are measured and buffered, and the circadian rhythm of the pulse waves is estimated (see, for example, Patent Document 1). A method of estimating a baseline state using the circadian rhythm estimated by such a method is conceivable.

[0004] Patent No. 7228820

[0005] However, the method described in Patent Literature 1 requires continuous measurement of biological signals such as pulse waves over a long period of time in order to estimate circadian rhythms with sufficient accuracy. A decrease in the accuracy of the circadian rhythm estimation may result in a decrease in the accuracy of the baseline state estimation performed using the circadian rhythm. Furthermore, a decrease in the accuracy of the baseline state estimation may result in a decrease in the accuracy of the emotional state estimation performed using the baseline state. In other words, there is a risk that the measurement time of biological signals may be increased in order to prevent a decrease in the accuracy of the emotional state estimation.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in the measurement time of a biological signal while suppressing a decrease in the estimation accuracy of an emotional state.

[0007] An information processing device according to one aspect of the present technology is an information processing device including: a feature derivation unit that derives features of a biosignal of a user; a baseline feature estimation unit that estimates baseline features, which are features of the user in a reference state, using the derived features and a derivation function that reflects a circadian rhythm related to the features; a feature normalization unit that normalizes the derived features using the estimated baseline features; and an emotional state determination unit that determines an emotional state of the user using the normalized features.

[0008] An information processing method according to one aspect of the present technology is an information processing method that derives features of a biosignal of a user, estimates baseline features that are features of a reference state of the user using the derived features and a derived function that reflects a circadian rhythm related to the features, normalizes the derived features using the estimated baseline features, and determines an emotional state of the user using the normalized features.

[0009] In an information processing device and method according to one aspect of the present technology, features of a user's biosignal are derived, and baseline features, which are features of the user's reference state, are estimated using the derived features and a derivative function that reflects a circadian rhythm related to the features. The estimated baseline features are then used to normalize the derived features, and the normalized features are used to determine the user's emotional state.

[0010] FIG. 1 is a diagram showing an example of a Russell annular model. FIG. 2 is a diagram showing an example of baseline feature estimation using circadian rhythm. FIG. 3 is a block diagram showing an example of the main configuration of an emotion estimation device. FIG. 4 is a block diagram showing an example of the main configuration of a baseline feature derivation unit. FIG. 5 is a diagram for explaining an example of data selection. FIG. 6 is a diagram for explaining an example of how baseline feature estimation is performed. FIG. 7 is a block diagram showing an example of the main configuration of a derived function generation device. FIG. 8 is a diagram showing an example of how a derived function is generated. A flowchart showing an example of the flow of emotion estimation processing. A flowchart showing an example of the flow of baseline feature derivation processing. A block diagram showing an example of the main configuration of an emotion estimation device. A flowchart showing an example of the flow of emotion estimation processing. FIG. 9 is a block diagram showing an example of the main configuration of a computer.

[0011] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be made in the following order: 1. Literature supporting technical content and technical terminology 2. Estimation of emotional state 3. First embodiment (emotion estimation device) 4. Second embodiment (emotion estimation device) 5. Supplementary notes

[0012] <1. Literature, etc. supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents and patent documents that were publicly known at the time of filing, as well as the content of other documents referenced in the following non-patent documents and patent documents.

[0013] Non-patent document 1: Posner, Jonathan, James A. Russell, and Bradley S. Peterson. "The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology." Development and psychopathology 17.3 (2005): 715-734. Non-patent document 2: Laborde, Sylvain, et al., "Heart rate variability and cardiac vagal tone in psychophysiological research-recommendations for experiment planning, data analysis, and data reporting." Frontiers in psychology 8 (2017): 213. Non-patent document 3: Laborde, Sylvain, et al., "Vagal tank theory: the three Rs of cardiac vagal control functioning-resting, reactivity, and recovery." Frontiers in neuroscience 12 (2018): 458. Non-patent document 4: Aston-Jones, Gary, et al. "A neural circuit for circadian regulation of arousal." Nature neuroscience 4.7 (2001): 732-738.

[0014] Patent Document 1: JP 2010-227029 A Patent Document 2: JP 2022-515374 A Patent Document 3: (mentioned above)

[0015] In other words, the contents of the above-mentioned non-patent documents and patent documents, as well as the contents of other documents referenced in the above-mentioned non-patent documents and patent documents, are also used as the basis for determining support requirements.

[0016] <2. Estimation of Emotional State> <Biological Signal Measurement Time and Emotion Estimation Accuracy> Conventionally, there have been methods that detect a user's physiological responses, etc., as biological signals using a sensor device and estimate the user's affect from the resulting feature values. Changes in a person's affect are expressed on the body surface as physiological responses such as brain waves, heart rate, and sweating. Non-Patent Document 1 discloses a method that reads these physiological responses as biological signals using a sensor device, extracts feature values ​​such as physiological indices that contribute to the emotional response through signal processing, and estimates the user's affect from the feature values ​​using a model formula obtained through machine learning. Emotions are classified, for example, along two axes: pleasant / unpleasant and arousal, as in the Russell ring model shown in Figure 1.

[0017] In order to realize a highly accurate emotion estimation model that takes into account individual differences in emotional physiological responses, it is important to reduce individual differences in the baseline state of human emotional physiological responses. Human physiological responses, as described in Non-Patent Documents 2 to 4, for example, have a sleep-wake rhythm, which is regulated to an approximately daily rhythm by the body clock, similar to the autonomic nervous system, endocrine hormone system, immune and metabolic system, etc., such as body temperature. Such rhythms of human physiological responses with an approximately daily period are also called circadian rhythms.

[0018] When estimating emotions in a real environment, particularly arousal level estimation such as stress state estimation, concentration / relaxation estimation, etc., it is possible to improve the accuracy of handling individual differences in human physiological responses by understanding the physiological and neurological characteristics as shown in Non-Patent Documents 2 to 4. In particular, to capture the subtleties of norms of high and low arousal levels depending on the application and to achieve high emotion estimation accuracy, the system is required to control changes in sensitivity of physiological responses due to behavioral contexts as shown in the above-mentioned Non-Patent Documents.

[0019] As a method of utilizing such circadian rhythms, Patent Document 1 discloses a method for detecting a phase shift of a biological rhythm from the expression level of the Gm129 gene at a predetermined time. Patent Document 2 discloses a method for estimating a circadian phase using a central sleep time. Patent Document 3 discloses a method for estimating a diurnal phase from changes over time in pulse waves (heart rate, pulse rate, etc.).

[0020] As described above, Patent Documents 2 and 3 disclose methods for estimating circadian rhythms. However, both methods are based on the premise that biological signals, such as pulse waves, are continuously measured over a long period of time using a wristband or a chest-mounted sensor. Therefore, reducing the measurement time of the biological signals may reduce the accuracy of the circadian rhythm estimation. A reduction in the accuracy of the circadian rhythm estimation may reduce the accuracy of the baseline state estimation performed using that circadian rhythm. Furthermore, a reduction in the accuracy of the baseline state estimation may reduce the accuracy of the emotional state estimation performed using that baseline state. In other words, reducing the measurement time of the biological signals may reduce the accuracy of the emotional state estimation.

[0021] In other words, to estimate the circadian rhythm with sufficient accuracy, it was necessary to continuously measure the pulse wave for a long period of time, which meant that there was a risk of increasing the measurement time of the biological signal in order to prevent a decrease in the accuracy of the estimation of the emotional state.

[0022] 3. First Embodiment Estimation of Baseline Feature Amount Using Derivation Function Therefore, a derivative function is used to estimate a baseline feature amount from the feature amount of a biological signal measured over a relatively short period of time, and the baseline feature amount is used to estimate an emotion (affect).

[0023] Here, emotion basically refers to feelings related to instinctive desires such as appetite and sexual desire. For example, emotion includes parameters such as arousal level, which indicates the user's level of arousal, and the user's pleasant or unpleasant feelings. For example, arousal level may include concentration level, which indicates the user's mental concentration level. Arousal level may also include stress level, which indicates the level of stress felt by the user. Arousal level may also include relaxation level, which indicates the user's level of relaxation. For example, heart rate, pulse rate, etc. may be used as parameters indicating arousal level.

[0024] The biosignal refers to a signal emitted from within the body due to a user's biological reaction (e.g., a biological phenomenon such as heart rate, brain waves, pulse rate, breathing, or sweating), or a signal obtained by detecting such a biological reaction. For example, the biosignal may include a photoplethysmogram (PPG) measured by photoplethysmography.

[0025] A feature refers to the amount of a feature corresponding to a biosignal that contributes to emotion estimation. In other words, a feature is information derived from a biosignal and used to estimate an emotion. For example, the feature may include a parameter value directly included in the biosignal. The feature may also include a statistical value of the biosignal (or a portion of information included in the biosignal). The feature may also include other parameter values ​​related to the biosignal (or a portion of information included in the biosignal) (i.e., other parameter values ​​derivable from the biosignal). The feature may be singular or plural. In other words, it may be a collection of feature values ​​(also referred to as a feature vector).

[0026] The baseline feature indicates the feature when the user is in a reference state. The reference state indicates a predetermined state in the circadian rhythm (the user's emotional state during a predetermined time period). For example, the user's emotional state during a time period considered to be when the user wakes up (e.g., around 6:00 AM) may be the reference state. In this specification, the user's emotional state during a time period from around 6:00 AM to around 8:00 AM will be described as the reference state. In other words, the baseline feature indicates the feature of that time period in the circadian rhythm.

[0027] The derivative function is a function for deriving a baseline feature from a feature of a biosignal measured over a relatively short period of time. This derivative function is based on (reflects) the circadian rhythm of the feature. This derivative function is generated in advance (before use). This derivative function may be generated by any method. For example, this derivative function may be derived using a set of pairs of baseline feature and feature of a biosignal measured over a relatively long period of time. For example, this derivative function may be derived by the least squares method using the set of pairs. This derivative function may be any function. For example, it may be a linear function, or a quadratic or higher order function. Furthermore, this derivative function may be table information indicating the correspondence between the feature and the baseline feature. Note that this derivative function may be generated for each detection time period of the biosignal. For example, a biosignal may be detected during a certain time period of a day (hereinafter also referred to as a detection time period), and the baseline feature may be derived using the feature of the biosignal during that detection time period. In this case, a common derivation function may be applied to derive the baseline feature regardless of the detection time period, or a derivation function created for the detection time period may be applied.

[0028] In other words, features are derived from the user's biosignals measured over a relatively short period of time, and baseline features, which are features of the user's reference state, are estimated using the derived features and a derivative function that reflects the circadian rhythm related to the features.The estimated baseline features are then used to normalize the derived features, and the normalized features are used to determine the user's emotional state.

[0029] For example, suppose the circadian rhythm of relaxation level is as shown in the graph in Figure 2. In other words, in the graph in Figure 2, the horizontal axis represents time, and the vertical axis represents relaxation level. The relaxation level fluctuates throughout the day, as shown by curve 21. Dotted lines 21A and 21B indicate the fluctuation range of the relaxation level (maximum and minimum values ​​at each time). This relaxation level may be inversely proportional to, for example, alertness or heart rate. In other words, in Figure 2, the lower the position along the vertical axis, the lower the state of relaxation (= high heart rate = high alertness). In other words, the higher the position along the vertical axis, the higher the state of relaxation (= low heart rate = low alertness).

[0030] A derivative function that reflects such a circadian rhythm is applied, and the feature (level of relaxation) for the time period indicated by square 23, i.e., the baseline feature, is estimated from the feature (level of relaxation) detected for the time period indicated by square 22.

[0031] In this way, by deriving baseline features using a derivative function that reflects circadian rhythms and using these baseline features, more accurate baseline features can be derived using features of biosignals measured over a relatively short period of time. Therefore, it is possible to suppress an increase in the measurement time of biosignals while suppressing a decrease in the estimation accuracy of emotional states. In other words, it is possible to suppress a decrease in the estimation accuracy of emotional states while suppressing an increase in the measurement time of biosignals.

[0032] If it becomes possible to estimate physiological indices in a baseline state without requiring users to wear sensor devices for long periods of time, it will be possible to improve the accuracy of emotion estimation in a variety of everyday situations, and we can expect to see an expansion of applications using emotion estimation.

[0033] For example, when detecting biosignals using a battery-powered device such as TWS (True Wireless Stereo) (wireless earphones, speakers, etc.), the shorter the measurement time (operating time), the better, since such devices have a limit on the continuous operating time. In other words, shortening the measurement time relaxes the conditions required for measurement, making it possible to measure biosignals using a wider variety of devices.

[0034] In this specification, superscripts are represented by "^". For example, "A^B" indicates that B is a superscript attached to A. Subscripts are represented by "_". For example, "A_B" indicates that B is a subscript attached to A. Estimated values ​​are represented by "*". For example, "A*" indicates an estimated A (the estimated value of A).

[0035] <Emotion Estimation Device> Fig. 3 is a block diagram showing an example of the configuration of an emotion estimation device, which is one aspect of an information processing device to which the present technology is applied. The emotion estimation device 100 shown in Fig. 3 is a device that estimates an emotional state of a user based on detected biosignals of the user, etc. For example, the emotion estimation device 100 may acquire the biosignals and movement detection results of the user.

[0036] The biosignal may be, for example, a signal detected by a vital sensor (biosensor) mounted on the mobile device. The biosignal may include any information. For example, it may include a photoplethysmogram (PPG) (i.e., heart rate, pulse rate, etc.), an electroencephalogram (EEG), or other information. The movement detection result may be, for example, an output signal from an inertial measurement unit (IMU). The movement detection result may include an output signal from an acceleration sensor (ACC). The movement detection result may include an output signal from an angular velocity sensor. The movement detection result may also be an output signal from a motion sensor. For example, the movement detection result may include a result of analyzing the movement of the user captured using a camera or the like.

[0037] Furthermore, the emotion estimation device 100 may use the information to determine (estimate) an emotional state and output information indicating the determined (estimated) emotional state. For example, emotional state parameters may be output as the information indicating the emotional state. The emotional state parameters include any parameters related to the emotional state. In other words, the emotional state parameters indicate the estimation result of the emotional state. For example, the emotional state parameters may include an estimated arousal level (concentration level, stress level, relaxation level, etc.).

[0038] Fig. 3 shows the main processing units, data flows, etc., but is not limited to what is shown in Fig. 3. In other words, in the emotion estimation device 100, there may be processing units not shown as blocks in Fig. 3, or there may be processing or data flows not shown as arrows or the like in Fig. 3.

[0039] As shown in FIG. 3 , the emotion estimation device 100 includes a feature deriving unit 111, a signal quality determining unit 112, an attachment determining unit 113, an activity determining unit 114, a baseline feature deriving unit 115, a feature normalizing unit 116, a normalization coefficient holding unit 117, an emotional state determining unit 118, and a model parameter holding unit 119.

[0040] The feature derivation unit 111 executes processing related to the derivation of features. For example, the feature derivation unit 111 may acquire a biosignal input to the emotion estimation device 100. The biosignal may be detected (measured) over a relatively short period of time. For example, the biosignal may be a signal detected during a detection time period Tk. The feature derivation unit 111 may derive a feature x as a model input variable for estimating an emotional state based on the biosignal. That is, the feature derivation unit 111 may derive a feature of the user's biosignal. For example, the feature derivation unit 111 may derive multiple feature values ​​(x_1, x_1, ..., x_N) based on the biosignal and derive a set of the feature values ​​x (a feature vector X (=(x_1, x_1, ..., x_N))). In the following description, it is assumed that the feature derivation unit 111 derives multiple feature values ​​x (i.e., a feature vector X). In the following description, the feature vector X of the biological signal detected in the detection time period Tk is also referred to as the feature vector X_Tk. The feature deriving unit 111 may supply the derived feature vector X_Tk to the feature normalizing unit 116. The feature deriving unit 111 may supply the derived feature vector X_Tk to the baseline feature deriving unit 115.

[0041] The signal quality determination unit 112 executes processing related to determination of the quality of a biosignal. For example, the signal quality determination unit 112 may acquire a biosignal input to the emotion estimation device 100. This biosignal may be detected (measured) over a relatively short period of time. For example, this biosignal may be a signal detected during a detection time period Tk. The signal quality determination unit 112 may determine the quality of the biosignal. Any method may be used to determine this signal quality. For example, the signal quality determination unit may determine the quality of the biosignal based on noise components contained in the biosignal. For example, the signal quality determination unit 112 may calculate a signal quality score (SQE (Supplier Quality Engineering) class). The signal quality determination unit 112 may supply the quality determination result (SQE class) to the wearing determination unit 113. At that time, the signal quality determination unit 112 may also supply the acquired biosignal to the wearing determination unit 113. Note that the signal quality determination unit 112 may be omitted.

[0042] The wearing determination unit 113 performs processing related to determining the wearing state of the sensor device that detects the biosignal. For example, the wearing determination unit 113 may acquire a quality determination result (or the quality determination result and the biosignal corresponding to the quality determination result) supplied from the signal quality determination unit 112. The wearing determination unit 113 may determine the wearing state of the sensor device that detects the biosignal by the user. For example, the wearing determination unit 113 may determine the wearing state based on the quality of the biosignal (i.e., the acquired quality determination result). The wearing determination unit 113 may also separately acquire a detection result of contact between the user and the sensor device and determine the wearing state based on the detection result. In other words, the wearing determination unit 113 may acquire a contact determination result from a contact sensor that detects whether the user is in contact with the sensor device and determine the wearing state based on the contact determination result. The wearing determination unit 113 may supply the wearing state determination result (also referred to as the wearing determination result) to the baseline feature derivation unit 115 together with the acquired quality determination result. At this time, the wearing determination unit 113 may further supply the acquired biological signal to the baseline feature amount derivation unit 115. Note that the wearing determination unit 113 may be omitted.

[0043] The activity determination unit 114 executes processing related to determining the user's activity state. For example, the activity determination unit 114 may acquire a user's motion detection result input to the emotion estimation device 100. The motion detection result may include an output signal from an inertial measurement unit (IMU). For example, the motion detection result may include an output signal from an acceleration sensor (ACC (Acceleration)) or an output signal from an angular velocity sensor. The motion detection result may also include an output signal from a motion sensor. Note that the motion sensor may be any device or system. For example, it may be a system that captures an image of the user using a camera or the like, analyzes the user's motion based on the captured image, and outputs the analysis result. In other words, the motion detection result may be any information as long as it includes information indicating the user's motion. The activity determination unit 114 may determine the user's activity state (whether the user is active) based on the user's motion detection result. Any method for determining the activity state may be used. For example, the activity determination unit 114 may identify the user's motion based on a pattern of the motion detection result and determine the activity state based on the identified motion. For example, a movement pattern may be recognized, and the type of activity the user is performing (such as the type of movement or the type of vehicle) may be determined based on the pattern (see, for example, https: / / www.sonynetwork.co.jp / corporation / release / 2013 / pub20130314_2962.html). The activity determination unit 114 may supply the activity determination result (activity class (AEP)) to the baseline feature derivation unit 115. The activity determination unit 114 may also supply information such as smartphone application usage as context information to the baseline feature derivation unit 115. Note that the activity determination unit 114 may be omitted.

[0044] The baseline feature deriving unit 115 executes processing related to the derivation of baseline features. For example, the baseline feature deriving unit 115 may acquire a feature vector X_Tk supplied from the feature deriving unit 111. The baseline feature deriving unit 115 may also acquire a quality assessment result or an wearing assessment result supplied from the wearing assessment unit 113. The baseline feature deriving unit 115 may also acquire an activity assessment result supplied from the activity assessment unit 114. The baseline feature deriving unit 115 may estimate a baseline feature X*_baseline, which is a feature of the user's baseline state, using the information. For example, the baseline feature deriving unit 115 may estimate a baseline feature X*_baseline, which is a feature of the user's baseline state, using the feature vector X_Tk and a derivation function f that reflects a circadian rhythm related to the feature. The baseline feature deriving unit 115 may supply the estimated baseline feature X*_baseline to the feature normalization unit 116.

[0045] The feature normalization unit 116 performs processing related to feature normalization. For example, the feature normalization unit 116 may acquire a feature vector X_Tk supplied from the feature derivation unit 111. Alternatively, the feature normalization unit 116 may acquire a baseline feature X*_baseline supplied from the baseline feature derivation unit 115. Alternatively, the feature normalization unit 116 may acquire a normalization coefficient (a feature range conversion table previously stored as a model parameter) stored in the normalization coefficient storage unit 117. The feature normalization unit 116 may normalize the feature vector X_Tk using the baseline feature X*_baseline. In this case, the feature normalization unit 116 may apply the acquired normalization coefficient (a feature range conversion table previously stored as a model parameter). Any method may be used to normalize the feature vector X_Tk. For example, the feature normalization unit 116 may subtract or divide the baseline feature X*_baseline from the feature vector X_Tk and normalize the distribution of the calculation results. For example, the feature normalization unit 116 may apply min-max normalization to normalize the distribution of the calculation results by setting the maximum value of the calculation results to 1 and the minimum value of the calculation results to 0. Alternatively, the feature normalization unit 116 may apply Z-standardization to normalize the distribution of the calculation results by setting the average of the calculation results to 0 and the variance of the calculation results to 1. The feature normalization unit 116 may supply the normalized feature vector X_Tk to the emotional state determination unit 118.

[0046] The normalization coefficient holding unit 117 holds normalization coefficients (feature amount range conversion table as model parameters) and supplies the normalization coefficients to the feature amount normalization unit 116 in response to a request from the feature amount normalization unit 116 .

[0047] The emotional state determination unit 118 executes processing related to determining the emotional state of the user. For example, the emotional state determination unit 118 may acquire a normalized feature vector X_Tk supplied from the feature normalization unit 116. Furthermore, the emotional state determination unit 118 may acquire model parameters stored in the model parameter storage unit 119. The emotional state determination unit 118 may determine the emotional state of the user using the normalized feature vector X_Tk. In this case, the emotional state determination unit 118 may apply model parameters acquired from the model parameter storage unit 119. For example, the emotional state determination unit 118 may derive a desired emotional state parameter Y* from the normalized feature vector X_Tk based on the model parameters. Any method may be used to derive this emotional state parameter. For example, the emotional state determination unit 118 may analyze the pattern of time-series data of the normalized feature vector X_Tk to recognize the emotional state of the user. Alternatively, the emotional state determination unit 118 may determine the user's emotional state using a machine learning model (pre-constructed machine learning model) that inputs a normalized feature vector X_Tk (=(x_1, x_2, ..., x_n)) within a sliding window and outputs an emotional state parameter Y* (=(y_1, y_2, ..., y_n)) related to the emotional state (the emotional state sequence is labeled as Y*=(y_1, y_2, ..., y_n)). Here, the emotional state parameter Y is a set (vector) of arbitrary parameters y related to the emotional state. In other words, the emotional state parameter Y is a vector of parameters related to the estimation result of the emotional state. For example, the emotional state parameter Y may include a parameter indicating the arousal level (arousal level estimation result). The value of this parameter indicating the arousal level may be any value, for example, a labeling result such as "low" or "high," or a value within a predetermined range such as "0" to "1." The emotional state determination unit 118 may output the emotional state parameter Y* to the outside of the emotion estimation device 100 as the determination (estimation) result of the emotional state.

[0048] The model parameter holding section 119 holds the model parameters and supplies the model parameters to the emotional state determination section 118 in response to a request from the emotional state determination section 118 .

[0049] The emotional state parameter output from the emotion estimation device 100 can be used in any application or service. For example, the emotional state parameter may be used in an application related to vital signs or a service using vital signs. For example, the emotional state parameter may be used to generate a log of concentration levels for work evaluation. Furthermore, the emotional state parameter may be used to evaluate reactions to provided services (music, video, etc.). Furthermore, the emotional state parameter may be used to customize provided services (music selection, etc.) according to the user's current level of relaxation.

[0050] As described above, the baseline feature derivation unit 115 estimates the baseline feature X*_baseline, which is a feature of the user's baseline state, using the feature vector X_Tk of a biosignal detected (measured) within a relatively short period of time (within the detection time Tk) and the derivation function f that reflects the circadian rhythm related to the feature vector X. This allows the emotion estimation device 100 to derive more accurate baseline feature values ​​using the feature values ​​of the biosignals measured within a relatively short period of time. Therefore, the emotion estimation device 100 can suppress an increase in the measurement time of the biosignals while suppressing a decrease in the estimation accuracy of the emotional state. In other words, the emotion estimation device 100 can suppress a decrease in the estimation accuracy of the emotional state while suppressing an increase in the measurement time of the biosignals.

[0051] <Baseline Feature Derivation Unit> Fig. 4 is a block diagram showing an example of the main configuration of the baseline feature derivation unit 115 in Fig. 3. As shown in Fig. 4, the baseline feature derivation unit 115 includes a data selection unit 151, a baseline feature estimation unit 152, a baseline feature derivation function holding unit 153, and a stabilization unit 154.

[0052] Note that Fig. 4 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 4. In other words, the baseline feature derivation unit 115 may include processing units that are not shown as blocks in Fig. 4, or processes or data flows that are not shown as arrows or the like in Fig. 4.

[0053] The data selection unit 151 performs processing related to the selection of features to be applied to the estimation of baseline features. For example, the data selection unit 151 may acquire a feature vector X_Tk supplied from the feature derivation unit 111. The data selection unit 151 may acquire a quality assessment result (SQE class) of a biosignal (or a quality assessment result and a biosignal corresponding to the quality assessment result) supplied from the signal quality assessment unit 112 (via the wearing assessment unit 113). The data selection unit 151 may acquire a wearing assessment result supplied from the wearing assessment unit 113. The data selection unit 151 may acquire a user activity assessment result (behavior class) supplied from the activity assessment unit 114. For example, the data selection unit 151 may select data to be applied to the estimation of baseline features from the acquired feature vector X_Tk based on information such as the acquired quality assessment result, wearing assessment result, and activity assessment result (all or part of this information). For example, the data selection unit 151 may select a feature vector X_Tk of a biosignal having a quality higher than a predetermined standard based on the quality assessment result of the biosignal. Furthermore, the data selection unit 151 may select a feature vector X_Tk of a biosignal detected when the sensor device is worn by the user based on the wearing assessment result of the sensor device. Furthermore, the data selection unit 151 may select a feature vector X_Tk of a biosignal detected when the user is in an inactive state based on the user activity assessment result. For example, as shown in FIG. 5 , the data selection unit 151 may select a feature vector X_Tk of a biosignal having good signal quality that is detected when the sensor device is worn and the user is in an inactive state. For example, the selected feature vector may be buffered in the recording unit. If a time zone transition is not determined, buffering may be continued, and if a time zone transition is determined, buffering may be stopped. The data selection unit 151 may supply the selected feature vector (feature vector X_Tk) to the baseline feature estimation unit 152. The data selection unit 151 may also supply the baseline feature estimation unit 152 with information such as the acquired quality assessment result, wearing assessment result, and activity assessment result (all or part of this information).

[0054] Note that the data selection unit 151 may be omitted. In other words, data selection may be omitted. In that case, the feature vector X_Tk supplied from the feature derivation unit 111 may be supplied to the baseline feature estimation unit 152. In that case, the signal quality determination unit 112, the wearing determination unit 113, and the activity determination unit 114 may also be omitted. Alternatively, without omitting these units, information such as the quality determination result, the wearing determination result, and the activity determination result supplied therefrom (all or part of this information) may be supplied to the baseline feature estimation unit 152.

[0055] The baseline feature estimation unit 152 executes processing related to the estimation of baseline features. For example, the baseline feature estimation unit 152 may acquire a feature vector X_Tk supplied from the feature derivation unit 111 (via the data selection unit 151) (or a feature vector X_Tk selected by the data selection unit 151). Note that, when the data selection unit 151 is omitted, the baseline feature estimation unit 152 may acquire information such as a quality assessment result, an attachment assessment result, and an activity assessment result (all or part of this information). Furthermore, the baseline feature estimation unit 152 may acquire a derivative function f stored in the baseline feature derivation function storage unit 153.

[0056] This derivative function f is a function that reflects the circadian rhythm related to the feature vector X and is a function for deriving a baseline feature from the feature vector X. Note that this derivative function f may be a function f_Tk corresponding to the detection time period Tk. In other words, this derivative function f may be a function f_Tk for deriving a baseline feature from the feature of the detection time period.

[0057] The baseline feature estimation unit 152 may estimate baseline features, which are features of the user's reference state, using the acquired feature vector X_Tk and a derived function f that reflects the circadian rhythm related to the feature vector X. The baseline feature estimation unit 152 may estimate baseline features using biosignal features (feature vector X_Tk) of higher quality than a predetermined standard, selected by the data selection unit 151. The baseline feature estimation unit 152 may also estimate baseline features using biosignal features (feature vector X_Tk) selected by the data selection unit 151 and detected when the sensor device is worn by the user. The baseline feature estimation unit 152 may also estimate baseline features using biosignal features (feature vector X_Tk) selected by the data selection unit 151 and detected in an inactive state.

[0058] For example, the baseline feature estimation unit 152 may estimate a feature X*_(baseline-day, k) 222 of a baseline time period from a feature (X_Tk) 221 of a detection time period Tk, as shown in FIG. 6 .

[0059] In this case, the baseline feature estimation unit 152 may derive a representative value of the feature vector X_Tk for the detection time period Tk when the biological signal was detected, and may use this representative value to estimate a baseline feature X*_(baseline-day, k) corresponding to the detection date when the feature was detected. In this case, the feature normalization unit 116 may normalize the feature using the baseline feature X*_(baseline-day, k) corresponding to the detection date estimated in this manner. Note that any method may be used to derive the representative value of the feature vector X_Tk for the detection time period Tk. For example, this representative value may be the average, mode, median, maximum value, minimum value, first detected value, intermediate detected value, or last detected value of the buffered data (i.e., the feature vector X_Tk), or any other statistical value.

[0060] That is, in this case, the baseline feature X*_(baseline-day,k) is estimated as information corresponding to the detection time period Tk (i.e., "baseline feature corresponding to the detection date" corresponding to the detection time period). The baseline feature estimation unit 152 may store the baseline feature thus derived as table information, corresponding to the detection time period Tk. That is, the baseline feature estimation unit 152 may buffer the derived baseline feature as a different estimation result each time the detection time period Tk transitions. That is, the baseline feature estimation unit 152 may estimate the baseline feature X*_(baseline-day,k) for each detection time period Tk.

[0061] As described above, the baseline feature estimation unit 152 may estimate the baseline feature X*_(baseline-day,k) from (the representative value of) the feature vector X_Tk using a function f_Tk corresponding to the detection time period Tk. That is, the baseline feature estimation unit 152 may estimate the baseline feature X*_(baseline-day,k) using a function f_Tk that is a function for deriving a baseline feature from the feature (feature vector X_Tk) of the detection time period Tk. For example, the baseline feature derivation function storage unit 153 may store a function f_Tk for each detection time period Tk, and the baseline feature estimation unit 152 may acquire the function f_Tk corresponding to the detection time period Tk from the baseline feature derivation function storage unit 153, and use the acquired function f_Tk to derive (estimate) the baseline feature X*_(baseline-day,k) as shown in the following formula (1).

[0062] X*_(baseline-day,k) = f(X_Tk) ...(1)

[0063] When multiple baseline features X*_(baseline-day,k) are estimated for the same detection date (i.e., when multiple baseline features X*_(baseline-day,k) are estimated for one day), the baseline feature estimation unit 152 may use the multiple baseline features X*_(baseline-day,k) to derive (estimate) one baseline feature X*_baseline-day corresponding to the detection date (i.e., a representative value of the baseline features corresponding to the detection date). For example, the baseline feature estimation unit 152 may estimate baseline features X*_(baseline-day,k) for multiple detection time periods Tk for a certain detection date, and then use the multiple baseline features X*_(baseline-day,k) to estimate one baseline feature X*_baseline-day corresponding to the detection date. In other words, the baseline feature estimation unit 152 may estimate a baseline feature corresponding to the detection date using multiple baseline features corresponding to representative values ​​of different detection time periods.

[0064] In this case, the baseline feature estimation unit 152 may derive a weighted average of multiple baseline feature amounts as the baseline feature amount corresponding to the detection date. In other words, if there are multiple time periods in a day that contain actual measurement data that can be used to estimate baseline feature amounts, the baseline feature estimation unit 152 may take a weighted average of the estimated values ​​of the baseline feature amount X*_(baseline-day,k) as shown in the following equation (2) to estimate one baseline feature amount X*_(baseline_day) corresponding to the detection date.

[0065] ... (2)

[0066] In equation (2), the weight value w_k indicates the reliability of estimation for each detection time slot Tk. This weight value w_k may be adjusted based on the approximation error of the derivative function f for that detection time slot Tk, the actual measurement time, the signal quality, and the like. For example, the baseline feature estimation unit 152 may acquire a quality assessment result (SQE class) of the biosignal (or the quality assessment result and the biosignal corresponding to that quality assessment result) supplied from the signal quality determination unit 112 (via the wearing determination unit 113 or the data selection unit 151), and set the weight value w_k for that detection time slot Tk based on the quality assessment result. Alternatively, the baseline feature estimation unit 152 may acquire a wearing determination result supplied from the wearing determination unit 113 (via the data selection unit 151), and set the weight value w_k for that detection time slot Tk based on the wearing determination result. Furthermore, the baseline feature estimation unit 152 may acquire the user's activity determination result (behavior class) supplied from the activity determination unit 114 (via the data selection unit 151) and set the weight value w_k for the detection time period Tk based on the activity determination result. Of course, the method for setting the weight value w_k may be any method and is not limited to these examples. For example, the weight value w_k may be set based on attribute information such as a person's gender, age, and genes. The baseline feature estimation unit 152 may associate the weight value w_k set in this manner with the detection time period Tk and store it in advance as table information. The baseline feature estimation unit 152 may apply the weight value w_k corresponding to the detection time period Tk of the baseline feature X*_(baseline-day,k) to the above-described formula (2) to estimate one baseline feature X*_(baseline-day) corresponding to the detection date.

[0067] The baseline feature estimation unit 152 may supply the estimated baseline feature X*_(baseline-day) to the stabilization unit 154 .

[0068] The baseline feature amount derivation function holding unit 153 holds a derivative function f (function f_Tk) and supplies the derivative function f to the baseline feature amount estimating unit 152 in response to a request from the baseline feature amount estimating unit 152 .

[0069] The stabilization unit 154 performs processing related to stabilization of baseline features. For example, the stabilization unit 154 may acquire a baseline feature X*_(baseline-day) supplied from the baseline feature estimation unit 152. This baseline feature is a single baseline feature corresponding to the detection date. The stabilization unit 154 may estimate a baseline feature X*_(baseline) corresponding to a longer period than one day using baseline features X*_(baseline-day) corresponding to different days. In this case, the feature normalization unit 116 may normalize the derived feature using the baseline feature corresponding to that longer period. That is, the stabilization unit 154 may buffer the estimated values ​​of the baseline features calculated by the baseline feature estimation unit 152 and calculate a representative value at a certain frequency. For example, if the user's work days and holidays cycle is one week, the stabilization unit 154 may calculate an average value for that one week. Of course, this is just an example, and the target period and the method for calculating the representative value are arbitrary. In use cases where the actual measurement time is limited, the analysis window size and update frequency for buffering the representative value calculation can be adjusted. The stabilization unit 154 may supply the derived baseline feature X*_(baseline) to the feature normalization unit 116 (FIG. 3). This baseline feature X*_(baseline) is a baseline feature corresponding to a long period longer than one day.

[0070] <Derivative Function Generation Device> The derivative function f (derivative function f_Tk) is derived in advance and stored in the baseline feature estimation unit 152. This derivative function f may be derived, for example, based on features of a long-term biological signal. For example, it may be derived by a derivative function generation device 300 as shown in FIG. 7. This derivative function generation device 300 is a device that derives the derivative function f based on features of a long-term biological signal. As shown in FIG. 7, the derivative function generation device 300 includes a long-term data generation unit 311, a baseline feature generation unit 312, and a derivative function generation unit 313.

[0071] The long-term data generating unit 311 executes processing related to the generation of feature quantities of long-term biological signals. For example, the long-term data generating unit 311 may continuously measure biological signals (e.g., heart rate data) and generate feature quantities of long-term biological signals including circadian rhythms using the biological signals. The long-term data generating unit 311 may supply the generated feature quantities to the baseline feature generating unit 312.

[0072] The baseline feature generator 312 executes processing related to the generation of baseline features. For example, the baseline feature generator 312 may acquire features of long-term biological signals including circadian rhythms supplied from the long-term data generator 311. The baseline feature generator 312 may generate a baseline feature X*_(baseline-day) based on the features of the long-term biological signals (feature vector X_Tk for detection time period Tk). The baseline feature generator 312 may supply a combination of the feature vector X_Tk and the baseline feature X*_(baseline-day) to the derivative function generator 313 as generated data.

[0073] The derivative function generation unit 313 executes processing related to the generation of a derivative function. For example, the derivative function generation unit 313 may acquire generated data (a combination of the feature vector X_Tk and the baseline feature X*_(baseline-day)) supplied from the baseline feature generation unit 312. The derivative function generation unit 313 may generate the derivative function f using the generated data.

[0074] The baseline feature derivation function storage unit 153 stores the derived function f thus generated in advance. For example, the baseline feature derivation function storage unit 153 may store this derived function f as a table. For example, the baseline feature derivation function storage unit 153 may store a table indicating the relationship between the baseline feature and the feature measured in the time period Tk in order to utilize circadian rhythm characteristics. For example, the baseline feature derivation function storage unit 153 may store a correction table that quantifies the relationship, such as low arousal immediately after waking up and high arousal during the day.

[0075] For example, when the average heart rate and heart rate variability are used as features, it is generally known that the average heart rate tends to decrease and the heart rate variability tends to increase during periods of low arousal, and that the average heart rate tends to increase and the heart rate variability tends to decrease during periods of high arousal. It is also generally known that a circadian rhythm occurs in which low arousal occurs immediately after waking up and high arousal peaks during the day (e.g., around 2:00 p.m.). Given these properties, the relationship between the feature X_T and the baseline feature X_(baseline-day) can be expected to show a monotonically decreasing trend in the average heart rate and a monotonically increasing trend in the heart rate variability, as shown in the graph of FIG. 8 . In the graph of FIG. 8 , a straight line 331 indicates the relationship between the feature X_T and the baseline feature X_(baseline-day).

[0076] In this example, the relationship between these features is described as a linear function, but this is merely an example and is not limited to this example. That is, the relationship between the feature X_T and the baseline feature X_(baseline-day) may be expressed by any function. Also, in this example, the feature is described using the average heart rate and heart rate variability as examples, but these are merely examples and the feature is not limited to this example. For example, the feature may be electroencephalogram, sweating, blood pressure, or the like. Also, any information obtained from other physiological signals that contribute to the wakefulness state may be used as the feature.

[0077] Such a baseline feature derivation function f (e.g., a table) may be derived and stored for each feature and for each time period. In this case, the table to be applied may be switched during inference depending on the measurement time period Tk of the actual measurement value.

[0078] With the above configuration, the emotion estimation device 100 can suppress an increase in the measurement time of a biological signal while suppressing a decrease in the estimation accuracy of an emotional state.

[0079] <Processing Flow> An example of the flow of emotion estimation processing executed by the emotion estimation device 100 will be described with reference to the flowchart in Fig. 9. When the emotion estimation processing starts, in step S101, the feature derivation unit 111 derives features using a biological signal.

[0080] In step S102, the signal quality determining unit 112 determines the signal quality of the biological signal.

[0081] In step S103, the attachment determination unit 113 determines whether or not the device is attached.

[0082] In step S104, the activity determination unit 114 determines whether or not an activity is occurring.

[0083] In step S105, the baseline feature deriving unit 115 executes a baseline feature deriving process, and derives baseline features based on the information obtained by the processes in steps S101 to S104.

[0084] In step S106, the feature normalization unit 116 normalizes the feature using the baseline feature obtained by the process in step S105.

[0085] In step S107, the emotional state determination unit 118 determines the emotional state based on the "normalized feature amount" obtained by the processing in step S106.

[0086] In step S108, the emotional state determination unit 118 supplies the emotional state determination result to the outside.

[0087] When the process of step S108 ends, the emotion estimation process ends.

[0088] Next, an example of the flow of the baseline feature derivation process executed in step S105 of FIG. 9 will be described with reference to the flowchart of FIG.

[0089] When the baseline feature derivation process is started, the data selection unit 151 of the baseline feature derivation unit 115 selects features in step S121.

[0090] In step S122, the baseline feature estimation unit 152 derives a representative value of the selected feature for each time period.

[0091] In step S123, the baseline feature estimation unit 152 estimates baseline features using the time-slot representative values ​​of the derived features and the derivative function f for each time slot. In this case, the baseline feature estimation unit 152 may estimate baseline features using information obtained by the processes of steps S102 to S104 in FIG. 9 .

[0092] In step S124, the baseline feature estimation unit 152 derives a daily representative value of the baseline feature using the baseline feature corresponding to the time period representative value of the feature.

[0093] In step S125, the stabilization unit 154 derives a long-term representative value of the baseline feature using the one-day representative value of the baseline feature. When the process of step S125 ends, the process returns to FIG.

[0094] By performing each process in this manner, the emotion estimation device 100 can suppress an increase in the measurement time of the biological signal while suppressing a decrease in the estimation accuracy of the emotional state.

[0095] 4. Second Embodiment Emotion Estimation Device A detection unit 411 may be included, as in the emotion estimation device 400 shown in FIG. 11 . Like the emotion estimation device 100 shown in FIG. 3 , this emotion estimation device 400 is a device that estimates an emotional state of a user based on detected biosignals of the user, etc. However, the emotion estimation device 400 also detects biosignals of the user, etc. As shown in FIG. 11 , the emotion estimation device 400 includes a detection unit 411, an emotion estimation unit 412, a storage unit 413, and a communication unit 414.

[0096] The detection unit 411 executes processing related to detection of a biosignal of the user. For example, the detection unit 411 may have a biosensor 421 and detect the biosignal using the biosensor 421. The detection unit 411 may also have an acceleration sensor 422 and detect the movement of the user. The detection unit 411 may supply the detected information to the emotion estimation unit 412.

[0097] The emotion estimation unit 412 has the same configuration and executes the same processing as the emotion estimation device 100 of FIG. 3 . For example, the biosensor 421 of the detection unit 411 may supply the detected biosignal to the feature derivation unit 111 and the signal quality determination unit 112 of the emotion estimation unit 412. That is, the feature derivation unit 111 of the emotion estimation unit 412 may acquire the biosignal supplied from the biosensor 421 and derive its feature. Furthermore, the signal quality determination unit 112 of the emotion estimation unit 412 may acquire the biosignal supplied from the biosensor 421 and determine its quality. Furthermore, the acceleration sensor 422 of the detection unit 411 may supply information indicating the detected user movement to the activity determination unit 114 of the emotion estimation unit 412. That is, the activity determination unit 114 of the emotion estimation unit 412 may acquire the user movement detection result supplied from the acceleration sensor 422. The activity determination unit 114 may then determine the user's activity state (whether or not the user is active) based on the result of the user's motion detection. Furthermore, the emotion estimation unit 412 may supply the emotional state parameter Y* to the storage unit 413 or the communication unit 414 as the emotional state determination (estimation) result.

[0098] The storage unit 413 may store the emotion estimation result (emotional state parameters) supplied from the emotion estimation unit 412. Furthermore, the storage unit 413 may supply the stored emotion estimation result (emotional state parameters) to the communication unit 414 at any timing or based on any request.

[0099] The communication unit 414 may acquire the emotion estimation result (emotional state parameter) supplied from the emotion estimation unit 412. Furthermore, the communication unit 414 may acquire the emotion estimation result (emotional state parameter) stored in the storage unit 413 at any timing or based on any request. The communication unit 414 may supply the acquired emotion estimation result (emotional state parameter) to an outside of the emotion estimation device 400 (another device).

[0100] With this configuration, the emotion estimation device 400 can suppress an increase in the measurement time of the biological signal while suppressing a decrease in the estimation accuracy of the emotional state.

[0101] <Processing Flow> An example of the flow of sensor device processing executed by the emotion estimation device 400 will be described with reference to the flowchart in Fig. 12. When the sensor device processing starts, the detection unit 411 detects biometric information (biological signal) using the biometric sensor 421 in step S401.

[0102] In step S402, the detection unit 411 detects acceleration information (such as the user's movement) using the acceleration sensor 422.

[0103] In step S403, the emotion estimation unit 412 executes the emotion estimation process (FIG. 9) to determine (estimate) the emotional state.

[0104] In step S404, the storage unit 413 stores the emotional state determination result obtained by the process in step S403.

[0105] In step S405, the communication unit 414 supplies the emotional state determination result obtained in the process of step S403 to the other device. When the process of step S405 ends, the sensor device process ends.

[0106] By performing each process as described above, the emotion estimation device 400 can suppress an increase in the measurement time of a biological signal while suppressing a decrease in the estimation accuracy of an emotional state.

[0107] 5. Supplementary Notes Computer The above-described series of processes can be executed by hardware or software. When the series of processes are executed by software, the programs that make up the software are installed on a computer. Here, the term computer includes computers built into dedicated hardware, and general-purpose personal computers, for example, that can execute various functions by installing various programs.

[0108] FIG. 13 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0109] In a computer 900 shown in FIG. 13, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0110] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0111] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0112] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0113] The program executed by the computer can be applied by recording it on, for example, a removable medium 921 such as a package medium. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by inserting the removable medium 921 into the drive 915.

[0114] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0115] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0116] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0117] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0118] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0119] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0120] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0121] <Others> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in a bitstream, but also in a bitstream that includes differential information of the identification information relative to certain reference information. Therefore, in this specification, "flag" and "identification information" encompass not only the information itself, but also differential information relative to the reference information.

[0122] Furthermore, various types of information (e.g., metadata) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, the associated pieces of data may be combined into one piece of data or may be separate pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Furthermore, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer not to the entire data, but to only part of the data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0123] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0124] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0125] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0126] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0127] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0128] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0129] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0130] The present technology may also be configured as follows. (1) An information processing device comprising: a feature derivation unit that derives features of a biosignal of a user; a baseline feature estimation unit that estimates baseline features, which are the features of the user in a reference state, using the derived features and a derivation function that reflects a circadian rhythm related to the features; a feature normalization unit that normalizes the derived features using the estimated baseline features; and an emotional state determination unit that determines an emotional state of the user using the normalized features. (2) The information processing device described in (1), wherein the baseline feature estimation unit derives a representative value of the features for a detection time slot in which the biosignal is detected, and estimates the baseline features corresponding to a detection date on which the features were detected using the representative value, and the feature normalization unit normalizes the derived features using the baseline features corresponding to the detection date. (3) The information processing device described in (2), wherein the derivation function is a function for deriving the baseline features from the features for the detection time slot. (4) The information processing device according to (2) or (3), wherein the baseline feature estimation unit estimates the baseline feature corresponding to the detection date using a plurality of baseline feature amounts corresponding to the representative values ​​of the detection time periods that are different from each other. (5) The information processing device according to (4), wherein the baseline feature estimation unit derives a weighted average of the plurality of baseline feature amounts as the baseline feature corresponding to the detection date. (6) The information processing device according to any of (2) to (5), wherein the baseline feature estimation unit estimates the baseline feature corresponding to a long period longer than one day using the baseline feature amounts corresponding to different days from each other, and the feature normalization unit normalizes the derived feature amount using the baseline feature amount corresponding to the long period. (7) The information processing device according to any of (1) to (6), further comprising a signal quality determination unit that determines quality of the biological signal, and wherein the baseline feature estimation unit is configured to estimate the baseline feature based on the determined quality of the biological signal.(8) The information processing device according to (7), wherein the signal quality determination unit determines the quality of the biological signal based on a noise component included in the biological signal. (9) The information processing device according to (7) or (8), wherein the baseline feature estimation unit estimates the baseline feature using the feature of the biological signal having a quality higher than a predetermined standard. (10) The information processing device according to any of (7) to (9), wherein the baseline feature estimation unit weights the estimated baseline feature based on the quality of the biological signal and derives a weighted average of multiple baseline feature values ​​corresponding to representative values ​​of feature values ​​in different detection time periods as the baseline feature corresponding to the detection date on which the feature was detected. (11) The information processing device according to any of (1) to (10), further comprising a wearing determination unit that determines a wearing state of a sensor device that detects the biological signal by the user, wherein the baseline feature estimation unit is configured to estimate the baseline feature based on the determined wearing state. (12) The information processing device according to (11), wherein the wearing determination unit determines the wearing state based on the quality of the biological signal. (13) The information processing device according to (11) or (12), wherein the wearing determination unit determines the wearing state based on a detection result of contact between the user and the sensor device. (14) The information processing device according to any of (11) to (13), wherein the baseline feature estimation unit estimates the baseline feature using the feature of the biological signal detected when the sensor device is worn by the user. (15) The information processing device according to any of (11) to (14), wherein the baseline feature estimation unit weights the estimated baseline feature based on the wearing state, and derives a weighted average of multiple baseline feature amounts corresponding to representative values ​​of feature amounts in different detection time periods as the baseline feature corresponding to a detection date on which the feature was detected.(16) The information processing device according to any of (1) to (15), further comprising an activity determination unit that determines an activity state of the user based on a result of motion detection of the user, wherein the baseline feature estimation unit is configured to estimate the baseline feature based on the determined activity state. (17) The information processing device according to (16), wherein the motion detection result includes an output signal of an inertial measurement unit. (18) The information processing device according to (16) or (17), wherein the motion detection result includes an output signal of a motion sensor. (19) The information processing device according to any of (16) to (18), wherein the activity determination unit identifies a state of movement of the user based on a pattern of the motion detection result and determines the activity state based on the identified state of movement. (20) The information processing device according to any of (16) to (19), wherein the baseline feature estimation unit estimates the baseline feature using the feature of the biological signal detected in an inactive state. (21) The information processing device according to any of (16) to (20), wherein the baseline feature estimation unit weights the estimated baseline feature based on the activity state, and derives a weighted average of multiple baseline feature values ​​corresponding to representative values ​​of feature values ​​in different detection time periods as the baseline feature corresponding to the detection date on which the feature was detected. (22) The information processing device according to any of (1) to (21), wherein the feature normalization unit subtracts or divides the estimated baseline feature from the derived feature, and normalizes a distribution of the calculation results. (23) The information processing device according to (22), wherein the feature normalization unit normalizes the distribution of the calculation results by setting a maximum value of the calculation results to 1 and a minimum value of the calculation results to 0. (24) The information processing device according to (22), wherein the feature normalization unit normalizes the distribution of the calculation results by setting an average of the calculation results to 0 and a variance of the calculation results to 1. (25) The information processing device according to any one of (1) to (24), wherein the emotional state determination unit determines the emotional state of the user using a learning model that receives the normalized feature amount and outputs an emotional state parameter related to the emotional state.(26) The information processing device according to (25), wherein the emotional state parameter includes a level of arousal. (27) The information processing device according to any of (1) to (26), further comprising a biological signal detection unit that detects the biological signal, wherein the feature derivation unit is configured to derive the feature of the detected biological signal. (28) An information processing method, comprising: deriving a feature of a user's biological signal; estimating a baseline feature that is the feature of a reference state of the user using the derived feature and a derivative function that reflects a circadian rhythm related to the feature; normalizing the derived feature using the estimated baseline feature; and determining the emotional state of the user using the normalized feature.

[0131] REFERENCE SIGNS LIST 100 Emotion estimation device, 111 Feature derivation unit, 112 Signal quality determination unit, 113 Wear determination unit, 114 Activity determination unit, 115 Baseline feature derivation unit, 116 Feature normalization unit, 117 Normal coefficient storage unit, 118 Emotion state determination unit, 119 Model parameter storage unit, 151 Data selection unit, 152 Baseline feature estimation unit, 153 Baseline feature derivation function storage unit, 154 Stabilization unit, 300 Derived function generation device, 311 Long-term data generation unit, 312 Baseline feature generation unit, 313 Derived function generation unit, 400 Emotion estimation device, 411 Detection unit, 412 Emotion estimation unit, 413 Storage unit, 414 Communication unit, 421 Biometric sensor, 422 Acceleration sensor, 900 computer

Claims

1. An information processing device comprising: a feature derivation unit that derives features of a user's biometric signal; a baseline feature estimation unit that estimates baseline features, which are features of the user in a reference state, using the derived features and a derivation function that reflects a circadian rhythm related to the features; a feature normalization unit that normalizes the derived features using the estimated baseline features; and an emotional state determination unit that determines the emotional state of the user using the normalized features.

2. The information processing device described in claim 1, wherein the baseline feature estimation unit derives a representative value of the feature for the detection time period when the biological signal was detected, and uses the representative value to estimate the baseline feature corresponding to the detection date when the feature was detected, and the feature normalization unit normalizes the derived feature using the baseline feature corresponding to the detection date.

3. The information processing device according to claim 2, wherein the derivation function is a function for deriving the baseline feature amount from the feature amount of the detection time period.

4. The information processing device according to claim 2, wherein the baseline feature estimation unit estimates the baseline feature corresponding to the detection date using a plurality of baseline feature values ​​corresponding to the representative values ​​of the detection time periods that differ from one another.

5. The information processing device according to claim 4, wherein the baseline feature estimation unit derives a weighted average of a plurality of the baseline feature values ​​as the baseline feature value corresponding to the detection date.

6. The information processing device according to claim 2, wherein the baseline feature estimation unit estimates the baseline feature corresponding to a long period longer than one day using the baseline feature corresponding to different days, and the feature normalization unit normalizes the derived feature using the baseline feature corresponding to the long period.

7. The information processing device according to claim 1, further comprising a signal quality determination unit that determines the quality of the biological signal, wherein the baseline feature estimation unit is configured to estimate the baseline feature based on the determined quality of the biological signal.

8. The information processing device according to claim 7, wherein the signal quality determining unit determines the quality of the biological signal based on a noise component contained in the biological signal.

9. The information processing device according to claim 7, wherein the baseline feature estimation unit estimates the baseline feature using the feature of the biological signal having a quality higher than a predetermined standard.

10. The information processing device according to claim 7, wherein the baseline feature estimation unit weights the estimated baseline feature based on the quality of the biological signal, and derives a weighted average of multiple baseline feature values ​​corresponding to representative values ​​of feature values ​​in different detection time periods as the baseline feature corresponding to the detection date on which the feature was detected.

11. The information processing device according to claim 1, further comprising a wearing determination unit that determines a state in which the sensor device that detects the biosignal is worn by the user, and the baseline feature estimation unit is configured to estimate the baseline feature based on the determined wearing state.

12. The information processing device according to claim 11, wherein the baseline feature estimation unit estimates the baseline feature using the feature of the biological signal detected when the sensor device is attached to the user.

13. The information processing device according to claim 11, wherein the baseline feature estimation unit weights the estimated baseline feature based on the wearing state, and derives a weighted average of multiple baseline feature values ​​corresponding to representative values ​​of feature values ​​from different detection time periods as the baseline feature corresponding to the detection date on which the feature was detected.

14. The information processing device according to claim 1, further comprising an activity determination unit that determines an activity state of the user based on a result of the user's movement detection, and wherein the baseline feature estimation unit is configured to estimate the baseline feature based on the determined activity state.

15. The information processing device according to claim 14, wherein the activity determination unit identifies the state of movement of the user based on a pattern of the movement detection result, and determines the activity state based on the identified state of movement.

16. The information processing device according to claim 14, wherein the baseline feature estimation unit estimates the baseline feature using the feature of the biological signal detected in an inactive state.

17. The information processing device according to claim 14, wherein the baseline feature estimation unit weights the estimated baseline feature based on the activity state, and derives a weighted average of multiple baseline feature values ​​corresponding to representative values ​​of feature values ​​from different detection time periods as the baseline feature corresponding to the detection date on which the feature was detected.

18. The information processing device according to claim 1, wherein the feature normalization unit subtracts or divides the estimated baseline feature from the derived feature to normalize the distribution of the calculation result.

19. The information processing device according to claim 1, wherein the emotional state determination unit determines the emotional state of the user using a learning model that receives the normalized feature amount as input and outputs an emotional state parameter related to the emotional state.

20. An information processing method comprising: deriving features of a user's biosignal; estimating baseline features, which are the features of the user in a reference state, using the derived features and a derived function that reflects a circadian rhythm related to the features; normalizing the derived features using the estimated baseline features; and determining the emotional state of the user using the normalized features.

Images