Signal processing device and method

Abstract

The present art relates to a signal processing device and method that can improve robustness against noise of emotion estimation. A signal processing device extracts, as a feature value on the basis of a measured biological signal, a physiological index contributing to an emotion, outputs, for time series data on the feature value, time series data on the prediction label of an emotion status according to an identification model constructed in advance, and outputs an emotion estimation result on the basis of the result of a weighted sum performed on the prediction label by using a prediction label reliability that is the reliability of the prediction label. The present art can be applied to emotion estimation systems.

Description

[Technical Field]

[0001] The present technology relates to a signal processing device and method, and more particularly to a signal processing device and method that can improve robustness against noise in emotion estimation. [Background technology]

[0002] When a person's emotions change, physiological responses such as brain waves, heart rate, and sweating are expressed on the body surface. An emotion estimation system that estimates a person's emotions reads these physiological responses as biosignals using sensor devices, extracts feature quantities such as physiological indices that contribute to emotions through signal processing, and estimates the user's emotions from these feature quantities using a model derived through machine learning.

[0003] However, when deploying such technology in practical applications, body movement noise occurs due to user movements, which reduces signal quality and leads to errors in the emotion estimation output results.

[0004] In response to this, an emotion estimation technology that takes into account the influence of noise is described in Non-Patent Document 1. However, the technology described in Non-Patent Document 1 does not include an algorithm that takes into account the reliability of noise removal, and is limited to evaluating a combination of noise removal and emotion estimation.

[0005] Furthermore, an example of a conventional signal quality determination technique is the technique described in Patent Document 1. In the technique described in Patent Document 1, the valid biometric state is determined taking reliability into consideration and is finally output. However, the method for determining the valid biometric state in Patent Document 1 is limited to the level of illuminance and the face direction, which restricts the applications and situations in which it can be applied. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Val-Calvo, Mikel, et al. "Optimization of real-time EEG artifact removal and emotion estimation for human-robot interaction applications." Frontiers in Computational Neuroscience 13 (2019), Internet search <https: / / www.frontiersin.org / articles / 10.3389 / fncom.2019.00080 / full, searched on June 15, 2021>

Patent Document

[0007]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0008] As described above, when the technology of estimating emotions from biological signals is applied to actual applications, body movement noise is generated due to the movement of the user, resulting in a decrease in signal quality and leading to errors in the output results of emotion estimation.

[0009] This technology has been made in view of such a situation, and aims to improve the robustness to noise in emotion estimation.

Means for Solving the Problems

[0010] A signal processing device according to one aspect of this technology includes a feature quantity extraction unit that extracts, as feature quantities, physiological indexes contributing to emotions based on the measured biological signals, an emotion state time-series labeling unit that outputs time-series data of predicted labels of emotion states for the time-series data of the feature quantities by using a pre-constructed discrimination model, and a stabilization processing unit that outputs an emotion estimation result based on the result of weighted addition of the predicted labels using the predicted label reliability that is the reliability of the predicted labels.

[0011] In one aspect of the present technology, physiological indices contributing to emotions are extracted as features based on measured biosignals, and time-series data of predicted labels of emotional states is output using a pre-constructed discriminant model for time-series data of the features. Then, the predicted labels are weighted and added using predicted label reliability, which is the reliability of the predicted labels, and an emotion estimation result is output based on the result. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an emotion estimation processing system according to an embodiment of the present technology. [Figure 2] FIG. 2 is a diagram showing a state in which a biometric information processing device is attached to a living body. [Figure 3] FIG. 2 is a diagram showing a state in which a biometric information processing device is attached to a living body. [Figure 4] FIG. 10 is a diagram showing another aspect of the biometric information processing apparatus. [Figure 5] FIG. 10 is a diagram showing another aspect of the biometric information processing apparatus. [Figure 6] FIG. 10 is a diagram showing another aspect of the biometric information processing apparatus. [Figure 7] FIG. 10 is a diagram showing another aspect of the biometric information processing apparatus. [Figure 8] FIG. 1 is a block diagram showing a first configuration example of a biometric information processing device. [Figure 9] FIG. 10 is a diagram illustrating an image of time-series labeling of emotional states within a sliding window. [Figure 10] FIG. 10 is a diagram illustrating a calculation method for calculating a representative value of predicted labels of emotional states within a sliding window. [Figure 11] 9 is a flowchart illustrating processing by the biometric information processing device of FIG. 8. [Figure 12] FIG. 10 is a block diagram showing a second configuration example of the biometric information processing device. [Figure 13] FIG. 10 is a diagram illustrating an image of time-series labeling of emotional states within a sliding window. [Figure 14] FIG. 10 is a diagram illustrating a calculation method for calculating a representative value of predicted labels of emotional states within a sliding window. [Figure 15] FIG. 10 is a diagram showing an example of the waveform of an electroencephalogram signal over time. [Figure 16] FIG. 10 is a diagram illustrating an example of quality stability determination using electroencephalograms. [Figure 17] FIG. 10 is a diagram illustrating an example of calculating weights for signal quality scores from each type and each channel. [Figure 18] 13 is a flowchart illustrating processing by the biometric information processing device of FIG. 12. [Figure 19] FIG. 1 is a block diagram illustrating an example of the configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present technology will be described in the following order. 1. Conventional Technology 2. System Configuration 3. First Embodiment 4. Second Embodiment 5.Other

[0014] <1. Conventional Technology> With the recent boom in healthcare and wellness, wearable devices that measure people's physiological responses in daily life and sense their health and psychological state have been attracting attention.

[0015] When a person's psychological state changes, signals are transmitted from the brain via the autonomic nervous system, causing changes in various functions such as breathing, skin temperature, sweating, cardiac and vascular activity. In particular, central nervous activity such as electroencephalograms, which can be measured non-invasively in the human head, and autonomic nervous activity such as heart rate and sweating are known as physiological responses that indicate a person's emotions (e.g., arousal level).

[0016] Electroencephalograms (EEG) are generally measured using electrodes attached to the scalp, by measuring the brain's action potentials that leak through the scalp, skull, etc. There are more than several million neurons within the detection range of one electrode, and the electrode detects the sum of the action potentials emitted by a huge number of neurons.

[0017] It is known that physiological indices of electroencephalograms that contribute to human emotions are characterized by frequency components of the signal, such as theta waves, alpha waves, and beta waves.

[0018] Heart rate can be measured by measuring the electrical activity of the heart (electrocardiogram (ECG)) or by optically measuring the changes in blood vessel volume that occur as the heart pumps blood (photoplethysmography (PPG)). The commonly known heart rate is calculated by averaging the reciprocal of the intervals between each beat (average heart rate). The degree of variability in the intervals between each beat is measured as heart rate variability, and various physiological indices that represent this variability have been defined (LF, HF, LF / HF, percentage of adjacent normal-to-normal intervals (pNN)50, root mean square successive difference (RMSSD), etc.).

[0019] Sweating (emotional sweating) is expressed on the body surface as a change in skin conductivity (electrodermal activity (EDA)) and can be measured electrically as a change in skin conductance value. This is called skin conductance. Physiological indices extracted from skin conductance signals are broadly divided into skin conductance response (SCR), which indicates instantaneous sweating activity, and skin conductance level (SCL), which indicates gradual changes in the condition of the skin surface. In this way, multiple features representing autonomic nervous activity are defined.

[0020] The emotion estimation system, which estimates the user's emotions, reads these physiological responses as biosignals using sensor devices, extracts features such as physiological indicators that contribute to the emotional response through signal processing, and estimates the user's emotions from the features using a model obtained through machine learning.

[0021] With the recent advances in machine learning technology, research into emotion estimation has progressed, combining traditional physiological psychology with computing and in interdisciplinary fields. For example, in the academic field of emotion estimation, which is centered around engineering, research into affective computing has become active. In addition, in academic fields centered around science, research into affective science has become active.

[0022] Toward applications in daily life and real environments, emotion estimation by sensing autonomic nervous activity is expected to be realized using wearable devices that can be worn on the wrist, ear, etc. Furthermore, emotion estimation by sensing EEG is expected to be realized using wearable devices that naturally blend into the user's experience, such as VR (virtual reality) head-mounted displays that can be worn naturally on the head, and earphone- or headphone-type wearable devices that use in-ear EEG / around-ear EEG, a technology that has been researched in recent years.

[0023] In addition, it is expected that applications that realize these will enable emotion estimation through multimodal analysis of both electroencephalograms and autonomic nervous activity.

[0024] On the other hand, applications in daily life and real environments are affected by the user's body movements. Generally, the signal strength of biological signals is weak, and accurate sensing in daily life and real environments requires improved robustness against the effects of noise associated with the user's body movements.

[0025] Technologies for reducing the effects of noise caused by user body movements can be broadly divided into two categories: (1) methods that separate noise components using adaptive filters and signal separation techniques such as principal component analysis and independent component analysis, and (2) methods that analyze signal waveforms and identify classes according to signal quality. These methods have been researched and developed primarily in the fields of medical engineering and signal processing, and are also applied to general products such as wearable heart rate sensors.

[0026] In this way, commercialization and research and development are being conducted to improve robustness against the effects of noise associated with the user's body movements. However, completely eliminating noise is often difficult in principle, making it difficult to realize emotion estimation using the above-mentioned wearable devices.

[0027] A specific example of conventional technology is the technology described in Non-Patent Document 1. In the technology described in Non-Patent Document 1, a noise reduction technique is introduced to improve robustness against noise that occurs in a real environment, and emotion estimation performance is evaluated.

[0028] However, the technology described in Non-Patent Document 1 merely introduces noise removal technology in the preprocessing stage of emotion estimation. Therefore, while a user's emotion can be robustly estimated when noise removal is sufficient, in cases where statistical modeling is difficult, noise removal generally does not work sufficiently, which can result in a decrease in the accuracy of emotion estimation.

[0029] For example, noise can be reduced by obtaining a noise reference signal and utilizing statistical properties such as correlation analysis, principal component analysis, and independent component analysis. However, in real life, statistical processes are not constant, so complete noise removal is generally difficult.

[0030] A related technology is that described in Patent Document 1. In the technology described in Patent Document 1, the bio-valid state is determined taking into account reliability and is finally output. However, the technology described in Patent Document 1 is limited to vehicle technology and remote optical vital sensors, and the method of determining the bio-valid state is limited to the level of illuminance and the direction of the face, which restricts the applications and situations in which it can be applied.

[0031] Therefore, in the present technology, an emotion estimation result is output based on the result of weighting and adding the predicted labels using the predicted label reliability, which is the reliability of the predicted labels.

[0032] This will improve the robustness of emotion estimation against noise, improve the accuracy of emotion estimation, and is expected to expand the range of applications to which emotion estimation can be applied.

[0033] Specifically, it is expected that this technology can be used in a variety of applications involving body movement, such as monitoring stress levels in everyday life, visualizing concentration levels in an office environment, analyzing user engagement while watching video content, and analyzing excitement levels while playing games.

[0034] <2. System Configuration> <Configuration example of emotion estimation system> FIG. 1 is a diagram illustrating an example of the configuration of an emotion estimation processing system according to an embodiment of the present technology.

[0035] The emotion estimation processing system 1 in FIG.

[0036] The emotion estimation processing system 1 may include a server 12, a terminal device 13, and a network 14. In this case, in the emotion estimation processing system 1, the biometric information processing device 11, the server 12, and the terminal device 13 are connected to each other via the network 14.

[0037] The emotion estimation processing system 1 is a system that detects a signal related to the state of a living organism (hereinafter referred to as a biosignal) and estimates the emotion of the living organism based on the detected biosignal. For example, at least a bio-information processing device 11 of the emotion estimation processing system 1 is directly attached to a living organism in order to detect the biosignal.

[0038] Specifically, the biometric information processing device 11 is used, for example, as shown in FIGS. 2 and 3, to estimate the emotion of a living body.

[0039] 2 and 3 are diagrams showing how the biometric information processing device 11 is attached to a living body.

[0040] In the example of FIG. 2, a wristband-type biometric information processing device 11, such as a wristwatch-type device, is worn on the wrist of a user U1.

[0041] In the example of FIG. 3, a headband-type biometric information processing device 11, such as a forehead contact type, is wrapped around the head of a user U1.

[0042] The bio-information processing device 11 includes a bio-sensor that detects bio-signals for estimating the emotional state of the living body of the user U1, such as sweating state, pulse wave, electromyography, blood pressure, blood flow, or body temperature, and estimates the emotional state of the user U1 based on the bio-signals detected by the bio-sensor. Based on these emotional states, the user's concentration state, wakefulness state, etc. can be confirmed.

[0043] 2 and 3 show an example in which the biometric information processing device 11 is worn on the arm or head, the wearing position of the biometric information processing device 11 is not limited to the examples in FIGS.

[0044] For example, the biometric information processing device 11 may be realized in a form that can be worn on a part of a hand, such as a wristband, glove, smartwatch, or ring. Furthermore, when the biometric information processing device 11 comes into contact with a part of a living body, such as a hand, the biometric information processing device 11 may be provided on, for example, an object that may come into contact with a user. For example, the biometric information processing device 11 may be provided on the surface of or inside an object that may come into contact with a user, such as a mobile terminal, smartphone, tablet, mouse, keyboard, handle, lever, camera, sports equipment (such as a golf club, tennis racket, or archery bow), or writing implement.

[0045] Furthermore, for example, the biometric information processing device 11 may be realized in a form that can be worn on a part of the user's head or ear, such as a head-mounted display (FIG. 4), headphones (FIG. 5), earphones (FIG. 6), a hat, an accessory, goggles, or glasses (FIG. 7).

[0046] 4 to 7 are diagrams showing other aspects of the biometric information processing device 11. FIG.

[0047] In FIG. 4, a head-mounted display type biometric information processing device 11 is shown.

[0048] In the head-mounted display type biometric information processing device 11, the pad unit 21, the band unit 22, etc. are worn on the user's head.

[0049] In FIG. 5, a headphone-type biometric information processing device 11 is shown.

[0050] In the headphone-type biometric information processing device 11, a band portion 31, ear pads 32, etc. are worn on the head and ears of the user.

[0051] In FIG. 6, an earphone-type biometric information processing device 11 is shown.

[0052] In the earphone-type biometric information processing device 11, earpieces 41 are attached to the ears of the user.

[0053] In FIG. 7, a glasses-type biometric information processing device 11 is shown.

[0054] In the eyeglass-type biometric information processing device 11, temples 51 are worn above the ears of the user.

[0055] The biometric information processing device 11 may also be provided in clothing such as sportswear, socks, underwear, protective gear, or shoes.

[0056] The position and method of wearing the biometric information processing device 11 are not particularly limited as long as the biometric information processing device 11 can detect signals related to the state of the living body. For example, the biometric information processing device 11 does not need to be in direct contact with the body surface of the living body. For example, the biometric information processing device 11 may be in contact with the surface of the living body via clothing or a detection sensor protective film, etc.

[0057] Furthermore, the above-described biometric information processing device 11 does not necessarily have to perform processing by itself in the emotion estimation processing system 1. For example, the biometric information processing device 11 may be equipped with a biometric sensor that comes into contact with a living organism, and the biometric signal detected by the biometric sensor may be transmitted to another device such as the server 12 or the terminal device 13, and the other device may perform information processing based on the received biometric signal to estimate the emotion of the living organism.

[0058] For example, when a biosensor is attached to the user's arm or head, the bioinformation processing device 11 may transmit the biosignal acquired from the biosensor to a server 12 or a terminal device 13 such as a smartphone, and the server 12 or the terminal device 13 may process the information and estimate the emotions of the living body.

[0059] The biosensor provided in the bio-information processing device 11 detects biosignals by contacting the surface of a living organism in various ways as described above. Therefore, the measurement results of the biosensor are easily affected by fluctuations in contact pressure between the biosensor and the living organism due to the body movement of the living organism. For example, the biosignal acquired from the biosensor contains noise caused by the body movement of the living organism. It is desirable to accurately estimate the emotions of the living organism from the biosignal containing such noise.

[0060] The biological body movement refers to the general movement of the living body, such as twisting the wrist, bending and straightening the fingers, etc., when the user U1 wears the biometric information processing device 11 on his / her wrist. Such user movements may cause a change in the contact pressure between the biometric sensor included in the biometric information processing device 11 and the user U1.

[0061] In addition to the above-described biosensor, the bio-information processing device 11 may be provided with a second sensor and a third sensor, which will be described next, in order to improve the accuracy of the biosignal obtained by the biosensor.

[0062] For example, the second sensor is configured to detect a change in body movement of the living body, and the third sensor is configured to detect a change in pressure of the living body in a detection area of ​​the biosensor.

[0063] In this case, the biological information processing device 11 can accurately reduce body movement noise from the biological signal detected by the biological sensor by using the body movement signal and pressure signal detected by the second sensor and the third sensor. The biological information processing device 11 may perform emotion estimation processing of the present technology described below by using the biological signal corrected in this way.

[0064] 1, in the emotion estimation processing system 1, the server 12 is configured by a computer, etc. The terminal device 13 is configured by a smartphone, a mobile terminal, a personal computer, etc.

[0065] The server 12 and the terminal device 13 receive information and signals transmitted from the biometric information processing device 11 and transmit information and signals to the biometric information processing device 11 via the network 14 .

[0066] For example, as described above, the server 12 and the terminal device 13 receive from the biometric information processing device 11 a biometric signal obtained by a biometric sensor included in the biometric information processing device 11, and estimate the emotions of the living body by performing signal processing on the received biometric signal.

[0067] The network 14 is configured by the Internet, a wireless LAN (Local Area Network), or the like.

[0068] 3. First embodiment (basic configuration) <First Configuration Example of Biometric Information Processing Device> FIG. 8 is a block diagram showing a first configuration example of the biometric information processing device 11. As shown in FIG.

[0069] 8, the biometric information processing device 11 includes a filter pre-processing unit 101, a feature extraction unit 102, an emotional state time-series labeling unit 103, and a stabilization processing unit 104.

[0070] The filter pre-processing unit 101 performs pre-processing such as band-pass filtering and noise removal on the measured biological signal. The filter pre-processing unit 101 outputs the pre-processed biological signal to the feature extraction unit 102.

[0071] For example, when the biological signal is an electroencephalogram (EEG), the EEG is measured by attaching electrodes to the scalp and detecting brain action potentials leaking through the scalp, skull, etc. One of the characteristics of EEG measured in this way is that it is known to have a very low signal-to-noise ratio. Therefore, it is necessary to remove unnecessary frequency components from the EEG, which is a time-series signal, and a band-pass filter is applied (for example, a passband of 0.1 Hz to 40 Hz).

[0072] In addition, body movement components caused by human body movements are superimposed on biosignals as artifacts (noise other than the target signal, etc.). To address this, signal processing techniques such as adaptive filters and independent component analysis are applied. Pre-filtering is also performed for biosignals such as emotional sweating (EDA), pulse waves (PPG), and blood flow (LDF).

[0073] The feature extraction unit 102 uses the biological signals supplied from the filter pre-processing unit 101 to extract a feature vector x=(x1, x2, ..., x n The feature extraction unit 102 extracts the extracted feature vector x=(x1, x2, ..., x n ) to the emotional state time-series labeling unit 103.

[0074] Specifically, the feature extraction unit 102 observes signals from vital sensors (biological sensors) such as electroencephalogram (EEG), psychoacoustic dehydration (EDA), pulse wave (PPG), and blood flow (LDF) as time-series data, and extracts physiological indices that contribute to changes in emotions as features. Here, the method for extracting each feature is not limited, but an example of feature extraction is shown below.

[0075] For example, as mentioned above, electroencephalograms (EEG) can be measured by measuring brain action potentials leaking through the scalp, skull, etc. using electrodes attached to the scalp. Aftanas, LI, and SA Golocheikine. "Human anterior and frontal midline theta and lower alpha reflect emotionally positive state and internalized attention: high-resolution EEG investigation of meditation." Neuroscience Letters 310.1 (2001): 57-60 (hereinafter referred to as Cited Reference 1) describes that characteristics of frequency components of EEG signals, such as theta waves, alpha waves, and beta waves, can be extracted as physiological indices that contribute to human emotions.

[0076] On the other hand, as mentioned above, emotional sweating (EDA) is observed as a time-series signal of skin conductance (hereinafter referred to as skin conductance signal). Benedek, M. & Kaernbach, C. (2010). A continuous measure of phasic electrodermal activity. Journal of Neuroscience Methods, 190, 80-91. (hereinafter referred to as Cited Reference 2) describes that physiological indices extracted from skin conductance signals are separated into Skin Conductance Response (SCR), which represents instantaneous sweating activity, and Skin Conductance Level (SCL), which represents gradual changes in the condition of the skin surface.

[0077] In addition, heart rate and heart rate variability can be extracted from the pulse wave (PPG), and heart rate variability includes physiological indices such as mHR, LF, HF, LF / HF, pNN50, and RMSSD.

[0078] Typically, features are extracted using an analysis window (sliding window) of about five minutes. For example, Salahuddin, Lizawati, et al. "Ultra short-term analysis of heart rate variability for monitoring mental stress in mobile settings," 2007 29th annual international conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2007 (hereinafter, Reference 3) shows the validity of HRV over a short window of about several tens of seconds as ultra-short-term HRV, taking into consideration more real-time performance.

[0079] Note that the feature amounts are not limited to these physiologically known feature amounts. The feature amount extraction unit 102 can also perform signal processing to extract feature amounts that contribute to emotions in a data-driven manner, for example, by deep learning or an autoencoder.

[0080] The emotional state time series labeling unit 103 extracts the time series feature vectors X within the sliding window from the feature vectors x supplied from the feature extraction unit 102. k =(x1,x2,…,x k ) is input. The emotional state time series labeling unit 103 identifies the predicted label of the emotional state in the time series using a discrimination model, which is a machine learning model constructed in advance, and k =(y1,y2,…,y k )

[0081] The emotional state time series labeling unit 103 generates time series data Y of predicted labels, which are the labeling results of the time series emotional states. k =(y1,y2,…,y k ) to the stabilization processor 104. At this time, the reliability of the predicted label obtained from the discriminative model is also output.

[0082] Methods for time-series labeling of emotional states are generally based on discriminative models used in time-series data analysis and natural language processing. Specific examples include Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Linear Discriminant Analysis (LDA), Hidden Markov Models (HMM), Conditional Random Fields (CRF), Structured Output Support Vector Machine (SOSVM), Bayesian Network, Recurrent Neural Network (RNN), and Long Short Term Memory (LSTM). However, the methods are not limited.

[0083] The stabilization processing unit 104 receives the time series data Y of the predicted labels of the emotional states supplied from the emotional state time series labeling unit 103. t =(y1,y2,…,y t ), the predicted labels of the time series emotional states are weighted and added by the reliability of the predicted labels of the emotional states within the sliding window, and the representative value of the predicted labels (e.g., arousal level, described later) z and the reliability r of the representative value of the predicted labels are output as the emotion estimation result. The reliability r of the representative value of the predicted labels is the reliability when the representative value of the predicted labels is calculated.

[0084] Specifically, the stabilization processor 104 calculates the reliability r of the representative value of the predicted labels within the sliding window by weighting and adding the predicted labels of the emotional states within the sliding window according to the reliability of the predicted labels. Furthermore, the stabilization processor 104 performs threshold processing on the reliability r of the representative value of the predicted labels and outputs the representative value z of the predicted labels as the emotion estimation result.

[0085] FIG. 9 is a diagram showing an image of how the stabilization processor 104 performs time-series labeling of emotional states within a sliding window.

[0086] In Fig. 9, the inside of the sliding window for each subsequence is shown. In the figure, y is the predicted label of the emotional state, c is the reliability of the predicted label obtained from the discriminative model, and Δt i represents the duration of the i-th event.

[0087] As shown in FIG. 9, within the sliding window, for each event, a predicted label y of the emotional state is calculated along with the confidence c of the predicted label.

[0088] When the predicted label y of the emotional state is identified by, for example, arousal level, low / high arousal level is defined as two classes, 0 or 1.

[0089] Also, for the sake of convenience in explanation, FIG. 9 shows an example in which there is a gap between the durations of the events, but there is no need to leave a gap between the durations of the events.

[0090] FIG. 10 is a diagram showing a calculation method in which the stabilization processing unit 104 calculates a representative value of the predicted label according to the predicted label of the time-series emotional state within the sliding window and its reliability.

[0091] The reliability r of the representative value of the predicted label is calculated by the following formula (1).

[0092]

number

[0093] Using the above formula (1), the reliability of the representative value of the predicted labels within the sliding window is calculated as a continuous value [-1 1] for the reliability of the predicted labels of the emotional states of multiple events detected within the sliding window.

[0094] Furthermore, the output r of Equation (1) is subjected to threshold processing and substituted into the following Equation (2), whereby a representative value z of the predicted label is calculated as the emotion estimation result.

[0095]

number

[0096] For example, if the predicted label of the emotional state is determined as an arousal level, the predicted label of the emotional state is defined as two classes, 0 or 1, representing low or high arousal. In this case, when the representative value z of the predicted label of the emotion estimation result is 0, the user's emotional state at the corresponding time is identified as low arousal (relaxed state). On the other hand, when the representative value z of the predicted label of the emotion estimation result is 1, the user's emotional state at the corresponding time is identified as high arousal (awake and focused state).

[0097] <Device processing> FIG. 11 is a flowchart illustrating the processing of the biometric information processing device 11 of FIG.

[0098] In step S101, the filter pre-processing unit 101 performs pre-processing on a biosignal measured by a biosensor. The filter pre-processing unit 101 outputs the biosignal after the pre-processing to the feature extraction unit .

[0099] In step S102, the feature extraction unit 102 extracts features based on the biological signal supplied from the filter pre-processing unit 101. The feature extraction unit 102 extracts the extracted feature vector x=(x1, x2, ..., x n ) to the emotional state time-series labeling unit 103.

[0100] In step S103, the emotional state time-series labeling unit 103 performs time-series labeling of emotional states using, as input, feature quantities within the sliding window among the feature quantities supplied from the feature extraction unit 102. The emotional state time-series labeling unit 103 outputs predicted labels of the time-series emotional states, which are the labeling results of the emotional state time series, to the stabilization processing unit 104.

[0101] In step S104, the stabilization processing unit 104 receives the predicted labels of the time-series emotional states supplied from the emotional state time-series labeling unit 103 as input, and calculates the reliability r of the representative value of the predicted labels within the sliding window using the above-mentioned equation (1).

[0102] In step S105, the stabilization processing unit 104 performs threshold processing on the reliability r of the representative value of the predicted label using the above-mentioned equation (2), and outputs the representative value z of the predicted label as the emotion estimation result.

[0103] As described above, in the first embodiment of the present technology, an emotion estimation result is output based on the result of weighting and adding predicted labels using predicted label reliability, which is the reliability of the predicted labels. This improves the robustness of the estimation accuracy of emotion estimation.

[0104] 4. Second embodiment (additional configuration) <Second Configuration of the Device> FIG. 12 is a block diagram showing a second configuration example of the biometric information processing device 11. As shown in FIG.

[0105] In the second embodiment, a signal quality determination unit 201 is added to further improve noise robustness of emotion estimation when noise occurs due to body movement or the like in a real environment.

[0106] That is, the biometric information processing device 11 in Fig. 12 differs from the biometric information processing device 11 in Fig. 8 in that a signal quality determination unit 201 is added and the stabilization processing unit 104 is replaced with a stabilization processing unit 202. In Fig. 12, the same reference numerals are used to designate the same units as in Fig. 8.

[0107] The signal quality determination unit 201 analyzes the waveform of a biological signal measured by a biological sensor and identifies the type of artifact. The signal quality determination unit 201 determines the signal quality based on the identification result and calculates a signal quality score as the signal quality determination result.

[0108] The stabilization processing unit 202 performs weighted addition using the reliability of the predicted label of the emotional state and the signal quality score, which is the judgment result of the signal quality judgment unit 201, and outputs a representative value z of the predicted label and the reliability r of the representative value of the predicted label as the emotion estimation result.

[0109] FIG. 13 is a diagram showing an image of the stabilization processor 202 performing time-series labeling of emotional states within a sliding window.

[0110] In Fig. 13, the inside of the sliding window in subsequence units is shown, as in Fig. 9. In the figure, y is the predicted label of the emotional state, c is the reliability of the predicted label obtained from the discriminative model, and Δt i represents the duration of the i-th event.

[0111] Furthermore, FIG. 13 shows the signal quality score s, which is the output of the signal quality determining unit 201 at the same time as the sliding window.

[0112] 13 calculates, for each event within the sliding window, a predicted label y of the emotional state along with the reliability c of the predicted label, as in the case of Fig. 9. Furthermore, at the same time as the sliding window, the signal quality determination unit 201 outputs a signal quality score s to the stabilization processing unit 202.

[0113] FIG. 14 is a diagram showing a calculation method in which the stabilization processing unit 202 calculates a representative value of the predicted label according to the predicted label of the emotional state within the sliding window, its reliability, and the signal quality score.

[0114] The signal quality determination unit 201 calculates time-series data of the signal quality score s and outputs it to the stabilization processing unit 202. As shown in Fig. 14, the stabilization processing unit 202 uses this signal quality score s to calculate the reliability of the representative value of the predicted label, feeding back the signal quality as a weight. The method for calculating the reliability r of the representative value after feeding back the signal quality can be defined as the following equation (3) based on the above-mentioned equation (1).

[0115]

number

[0116] Using the above formula (3), the reliability of the representative value of the predicted label within the sliding window is calculated as a continuous value [-1 1] for the reliability of the predicted label of the emotional state of multiple events detected within the sliding window.

[0117] Furthermore, similarly to the first embodiment, the output r of equation (3) is subjected to threshold processing and substituted into the above-described equation (2), thereby calculating a representative value z of the predicted label as an emotion estimation result.

[0118] It should be noted that the following equation (4) may be used instead of the above equation (3).

[0119]

number

[0120] Equation (3) has the property that the reliability r becomes small in a sliding window where the signal quality is low. In contrast to Equation (3) which has this property, Equation (4) has s in the denominator. i By including the ,normalization can be performed. This allows for consistent emotion determination even between sliding windows with different signal qualities.

[0121] Hereinafter, when formula (3) is used, formula (4) can be used instead.

[0122] <General signal quality judgment process> A general technique underlying the signal quality determination in the signal quality determination unit 201 is described in Lawhern, Vernon, W. David Hairston, and Kay Robbins, "DETECT: A MATLAB toolbox for event detection and identification in time series, with applications to artifact detection in EEG signals." PloS one 8.4 (2013): e62944 (hereinafter referred to as Reference 4).

[0123] In the technique described in Cited Document 4, as shown in FIG. 15, electroencephalogram signals consisting of multiple channels are input and the waveforms are analyzed.

[0124] FIG. 15 is a diagram showing an example of the waveform of an electroencephalogram signal over time.

[0125] When electroencephalograms are measured in a real environment, various noises are superimposed on the electroencephalogram signals due to the movements of the user, as shown in FIG.

[0126] No noise is superimposed on the EEG signals from t=63 to t=66 and t=71 to t=74. Note that signals with no noise superimposed on them are indicated as Clean in the figure.

[0127] Artifact 1 (eye movement noise) is superimposed on the EEG signal from t=66 to t=68. Eye movement noise is noise that occurs on EEG signals, especially those placed on the forehead, when a person moves their eyes.

[0128] Artifacts 2 and 3 (electromyographic noise) (muscular in the figure) are superimposed on the EEG signal from t = 68 to t = 70. Electromyographic noise is noise that occurs in EEG signals when, for example, a person changes facial expression or moves their muscles while talking.

[0129] Artifacts 4 and 5 (eyeblink noise) are superimposed on the EEG signal from t = 70 to t = 72. Eyeblink noise is noise that occurs in EEG signals, particularly in the frontal lobe, when the frequency or strength of a person's blinks changes.

[0130] Artifact 6, or electrocardiographic noise (Cardiac in the figure), is superimposed on the EEG signal from t = 170 to t = 180. Changes due to a person's heartbeat appear as potential changes, and these potential variations can appear as electrocardiographic noise on the EEG signal.

[0131] As described above, EEG signals with various types of noise superimposed on them have different waveforms, so by analyzing the waveforms, the signal waveforms in the areas where each artifact occurs can be identified, and the type of artifact can be identified. In this case, pattern matching, signal processing for waveform identification, machine learning technology, etc. are used to determine the signal quality.

[0132] For example, Khatwani, Mohit, et al. "Energy efficient convolutional neural networks for eeg artifact detection," 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), IEEE, 2018 (hereinafter referred to as Reference 5) describes a machine learning technique used to determine signal quality.

[0133] In the technology described in Cited Document 5, classification is performed using a classification model, which is a machine learning model, and class classification is performed for each waveform of a biological signal, and signal quality is determined based on the class classification result.

[0134] In signal quality determination section 201, a signal quality score (SQE score) specialized for the processing in stabilization processing section 202 (equation (3)) is calculated based on the above-mentioned signal quality determination technique.

[0135] <Signal quality determination using EEG> FIG. 16 is a diagram showing an example of quality stability determination using electroencephalograms.

[0136] When measuring electroencephalograms in a real environment, as described above with reference to FIG. 15, due to the user's movements, a signal (observed EEG signal in the figure) is detected in which various noises (Eye-movement, Muscular, Eyeblink, Cardiac, etc. in the figure) are superimposed on the electroencephalogram signal (EEG in the figure).

[0137] These noises naturally occur when a user experiences an application in an everyday or real-world environment, and in many cases cannot be completely removed by filtering due to factors such as the user's situation and device limitations.

[0138] Therefore, in the second embodiment, a discrimination class for each type of noise is defined in advance, and a discrimination model is constructed by supervised learning. Hereinafter, the discrimination model for quality assessment will be referred to as an SQE discrimination model, which stands for Signal Quality Estimation, and the predefined discrimination class will be referred to as an SQE discrimination class.

[0139] The signal quality determination unit 201 identifies the waveform type using the SQE identification model. Then, the signal quality determination unit 201 calculates a signal quality score s that is specialized for the signal processing method defined by the above-mentioned equation (3). The signal quality score s is calculated by the following equation (5).

[0140]

number

[0141] α is an adjustment term that takes into account the difference in noise removal performance in the filter pre-processing unit 101 depending on the type of noise identified by the SQE identification class.

[0142] In this technology, when the SQE discrimination model determines that the EEG signal is clean, in equation (3), the more reliable the class label obtained from the SQE discrimination model, the larger the weight is so that it is classified as a positive class, and f() is set to a monotonically increasing look-up table.

[0143] Here, the positive class is a class whose signal quality is identified as being better than a predetermined threshold, and the negative class is a class whose signal quality is identified as being worse than a predetermined threshold and containing noise.

[0144] When the EEG signal is clean, the weight is set to a maximum of α = 1.0. When noise occurs in the EEG signal, in equation (3), f() is set to a monotonically decreasing look-up table so that the more reliable the class label obtained from the SQE discrimination model, the smaller the weight is assigned to the positive class.

[0145] α is adjusted depending on the SQE discrimination class and the performance difference of the filter pre-processing unit 101. For example, α is adjusted to α for eyeblink noise, which is relatively easy to remove by signal processing. m α is set to a large value, such as α = 0.9. Myoelectric noise, which is difficult in principle to remove by signal processing in the filter pre-processing unit 101, is set to α m =0.2 or similar.

[0146] In this technology, α m is positioned as an adjustment term and there are no constraints on its value.

[0147] f(d m ) monotonically increases when m is the main signal, and monotonically decreases when m is noise.

[0148] As described above, by defining the above-mentioned equation (5), the signal quality score s[0.0 1.0] becomes larger as the signal quality increases and becomes smaller as the signal quality decreases, making it a signal processing method specialized for equation (3).

[0149] Furthermore, the above description has been given of an example in which the SQE identification model determines the signal quality at each time from all channels, but the present technology also contemplates a case in which the signal quality is determined by the SQE identification model on a channel-by-channel basis.

[0150] Furthermore, this technology also targets the processing of emotion estimation from multiple types of biosignal modals (e.g., multimodal signals such as electroencephalogram (EEG), pulse wave (PPG), and sweating (EDA)), and also assumes the case where an SQE discrimination model is constructed for each biosignal modal and signal quality is determined.

[0151] Therefore, a further expanded modification of the above-mentioned formula (5) will be described to deal with such a case.

[0152] <Signal Quality Score Weight> FIG. 17 is a diagram showing an example of calculating the weight of the signal quality score (SQE score) from each type and each channel.

[0153] On the right side of FIG. 17, a graph showing the contribution of features corresponding to each type and channel of the signal to the discrimination of the emotional state is shown.

[0154] The graph shows the features calculated from EEG (theta waves, alpha waves, and beta waves), the features calculated from pulse waves (average heart rate, RMSSD, and LF / HF), and the features calculated from sweating (SCL and SCR).

[0155] The graph shows an example of an emotion estimation model in which the machine learning model consists of three (j=3) types of biological signals.

[0156] At this time, as shown on the right side of FIG. 17, it is necessary to calculate an integrated signal quality score s as a scalar value for j types of biological signals in order to apply it to the above-mentioned equation (3).

[0157] In this technology, an overall signal quality score s of the estimation results is calculated by weighting the SQE scores for each of the j types of signals. Specifically, the signal quality score s is calculated using the following equation (6).

number

[0158] In addition, the total model contribution W j is expressed by the following equation (7).

number

[0159] That is, W j The sum of the model contributions of the features belonging to signal j is calculated once as W j is added as a weight for the signal quality score of signal j, and a scalar value of the overall signal quality score at each time is calculated.

[0160] As described above, this technology can also be applied to cases where the SQE score weight is calculated from each type and each channel.

[0161] <Device processing> FIG. 18 is a flowchart illustrating the processing of the biometric information processing device 11 of FIG.

[0162] Steps S201 to S203 in FIG. 18 are similar to steps S101 to S103 in FIG. 11, and therefore a description thereof will be omitted.

[0163] In FIG. 18, the processes of steps S204 and S205 are performed in parallel with the processes of steps S201 to S203.

[0164] In step S204, the signal quality determining unit 201 analyzes the signal waveform of each biosensor and identifies the waveform type.

[0165] In step S205, the signal quality determining unit 201 calculates a signal quality score according to the waveform type and outputs the calculated signal quality score to the stabilization processing unit 202.

[0166] In step S206, the stabilization processing unit 202 receives as input the time-series emotional state labels supplied from the emotional state time-series labeling unit 103 and the signal quality scores supplied from the stabilization processing unit 202, and calculates the reliability r of the representative value of the predicted label within the sliding window using the above-mentioned equation (3).

[0167] In step S207, the stabilization processing unit 202 performs threshold processing on the reliability r of the representative value of the predicted label using the above-described equation (2), and outputs the representative value z of the predicted label as the emotion estimation result.

[0168] As described above, in the second embodiment of the present technology, an emotion estimation result is output based on the result of weighted addition of the reliability of the predicted label and the result of signal quality determination. Therefore, the estimation accuracy of emotion estimation is further improved in robustness compared to the first embodiment.

[0169] In the above description, an example has been described in which signal quality is determined by a machine learning technique, but it is also possible to determine signal quality by a technique other than machine learning.

[0170] For example, a heart rate sensor (PPG, photoplethysmography) is considered. When the pulse wave is measured normally, it has a strong periodicity corresponding to the pulsation. When noise such as body movement noise occurs, the periodicity of the signal decreases.

[0171] Therefore, International Publication No. 2017 / 199597 (hereinafter referred to as Reference 6) describes a method for evaluating the periodicity of a pulse wave signal by analyzing the autocorrelation of the signal (the relationship between the shift amount of the signal itself and the correlation value). With the technology of Reference 6, for example, a low autocorrelation value is recognized as a low periodicity, and signal quality can be determined.

[0172] In view of the above, the signal quality determining unit 201 may output a signal quality score according to the strength of the periodicity of the signal without using machine learning.

[0173] This technology can be applied not only to pulse waves but also to other highly periodic biological signals such as blood flow and continuous blood pressure. <5.Other>

[0174] <Effects of this technology> Recently, wearable devices such as wristbands, headbands, and earphones that can be used in daily life environments can measure neural activity that contributes to emotional changes with little burden on the user. In particular, it is expected that these devices will be able to easily measure autonomic nervous activity (e.g., pulse wave and sweating).

[0175] In addition, it is expected that emotion estimation by sensing brain waves will be realized through wearable devices that naturally blend into the user's experience, such as wearable devices that can be worn naturally on the head, such as VR head-mounted displays, and earphone- or headphone-type wearable devices that use technology that measures brain waves in or around the ear, such as in-ear EEG / around-ear EEG, which has been researched in recent years.

[0176] On the other hand, applications in daily life and real environments are affected by the user's body movements. Generally, the signal strength of biological signals is weak, and accurate sensing in daily life and real environments requires improved robustness against the effects of noise associated with the user's body movements.

[0177] Therefore, in the present technology, an emotion estimation result is output based on the result of weighting and adding the predicted labels using the predicted label reliability, which is the reliability of the predicted labels.

[0178] This leads to improved robustness and accuracy of emotion estimation against noise.

[0179] Furthermore, in this technology, an emotion estimation result is output based on the result of weighting and adding the reliability of the predicted label of the emotional state and the result of the signal quality determination.

[0180] Therefore, it is expected that the number of practical applications involving user movement will increase.

[0181] This technology is expected to be used in a variety of applications involving body movement, such as monitoring stress levels in everyday life, visualizing concentration levels in an office environment, analyzing user engagement while watching video content, and analyzing excitement levels during gameplay.

[0182] <Computer configuration example> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0183] FIG. 19 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program.

[0184] A CPU (Central Processing Unit) 301 , a ROM (Read Only Memory) 302 , and a RAM (Random Access Memory) 303 are interconnected by a bus 304 .

[0185] An input / output interface 305 is also connected to the bus 304. An input unit 306 including a keyboard, a mouse, etc., and an output unit 307 including a display, a speaker, etc. are connected to the input / output interface 305. In addition, a storage unit 308 including a hard disk, a nonvolatile memory, etc., a communication unit 309 including a network interface, etc., and a drive 310 that drives removable media 311 are also connected to the input / output interface 305.

[0186] In the computer configured as above, the CPU 301 loads a program stored in the storage unit 308 into the RAM 303 via the input / output interface 305 and the bus 304 and executes the program, thereby performing the above-described series of processes.

[0187] The program executed by the CPU 301 is installed in the storage unit 308 by being recorded on a removable medium 311, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0188] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0189] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0190] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0191] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0192] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0193] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0194] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0195] <Configuration combination example> The present technology can also be configured as follows. (1) a feature extraction unit that extracts physiological indices that contribute to emotions as features based on the measured biosignals; an emotional state time-series labeling unit that outputs time-series data of predicted labels of emotional states using a pre-constructed discrimination model for the time-series data of the feature amounts; a stabilization processing unit that outputs an emotion estimation result based on a result of weighting and adding the predicted labels using a predicted label reliability that is a reliability of the predicted labels; A signal processing device comprising: (2) The emotional state time-series labeling unit outputs time-series data of the predicted labels of the emotional states within the sliding window using the discriminative model for the time-series data of the feature amounts within the sliding window. The signal processing device according to (1) above. (3) The stabilization processing unit A representative value reliability, which is the reliability of a representative value of the predicted label, is calculated by weighting and adding the predicted label and the predicted label reliability, and a threshold process is performed on the representative value reliability to output the representative value of the predicted label as the emotion estimation result. The signal processing device according to (1) or (2). (4) The stabilization processing unit calculates the representative value reliability as follows:

number

number

number

number

number

number

number

[0196] 1 emotion estimation processing system, 11 biometric information processing device, 12 server, 13 terminal device, 14 network, 101 filter pre-processing unit, 102 feature extraction unit, 103 emotional state time series labeling unit, 104 stabilization processing unit, 201 signal quality determination unit, 202 stabilization processing unit

Claims

1. a feature extraction unit that extracts physiological indices that contribute to emotions as features based on the measured biosignals; an emotional state time-series labeling unit that outputs time-series data of predicted labels of emotional states within a sliding window using a pre-constructed discrimination model for the time-series data of the feature amounts within the sliding window; a stabilization processing unit that outputs an emotion estimation result based on a result of weighting and adding the predicted labels using a predicted label reliability that is a reliability of the predicted labels; A signal processing device comprising:

2. The stabilization processing unit A representative value reliability, which is the reliability of a representative value of the predicted label, is calculated by weighting and adding the predicted label and the predicted label reliability, and a threshold process is performed on the representative value reliability to output the representative value of the predicted label as the emotion estimation result. The signal processing device according to claim 1 .

3. The stabilization processing unit calculates the representative value reliability as follows: [Equation 1] Calculate it as follows: The yi is the predicted label labeled for the i-th event in the time-series data of the predicted labels of the emotional states labeled for each time-series event within the sliding window, the ci is the predicted label reliability of the predicted label for the i-th event, and the Δt i represents the duration of the i-th event, and wi represents the forgetting weight of the predicted label of the i-th event. The signal processing device according to claim 2 .

4. a signal quality determination unit that determines the signal quality of the biological signal; The stabilization processing unit outputs the emotion estimation result based on a result of weighting and adding the predicted label using the predicted label reliability and the signal quality determination result. The signal processing device according to claim 1 .

5. The stabilization processing unit calculates a representative value reliability, which is a reliability of a representative value of the predicted labels, by performing weighted addition of the predicted labels using the predicted label reliability and the determination result of the signal quality, and outputs the representative value of the predicted labels as the emotion estimation result by thresholding the representative value reliability. The signal processing device according to claim 4 .

6. The stabilization processing unit calculates the representative value reliability as follows: [Equation 3] Calculate it as follows: The yi represents the predicted label labeled for the i-th event in the time-series data of the predicted labels of the emotional states labeled for each time-series event within the sliding window, the ci represents the predicted label reliability of the predicted label of the i-th event, the si represents the signal quality score that is the determination result of the signal quality of the predicted label of the i-th event, the Δti represents the duration of the i-th event, and the wi represents the forgetting weight of the predicted label of the i-th event. The signal processing device according to claim 5 .

7. the signal quality determination unit calculates a signal quality using a class m defined for each type for which a quality determination discrimination model is constructed, two or more types of class labels α m discriminated by the quality determination discrimination model for the class m, a reliability d m of the class label α m corresponding to the class m, and a function f() for adjusting the reliability d m that is set in response to a comparison between the signal quality and a predetermined threshold value, [Equation 5] and outputs the signal quality score sm for the class m. The signal processing device according to claim 6 .

8. The function f() is monotonically increasing for a class whose signal quality is identified as being better than a predetermined threshold, and is monotonically decreasing for a class whose signal quality is identified as being worse than a predetermined threshold and containing noise, for each of one or more classes. The signal processing device according to claim 7 .

9. The signal quality determination unit outputs the signal quality score according to the strength of the periodicity of the signal. The signal processing device according to claim 6 .

10. When the feature quantities extracted from j types of biosignals are used as input variables, the signal quality judgment unit uses a class m defined for each type for which a discriminant model for quality judgment is constructed, two or more types of class labels α m,j discriminated by the discriminant model for quality judgment of the class m, a reliability dm,j of the class label α m,j corresponding to the class m, a function f() for adjusting the reliability dm,j set in accordance with a comparison between the signal quality and a predetermined threshold, and a weight W j for weighted addition of the signal quality scores for each of the j types of biosignals, [Equation 6] outputting a signal quality score sm for said class m by The weight W j of the weighted sum of the signal quality scores is calculated using the model contribution w jk of the feature value k belonging to the j type of signal F j as follows: [Equation 7] is expressed as The signal processing device according to claim 6 .

11. The stabilization processing unit calculates the representative value reliability as follows: [Equation 4] Calculate it as follows: The yi represents the predicted label labeled for the i-th event in the time-series data of the predicted labels of the emotional states labeled for each time-series event within the sliding window, the ci represents the predicted label reliability of the predicted label of the i-th event, the si represents the signal quality score that is the determination result of the signal quality of the predicted label of the i-th event, the Δt i represents the duration of the i-th event, and the wi represents the forgetting weight of the predicted label of the i-th event. The signal processing device according to claim 5 .

12. The biological signal is at least one of a signal obtained by measuring an electroencephalogram, mental sweating, a pulse wave, a blood flow, or a continuous blood pressure. The signal processing device according to claim 1 .

13. The device further includes a biosensor that measures the biosignal. The signal processing device according to claim 1 .

14. The housing is configured to be wearable. The signal processing device according to claim 1 .

15. The signal processing device Based on the measured biosignals, physiological indices that contribute to emotions are extracted as features, outputting time series data of predicted labels of emotional states within the sliding window using a pre-constructed discriminative model for the time series data of the feature quantities within the sliding window; The emotion estimation result is output based on the result of weighting and adding the predicted label using a predicted label reliability, which is the reliability of the predicted label. Signal processing methods.

Images