Biological signal processing device, method, program, and emotion estimation device

Abstract

To appropriately remove noise in biological signals acquired by using a biological sensor.SOLUTION: A biological signal processing device (1) includes a calculation processing unit (20) that acquires a first biological signal (S1) in a time domain based on a measurement result of biological information of a subject (U1) by a biological sensor (SNS) and converts the first biological signal into a second biological signal (S2) in a frequency domain. The calculation processing unit determines whether or not noise is superimposed on the first biological signal on the basis of a change in the intensity of the first biological signal, and upon determining that the noise is superimposed, reduces and corrects each frequency component of the second biological signal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a biological signal processing device, a method and a program, and a feeling estimation device. [Background technology]

[0002] A method has been proposed in which a biosensor measures the bioinformation of a subject and estimates the subject's emotions based on the measurement results. Electroencephalograms are often used as the bioinformation for emotion estimation. For example, a technology is being developed that performs vehicle driving assistance control based on the emotion estimation results when the subject is a vehicle driver, or that controls the progress of a game based on the emotion estimation results when the subject is a game player.

[0003] The EEG sensor as a biosensor has one or more electrodes. The EEG sensor is attached to a subject so that each electrode is in contact with the subject's skin to read the electrical signal of the EEG. The following Patent Document 1 proposes a method for distinguishing between sections containing noise and sections with little noise based on the EEG and for judging drowsiness. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-283 A Summary of the Invention [Problem to be solved by the invention]

[0005] Since brainwaves are minute electrical signals, they are easily affected by noise. In particular, when applying an EEG sensor to a vehicle driver or a game player, ease and comfort of attachment of the electrodes is required. If priority is given to ease and comfort of attachment of the electrodes, the effect of noise tends to be greater compared to medical EEG sensors in which the electrodes are attached precisely. In addition, the movement of the subject, such as the driver, can easily introduce large noise into the measurement results.

[0006] Noise contained in EEG measurement results reduces the accuracy of emotion estimation, so technology to appropriately remove noise is required.

[0007] For the sake of concrete explanation, the present invention focuses on electroencephalograms and uses the results of electroencephalogram estimation for emotion estimation. However, even when the results of electroencephalogram measurement are used for purposes other than emotion estimation, or when the biological information to be acquired is other than electroencephalograms, it is necessary to appropriately remove noise. Note that the technology disclosed in Patent Document 1 cannot remove noise.

[0008] An object of the present invention is to provide a biological signal processing device, method, and program, as well as a feeling estimation device, that contribute to appropriate removal of noise contained in measurement results of biological information. [Means for solving the problem]

[0009] The biosignal processing device of the present invention acquires a first biosignal in the time domain based on the measurement results of a subject's bioinformation by a biosensor, converts the first biosignal into a second biosignal in the frequency domain, determines whether or not noise is superimposed on the first biosignal based on changes in intensity of the first biosignal, and when it is determined that noise is superimposed, reduces and corrects each frequency component of the second biosignal. Effect of the Invention

[0010] According to the present invention, it is possible to provide a biological signal processing device, method, program, and emotion estimation device that contribute to appropriately removing noise contained in measurement results of biological information. [Brief description of the drawings]

[0011] [Figure 1] 1 is a configuration diagram of a biological signal processing device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a relationship diagram of a plurality of unit periods arranged in a time series according to the first embodiment of the present invention. [Diagram 3]FIG. 1 is a diagram showing a waveform (power spectrum) of a biological signal in the time domain and a biological signal in the frequency domain (without noise superposition) according to the first embodiment of the present invention. [Figure 4] FIG. 2 is a waveform diagram of a biological signal in the time domain on which noise is superimposed according to the first embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram illustrating the influence of noise superimposition on a biological signal in the frequency domain according to the first embodiment of the present invention. [Figure 6] FIG. 4 is a diagram showing a change in intensity of a biological signal output from an AFE in the first embodiment of the present invention. [Figure 7] FIG. 1 is a conceptual diagram illustrating a noise removal method for a biological signal according to a first embodiment of the present invention. [Figure 8] FIG. 11 is a diagram showing a relationship between a target period and a reference period according to an example EX1_1 belonging to the first embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing the relationship between the moving average of the intensity of the biological signal in the reference period and the intensity of the biological signal in the target period according to Example EX1_1 belonging to the first embodiment of the present invention. [Figure 10] 11 is an operation flowchart of the biological signal processing apparatus according to Example EX1_1 belonging to the first embodiment of the present invention. [Figure 11] FIG. 11 is a configuration diagram of a feeling estimation device according to a third embodiment of the present invention. [Figure 12] FIG. 11 is a functional block diagram of one arithmetic processing unit in the feeling estimation device according to the third embodiment of the present invention. [Figure 13] FIG. 11 is a functional block diagram of one arithmetic processing unit in the feeling estimation device according to the third embodiment of the present invention. [Figure 14] FIG. 13 is a diagram showing an example of a two-dimensional model related to emotion estimation according to the third embodiment of the present invention. [Figure 15] FIG. 13 is a diagram showing another example of a two-dimensional model related to emotion estimation in the third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, examples of embodiments of the present invention will be described in detail with reference to the drawings. In each of the drawings, the same parts are given the same reference numerals, and duplicated descriptions of the same parts are omitted as a rule. In this specification, for the sake of simplicity, by writing a symbol or code referring to information, signal, physical quantity, functional part, circuit, element, or part, the name of the information, signal, physical quantity, functional part, circuit, element, or part corresponding to the symbol or code may be omitted or abbreviated.

[0013] <<First embodiment>> A first embodiment of the present invention will be described. Fig. 1 shows a configuration of a biosignal processing device 1 according to this embodiment. Fig. 1 also shows a subject U1 and a biosensor SNS. The biosignal processing device 1 and the biosensor SNS form a biosignal measuring device. The biosignal processing device 1 includes an AFE10 which is an analog front end, an arithmetic processing unit 20, a storage unit 30, and an alarm unit 40.

[0014] The biosensor SNS measures the bioinformation of the subject U1. The biosensor SNS is attached to the subject U1 to acquire the bioinformation of the subject U1. The biosensor SNS measures the bioinformation of the subject U1, and generates and outputs a signal S0 representing the measurement result of the bioinformation of the subject U1. The output signal S0 of the biosensor SNS is input to the biosignal processing device 1.

[0015] In the first embodiment, the bioinformation of the subject U1 measured by the biosensor SNS is the brainwaves of the subject U1. Therefore, the biosensor SNS according to the first embodiment is an brainwave sensor. The brainwave sensor as the biosensor SNS measures the brainwaves of the subject U1, and generates and outputs a signal S0 indicating the measurement result of the brainwaves. The brainwave sensor has one or more electrodes, and the brainwave sensor is attached to the subject U1 so that each electrode is in contact with the skin of the subject U1 (for example, in contact with the scalp). The brainwave sensor as the biosensor SNS reads electrical signals emitted by nerve cells constituting the brain of the subject U1, and outputs the read electrical signals as signal S0 to the biosignal processing device 1.

[0016] The signal S0 is input to the AFE10. The AFE10 generates a signal S1 by performing a predetermined signal processing on the signal S0. Both the signals S0 and S1 are voltage signals in the time domain. The signal S1 is hereinafter referred to as a biosignal S1. In the first embodiment, the biosignal S1 and biosignals S2 to S4 described below are electroencephalogram signals.

[0017] The signal processing in the AFE 10 includes amplification and AD conversion. Therefore, the AFE 10 has an amplifier circuit 11 that performs amplification and an AD conversion circuit 12 that performs AD conversion. The amplifier circuit 11 amplifies the signal S0 from the biosensor SNS and outputs the amplified signal S0. The signal S0 before and after amplification by the amplifier circuit 11 is an analog voltage signal. The AD conversion circuit 12 generates a biosignal S1 by converting the output signal of the amplifier circuit 11 (i.e., the amplified signal S0) into a digital signal. Therefore, the biosignal S1 has a digital value proportional to the value of the output signal S0 of the biosensor SNS. The value (digital value) of the biosignal S1 increases if the value of the output signal S0 of the biosensor SNS increases, and decreases if the value of the output signal S0 of the biosensor SNS decreases.

[0018] The signal processing in the AFE 10 may include processing other than the amplification processing and the AD conversion processing (filtering processing, etc.). The whole or part of the AFE 10 may be provided in the biosensor SNS. When the whole of the AFE 10 is provided in the biosensor SNS, the biosignal S1 is supplied from the biosensor SNS to the biosignal processing device 1.

[0019] The biosignal S1 generated by the AFE 10 is input to the calculation processing unit 20. The calculation processing unit 20 is a so-called controller, and includes a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) as hardware resources, and controls the overall operation of the biosignal processing device 1.

[0020] The storage unit 30 is configured to have a non-volatile memory such as a ROM (Read only memory) or a flash memory, and a volatile memory such as a RAM (Random access memory). The storage unit 30 stores various data referenced by the arithmetic processing unit 20, as well as various programs executed by the arithmetic processing unit 20.

[0021] The notification unit 40 performs various notifications to the user. The user is a user of the biological signal processing device 1. The user may be the subject U1 himself, or may include a person other than the subject U1.

[0022] The following describes in detail the functions of the arithmetic processing unit 20. The arithmetic processing unit 20 has function blocks 21 to 24. The function blocks 21, 22, 23, and 24 are a frequency analysis unit, a noise determination unit, a correction processing unit, and an inverse conversion unit, respectively.

[0023] The arithmetic processing unit 20 includes a program execution device (computer) capable of executing any program. The arithmetic processing unit 20 executes a predetermined program for biological signal processing, thereby realizing each function of the arithmetic processing unit 20 (including each function of the functional blocks 21 to 24). All of the operations of the arithmetic processing unit 20 described in this embodiment may be operations that are realized by the arithmetic processing unit 20 executing the program for biological signal processing. The program for biological signal processing may be composed of multiple programs.

[0024] The frequency analysis unit 21 converts the time domain biosignal S1 supplied from the AFE 10 into a frequency domain biosignal S2 using a fast Fourier transform. See FIG. 2. Any time after the biosensor SNS and the biosignal processing device 1 are activated and the biosensor SNS is able to measure bioinformation and the biosignal processing device 1 is able to process signals is called time t0. As shown in FIG. 2, in the biosignal processing device 1, the period after time t0 is divided into a plurality of unit periods that are arranged consecutively in a time series. The i-th unit period is called unit period P[i]. i represents any natural number. FIG. 2 also shows unit period P[M], which is the M-th unit period. M represents an integer of 3 or more, and may be interpreted as having a sufficiently large integer value (for example, 10000).

[0025] The start time of the unit period P[1] coincides with time t0. Each unit period has a length of a predetermined unit time. The unit time coincides with L times the reciprocal of the sampling frequency. The sampling frequency here is the sampling frequency of AD conversion (analog / digital conversion) in the AD conversion circuit 12. L has an integer value of 2 or more (for example, 256 or 1024). Then, the biosignal S1 during one unit period has L digital values ​​arranged in a time series. The frequency analysis unit 21 generates the biosignal S2 by applying a fast Fourier transform to the result obtained by multiplying the biosignal S1 by a window function having a length of the unit time. At this time, the window function is shifted in the time axis direction by unit time. As a result, the biosignal S1 is converted into the biosignal S2 for each unit period. That is, the biosignal S2 for each unit period is generated sequentially as time progresses based on the biosignal S1.

[0026] FIG. 3(a) shows the waveform of a biosignal S1 in a certain unit period, and FIG. 3(b) shows the waveform of a biosignal S2 in a certain unit period. In the waveform of the biosignal S1, the horizontal axis corresponds to time, and the vertical axis corresponds to voltage (digital value representing the voltage value of the biosignal S1). The biosignal S1 is a digital voltage signal expressed in the time domain. In the waveform of the biosignal S2, the horizontal axis corresponds to frequency, and the vertical axis corresponds to intensity. The frequency analysis unit 21 generates the power spectrum of the biosignal S1 as the biosignal S2. Therefore, the intensity in the biosignal S2 is power. The frequency analysis unit 21 divides the frequency range of the biosignal S1 into 1st to Fth subdivision bands, and calculates the power of the biosignal S1 for each subdivision band to generate the power spectrum of the biosignal S1. F represents an integer of 2 or more. The 1st to Fth subdivision bands are mutually different bands, and the frequency belonging to the (i+1)th subdivision band is higher than the frequency belonging to the ith subdivision band. The frequency range of the biosignal S1 is the entire band of the signal components of the biosignal S1, and is actually a band (for example, a band of 0.1 Hz or more and 100 Hz or less) defined by the frequency analysis unit 21. The power spectrum of the biosignal S1 (i.e., the biosignal S2) represents the power of the biosignal S1 for each subdivided band.

[0027] The frequency analysis unit 21 may generate an amplitude spectrum of the biosignal S1 as the biosignal S2. In this case, the intensity of the biosignal S2 is the amplitude.

[0028] The noise determination unit 22 determines whether noise is superimposed on the biosignal S1. Superimposition of noise on the biosignal S1 is, in other words, superimposition of noise on the biosignal S2. In this embodiment, the noise that is of particular interest is noise caused by the body movement of the subject U1. When the subject U1 moves his / her body in such a manner that a force is applied to the contact surface between the subject U1 and the electrode of the biosensor SNS, a signal caused by the movement is added to the output signal S0 as noise, and as a result, the noise component is superimposed on the biosignal S1.

[0029] FIG. 4 shows an example of the waveform of the biosignal S1 in a case where noise due to the body movement of the subject U1 is superimposed on the biosignal S1. Hereinafter, the noise due to the body movement of the subject U1 is particularly referred to as body movement noise. In FIG. 5, the solid line spectrum 621 is the power spectrum of the biosignal S1 in a case where the body movement noise is superimposed on the biosignal S1. In FIG. 5, the dashed line spectrum 622 is the power spectrum of the biosignal S1 when the body movement noise component is removed from the solid line power spectrum 621. In FIG. 5, the shaded area 623 represents the body movement noise component. If the body movement noise component 623 can be subtracted from the solid line spectrum 621, a spectrum from which the influence of the body movement noise has been removed (dashed line spectrum 622) can be obtained.

[0030] The noise determination unit 22 derives the intensity of the biological signal S1 for each unit period. The noise determination unit 22 determines the power (in other words, the energy) of the biological signal S1 as the intensity of the biological signal S1. The noise determination unit 22 determines the sum of the power of the biological signal S1 in the unit period P[i] as the power PW[i] of the biological signal S1 in the unit period P[i]. Therefore, the noise determination unit 22 derives the power of the biological signal S1 for each unit period P[1] to P[M], thereby determining the power PW[1] to PW[M].

[0031] Fig. 6(a) shows the time series change in the strength (power) of the biosignal S1 during a period without body movement noise, and Fig. 6(b) shows the time series change in the strength (power) of the biosignal S1 during a period including the timing when the biosignal S1 is superimposed with body movement noise.

[0032] During the period when body movement noise is superimposed on the biosignal S1, the magnitude of the biosignal S1 (and therefore the power of the biosignal S1) becomes abnormally large compared to other periods (see Figures 4 and 6(b)). Taking this into consideration, the noise determination unit 22 determines whether noise is superimposed on the biosignal S1 based on changes in the intensity (power) of the biosignal S1. The noise determined by the noise determination unit 22 to be superimposed is mainly body movement noise, but it may be noise other than body movement noise. The result of the determination by the noise determination unit 22 as to whether noise is superimposed is transmitted to the correction processing unit 23 (see Figure 1).

[0033] When noise is superimposed on the biological signal S1, the noise components are also superimposed on the power spectrum of the biological signal S1, and the noise increases each frequency component of the biological signal S2 over a wide frequency range. This effect corresponds to the change from the dashed spectrum 622 to the solid spectrum 621 in FIG. 5.

[0034] Therefore, when the noise determination unit 22 determines that the biosignal S1 of the unit period P[i] has noise superimposed thereon, the correction processing unit 23 executes a correction process to reduce each frequency component of the biosignal S2 of the unit period P[i]. The correction process can remove the noise components superimposed on the biosignal S2. When the noise determination unit 22 determines that the biosignal S1 of the unit period P[i] has no noise superimposed thereon, the correction processing unit 23 does not execute the correction process. The correction processing unit 23 outputs a biosignal S3 based on the biosignal S2 for each unit period. The biosignal S3 is a biosignal in the frequency domain, similar to the biosignal S2. When it is determined that the biosignal S1 of the unit period P[i] has noise superimposed thereon, the biosignal S2 after the correction process in the unit period P[i] is the biosignal S3 of the unit period P[i]. When it is determined that no noise is superimposed on the biosignal S1 for the unit period P[i], the biosignal S2 for the unit period P[i] is used as it is as the biosignal S3 for the unit period P[i] without undergoing correction processing.

[0035] A biosignal S3 is supplied to the inverse transform unit 24. The inverse transform unit 24 uses an inverse fast Fourier transform to transform the biosignal S3 in the frequency domain into a biosignal S4, which is a biosignal in the time domain. Therefore, when noise is superimposed on the biosignal S1, a biosignal in the time domain from which the noise components have been removed is obtained as the biosignal S4. In this specification, the removal of noise components means that the noise components are completely or partially removed, and both belong to the reduction of noise components. With regard to noise, removal and reduction may be interpreted as interchangeable.

[0036] FIG. 7 is a conceptual diagram showing how a biosignal S4 is obtained from the biosignal S1 through the generation of biosignals S2 and S3 when noise is superimposed on the biosignal S1. The biosignal S1 on which noise is superimposed is converted into a biosignal in the frequency domain, and a biosignal S2 with a solid line spectrum 621 is obtained. The solid line spectrum 621 shown in FIG. 7 corresponds to the biosignal S2 during the noise superimposition period (a unit period during which it is determined that noise is superimposed). It is determined that noise is superimposed on the biosignal S1, and based on the determination result, the correction processing unit 23 reduces and corrects each frequency component of the biosignal S2 during the noise superimposition period. As a result, for the noise superimposition period, a biosignal S3 with a dashed line spectrum 622 is obtained from the biosignal S2 with the solid line spectrum 621. The biosignal S3 in the frequency domain is converted into a biosignal in the time domain, and a biosignal S4 is obtained. The biosignal S4 in FIG. 7 corresponds to the biosignal S1 in FIG. 7 from which the noise component has been removed.

[0037] The biosignal S3 may be output to an external device (not shown) as a biosignal (biological signal in the frequency domain) of the subject U1 to be generated by the biosignal processing device 1. Various estimation processes (such as estimation of the health condition or emotion of the subject U1) based on the biosignal S3 may be performed in a device (not shown) that includes the biosignal processing device 1. The biosignal S4 may be output to an external device (not shown) as a biosignal (biological signal in the time domain) of the subject U1 to be generated by the biosignal processing device 1. Various estimation processes (such as estimation of the health condition or emotion of the subject U1) based on the biosignal S4 may be performed in a device (not shown) that includes the biosignal processing device 1.

[0038] In this way, whether or not noise is superimposed on the biosignal S1 is determined based on the change in the intensity of the biosignal S1, and when it is determined that noise is superimposed, each frequency component of the biosignal S2 is reduced and corrected. Considering the characteristic that noise is superimposed on the biosignal S1 over a wide frequency range, a method is adopted in which the biosignal is corrected in the frequency domain (i.e., each frequency component of the biosignal S2 is reduced and corrected). This makes it possible to obtain bioinformation (brainwave information in this case) in which the influence of noise is appropriately suppressed. In medical examinations and the like, the subject is required to remain still when measuring bioinformation. However, in an environment in which bioinformation is measured in daily life using wearable devices, the subject U1 as a subject moves his or her body in various ways. According to this embodiment, even in such an environment, it is possible to obtain effective bioinformation (brainwave signal in this case) in which the influence of noise is suppressed.

[0039] Furthermore, by providing the inverse conversion unit 24, it is possible to obtain a time-domain biosignal S4 in which the influence of noise is suppressed. When the time-domain biosignal is required in the subsequent processing, the provision of the inverse conversion unit 24 is beneficial.

[0040] However, the inverse conversion unit 24 is not required (for example, the inverse conversion unit 24 is not required when the subsequent processing handles biological signals in the frequency domain), and therefore the inverse conversion unit 24 may be omitted in the biological signal processing device 1.

[0041] In the case where the biosensor SNS is an electroencephalogram sensor as in this embodiment, the output signal S0 of the electroencephalogram sensor has a weak intensity, so that the influence of body movement noise becomes significant, and therefore the correction method according to this embodiment works particularly effectively.

[0042] The first embodiment includes the following Examples EX1_1 to EX1_3. The matters described above in the first embodiment are applied to the following Examples EX1_1 to EX1_3 unless otherwise specified and unless there is a contradiction. However, in each example, for matters that contradict the matters described above in the first embodiment, the description in each example may take precedence. Furthermore, unless there is a contradiction, matters described in any of Examples EX1_1 to EX1_3 can also be applied to any other example (i.e., any two or more of the multiple examples can be combined).

[0043] [Example EX1_1] Example EX1_1 will be described. In Example EX1_1, a specific example of a method for determining whether or not noise is superimposed by the noise determining unit 22 and a specific example of a correction process by the correction processing unit 23 will be described.

[0044] Referring to FIG. 8, a period for which the presence or absence of noise superimposed on the biological signal S1 is to be determined is referred to as a target period. The noise determination unit 22 sets a reference period to a period prior to the target period, and determines the presence or absence of noise superimposed on the biological signal S1 during the target period by referring to the biological signal S1 during the reference period. For the sake of concrete explanation, the target period is assumed to be a unit period P[M]. In this case, the reference period is a combined period of unit periods P[MN] to P[M-1]. That is, the most recent N consecutive unit periods from the target period are set as the reference period. N represents an integer of 2 or more, and may be, for example, several tens to several hundreds.

[0045] As described above, the noise determination unit 22 obtains the power of the biological signal P1 as the intensity of the biological signal P1 for each unit period based on the biological signal P1. Therefore, for the reference period and the target period, the power PW[i] of the biological signal P1 is obtained for each integer i that satisfies "MN≦i≦N".

[0046] The noise determination unit 22 executes a moving average derivation process to obtain a moving average of the intensity (power) of the biological signal P1. After the moving average derivation process is enabled, the moving average derivation process is constantly executed, and the value of the moving average is updated every time the power of the biological signal P1 is obtained. The moving average obtained by the noise determination unit 22 is a moving average of the most recent N sample values. The moving average obtained by the noise determination unit 22 may be a simple moving average, but may also be another type of moving average (for example, a weighted moving average or an exponential moving average).

[0047] The sample value in the noise determination unit 22 is the intensity of the biological signal P1, which is the power PW[i] of the biological signal P1 in this case. When the power PW[i] is calculated, the most recent N sample values ​​are powers PW[i-N+1] to PW[i].

[0048] Therefore, at the stage where the power PW[M] for the target period is calculated, the most recent N sample values ​​are the powers PW[MN] to PW[M-1] for the reference period. Therefore, when focusing on the target period, the noise determination unit 22 derives the moving average of the intensity of the biosignal S1 based on the biosignal S1 in the reference period. Then, the noise determination unit 22 calculates the difference DIF between the moving average value MV[M-1] just before the target period and the power PW[M], which is the intensity of the biosignal S1 in the target period. Here, the difference DIF is the difference (PW[M]-MV[M-1]), and therefore the difference DIF has a positive value only when the power PW[M] is greater than the moving average value MV[M-1].

[0049] Fig. 9 shows the relationship between the moving average value MV[M-1] and the power PW[M]. In Fig. 9, the waveform 660 is the moving average line of the power of the biological signal S1 obtained before the target period. Since the most recent N sample values immediately before the target period are the powers PW[M-N] to PW[M-1], the moving average value MV[M-1] immediately before the target period is the moving average value of the powers PW[M-N] to PW[M-1]. For example, when a simple moving average is used as the moving average, the moving average value MV[M-1] corresponds to the average value (simple average value) of the powers PW[M-N] to PW[M-1]. Or, for example, when a weighted moving average is used as the moving average, the moving average value MV[M-1] corresponds to the weighted average value of the powers PW[M-N] to PW[M-1]. At this time, the weights in the weighted average are set in advance. Here, it is assumed that there is no superposition of noise (body movement noise) on the biological signal S1 during the reference period.

[0050] The noise determination unit 22 determines whether there is superposition of noise on the biological signal S1 during the target period based on the difference DIF. When noise is superimposed on the biological signal S1 during the target period, the difference DIF is considered to have a correspondingly large value. Therefore, the noise determination unit 22 compares the difference DIF with the threshold value TH, and when the difference DIF is greater than or equal to the threshold value TH, it determines that noise is superimposed on the biological signal S1 during the target period. That is, when "DIF≧TH" holds, the noise determination unit 22 determines that there is superposition of noise on the biological signal S1 during the target period. The noise determination unit 22 determines that there is no superposition of noise on the biological signal S1 during the target period when the difference DIF is less than the threshold value TH. That is, when "DIF<TH" holds, the noise determination unit 22 determines that there is no superposition of noise on the biological signal S1 during the target period.

[0051] It is also possible to set the threshold value TH to a predetermined fixed value. However, the intensity level and variation of the biological signal S1 vary depending on individual differences of the person wearing the biological sensor SNS and the wearing status of the biological sensor SNS. Therefore, the noise determination unit 22 may set the threshold value TH based on the statistic of the intensity (power) of the biological signal S1 during the reference period.

[0052] Specifically, for example, the noise determination unit 22 may set the threshold TH (threshold for the degree of fluctuation in the intensity of the biological signal S1) according to the following formula (1). In formula (1), σ is the standard deviation of the intensity of the biological signal S1 in the reference period, that is, the standard deviation of the powers PW[MN] to PW[M-1]. In formula (1), k1 is a coefficient and has a predetermined positive value (such as 2.0 or 3.0). Since power fluctuations occur by about the standard deviation σ even in the absence of noise, it is preferable that the coefficient k1 is greater than 1. TH = k1 × σ (1)

[0053] Alternatively, for example, the noise determination unit 22 may set the threshold TH (threshold for the intensity level of the biological signal S1) according to the following formula (2). AVE in formula (2) is the average value of the intensity of the biological signal S1 in the reference period, that is, the average value of the powers PW[MN] to PW[M-1]. k2 in formula (2) is a coefficient and has a predetermined positive value (such as 1.2 or 2.0). TH = k2 × AVE (2)

[0054] A case in which it is determined that noise is superimposed on the biological signal S1 during the target period is referred to as a noisy case. In the noisy case, the correction processing unit 23 executes a correction process. In the correction process for the noisy case, the correction processing unit 23 reduces and corrects each frequency component of the biological signal S2 during the target period by an amount corresponding to the difference DIF, and outputs the reduced and corrected biological signal S2 as the biological signal S3 for the target period.

[0055] As described above, the frequency analysis unit 21 divides the frequency range of the biosignal S1 into 1st to Fth sub-bands, obtains the power of the biosignal S1 for each sub-band, and generates a biosignal S2, which is a power spectrum of the biosignal S1. The biosignal S2 has 1st to Fth frequency components, and the i-th frequency component in the biosignal S2 represents the power of the i-th sub-band in the biosignal S1. Reducing and correcting each frequency component of the biosignal S2 by an amount corresponding to the difference DIF means reducing each of the 1st to Fth frequency components in the biosignal S2 by an amount corresponding to the difference DIF.

[0056] The sum of the power of all frequency components in the biosignal S3 during the target period is the corrected power sum PW3 SUM The sum of the power of all frequency components in the biological signal S2 during the target period is called the pre-correction power sum PW2 SUM In the case where noise is present, the correction processing unit 23 calculates the corrected power sum PW3 SUM is the total power before correction PW2 SUM than the value (k A The correction process is performed so that the difference is reduced by k × DIF. A is a given coefficient, which has a positive value less than or equal to 1. Typically, “k A = 1”, but k A The value of k can be slightly smaller than 1. A It is possible to set the value of k slightly larger than 1. A =1".

[0057] Body movement noise is considered to have characteristics similar to impulse noise, and in this case, the body movement noise functions as white noise. Therefore, in the case where noise is present, the correction processing unit 23 should assume that the noise superimposed on the biosignal S1 is white noise, and reduce each frequency component of the biosignal S2 so that the white noise is removed from the biosignal S2. For example, in the case of “PW3 SUM =PW2 SUM -k A ×DIF" reduction correction "PW3 SUM =PW2 SUM / p", then the first to Fth frequency components in the biosignal S2 should be reduced by 1 / p times in the correction process, where p is greater than 1. The shaded area 623 in FIG. 7 represents noise components when it is assumed that white noise is superimposed on the biosignal S1, and the noise components are subtracted from the biosignal S2 (spectrum 621) to obtain the biosignal S3 (spectrum 622).

[0058] However, in the case where noise is present, the correction processing unit 23 may assume that the noise superimposed on the biological signal S1 is colored noise, and reduce each frequency component of the biological signal S2 so that the colored noise is removed from the biological signal S2. The colored noise is, for example, pink noise, brown noise, or blue noise.

[0059] A case in which it is determined that there is no noise superimposed on the biological signal S1 during the target period is referred to as a noise-free case. In the noise-free case, the correction processing unit 23 does not execute the correction process. In the noise-free case, the correction processing unit 23 outputs the biological signal S2 during the target period derived by the frequency analysis unit 21 as the biological signal S3 during the target period without performing the correction process.

[0060] Fig. 10 is an operation flowchart of the biosignal processing device 1. The flow of operation of the biosignal processing device 1 will be described with reference to Fig. 10. The operation of Fig. 10 starts when the biosignal processing device 1 is started, and first, a preparatory operation of step S11 is executed. The biosensor SNS starts up simultaneously with the biosignal processing device 1 or starts up before the biosignal processing device 1.

[0061] In the preparatory operation of step S11, a predetermined initialization sequence is performed in the biosignal processing device 1, and then acquisition of the biosignal S1 is continued for a time period equal to or longer than N times the unit period. In the preparatory operation of step S11, the noise determination unit 22 derives the power of each unit period, and starts the above-mentioned moving average derivation process. After step S11, the process proceeds to step S12. At the stage of proceeding from step S11 to step S12, the moving average derivation process has already started, and thereafter, the value of the moving average is updated every time a unit period elapses (every time the power of the biosignal P1 is calculated).

[0062] In step S12, the calculation processing unit 20 checks whether a biosignal S1 for one unit period has been newly acquired (whether it has been input from the AFE 10). The transition from step S12 to step S13 is suspended until a biosignal S1 for one unit period is newly acquired, and when a biosignal S1 for one unit period is newly acquired (Y in step S12), the process proceeds from step S12 to step S13.

[0063] In step S13, the calculation processing unit 20 sets the latest unit period as the target period. The biosignal S1 in the latest unit period is the newly acquired biosignal S1 for one unit period. Here, the latest unit period is considered to be the unit period P[M]. In step S13, the noise determination unit 22 derives the intensity of the biosignal S1 in the target period, that is, the power PW[M] of the biosignal S1 in the target period (unit period P[M]). Also in step S13, the frequency analysis unit 21 converts the biosignal S1 in the target period into a biosignal S2 in the frequency domain. After step S13, proceed to step S14.

[0064] In step S14, the noise determination unit 22 derives the difference DIF between the moving average value MV[M-1] immediately before the target period and the intensity (i.e., power PW[M]) of the biological signal S1 during the target period. As described above, "DIF=PW[M]-MV[M-1]" (see FIG. 9). After step S14, the process proceeds to step S15.

[0065] In step S14, the noise determination unit 22 determines whether "DIF≧TH" is true or false. A case in which "DIF≧TH" is true corresponds to the above-mentioned case in which noise is present. If "DIF≧TH" is true (Y in step S15), the process proceeds to step S16. If "DIF≧TH" is not true corresponds to the above-mentioned case in which noise is not present. If "DIF≧TH" is not true (N in step S15), the process proceeds to step S18.

[0066] In step S16, the correction processing unit 23 executes the above-mentioned correction process to reduce and correct each frequency component of the biological signal S2 in the target period. After step S16, the process proceeds to step S17.

[0067] In step S17, the correction processing unit 23 outputs the biosignal S2 after the reduction correction in step S16 as the biosignal S3 of the target period. In step S18, the correction processing unit 23 outputs the biosignal S2 of the target period generated by the frequency analysis unit 21 as it is as the biosignal S3 of the target period. After step S17 or S18, the process proceeds to step S19. In step S19, the inverse conversion unit 24 converts the biosignal S3 of the target period output from the correction processing unit 23 into a biosignal in the time domain to obtain the biosignal S4 of the target period. After step S19, the process returns to step S12. Thereafter, a loop process consisting of the process group of steps S13 to S19 is repeatedly executed every time a biosignal S1 for one unit period is newly acquired. Note that, when proceeding to step S18, the biosignal S1 of the target period may be output as the biosignal S4 of the target period in step S18, and then the process may return to step S12 without passing through step S19.

[0068] As shown in Example EX1_1, the noise determination unit 22 determines whether or not noise is superimposed on the biosignal S1 during the target period based on the intensity (power) of the biosignal S1 during the reference period and the intensity (power) of the biosignal S1 during the target period. The reference period is set before the target period (see Figs. 8 and 9). Then, when it is determined that noise is superimposed on the biosignal S1 during the target period, the correction processing unit 23 reduces and corrects each frequency component of the biosignal S2 during the target period. According to this method, it is possible to accurately determine whether or not noise is superimposed from the intensity change of the biosignal S1, and when noise is superimposed, it is possible to properly remove the noise from the biosignal in the frequency domain.

[0069] The moving average of the strength (power) of the biosignal S1 is calculated sequentially, and the strength (power) of the biosignal S1 during the target period is compared with the moving average value immediately before the target period to calculate the difference DIF between them (step S14). Since the difference DIF varies greatly depending on whether noise is superimposed on the biosignal S1 during the target period, it is possible to determine whether noise is superimposed on the biosignal S1 during the target period based on the difference DIF. In other words, based on the difference DIF, it is possible to properly determine whether noise is superimposed on the biosignal S1 during the target period.

[0070] Specifically, when "DIF≧TH" is established, it is determined that noise is superimposed on the biosignal S1 during the target period. This is because it is considered that the difference DIF has a correspondingly large value in cases where noise is superimposed on the biosignal S1 during the target period. Furthermore, the intensity level and fluctuation of the biosignal S1 depend on the individual differences of the person wearing the biosensor SNS and the wearing conditions of the biosensor SNS. Taking this into consideration, the threshold value TH is set based on the statistics of the intensity of the biosignal S1 during the reference period. This makes it possible to appropriately determine whether noise is superimposed on the biosignal S1 during the target period, adapting to the individual differences of the person wearing the biosensor SNS and the wearing conditions of the biosensor SNS.

[0071] The above-mentioned standard deviation σ can be used as a statistic. Since the standard deviation σ represents the variation in intensity in the biosignal S1, when an intensity fluctuation equal to or greater than the threshold TH corresponding to the standard deviation σ is observed, it can be accurately estimated that there is noise superimposition. In other words, by using the standard deviation σ as a statistic, it is possible to properly determine whether or not there is noise superimposition on the biosignal S1 during the target period. Alternatively, the above-mentioned average value AVE can be used as a statistic. Depending on the individual differences of the person wearing the biosensor SNS and the wearing condition of the biosensor SNS, the average value AVE varies even if there is no noise superimposition. For this reason, for example, when the average value AVE is twice as large as viewed from a certain reference state, it is appropriate to set the threshold value TH to about twice as large as viewed from the reference state, and such a setting is considered to contribute to proper determination of whether or not there is noise superimposition. In other words, by using the average value AVE as a statistic, it is possible to properly determine whether or not there is noise superimposition on the biosignal S1 during the target period. The statistic may be set based on both the standard deviation α and the average value AVE.

[0072] As described above, when the correction processing unit 23 determines that noise is superimposed on the biological signal S1 during the target period, it corrects each frequency component of the biological signal S2 during the target period by decreasing the amount according to the difference DIF. Since the main component of the difference DIF is considered to be superimposed noise, the noise component can be effectively removed by decreasing the amount according to the difference DIF.

[0073] [Example EX1_2] Example EX1_2 will be described. In example EX1_2, a modified method of the correction processing of the correction processing unit 23 will be described.

[0074] In the case where noise is present, the correction processing unit 23 may reduce each frequency component of the biosignal S2 in the target period by a predetermined amount (for example, an appropriate value calculated based on an experiment, etc.), and output the reduced and corrected biosignal S2 as the biosignal S3 in the target period. This also makes it possible to remove the noise components to a certain extent (i.e., the influence of the noise is suppressed). However, since the magnitude of the superimposed noise varies and the magnitude of the noise is reflected in the above-mentioned difference DIF, it is preferable to reduce the amount of the correction according to the difference DIF as shown in the embodiment EX1_1.

[0075] Incidentally, correcting each frequency component of the biological signal S2 by decreasing it by a predetermined amount means decreasing each of the first to Fth frequency components in the biological signal S2 by a predetermined amount.

[0076] [Example EX1_3] An embodiment EX1_3 will be described. When the noise determination unit 22 determines that noise is superimposed on the biological signal S1 during the target period, it may output a noise detection signal indicating that to the notification unit 40. When the notification unit 40 receives the noise detection signal, it executes a noise notification process to notify the user that noise has been detected in the measurement of biological information by the biological sensor SNS. The noise notification process allows the user to recognize the occurrence of noise in the measurement of biological information.

[0077] The notification may be implemented in any manner that is perceptible by the user. For example, the notification unit 40 may be provided with a light-emitting element or a display screen, and the notification may be realized by illuminating the light-emitting element or displaying a predetermined image on the display screen. Alternatively, the notification unit 40 may be provided with a buzzer or speaker, and the notification may be realized by outputting a predetermined sound from the buzzer or speaker.

[0078] <<Second embodiment>> The second embodiment of the present invention will be described. The second embodiment and the third and fourth embodiments described later are based on the first embodiment, and for matters not specifically mentioned in the second to fourth embodiments, the description of the first embodiment also applies to the second to fourth embodiments unless there is a contradiction. However, when interpreting the description of the second embodiment, the description of the second embodiment may take precedence for matters that contradict between the first and second embodiments (the same applies to the third and fourth embodiments described later). As long as there is no contradiction, any two or more of the first to third embodiments may be combined.

[0079] In the first embodiment, the measurement target of the biosensor SNS is an electroencephalogram. However, the measurement target of the biosensor SNS may be something other than an electroencephalogram. For example, the measurement target of the biosensor SNS may be the myoelectric potential of the subject U1, and among these, the cardiac potential of the subject U1 may be particularly good. When the measurement target of the biosensor SNS is the cardiac potential of the subject U1, the heart rate of the subject U1 can be measured by the biosensor SNS as the bioinformation of the subject U1.

[0080] In the second embodiment, it is assumed that the measurement target of the biosensor SNS is the cardiac potential of the subject U1, and differences from the first and second embodiments will be described below. A heartbeat sensor is used as the biosensor SNS. The heartbeat sensor as the biosensor SNS measures the heartbeat of the subject U1, and generates and outputs a signal S0 indicating the measurement result of the heartbeat. In detail, the heartbeat sensor detects the action potential (cardiac potential) of the subject U1, which is an electric potential generated on the skin surface of the subject U1, and generates a signal indicating the detected action potential as the signal S0. The signal S0 generated by the heartbeat sensor is input to the biosignal processing device 1. The heartbeat sensor has one or more electrodes, and the heartbeat sensor is attached to the subject U1 so that each electrode is in contact with the skin of the subject U1.

[0081] The processing performed by the AFE10 on the signal S0 and the processing performed by the calculation processing unit 20 on the biological signal S1 output from the AFE10 are the same as those described in the first embodiment. However, in the second embodiment, the biological signals S1 to S4 are cardiac potential signals based on the signal S0 generated by a heartbeat sensor.

[0082] <<Third embodiment>> A third embodiment of the present invention will be described. Fig. 11 shows a configuration of a feeling estimation device 100 according to the third embodiment. Fig. 11 also shows a subject U1 and biosensors SNS_A and SNS_B. The techniques shown in the first and second embodiments are applied to the feeling estimation device 100.

[0083] The biosensor SNS_A is the electroencephalogram sensor described in the first embodiment, and the matters described in the first embodiment regarding the electroencephalogram sensor are applied to the biosensor SNS_A. The output signal S0_A of the biosensor SNS_A is the same as the signal S0 in the first embodiment.

[0084] The biosensor SNS_B is the heart rate sensor described in the second embodiment, and the matters described in the second embodiment regarding the heart rate sensor are applied to the biosensor SNS_B. The output signal S0_B of the biosensor SNS_B is the same as the signal S0 in the second embodiment.

[0085] The feeling estimation device 100 includes AFEs 10_A and 10_B which are analog front-ends, a controller 120, a storage unit 130, and a notification unit 140. Each of the AFEs 10_A and 10_B is the same as the AFE 10 in the first embodiment, and outputs a biological signal S1 based on an input signal S0.

[0086] However, the input signal S0 in AFE10_A is the signal S0_A output from the biosensor SNS_A, and the biosignal S1 output from AFE10_A is the biosignal S1_A based on the signal S0_A. The biosignal S1_A is the biosignal S1 (see FIG. 1) of the first embodiment itself, and is a digital electroencephalogram signal expressed in the time domain. On the other hand, the input signal S0 in AFE10_B is the signal S0_B output from the biosensor SNS_B, and the biosignal S1 output from AFE10_B is the biosignal S1_B based on the signal S0_B. The biosignal S1_B corresponds to the biosignal S1 of the second embodiment, and is a digital cardiac potential signal expressed in the time domain.

[0087] The biological signals S1_A and S1_B generated by the AFEs 10_A and 10_B are input to the controller 120. The controller 120 has a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc. as hardware resources, and is provided with arithmetic processing units 20_A and 20_B and a feeling estimation unit 121 configured by the hardware resources.

[0088] The storage unit 130 is configured to include a non-volatile memory such as a ROM (Read only memory) or a flash memory, and a volatile memory such as a RAM (Random access memory). The storage unit 130 stores various data referenced by the controller 120, as well as various programs executed by the controller 120.

[0089] The notification unit 140 performs various notifications to the user. The user is a user of the feeling estimation device 100. The user may be the subject U1 itself, or may include a person other than the subject U1.

[0090] The controller 120 includes a program execution device (computer) capable of executing any program. Each function of the controller 120 is realized by the controller 120 executing a predetermined emotion estimation program. All operations of the controller 120 described in this embodiment may be operations realized by the controller 120 executing the emotion estimation program. The emotion estimation program may be composed of multiple programs.

[0091] The arithmetic processing unit 20_A is the same as the arithmetic processing unit 20 in the first embodiment, and therefore has a frequency analysis unit 21, a noise judgment unit 22, a correction processing unit 23, and an inverse conversion unit 24 (see FIG. 1). The arithmetic processing unit 20_B corresponds to the arithmetic processing unit 20 in the second embodiment, and has a configuration equivalent to that of the arithmetic processing unit 20 in FIG. 1. Hereinafter, as shown in FIG. 12, the frequency analysis unit 21, the noise judgment unit 22, the correction processing unit 23, and the inverse conversion unit 24 in the arithmetic processing unit 20_A will be referred to as the frequency analysis unit 21_A, the noise judgment unit 22_A, the correction processing unit 23_A, and the inverse conversion unit 24_A, respectively. As shown in FIG. 13, the frequency analysis unit 21, the noise judgment unit 22, the correction processing unit 23, and the inverse conversion unit 24 in the arithmetic processing unit 20_B will be referred to as the frequency analysis unit 21_B, the noise judgment unit 22_B, the correction processing unit 23_B, and the inverse conversion unit 24_B, respectively.

[0092] The frequency analysis unit 21_A, the noise determination unit 22_A, the correction processing unit 23_A, and the inverse transformation unit 24_A are the same as the frequency analysis unit 21, the noise determination unit 22, the correction processing unit 23, and the inverse transformation unit 24 in the first embodiment, respectively. The biosignals S1, S2, S3, and S4 handled by the calculation processing unit 20_A are referred to as biosignals S1_A, S2_A, S3_A, and S4_A, respectively, in the third embodiment. The biosignals S1_A and S4_A are electroencephalogram signals in the time domain, and the biosignals S2_A and S3_A are electroencephalogram signals in the frequency domain. The inverse transformation unit 24_A may be omitted.

[0093] The frequency analysis unit 21_B, the noise determination unit 22_B, the correction processing unit 23_B, and the inverse conversion unit 24_B correspond to the frequency analysis unit 21, the noise determination unit 22, the correction processing unit 23, and the inverse conversion unit 24 in the second embodiment, respectively. The biosignals S1, S2, S3, and S4 handled by the calculation processing unit 20_B are referred to as the biosignals S1_B, S2_B, S3_B, and S4_B, respectively, in the third embodiment. The biosignals S1_B and S4_B are cardiac potential signals in the time domain, and the biosignals S2_B and S3_B are cardiac potential signals in the frequency domain. The inverse conversion unit 24_B may be omitted.

[0094] The emotion estimation unit 121 can estimate the emotion of the subject U1 based on information on the brainwaves and heartbeats of the subject U1. In this case, the emotion estimation unit 121 estimates the emotion based on two emotion indices that are indices indicating the mental and physical state of the subject U1. The mental and physical state of any person includes a psychological state, and therefore each emotion index can also be said to be an index indicating a psychological state. The value of the emotion index is called an emotion index value. One of the emotion indices used in this embodiment is the arousal level of the central nervous system (hereinafter referred to as the arousal level), and the other emotion index used in this embodiment is the activity level of the autonomic nervous system (hereinafter referred to as the activity level). That is, one emotion index value is expressed by the arousal level, and the other emotion index value is expressed by the activity level.

[0095] The emotion estimation unit 121 derives the level of arousal based on the electroencephalogram data and derives the level of activity based on the heartbeat data. Here, the emotion estimation unit 121 uses the biological signal S3_A as the electroencephalogram data and the biological signal S3_B as the heartbeat data.

[0096] The emotion estimation unit 121 is provided with an emotion estimation model 121m. The emotion estimation model 121m is a model for emotion estimation based on arousal level and activity level, and may be configured with a calculation formula or a conversion data table that identifies an emotion from arousal level and activity level.

[0097] The emotion estimation model 121m is a multi-dimensional model with arousal and activity as parameters, and here is a two-dimensional model with arousal and activity as two axes. The two-dimensional model is created based on medical evidence (papers, etc.) showing the relationship between arousal and activity and emotions. Alternatively, the two-dimensional model is created based on the results of a questionnaire given to many subjects. The results of the questionnaire given to the subjects include the arousal and activity of the subjects identified from the measured values ​​of the subjects' electroencephalograms and heartbeats, and the emotions declared by the subjects when the electroencephalograms and heartbeats were measured. Note that a multi-dimensional model of three or more dimensions may be used as the emotion estimation model 121m.

[0098] FIG. 14 shows an example of a two-dimensional model related to emotion estimation. FIG. 14 shows a two-dimensional psychological plane PP defined in the two-dimensional model. According to various medical evidence related to psychology, psychology can be estimated based on two types of indices that indicate mental and physical states. In the psychological plane PP shown in FIG. 14, a vertical axis corresponding to arousal level and a horizontal axis corresponding to activity level are defined.

[0099] Points indicating the awakening and activity of the subject U1 can be plotted (placed) on the psychological plane PP. The higher the awakening of the subject U1, the more the coordinates of the plotted (placed) points move in the positive direction of the vertical axis, and the lower the awakening of the subject U1, the more the coordinates of the plotted (placed) points move in the negative direction of the vertical axis. The higher the activity of the subject U1, the more the coordinates of the plotted (placed) points move in the positive direction of the horizontal axis, and the lower the activity of the subject U1, the more the coordinates of the plotted (placed) points move in the negative direction of the horizontal axis. From the origin of the psychological plane PP, the positive side of the vertical axis corresponds to an awakened state, and the negative side of the vertical axis corresponds to an unawakened state. From the origin of the psychological plane PP, the positive side of the horizontal axis corresponds to a state in which the sympathetic nerves are activated (a state in which a relatively strong emotion is present), and the negative side of the horizontal axis corresponds to a state in which the parasympathetic nerves are activated (a state in which a relatively weak emotion is present).

[0100] The point where the vertical axis and the horizontal axis intersect on the psychological plane PP is the origin. The psychological plane PP has first to fourth quadrants separated by the vertical axis and the horizontal axis. The first quadrant is an area where the vertical axis component has a positive value when viewed from the origin of the psychological plane PP, and the horizontal axis component also has a positive value. The second quadrant is an area where the vertical axis component has a positive value when viewed from the origin of the psychological plane PP, and the horizontal axis component also has a negative value. The third quadrant is an area where the vertical axis component has a negative value when viewed from the origin of the psychological plane PP, and the horizontal axis component also has a negative value. The fourth quadrant is an area where the vertical axis component has a negative value when viewed from the origin of the psychological plane PP, and the horizontal axis component has a positive value.

[0101] In the psychological plane PP, each quadrant is assigned a particular psychological state (in other words, an emotion). The first quadrant is assigned the psychological states of "fun, joy, anger, sadness." The second quadrant is assigned the psychological state of "melancholy." The third quadrant is assigned the psychological states of "boredom, relaxation, calmness." The fourth quadrant is assigned the psychological states of "anxiety, fear, unpleasantness." The distance from the origin indicates the intensity of the corresponding psychological state. For example, when a point indicating the arousal and activity of subject U1 is plotted in the second quadrant, the greater the distance between the plotted point and the origin, the stronger the psychological state of "melancholy" for subject U1.

[0102] In this way, the emotion of any person depends on the arousal level and activity level of the person. The position of each axis on the psychological plane PP may be appropriately set based on an experiment or the like. Each axis may be set based on the position of a neutral area described later.

[0103] The emotion estimation model 121m can estimate the psychological state of the subject U1 from coordinates obtained by plotting (arranging) points indicating the arousal and activity of the subject U1 on the psychological plane PP. The psychological state and its intensity of the subject U1 can be estimated based on which quadrant of the psychological plane PP the coordinates of the plotted point are in, where in the quadrant they are located, and the distance between the plotted point and the origin. The first to fourth quadrants on the psychological plane PP can be considered as first to fourth classes, respectively, and in this case, the emotion estimation model 121m can be said to classify the psychological state of the subject U1 into one of the first to fourth classes. Note that when the emotion estimation model 121m is a multidimensional model of three or more dimensions, a multidimensional psychological space of three or more dimensions is used depending on the number of types of emotion index values ​​to be used.

[0104] Incidentally, when emotion intensity is strong, that is, when the emotion index value swings significantly toward the maximum or minimum value, emotion estimation accuracy tends to increase. On the other hand, when emotion intensity is weak, that is, when the emotion index value is near the median, emotion estimation accuracy tends to decrease. For this reason, as shown in Figure 15, a method can be adopted in which the area near the median of the emotion index value is set as a neutral area (corresponding to the diagonal line area in Figure 15), and emotion index values ​​that fall into the neutral area are judged to be unable to be estimated or to have no estimated emotion.

[0105] The neutral regions include an awakening neutral region NR1 and an activity neutral region NR2. In the psychological plane PP, the region whose distance from the horizontal axis is equal to or less than a predetermined first boundary distance is the awakening neutral region NR1, and the region whose distance from the vertical axis is equal to or less than a predetermined second boundary distance is the activity neutral region NR2. The awakening neutral region NR1 and the activity neutral region NR2 overlap each other in a region including the origin of the psychological plane PP. The combined region of the awakening neutral region NR1 and the activity neutral region NR2 is hereinafter referred to as a neutral region NR. It may be understood that each position in the neutral region NR does not belong to any of the first to fourth quadrants of the psychological plane PP.

[0106] The emotion estimation method will be further explained. The emotion estimation unit 121 derives the awakening level of the subject U1 based on the electroencephalogram data of the subject U1. As described above, the biosignal S3_A is used as the electroencephalogram data. The awakening level is derived based on the beta / alpha waves of the electroencephalogram. That is, a value according to the beta / alpha waves of the electroencephalogram (for example, a value proportional to the beta / alpha waves of the electroencephalogram) is derived as the awakening level. The beta / alpha waves of the electroencephalogram are the ratio of the beta waves of the electroencephalogram to the alpha waves of the electroencephalogram. More specifically, for example, the beta / alpha waves of the electroencephalogram are "PW_β / PW_α". PW_β is the power of the beta waves of the electroencephalogram represented by the biosignal S3_A. That is, PW_β represents the sum of the power of the frequency components belonging to the beta waves among the frequency components of the biosignal S3_A. PW_α is the power of the alpha waves of the electroencephalogram represented by the biosignal S3_A. That is, PW_α represents the sum of the power of the frequency components belonging to α waves among the frequency components of the biosignal S3_A. Alpha waves have a frequency of 8 Hz or more and less than 13 Hz, and beta waves have a frequency of 13 Hz or more and less than 30 Hz.

[0107] The emotion estimation unit 121 derives the activity level of the subject U1 based on the heartbeat data of the subject U1. As described above, the biosignal S3_B is used as the heartbeat data. The biosignal S4_B may be used as the heartbeat data. More specifically, the activity level is calculated based on the LF component of the time interval between heartbeats (hereinafter referred to as the heartbeat interval). The heartbeat interval corresponds to the RR interval of the cardiac potential signal (however, it may be the interval between two adjacent Q waves in the cardiac potential signal). The emotion estimation unit 121 derives the LF component of the heartbeat interval based on the heartbeat data. The LF component of the heartbeat interval is a frequency component in a predetermined low frequency band (for example, a frequency band from 0.04 to 0.15 Hz) among the frequency components included in the heartbeat interval. For the subject U1, the emotion estimation unit 121 derives the activity level of the subject U1 by, for example, calculating the standard deviation in the amplitude of the LF component of the heartbeat interval based on the heartbeat data. At this time, a value according to the standard deviation (for example, a value proportional to the standard deviation) is derived as the activity. In the amplitude spectrum of the heartbeat interval, the frequency components in the low frequency band of the heartbeat interval consist of 1st to Kth frequency components (K is an integer of 2 or more), and the standard deviation corresponds to the standard deviation in the amplitudes of the 1st to Kth frequency components.

[0108] Coordinates obtained by plotting (arranging) points indicating the arousal and activity of the subject U1 on the psychological plane PP are referred to as plot coordinates. The plot coordinates represent the positions of the points indicating the arousal and activity of the subject U1 (positions on the psychological plane PP). The emotion estimation unit 121 (emotion estimation model 121m) estimates and identifies the emotion of the subject U1 by identifying in which quadrant of the psychological plane PP the plot coordinates are located. When the plot coordinates are located in the i-th quadrant, the emotion estimation unit 121 (emotion estimation model 121m) estimates that the emotion of the subject U1 corresponds to the psychological state assigned to the i-th quadrant.

[0109] The notification unit 140 may have the function of the notification unit 40 described in the first embodiment, and therefore may execute a noise notification process based on a noise detection signal. The notification unit 140 may further execute a sensory information notification process under the control of the controller 120. In the sensory information notification process, the notification unit 140 notifies the user of the emotion of the subject U1 estimated by the emotion estimation unit 121. The notification is performed in any manner that is perceptible by the user. For example, a display screen may be provided in the notification unit 40, and notification in the sensory information notification process may be realized by displaying an image on the display screen.

[0110] The subject U1 may be a person who operates a game device. In this case, the emotion estimation device 100 may be built into the game device, or the emotion estimation device 100 may be provided separately from the game device. For example, the emotion estimation result can be used for mental training of the subject U1 in e-sports. When the subject U1 feels an emotion (anxiety, anger, etc.) that is unfavorable to the game while playing a game, it is determined that intensive training corresponding to the emotional state is necessary. In the training, the emotion information of the subject U1 estimated by the emotion estimation device 100 is used.

[0111] In addition, there are types of e-sports, such as a type in which multiple players play together and a type in which multiple players play against each other. In such types of e-sports, if the emotional state of each player is displayed, it becomes possible to perform advanced gameplay, such as changing game tactics according to the emotional state. It is also possible for spectators of e-sports to understand the emotional state of each player while watching the game, which can increase the element of enjoyment in watching the game. Another possible use is to control the progress of the game (adjusting the difficulty level, etc.) based on the estimation results of emotions.

[0112] The subject U1 is not limited to a person who operates a game device. For example, the subject U1 may be a driver of a vehicle such as an automobile. In this case, driving assistance for the vehicle can be provided according to the estimation result of the driver's emotion.

[0113] According to the third embodiment, the emotion of the subject U1 is estimated based on the biometric data (electroencephalogram data and heartbeat data) in which the influence of noise is suppressed, so that the accuracy of emotion estimation can be improved. By using the biometric signal S3_A output from the correction processing unit 23_A as the electroencephalogram data, noise in the electroencephalogram data is reduced. Similarly, by using the biometric signal S3_B output from the correction processing unit 23_B as the heartbeat data, noise in the heartbeat data is reduced.

[0114] However, the biosignal S3_A may be used as the electroencephalogram data, while the biosignal S2_B may be used as the heart rate data (in this case, the correction processing unit 23_B and the inverse conversion unit 24_B can be omitted from the calculation processing unit 20_B). By using the biosignal S3_A as the electroencephalogram data, it is expected that the estimation accuracy of emotions can be improved based on at least noise reduction in the electroencephalogram data.

[0115] Conversely, the biosignal S2_A may be used as electroencephalogram data, while the biosignal S3_B may be used as heartbeat data (in this case, the correction processing unit 23_A and the inverse conversion unit 24_A can be omitted from the calculation processing unit 20_A). By using the biosignal S3_B as heartbeat data, it is expected that the emotion estimation accuracy can be improved based on at least noise reduction in the heartbeat data.

[0116] <<Fourth embodiment>> A fourth embodiment of the present invention will now be described. The techniques shown in the above-described embodiments may be used to generate learning data in machine learning.

[0117] For example, a first AI model that estimates (infers) the subject's alertness and activity level based on input data may be constructed by machine learning. The first AI model is composed of a neural network. Machine learning in the first AI model is supervised machine learning performed using the first learning data. The first learning data has input data and correct answer data. The alertness level derived based on the biological signal S3_A and the activity level derived based on the biological signal S3_B can be adopted as correct answer data in the first learning data. Machine learning is performed by a learning device consisting of a computer device. In machine learning, the first AI model estimates the subject's alertness and activity level from the input data in the first learning data, and the learning device updates the parameters (weights, etc.) of the first AI model so that the error between the estimated result and the correct answer data in the first learning data is reduced.

[0118] The input data in the first learning data is any type of data that reflects the subject's alertness and activity, and includes, for example, image data of the subject's face. This is because it is believed that there is a correlation between the subject's alertness and activity and the subject's facial expression. If the subject is an e-sports player, the input data in the first learning data may include the operation content of the subject on the game device. This is because it is believed that there is a correlation between the subject's alertness and activity and the operation content of the subject on the game device. If the subject is a driver of a vehicle, the input data in the first learning data may include the driving operation content of the subject on the vehicle. This is because it is believed that there is a correlation between the subject's alertness and activity and the driving operation content of the subject on the vehicle.

[0119] Also, for example, a second AI model that estimates (infers) the subject's emotion based on the input data may be constructed by machine learning. The second AI model is composed of a neural network. Machine learning in the second AI model is supervised machine learning performed using the second learning data. The second learning data has input data and correct answer data. The emotion estimated by the emotion estimation unit 121 of FIG. 11 based on the biological signals S3_A and S3_B can be adopted as correct answer data in the second learning data. Machine learning is performed by a learning device consisting of a computer device. In machine learning, the second AI model estimates the subject's emotion from the input data in the second learning data, and the learning device updates the parameters (weights, etc.) of the second AI model so that the error between the estimation result and the correct answer data in the second learning data is reduced.

[0120] The input data in the second learning data is any type of data that reflects the subject's emotions, and includes, for example, image data of the subject's face. This is because it is believed that there is a correlation between the subject's emotions and the subject's facial expressions. If the subject is an e-sports player, the input data in the second learning data may include the operation content of the subject on the game device. This is because it is believed that there is a correlation between the subject's emotions and the operation content of the subject on the game device. If the subject is a driver of a vehicle, the input data in the second learning data may include the operation content of the subject on the vehicle. This is because it is believed that there is a correlation between the subject's emotions and the operation content of the subject on the vehicle.

[0121] <<Additional Information>> The embodiments of the present invention can be modified in various ways as appropriate within the scope of the technical ideas set forth in the claims. The above embodiments are merely examples of the present invention, and the meanings of the terms of the present invention and each component are not limited to those described in the above embodiments. The specific numerical values ​​shown in the above description are merely examples, and can be changed to various numerical values ​​as a matter of course.

[0122] The biological signal processing device 1 executes a biological signal processing method. In the biological signal processing method, when it is determined that noise is superimposed, each frequency component of the biological signal S2 is corrected by decreasing it. The noise components superimposed on the biological signal S2 are removed (reduced) by the decreasing correction. For this reason, the biological signal processing method can also be called a noise removal method or a noise reduction method.

[0123] A program for causing a computer device to execute any of the methods described in the embodiments of the present invention, and a non-volatile recording medium having the program recorded thereon, are included in the scope of the embodiments of the present invention. The program for causing a computer device to execute any of the methods described in the embodiments of the present invention may be a subprogram incorporated into any main program or called by any main program. Any of the processes in the embodiments of the present invention may be realized by hardware such as a semiconductor integrated circuit, software equivalent to the above program, or a combination of hardware and software. [Explanation of symbols]

[0124] SNS Biometric Sensor U1 Target S0 signal (output signal) S1~S4 Biosignals 1. Biological signal processing device 10 AFE 11 Amplification circuit 12 AD conversion circuit 20 Processing section 21 Frequency analysis section 22 Noise judgment section 23 Correction processing section 24 Inverse conversion section 30 Storage section 40 Notification Department SNS_A, SNS_B biosensor S0_A, S0_B signals (output signals) S1_A~S4_A, S1_B~S4_B Biological signals 100 Emotion estimation device 10_A, 10_B AFE 120 Controller 20_A, 20_B Processing section 21_A, 21_B Frequency analysis section 22_A, 22_B Noise judgment section 23_A, 23_B Correction processing section 24_A, 24_B Reverse conversion section 121 Emotion estimation part 121m Emotion estimation model 130 Storage section 140 Notification Department

Claims

1. Acquire a first biosignal in a time domain based on a measurement result of bioinformation of a subject by a biosensor, and convert the first biosignal into a second biosignal in a frequency domain; determining whether or not noise is superimposed on the first biological signal based on a change in intensity of the first biological signal; When it is determined that the noise is superimposed, each frequency component of the second biological signal is corrected by decreasing it. ,Biological signal processing device.

2. Based on the intensity of the first biological signal in a reference period and the intensity of the first biological signal in a target period, whether or not the noise is superimposed on the first biological signal during the target period is determined, and when it is determined that the noise is superimposed on the first biological signal during the target period, each frequency component of the second biological signal during the target period is reduced and corrected; The reference period is a period prior to the target period. The biological signal processing device according to claim 1 .

3. A moving average of the intensity of the first biological signal is derived based on the first biological signal in the reference period, and based on a difference between the moving average value immediately before the target period and the intensity of the first biological signal in the target period, it is determined whether the noise is superimposed on the first biological signal during the target period. The biological signal processing device according to claim 2 .

4. When the intensity of the first biological signal during the target period is greater than a moving average value immediately before the target period and the difference is equal to or greater than a threshold, it is determined that the noise is superimposed on the first biological signal during the target period; The threshold is set based on a statistic of the intensity of the first biological signal during the reference period. The biological signal processing device according to claim 3 .

5. The standard deviation or the average value of the intensity of the first biological signal during the reference period is used as the statistic. The biological signal processing device according to claim 4 .

6. When it is determined that the noise is superimposed on the first biological signal during the target period, each frequency component of the second biological signal during the target period is corrected by decreasing it by an amount corresponding to the difference.

6. A biological signal processing device according to claim 3.

7. When it is determined that the noise is superimposed on the first biological signal during the target period, each frequency component of the second biological signal during the target period is corrected by decreasing it by a predetermined amount.

6. A biological signal processing device according to claim 2.

8. generating a third biological signal in a frequency domain based on the second biological signal; When it is determined that the noise is superimposed, the second biological signal that has been subjected to the decrease correction is the third biological signal, When it is determined that the noise is not superimposed, the second biological signal is used as the third biological signal without being subjected to the attenuation correction.

6. A biological signal processing device according to claim 1.

9. Converting the third biosignal into a fourth biosignal in the time domain. The biological signal processing device according to claim 8 .

10. The biosensor measures the subject's brain waves, and the first biosignal is a time domain brain wave signal.

6. A biological signal processing device according to claim 1.

11. A feeling estimation device having the biological signal processing device according to claim 8, The subject's brain waves are measured by the biosensor, and the first biosignal is a time domain brain wave signal; Estimating the emotion of the subject based on the third biological signal and a heart rate signal of the subject measured by another biological sensor. , emotion estimation device.

12. acquiring a first biosignal in the time domain based on a measurement result of bioinformation of the subject by the biosensor; Converting the first biological signal into a second biological signal in the frequency domain; determining whether or not noise is superimposed on the first biological signal based on a change in intensity of the first biological signal; When it is determined that the noise is superimposed, each frequency component of the second biological signal is corrected by decreasing it. , A biological signal processing method.

13. acquiring a first biosignal in the time domain based on a measurement result of bioinformation of the subject by the biosensor; Converting the first biological signal into a second biological signal in the frequency domain; determining whether or not noise is superimposed on the first biological signal based on a change in intensity of the first biological signal; When it is determined that the noise is superimposed, the computer is caused to execute a method of reducing and correcting each frequency component of the second biological signal. , a biosignal processing program.

Images