Muscle activity estimation device, muscle activity estimation method, and muscle activity estimation program

Abstract

This muscular activity estimation device has a frequency dividing unit, an average calculating unit, a dispersion calculating unit, a reliability determining unit, and a decoding unit. The frequency dividing unit divides biopotential signals at each point in time into a plurality of frequency bands and calculates feature amounts. The average calculating unit calculates an average value of the feature amounts in each of the plurality of frequency bands. The dispersion calculating unit calculates a dispersion value of the feature amounts in each of the plurality of frequency bands. The reliability determining unit uses the average value and the dispersion value to calculate a reliability of each of the plurality of frequency bands. The decoding unit uses a model for calculating, from the feature amounts and the reliability, a probability of muscular activity on the basis of the feature amounts, and calculates a muscular activity.

Description

[Technical Field]

[0001] The present invention relates to a muscle activity estimation device, a muscle activity estimation method, and a muscle activity estimation program. [Background technology]

[0002] BACKGROUND ART Techniques are known for measuring bioelectric potential signals generated in association with brain activity as brain information, such as electroencephalography (EEG) and electrocorticogram (ECoG).

[0003] It is expected that decoding a user's intention based on the features calculated from such bioelectric potential signals will be applicable to device operation, etc.

[0004] Movements such as chewing and swallowing can cause noise in the measured bioelectrical signals.

[0005] As a countermeasure against noise, Non-Patent Document 1 proposes a method of selecting the frequency band to be analyzed, such as filtering out frequencies below 5 Hz as a noise band with a lot of noise.

[0006] Furthermore, Non-Patent Document 2 proposes a method of separating measured brain information into two components, a brain biopotential signal component and a noise component, and decoding the brain biopotential signal component only. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Hirai, A., Yoshida, K., & Miyaji, I. (2013). Comparative analysis of thinking and memory during learning using a simple electroencephalogram. Proceedings of the 2013 Multimedia, Distributed Collaboration and Mobile Symposium, 2013, 1441-1446. [Non-patent document 2] Matiko, JW, Beeby, S., & Tudor, J. (2013, July). [Real time eye blink noise removal from EEG signals using morphological component analysis]. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 13-16). IEEE. Summary of the Invention [Problem to be solved by the invention]

[0008] The method of filtering noise bands in Non-Patent Document 1 may bury information about brain activity that should be utilized. In addition, the signal division method in Non-Patent Document 2 has problems such as not guaranteeing the reproducibility of division and not being able to determine whether the divided signal is noise or not.

[0009] The present invention has been made in light of the above circumstances, and an object of the present invention is to provide a muscle activity estimation device, a muscle activity estimation method, and a muscle activity estimation program that are robust to noise. [Means for solving the problem]

[0010] One aspect of the present invention is a muscle activity estimation device. The muscle activity estimation device includes a frequency division unit, an average calculation unit, a variance calculation unit, a reliability determination unit, and a decoding unit. The frequency division unit calculates a mean value of a frequency at each time point. EEG signal into a plurality of frequency bands to calculate feature quantities. The average calculation unit calculates the average value of the feature quantities in each of the plurality of frequency bands. The variance calculation unit calculates the variance value of the feature quantities in each of the plurality of frequency bands. The reliability determination unit calculates the reliability in each of the plurality of frequency bands using the average value and the variance value. The reliability determination unit calculates the reliability r of each frequency band j at each time t according to the following formula: j Calculate (t).

number

[0011] One aspect of the present invention is a muscle activity estimation method. EEG signal into a plurality of frequency bands to calculate feature quantities, calculating the average value of the feature quantities in each of the plurality of frequency bands, calculating the variance value of the feature quantities in each of the plurality of frequency bands, calculating the reliability in each of the plurality of frequency bands using the average value and the variance value, and calculating muscle activity from the feature quantities and the reliability using a model that calculates the probability of muscle activity based on the feature quantities. The reliability calculation step calculates the reliability r of each frequency band j at each time t according to the following formula: j Calculate (t).

number

[0012] One aspect of the present invention is a muscle activity estimation program that causes a computer included in the muscle activity estimation device to execute the functions of the components of the muscle activity estimation device. [Effects of the Invention]

[0013] According to the present invention, a muscle activity estimation device, a muscle activity estimation method, and a muscle activity estimation program that are robust to noise are provided. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram illustrating an example of a functional configuration of a muscle activity estimation device according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the muscle activity estimation device according to the embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an example of a processing procedure and processing contents of muscle activity estimation executed by the muscle activity estimation device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] <Configuration example> (Functional configuration) First, the functional configuration of a muscle activity estimation device according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the functional configuration of a muscle activity estimation device 10 according to an embodiment.

[0017] The muscle activity estimation device 10 estimates a user's muscle activity from biopotential signals generated in association with the user's brain activity. In the following, the biopotential signals are electroencephalogram (EEG) signals, and an example is described in which the user's intention to click a mouse on a personal computer is decoded as the user's muscle activity. Furthermore, the electroencephalogram signals are assumed to contain noise due to chewing.

[0018] The muscle activity estimation device 10 includes a frequency division unit 11 , an average calculation unit 12 , a variance calculation unit 13 , a reliability determination unit 14 , and a decoding unit 15 .

[0019] An electroencephalogram signal is input to the frequency dividing unit 11. For example, the electroencephalogram signal is measured by attaching N electrodes to the scalp. In other words, the electroencephalogram signal is an N-channel signal. In the following, the N-channel electroencephalogram signal at each time t is referred to as S i This is written as (t)(i=1,…,N).

[0020] The frequency division unit 11 divides the input electroencephalogram signal S i For example, the frequency dividing unit 11 divides the electroencephalogram signal S (t) into a plurality of frequency bands to calculate the electroencephalogram feature amount. i A band-pass filter is applied to (t) to divide it into nine frequency bands: δ (1.5 to 4 Hz), θ (4 to 8 Hz), α (8 to 14 Hz), β1 (14 to 20 Hz), β2 (20 to 30 Hz), γ1 (30 to 50 Hz), γ2 (50 to 90 Hz), γ3 (90 to 120 Hz), and γ4 (120 to 150 Hz). As a result, the frequency dividing unit 11 divides the EEG feature quantity v into 9N (N channels × 9 frequency bands) dimensions. (i,j) (t) (i=1,...,N) (j=δ,...,γ4). The frequency division unit 11 calculates the calculated electroencephalogram feature value v (i,j) (t) (i=1, . . . , N) (j=δ, . . . , γ4) is output to the mean calculation unit 12, the variance calculation unit 13 and the decoding unit 15.

[0021] The average calculation unit 12 calculates the electroencephalogram feature value v input from the frequency division unit 11. (i,j)For (t), at each time, the EEG feature value v in each of the multiple, i.e., nine, frequency bands is calculated. (i,j) (t) average value m j (t). The average calculation unit 12 calculates the calculated average value m j (t) is output to the reliability determination unit 14.

[0022] The variance calculation unit 13 calculates the electroencephalogram feature value v input from the frequency division unit 11. (i,j) For (t), the EEG feature value v of all channels, i.e., N channels, in each of the multiple, i.e., nine frequency bands at each time t is (i,j) Variance value q of (t) j The variance calculation unit 13 calculates the calculated variance value q j (t) is output to the reliability determination unit 14.

[0023] The reliability determination unit 14 calculates the average value m of all channels, i.e., N channels, input from the average calculation unit 12. j (t) and the variance value q of all channels, i.e., N channels, input from the variance calculation unit 13 j (t) to calculate the reliability in each of the nine frequency bands j. r j For example, the reliability determination unit 14 calculates the reliability r (t) of each frequency band j at each time t according to the following formula: j Calculate (t).

[0024]

number

[0025] Here, α and β are predetermined variables. α and β are set based on the mean value and variance of the electroencephalogram signal containing noise, respectively. The reliability determination unit 14 calculates the calculated reliability r j (t) is output to the decoding unit 15.

[0026] The decoding unit 15 receives the electroencephalogram feature value v at each time t from the frequency dividing unit 11. (i,j)(t), and the reliability r at each time t input from the reliability determination unit 14 j From (t), muscle activity in each frequency band, i.e., the intention of the mouse click, is calculated.

[0027] First, the decoding unit 15 decodes the electroencephalogram feature value v (i,j) From (t), the probability p of muscle activity in each frequency band j at each time t, that is, the intention of mouse click, is j For this purpose, the decoding unit 15 calculates the electroencephalogram feature value v (i,j) Based on (t), the probability of mouse click intention is p j (t) is stored in the model M. This model M calculates the EEG feature value v for each frequency band j. (i,j) For (t), the probability of the mouse click operation intention is calculated. For example, the model M is expressed as follows:

[0028]

number

[0029] where p j (t) is the probability of mouse click operation intention calculated using frequency band j at each time t.

[0030] Next, the decoding unit 15 calculates the probability p of the mouse click operation intention in each frequency band j. j For (t), the reliability r j Taking (t) into consideration, the user's intention for the mouse click is calculated.

[0031] For example, the decoding unit 15 calculates the reliability r j (t) in descending order and select the top three frequency bands j with the highest reliability. The probability p jThe decoding unit 15 calculates the average value of (t) as the user's intention of the mouse click. The decoding unit 15 outputs an intention signal indicating the calculated intention of the mouse click.

[0032] Here, the probability p of the mouse click intention for the top three frequency bands is j Although an example has been described in which the average value of (t) is calculated as the user's mouse click operation intention, the target for calculating the average value is not limited to the top three frequency bands, but may be set to any multiple top frequency bands.

[0033] In addition, the probability p j Instead of using the mean value of (t) as the operation intention, we use the probability p j (t) may be calculated as the operation intention.

[0034] (Hardware configuration) Next, we will explain the hardware configuration of the muscle activity estimation device 10. The muscle activity estimation device 10 is configured by a computer. For example, the muscle activity estimation device 10 is configured by a personal computer or the like.

[0035] 2 is a block diagram showing an example of a hardware configuration of the muscle activity estimation device 10 according to the embodiment. As shown in FIG. 2, the muscle activity estimation device 10 includes a processor 21, a read-only memory (ROM) 22, a random access memory (RAM) 23, an auxiliary storage device 24, and an input / output interface 25.

[0036] The processor 21, ROM 22, RAM 23, auxiliary storage device 24, and input / output interface 25 are electrically connected to one another via a bus 26, and exchange data via the bus 26.

[0037] The processor 21 is configured by a general-purpose hardware processor including, for example, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), etc. The processor 21 controls the ROM 22, the RAM 23, the auxiliary storage device 24, and the input / output interface 25 as a whole.

[0038] The ROM 22 is a non-volatile memory. The ROM 22 non-temporarily stores a startup program required when the processor 21 is started. The processor 21 starts up by executing the program in the ROM 22. The ROM 22 is configured, for example, with an EPROM (Erasable Programmable Read Only Memory), and stores various startup settings in addition to the startup program.

[0039] The RAM 23 is a volatile memory that temporarily stores programs and the like required for processing by the processor 21. The RAM 23 is used by the processor 21 as a working memory.

[0040] The auxiliary storage device 24 is configured by a non-volatile memory such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The auxiliary storage device 24 non-temporarily stores programs to be executed by the processor 21 and data required for executing the programs. The processor 21 executes various functions by loading the programs and data in the auxiliary storage device 24 into the RAM 23 and executing them.

[0041] The input / output interface 25 is an interface that is connected to external input devices, output devices, etc., and enables the input of information from the input devices and the output of information to the output devices. For example, the input / output interface 25 includes a wireless communication interface. For example, the input devices are composed of a keyboard, a mouse, a touch panel, a receiving device, various drives, etc. The input devices are not limited to these, and may be composed of any other input devices. For example, the output devices are composed of a display, a transmitting device, etc. The output devices are not limited to these, and may be composed of any other output devices. The input and output devices may be composed of an input / output device that has the functions of both. For example, the input / output device is composed of a tablet, etc.

[0042] The program non-temporarily stored in the auxiliary storage device 24 is provided to the computer in a state where it is stored on a computer-readable recording medium. For example, the recording medium may be an optical disk (CD-ROM, CD-R, DVD-ROM, DVD-R, etc.), a magneto-optical disk (MO, etc.), a semiconductor memory, etc. In this case, the program in the recording medium is read via an input device such as a drive and non-temporarily stored in the auxiliary storage device 24. Alternatively, the program may be stored on a server on a network, downloaded from the server, and non-temporarily stored in the auxiliary storage device 24.

[0043] Upon startup, the processor 21 executes a program in the ROM 22 and loads and starts the OS into the RAM 23. Under control of the OS, the processor 21 monitors input instructions, connections to external devices, and the like. Under control of the OS, the processor 21 also sets up a program area and a data area in the RAM 23. In response to an instruction to start the muscle activity estimation device 10, the processor 21 loads a muscle activity estimation program and data from the auxiliary storage device 24 into the program area and data area of ​​the RAM 23, respectively. The data loaded into the data area of ​​the RAM 23 includes a model for calculating the probability of the user's mouse click operation intention. The processor 21 calculates the data in the data area according to the muscle activity estimation program and writes the calculation results to the data area. Through these operations, the processor 21 and the RAM work together to implement the functions of the frequency division unit 11, average calculation unit 12, variance calculation unit 13, reliability determination unit 14, and decoding unit 15 of the muscle activity estimation device 10.

[0044] <Example of operation> (Operation intention estimation process) Next, the muscle activity, i.e., operational intention estimation processing executed by the muscle activity estimation device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the operational intention estimation processing executed by the muscle activity estimation device 10 according to the embodiment.

[0045] In step S11, the frequency dividing unit 11 divides the N-channel electroencephalogram signals S at each input time t. i For example, the frequency dividing unit 11 divides the electroencephalogram signal S (t) of each channel into nine frequency bands. i (t) into nine frequency bands: δ (1.5 to 4 Hz), θ (4 to 8 Hz), α (8 to 14 Hz), β1 (14 to 20 Hz), β2 (20 to 30 Hz), γ1 (30 to 50 Hz), γ2 (50 to 90 Hz), γ3 (90 to 120 Hz), and γ4 (120 to 150 Hz). As a result, the frequency dividing unit 11 divides the 9N-dimensional EEG feature quantity v (i,j) Calculate (t) (i=1,…,N) (j=δ,…,γ4).

[0046] In step S12, the variance calculation unit 13 calculates the electroencephalogram feature value v calculated in step S11. (i,j) For (t), at each time t, the EEG feature value v in each of the nine frequency bands (i,j) (t) average value m j Calculate (t).

[0047] In step S13, the variance calculation unit 13 calculates the electroencephalogram feature value v calculated in step S11. (i,j) For (t), at each time t, the N-channel EEG feature value v (i,j) Variance value q of (t) j Calculate (t).

[0048] In step S14, the reliability determination unit 14 calculates the average value m of the N channels calculated in step S12. j (t) and the variance value q of N channels calculated in step S13 j (t) is used to calculate the reliability in each of the nine frequency bands j. r j For example, the reliability determination unit 14 calculates the reliability (t) using the following formula: r j Calculate (t).

[0049]

number

[0050] Here, α and β are predefined variables.

[0051] In step S15, the decoding unit 15 decodes the electroencephalogram feature value v calculated in step S11. (i,j) From (t), the EEG feature value v (i,j) Based on (t), the probability of mouse click intention is p j Using the model M that calculates (t), the probability p j For example, the decoding unit 15 calculates probability pj Calculate (t).

[0052]

number

[0053] Next, the decoding unit 15 calculates the probability p of the mouse click operation intention for each frequency band j at each time t. j For (t), the reliability r j For example, the decoding unit 15 calculates the user's intention to click the mouse by taking into account the reliability r j The probability p of mouse click operation intention in the top three frequency bands j with the highest (t) j The decoding unit 15 calculates the average value of (t) as the user's intention of the mouse click. The decoding unit 15 outputs an intention signal indicating the calculated intention of the mouse click.

[0054] <effect> In the embodiment, the EEG signal is divided into multiple frequency bands to calculate EEG feature amounts, the reliability in each frequency band is calculated, and the operational intention is calculated using a model that calculates the probability of the operational intention based on the EEG feature amounts and the reliability. In other words, since the operational intention is calculated taking into account the reliability in each frequency band, the muscle activity estimation device 10 can output an operational intention signal that is less affected by noise.

[0055] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention. [Explanation of symbols]

[0056] 10...Muscle activity estimation device 11...Frequency division section 12...Average calculation section 13...Variance calculation section 14...Reliability determination unit 15...Decoding section 21...Processor 22...ROM 23...RAM 24…Auxiliary storage device 25...Input / output interface 26...Bus

Claims

1. a frequency division unit that divides the electroencephalogram signal at each time point into a plurality of frequency bands and calculates a feature amount; an average calculation unit that calculates an average value of the feature amounts in each of the plurality of frequency bands; a variance calculation unit that calculates a variance value of the feature amount in each of the plurality of frequency bands; a reliability determination unit that calculates reliability for each of the plurality of frequency bands using the average value and the variance value; a decoding unit that calculates muscle activity from the feature amount and the reliability using a model that calculates a probability of muscle activity based on the feature amount; and The reliability determination unit calculates the reliability r of each frequency band j at each time t according to the following formula: ｊ (t) is calculated, [Equation 1] where α and β are variables set from the mean value and the variance value of the electroencephalogram signal containing noise, respectively, m j (t) is the mean value of the feature amount, and q j (t) is the variance value of the feature amount. The muscle activity estimation device, wherein the decoding unit calculates, as the muscle activity, an average value of the probability of the muscle activity in a plurality of frequency bands with the highest reliability.

2. a frequency division unit that divides the electroencephalogram signal at each time point into a plurality of frequency bands and calculates a feature amount; an average calculation unit that calculates an average value of the feature amounts in each of the plurality of frequency bands; a variance calculation unit that calculates a variance value of the feature amount in each of the plurality of frequency bands; a reliability determination unit that calculates reliability for each of the plurality of frequency bands using the average value and the variance value; a decoding unit that calculates muscle activity from the feature amount and the reliability using a model that calculates a probability of muscle activity based on the feature amount; and The reliability determination unit calculates the reliability r of each frequency band j at each time t according to the following formula: ｊ (t) is calculated, [Equation 2] where α and β are variables set from the mean value and the variance value of the electroencephalogram signal containing noise, respectively, m j (t) is the mean value of the feature amount, and q j (t) is the variance value of the feature amount. The muscle activity estimation device, wherein the decoding unit calculates, as the muscle activity, the probability of the muscle activity in the frequency band with the highest reliability.

3. the electroencephalogram signal is an electroencephalogram signal of a plurality of channels, and the frequency dividing unit calculates, as the feature amount, a feature amount of a combination of the plurality of channels and the plurality of frequency bands; The muscle activity estimation device according to claim 1 or 2.

4. A step of dividing the electroencephalogram signal at each time point into a plurality of frequency bands and calculating a feature amount; calculating an average value of the feature amounts in each of the plurality of frequency bands; calculating a variance value of the feature amount in each of the plurality of frequency bands; calculating a reliability for each of the plurality of frequency bands using the average value and the variance value; calculating muscle activity from the feature amount and the reliability using a model that calculates a probability of muscle activity based on the feature amount; and The step of calculating the reliability may include calculating the reliability r of each frequency band j at each time t according to the following formula: ｊ (t) is calculated, [Equation 3] where α and β are variables set from the mean value and the variance value of the electroencephalogram signal containing noise, respectively, m j (t) is the mean value of the feature amount, and q j (t) is the variance value of the feature amount. A computer-implemented muscle activity estimation method, wherein the step of calculating the muscle activity calculates, as the muscle activity, an average value of the probability of the muscle activity in a plurality of frequency bands with the highest reliability.

5. A step of dividing the electroencephalogram signal at each time point into a plurality of frequency bands and calculating a feature amount; calculating an average value of the feature amounts in each of the plurality of frequency bands; calculating a variance value of the feature amount in each of the plurality of frequency bands; calculating a reliability for each of the plurality of frequency bands using the average value and the variance value; calculating muscle activity from the feature amount and the reliability using a model that calculates a probability of muscle activity based on the feature amount; and The step of calculating the reliability may include calculating the reliability r of each frequency band j at each time t according to the following formula: ｊ (t) is calculated, [Equation 4] where α and β are variables set from the mean value and the variance value of the electroencephalogram signal containing noise, respectively, m j (t) is the mean value of the feature amount, and q j (t) is the variance value of the feature amount. A computer-implemented muscle activity estimation method, wherein the step of calculating the muscle activity calculates the probability of the muscle activity in the frequency band with the highest reliability as the muscle activity.

6. A muscle activity estimation program that causes a computer included in the muscle activity estimation device to execute functions of each component of the muscle activity estimation device according to claim 1 or 2.

Images