Muscle activity estimation device, method, and program

Abstract

To estimate muscle activity with higher accuracy by utilizing an estimation result of kinematics.SOLUTION: Brain activity data of a user is acquired, and first muscle activity and kinematics of the user are estimated based on the acquired brain activity data. Then, a kinematics feature amount is calculated from the estimation data of the kinematics, a purpose of motion corresponding to the calculated kinematics feature amount and a second muscle activity corresponding to the purpose of motion are estimated based on model data representing a relationship between the kinematics feature amount and the muscle activity prepared in advance for each purpose of motion, and a third muscle activity corresponding to the purpose of motion is estimated by reflecting the second muscle activity on the first muscle activity based on an estimation result of the purpose of motion.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] One aspect of the present invention relates to a muscle activity estimation device, method, and program for estimating muscle activity from brain activity, for example. [Background technology]

[0002] For example, it is known that spinal cord injury can prevent motor commands from reaching the muscles, resulting in limb paralysis. As one countermeasure, research is being conducted to restore the function of paralyzed limbs using BMI (Brain Machine Interface). In particular, technology to restore motor function of the upper arm is expected to be developed further, as it can significantly improve "well-being."

[0003] Generally, BMIs for motor function reconstruction estimate muscle activity from brain activity and apply functional electrical stimulation (FES) to muscles and nerves based on the results. In this case, the results of muscle activity estimation have a significant impact on the control accuracy of FES, so it is extremely important to estimate muscle activity with high accuracy.

[0004] Incidentally, methods such as sparse linear regression and Kalman filter are generally used to estimate muscle activity, and these methods can simultaneously estimate the kinematics that represent movements during exercise (see, for example, Non-Patent Document 1 or Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Yasuhiko Nakanishi, Takufumi Yanagisawa, Duk Shin, Hiroyuki Kambara, Natsue Yoshimura, Masataka Tanaka, Ryohei Fukuma, Haruhiko Kishima, Masayuki Hirata, and Yasuharu Koike. Mapping ecological signaling channel contributions to trajectory and muscle activity prediction in human sensorimotor cortex. Scientific Reports, Vol. 7, No. 1, p. 45486, 2017. [Non-patent document 2] Xuan Ma, Chaolin Ma, Jian Huang, Peng Zhang, Jiang Xu, and Jiping He. Decoding lower limb muscle activity and kinematics from cortical neural spike trains during monkey performing stand and squat movements. Frontiers in Neuroscience, Vol. 11, p. 44, 2017. Summary of the Invention [Problem to be solved by the invention]

[0006] However, muscle activity and kinematics estimated simultaneously from brain activity have been treated as separate estimation experiments and discussed separately. On the other hand, kinematics is known to be closely related to muscle activity, and it is expected that it can be used to improve the accuracy of muscle activity estimation. However, how to use the results of kinematics estimation to improve accuracy has not yet been explored, and there is a strong need for development of a technology that utilizes it.

[0007] The present invention has been made in light of the above circumstances, and aims to provide a technique that utilizes the results of kinematics estimation to enable muscle activity to be estimated with higher accuracy. [Means for solving the problem]

[0008] To solve the above problems, one aspect of a muscle activity estimation device or method according to the present invention estimates muscle activity based on a user's brain activity by acquiring the user's brain activity data, first estimating a first muscle activity and estimating kinematics based on the acquired brain activity data, calculating kinematic features from the estimated kinematics data, estimating an exercise purpose corresponding to the calculated kinematic features and a second muscle activity corresponding to the exercise purpose based on model data representing a relationship between the kinematic features and muscle activities, which are prepared in advance for each exercise purpose, and estimating a third muscle activity corresponding to the exercise purpose by reflecting the second muscle activity in the first muscle activity based on the estimation result of the exercise purpose.

[0009] According to one aspect of the present invention, muscle activity estimation data based on brain activity is corrected based on muscle activity data corresponding to the exercise purpose estimated from kinematics, thereby enabling the acquisition of highly accurate muscle activity estimation data. [Effects of the Invention]

[0010] That is, according to one aspect of the present invention, it is possible to provide a technology that utilizes the results of kinematics estimation to enable muscle activity estimation with higher accuracy. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a BMI system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a muscle activity estimation device included in the BMI system shown in FIG. [Figure 3] FIG. 3 is a block diagram showing an example of the software configuration of the muscle activity estimation device included in the BMI system shown in FIG. [Figure 4] FIG. 4 is a flowchart showing an example of the processing procedure and processing content of the model learning processing executed by the control unit of the muscle activity estimating device shown in FIG. 3 in the learning phase. [Figure 5] FIG. 5 is a flowchart showing an example of the processing procedure and processing content of the muscle activity estimation process executed by the control unit of the muscle activity estimation device shown in FIG. 3 in the estimation phase. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] [One embodiment] (overview) For example, when a person performs a movement limited to a specific direction, such as wrist movement, muscle activity and kinematics with a stable correlation are generated. In other words, by identifying the purpose of the movement, it becomes possible to accurately estimate muscle activity based on the kinematics at that time.

[0014] In one embodiment of the present invention, focusing on the above points, kinematic feature values ​​are calculated from a series of kinematics estimates over a certain period of time, and a motion purpose is estimated based on the kinematic feature values. Then, estimated values ​​of muscle activity corresponding to the estimated motion purpose are obtained, and the estimated values ​​of muscle activity for each motion purpose are reflected in estimated values ​​of muscle activity obtained from brain activity.

[0015] In other words, it is possible to correct the estimated muscle activity from brain activity using the estimated muscle activity for each exercise purpose estimated from kinematics, thereby obtaining highly accurate estimation results for muscle activity for each exercise purpose.

[0016] (Configuration example) (1) System FIG. 1 shows an example of a BMI system according to an embodiment of the present invention.

[0017] In one embodiment of the BMI system, an electroencephalogram sensor BS having a plurality of electrodes TM is worn on the head of a user US, and this electroencephalogram sensor BS is connected to a muscle activity estimation device CS via a signal cable, etc. Furthermore, an electrical stimulation generator ES is worn on a limb to be controlled by the user US, for example, an arm, and this electrical stimulation generator ES is connected to the muscle activity estimation device CS via a signal cable, etc.

[0018] The brain wave sensor BS uses, for example, an EEG (Electroencephalograph) sensor, and when worn on the head of the user US, measures brain waves generated from different parts of the user US's brain via multiple electrodes TM, and outputs each measured brain wave signal to the muscle activity estimation device CS.

[0019] The electrical stimulation generator ES is worn, for example, on the arm of the user US with the stimulation electrodes in contact with the skin, and applies tactile stimulation to the user's skin in accordance with the functional electrical stimulation (FES) signals output from the muscle activity estimation device CS.

[0020] (2) Muscle activity estimation device CS 2 and 3 are block diagrams showing an example of the hardware configuration and software configuration of the muscle activity estimation device CS, respectively.

[0021] The muscle activity estimation device CS is composed of an information processing device such as a wearable terminal that can be attached to a user US, and includes a control unit 1 that uses a hardware processor such as a central processing unit (CPU). A storage unit having a program storage unit 2 and a data storage unit 3, a sensor interface (hereinafter, interface will be abbreviated as I / F) unit 4, and an input / output I / F unit 5 are connected to the control unit 1 via a bus 6.

[0022] The electroencephalogram sensor BS is connected via a signal cable to the sensor I / F unit 4. The sensor I / F unit 4 receives the electroencephalogram signal output from the electroencephalogram sensor BS and converts it into a format that can be processed by the control unit 1.

[0023] An input device IN and an electrical stimulation generator ES are connected to the input / output I / F unit 5 via, for example, a signal cable. The input device IN includes, for example, a keyboard, a mouse, or a microphone for voice input. The input / output I / F unit 5 receives instruction data entered by the user US or an administrator at the input device IN and outputs electrical stimulation signals generated by the control unit 1 to the electrical stimulation generator ES. A display device for displaying various display data is also connected to the input / output I / F unit 5, making it possible to display various display data generated during or after processing by the muscle activity estimation device CS.

[0024] Note that, instead of signal cables, a low-power wireless interface such as Bluetooth (registered trademark) may be used as a connection means between the sensor I / F unit 4 and the EEG sensor BS, and between the input / output I / F unit 5 and the input device IN and electrical stimulation generator ES. Using a wireless interface can reduce the burden on the user when undergoing training.

[0025] The program storage unit 2 is, for example, a combination of a non-volatile memory such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) as a storage medium that can be written to and read from at any time, and a non-volatile memory such as a ROM (Read Only Memory), and stores application programs necessary to execute various processes related to the first embodiment of the present invention, in addition to middleware such as an OS (Operating System).

[0026] The data storage unit 3 is, for example, a combination of a non-volatile memory such as an HDD or SSD as a storage medium that can be written to and read from at any time, and a volatile memory such as RAM (Random Access Memory).The storage area includes a brain activity data storage unit 31, a muscle activity / kinematics estimation model storage unit 32, a kinematics feature-muscle activity model storage unit 33, and a muscle activity data storage unit 34.

[0027] The brain activity data storage unit 31 stores, in chronological order, brain activity measurement data generated based on the electroencephalogram signal output from the electroencephalogram sensor BS.

[0028] The muscle activity / kinematics estimation model storage unit 32 stores a muscle activity / kinematics estimation model. The muscle activity / kinematics estimation model simultaneously estimates muscle activity and kinematics from brain activity measurement data using a Kalman filter, and is generated using a machine learning model such as a CNN (Convolutional Neural Network). Note that the muscle activity / kinematics estimation model may also be generated using other machine learning models besides a CNN, such as an SVM (Support Vector Machine).

[0029] The kinematics feature-muscle activity model storage unit 33 stores the kinematics feature-muscle activity model generated for each type of exercise objective. Each kinematics feature-muscle activity model estimates muscle activity from kinematic features, and like the muscle activity-kinematics estimation model, is generated by training a machine learning model such as CNN.

[0030] The muscle activity data storage unit 34 stores the finally obtained estimated data of muscle activity.

[0031] The control unit 1 includes, as processing function units according to one embodiment of the present invention, a brain activity data acquisition processing unit 11, a muscle activity / kinematics estimation processing unit 12, a kinematics feature calculation processing unit 13, an exercise purpose determination processing unit 14, a muscle activity aggregation processing unit 15, and an electrical stimulation generation processing unit 16.

[0032] Each of the processing units 11 to 16 is realized by causing a hardware processor of the control unit 1 to execute an application program stored in the program storage unit 2. Note that some or all of the processing units 11 to 16 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0033] The brain activity data acquisition processing unit 11 receives the brain wave signal output from the brain wave sensor BS via the sensor I / F unit 4, generates brain activity measurement data based on the received brain wave signal, and stores the data in chronological order in the brain activity data storage unit 31.

[0034] The muscle activity / kinematics estimation processing unit 12 uses the muscle activity / kinematics estimation model stored in the muscle activity / kinematics estimation model storage unit 32 to estimate muscle activity and kinematics corresponding to the brain activity measurement data.

[0035] The kinematics feature calculation processing unit 13 calculates kinematics feature amounts based on the estimated kinematics data obtained by the muscle activity / kinematics estimation processing unit 12.

[0036] The exercise purpose determination processing unit 14 determines the exercise purpose by comparing the kinematic feature calculated by the kinematic feature calculation processing unit 13 with the kinematic feature estimated by each kinematic feature-muscle activity model for each exercise purpose.

[0037] The muscle activity summarization processing unit 15 acquires muscle activity estimation data corresponding to the exercise purpose from the kinematics feature-muscle activity model corresponding to the exercise purpose determined by the exercise purpose determination processing unit 14. The muscle activity summarization processing unit 15 then estimates the final muscle activity of the user US based on the muscle activity estimation data corresponding to the brain activity estimated by the muscle activity-kinematics estimation processing unit 12 and the muscle activity estimation data corresponding to the exercise purpose acquired from the kinematics feature-muscle activity model.

[0038] The electrical stimulation generation processing unit 16 generates an electrical stimulation signal based on the final muscle activity estimation data estimated by the muscle activity aggregation processing unit 15. Then, the electrical stimulation generation processing unit 16 outputs the generated electrical stimulation signal from the input / output I / F unit 5 to the electrical stimulation generator ES at a timing synchronized with the measurement time of the brain activity measurement data.

[0039] (Example of operation) Next, an example of the operation of the muscle activity estimation device CS configured as above will be described.

[0040] (1) Learning Phase The control unit 1 of the muscle activity estimation device CS sets a learning phase prior to estimating muscle activity, and learns a muscle activity-kinematics estimation model and a kinematics feature-muscle activity model.

[0041] FIG. 4 is a flowchart showing an example of the procedure and content of the learning process executed by the control unit 1 of the muscle activity estimating device CS in the learning phase.

[0042] That is, for example, when a system administrator inputs a learning execution instruction into the input device IN and this learning execution instruction is detected in step S10, the control unit 1 of the muscle activity estimation device CS first acquires learning data in step S11.

[0043] The training data consists of brain activity measurement data, muscle activity data, and kinematics data collected when, for example, the user US repeatedly performs a task of moving a cursor within a target by operating a lever with his or her wrist for D targets evenly arranged in a concentric circle, and is obtained, for example, from a terminal used by a system administrator.

[0044] As the brain activity measurement data, for example, when the user US performs the task of moving a cursor on a screen by operating an operating lever with his / her wrist as described above, N pieces of neuronal activity data generated from the electroencephalogram signal of the electroencephalogram sensor BS are used.

[0045] As the muscle activity data, muscle activity data measured for M muscles by an electromyographic sensor (not shown) when the above task is performed is used.

[0046] The kinematic data used is two-dimensional torque information obtained when performing the above tasks.

[0047] First, in step S12, the control unit 1 of the muscle activity estimation device CS uses the acquired learning data to learn a muscle activity / kinematics estimation model as follows.

[0048] That is, first, at time t k The vector that combines the muscle activity data and the two-dimensional torque that indicates the kinematics is x k =[u1(k),u2(k),…,u M (k),v x (k),v y (k)] T where u m (k), m=1,…,M is muscle activity, v x (k),v y (k) indicates the torque in the x and y directions, respectively.

[0049] In this case, the state transition model and the observation model are used to calculate the brain activity data zk Then, x k =Ax k-1 +w k , z k =Hx k +q k It is expressed as:

[0050] where A∈R (M+Z)×(M+Z) is the state transition matrix, H∈R C×(M+Z) indicates the observation matrix, and w k ~N(0,w) and q k ∼N(0,Q) represent noise. Furthermore, A and H are calculated from the training data using the least squares method, and Q and W represent the residuals.

[0051] Next, x k￣ ^ and x k If we define ^ as the prior estimate and ^ as the posterior estimate, respectively, the error covariance matrix in the prior estimate is P k￣ =E[(x k -x k￣ ^)(x k -x k￣ ^) T ] The error covariance matrix in the posterior estimation is P k =E[(x k -x k ^)(x k -x k ^) T ] are calculated as

[0052] The muscle activity and kinematics estimation model is trained using the data from time k=1, ..., C as training data, using the following calculations:

[0053] A=X2X1 T (X1X1 T ) -1 ,H=ZX T (XX T ) -1 however, X=(x1x2…x c ),X1=(x1x2…x c-1 ),X2=(x2x3…x c ), Z=(z1z2…z c ) Let's say.

[0054] Furthermore, the residual matrix is W=(X2-AX1)(X2-AX1) T / (C-1), Q=(Z-HX)(Z-HX) T / C It is calculated as:

[0055] The control unit 1 determines in step S13 whether the learning process of the muscle activity / kinematics estimation model described above has been completed, and if not, returns to step S12 to continue the learning process.

[0056] On the other hand, when the learning process is completed, the control unit 1 proceeds to step S14, where it learns a kinematic feature-muscle activity model, using the learning data acquired in step S11.

[0057] That is, if the training data is, for example, muscle activity data and torque data for a lever operation in a direction d(1,...,D), then S d It is assumed that the number of attempts is equal to the number of attempts.

[0058] Let the start time of each trial be 1 and the end time be end. Let the muscle activity data for the sth trial be u. m S (k), torque data v x S (k) ,v y S Then, as the kinematic feature - muscle activity, the average torque and muscle activity data at each time point for each movement direction is calculated over the entire trial period. The average torque and average muscle activity are calculated using the following formulas.

[0059]

number

[0060] The average torque data and average muscle activity data calculated by the kinematic feature-muscle activity model are set to be output to the exercise purpose determination processing unit 14 and the muscle activity summarization processing unit 15, respectively.

[0061] The control unit 1 determines whether the learning process of the kinematic feature-muscle activity model has been completed in step S15. If the learning process has not been completed, the control unit 1 returns to step S14 to continue the learning process. On the other hand, if the learning process has been completed, the control unit 1 ends all the learning processes in the learning phase.

[0062] (2) Estimation phase A brain wave sensor BS is attached to the head of the user US, and an electrical stimulation generator ES is attached to a body part used for exercise, for example, the right forearm.

[0063] In this state, suppose the user US imagines in his or her mind a task of inputting an estimation execution instruction into a microphone provided in the input device IN, for example, and then operating a control lever with his or her wrist to move a cursor on a screen. In this case, the control unit 1 of the muscle activity estimation device CS estimates the muscle activity of the user US when performing the above task as follows.

[0064] FIG. 5 is a flowchart showing an example of the processing procedure and processing content of the muscle activity estimation processing executed by the control unit 1 of the muscle activity estimation device CS.

[0065] (2-1) Acquisition of brain activity data When the control unit 1 of the muscle activity estimation device CS detects the above-mentioned estimation execution instruction in step S20, first, in step S21, under the control of the brain activity data acquisition processing unit 11, executes a process of acquiring measurement data related to the brain activity of the user US as follows.

[0066] That is, the brain activity data acquisition processing unit 11 first acquires N sequences of electroencephalogram signals measured by the electroencephalogram sensor BS during the period when the arm operation is being imagined, via the sensor I / F unit 4. Next, based on the acquired N sequences of electroencephalogram signals, the brain activity data acquisition processing unit 11 calculates firing sequences of N neurons (sequences of 0 (no firing) and 1 (firing) for each neuron) by spike sorting processing. Then, for the firing sequences of N neurons, discrete time t is calculated with a time window of Δt=10 msec. k =kΔt (k=1,2,…) and time t k N-dimensional vector Z represents the number of firings per second (expressed as firing rate Hz) of N neurons in k as the final brain activity measurement data. Finally, the brain activity data acquisition processing unit 11 stores the brain activity measurement data calculated as above in the brain activity data storage unit 31 in chronological order.

[0067] (2-2) Estimation of muscle activity and kinematics based on brain activity Next, in step S22, the control unit 1 of the muscle activity estimation device CS executes the following process of estimating muscle activities and kinematics based on the brain activity measurement data under the control of the muscle activity / kinematics estimation processing unit 12.

[0068] That is, the muscle activity / kinematics estimation processing unit 12 receives the brain activity measurement data Z for a certain period of time from the brain activity data storage unit 31. k are read out and input into the muscle activity and kinematics estimation model.

[0069] The muscle activity and kinematics estimation model estimates muscle activity and kinematics based on brain activity according to the estimation algorithm as follows:

[0070] That is, first, at time t k As a priori estimate of x k￣ ^=Ax k-1 ^,P k￣ =AP k-1 AT +W Next, we calculate the brain activity information Z k As an update step after observing x k ^=x k￣ ^+K k (Z k -Hx k￣ ^),P k =(IK k H)P k￣ Calculate. where K k =P k￣ H T (HP k￣ H T +Q) -1 is the Kalman gain.

[0071] The muscle activity / kinematics estimation processing unit 12 calculates the x calculated by the muscle activity / kinematics estimation model. k ^ contains the estimated muscle activity based on brain activity, u m ^(k) is output to the muscle activity summarization processor 15. Also, the two-dimensional torque estimate v x ^(k) and v y ^(k) is output to the kinematics feature amount calculation processing unit 13.

[0072] (2-3) Calculation of kinematic features Subsequently, in step S23, the control unit 1 of the muscle activity estimation device CS calculates the two-dimensional torque estimate v output from the muscle activity / kinematics estimation processing unit 12 under the control of the kinematics feature calculation processing unit 13. x ^(k) and v y ^(k) is received every certain period of time and the kinematic feature is calculated. In this example, the series of the torque estimates over a certain period of time is used as the feature directly. Note that the rate of change of torque may also be used as the kinematic feature.

[0073] For example, if a certain time from the start of estimation is Δg, the torque estimation value series in the X direction and the torque estimation value series in the y direction are {v x ^(1),v x ^(2),…,v x ^(Δg)} {v y ^(1),v y ^(2),…,v y ^(Δg)} The kinematic feature calculation processing unit 13 outputs the calculated kinematic feature to the exercise purpose determination processing unit 14.

[0074] (2-4) Determining the purpose of exercise Next, in step S24, the control unit 1 of the muscle activity estimation device CS, under the control of the exercise purpose determination processing unit 14, determines the exercise purpose of the operation imagined by the user US based on the kinematic feature calculated by the kinematic feature calculation processing unit 13 and the two-dimensional torque data and muscle activity data for each exercise purpose input from the kinematic feature-muscle activity model as follows:

[0075] That is, the motion purpose determination processing unit 14 first receives the torque estimation value series {v x ^(1),v x ^(2),…,v x ^(Δg)} and the torque estimate series in the y direction {v y ^(1),v y ^(2),…,v y ^(Δg)} and obtain two-dimensional torque data V for each movement direction d from the kinematic feature-muscle activity model. x d (k),V y d (k) and muscle activity data U m d Get (k), where k is (k=1,…,end) and d is (d=1,…,D).

[0076] Next, the exercise purpose determination processing unit 14 calculates the correlation between the two-dimensional torque estimation series and the two-dimensional torque acquired from the kinematic feature-muscle activity model for each movement direction d as follows:

[0077]

number

[0078] The exercise purpose determination processing unit 14 calculates the calculated cc x d ,cc y d Compare with the threshold α, and cc x d ,cc y d If both of the direction d and the direction d exceed the threshold value α, the direction d at this time is determined to be the movement direction of the operation of the user US. Then, the movement direction d is output to the muscle activity summarization processing unit 15 as the determination result of the exercise purpose.

[0079] (2-5) Collection of muscle activity data Subsequently, in step S25, the control unit 1 of the muscle activity estimation device CS, under the control of the muscle activity aggregation processing unit 15, aggregates the muscle activity data U based on the brain activity output from the muscle activity and kinematics estimation processing unit 12. m ^(k) and the muscle activity data U for each exercise purpose output from the kinematic feature-muscle activity model for each exercise purpose. m d Based on (k) and the exercise purpose determination result d output from the exercise purpose determination processing unit 14, the final muscle activity sequence is calculated as follows:

[0080] That is, the muscle activity summarization processing unit 15 first calculates muscle activity data U based on brain activity up to time 1, ..., Δg. m On the other hand, after the time Δg + 1 when the direction of movement is estimated, the muscle activity data U based on brain activity is used depending on the reliability of the estimation result of the direction of movement. m ^(k) and muscle activity data U according to the exercise purpose m dThe weighted average of (k) is calculated, and the calculated average is used as the final muscle activity data. For example, using the correlation coefficient calculated during the movement direction determination process, muscle activity is calculated using the following formula:

[0081]

number

[0082] In addition, by relying on the judgment result of the movement direction, muscle activity data U according to the movement purpose output from the kinematics feature-muscle activity model is m d (k) may be used as the final muscle activity data as is.

[0083] The muscle activity summarizing processing unit 15 stores the following muscle activity series estimated as described above in the muscle activity data storage unit 34.

[0084]

number

[0085] (3) Application of electrical stimulation In step S27, the control unit 1 of the muscle activity estimation device CS reads the finally obtained muscle activity series from the muscle activity data storage unit 34 under the control of the electrical stimulation generation processing unit 16, and generates an electrical stimulation signal for applying electrical stimulation to the corresponding muscle of the user US based on the read muscle activity series. Then, the electrical stimulation generation processing unit 16 outputs the generated electrical stimulation signal from the input / output I / F unit 5 to the electrical stimulation generator ES.

[0086] As a result, electrical stimulation is applied from the electrical stimulation generator ES to the corresponding muscle of the user US, and this electrical stimulation moves the wrist of the user US.

[0087] (effect) As described above, in one embodiment, based on the brain activity measurement data of the user US, muscle activity and kinematics based on brain activity are first estimated using a muscle activity-kinematics estimation model, and the exercise purpose of the user US is determined based on the correlation between the kinematic feature calculated from the two-dimensional torque of the estimated kinematics and the kinematics estimated value for each exercise purpose generated by the kinematic feature-muscle activity model. Then, depending on the exercise purpose determination result, final muscle activity estimation data is generated based on the muscle activity estimation data based on the brain activity estimated by the muscle activity-kinematics estimation model and the muscle activity estimation data corresponding to the exercise purpose generated by the kinematic feature-muscle activity model.

[0088] Therefore, the estimated muscle activity data based on brain activity is corrected based on the muscle activity data corresponding to the movement purpose estimated from the kinematics, which allows for accurate muscle activity estimation data to be obtained, and makes it possible to apply electrical stimulation to the muscles of the user US more accurately based on this muscle activity estimation data.

[0089] [Other embodiments] (1) In one embodiment, the muscle activity estimation device CS has been described as having a learning function for the muscle activity / kinematics estimation model and the kinematics feature value-muscle activity model. However, the model data of the trained muscle activity / kinematics estimation model and the kinematics feature value-muscle activity model may be downloaded from, for example, a personal computer or server computer used for system management and used.

[0090] (2) In the embodiment, the muscle activity estimation device CS is provided in a mobile information terminal such as a wearable device owned by the user US. However, the muscle activity estimation device CS may be provided in, for example, a personal computer used by a system administrator or a server computer located on the web or in the cloud. This allows multiple users US to use one muscle activity estimation device CS.

[0091] (3) In one embodiment, a so-called non-invasive case has been described, in which the electrodes of the EEG sensor are attached to the user's head and the electrodes for muscle stimulation are attached to the skin surface. However, this is not limited to this. Alternatively, the EEG sensor may be placed in direct contact with the user's cerebral cortex and the electrodes for electrical stimulation may be placed in direct contact with the user's muscle regions. While this is an invasive system configuration requiring surgery, it is possible to obtain EEG data directly from the cerebral cortex, thereby improving the accuracy of brain activity data and thereby improving the accuracy of muscle activity and kinematics estimation based on brain activity. Furthermore, by placing the electrodes for electrical stimulation directly in contact with the muscle regions, it is possible to reliably stimulate the desired muscle regions.

[0092] In addition, although the embodiment has been described using an example in which the target user is a human, the target user may be an animal other than a human. In addition, the functions of each processing unit of the muscle activity estimation device, the processing procedures and contents thereof, the configuration of each model, and the like can be modified in various ways without departing from the spirit and scope of the present invention.

[0093] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0094] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0095] CS…muscle activity estimation device BS: Brain wave sensor TM…Electrode ES...electrical stimulation generator 1...Control unit 2...Program memory section 3...Data storage unit 4...Sensor I / F section 5...Input / output interface 6...Bus 11...Brain activity data acquisition processing unit 12...Muscle activity and kinematics estimation processing unit 13...Kinematics feature calculation processing unit 14...exercise purpose determination processing unit 15...Muscle activity aggregation processing unit 16...Electrical stimulation generation processing unit 31...Brain activity data storage unit 32...Muscle activity and kinematics estimation model memory section 33...Kinematics feature-muscle activity model memory section 34...Muscle activity data storage unit

Claims

1. a first processing unit that acquires brain activity data of a user; a second processing unit that estimates a first muscle activity of the user and estimates kinematics based on the brain activity data; a third processing unit that calculates a kinematic feature amount from the estimated kinematic data; a fourth processing unit that estimates an exercise goal corresponding to the calculated kinematic feature amount and a second muscle activity corresponding to the exercise goal, based on model data that is prepared in advance for each exercise goal and that indicates a relationship between the kinematic feature amount and a muscle activity; a fifth processing unit that estimates a third muscle activity corresponding to the exercise purpose by reflecting the second muscle activity in the first muscle activity based on the estimation result of the exercise purpose; and A muscle activity estimation device comprising:

2. 2. The muscle activity estimation device according to claim 1, wherein the second processing unit acquires the first estimated muscle activity data and the estimated kinematics data corresponding to the brain activity data using a trained muscle activity / kinematics estimation model.

3. The fifth processing unit is performing a first process of selecting the estimated data of the first muscle activity as the third muscle activity during a period until the exercise purpose is estimated; During a period after the exercise purpose is estimated, a second process is performed to estimate the third muscle activity based on the estimated data of the first muscle activity and the estimated data of the second muscle activity, depending on the reliability of the exercise purpose estimation result. The muscle activity estimation device according to claim 1 .

4. 4. The muscle activity estimation device according to claim 3, wherein the fifth processing unit estimates the third muscle activity by calculating a weighted average of the estimated data of the first muscle activity and the estimated data of the second muscle activity in the second process.

5. 2. The muscle activity estimation device according to claim 1, further comprising: a sixth processing unit configured to generate, based on the third muscle activity estimated by the fifth processing unit, an electrical stimulation signal for applying electrical stimulation to a corresponding muscle site of the user.

6. A muscle activity estimation method in which an information processing device executes a process of estimating muscle activity based on a user's brain activity, acquiring brain activity data of the user; estimating a first muscle activity and kinematics of the user based on the brain activity data; calculating kinematic features from the estimated kinematics data; a step of estimating an exercise goal corresponding to the calculated kinematic feature amount and a second muscle activity corresponding to the exercise goal, based on model data representing a relationship between the kinematic feature amount and muscle activity, which is prepared in advance for each exercise goal; a step of estimating a third muscle activity corresponding to the exercise purpose by reflecting the second muscle activity in the first muscle activity based on the estimation result of the exercise purpose; A muscle activity estimation method comprising:

7. A program causing a processor included in the muscle activity estimation device to execute processing executed by at least one of the processing units included in the muscle activity estimation device according to claim 1 .

Images