Estimation device, estimation model creation device, estimation method, estimation model creation method, and program

Abstract

To estimate an action that a hand is going to do.SOLUTION: An estimation device estimates an action that a subject wants to do with a hand of the subject by inputting a muscle potential measured from the upper limb into an estimation model which uses the muscle potential as an input and uses an action that the subject wants to do with a hand as an output.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an estimation device, an estimation method, and a program. [Background technology]

[0002] Illnesses such as strokes can leave people with residual paralysis. Patients with residual paralysis undergo rehabilitation to treat the paralysis. Research is being conducted to improve the effectiveness of rehabilitation by analyzing movements and providing feedback. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Tan CK, Kadone H, Watanabe H, Marushima A, Hada Y, Yamazaki M, Sankai Y, Matsumura A, Suzuki K. Differences in Muscle Synergy Symmetry Between Subacute Post-stroke Patients With Bioelectrically-Controlled Exoskeleton Gait Training and Conventional Gait Training. Front Bioeng Biotechnol. 8:770. 2020 [Non-Patent Document 2] Tan CK, Kadone H, Watanabe H, Marushima A, Yamazaki M, Sankai Y, Suzuki K. Lateral Symmetry of Synergies in Lower Limb Muscles of Acute Post-stroke Patients After Robotic Intervention. Front Neurosci. 12:276. 2018 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are no fully established methods for improving motor paralysis of the hand. An object of the present invention is to provide an estimation device, an estimation method, and a program for estimating an intended hand movement. [Means for solving the problem]

[0005] One aspect of the present invention is an estimation device that estimates the movement that a subject wishes to have his or her hand performed by inputting the myoelectric potential measured from the upper limb into an estimation model that outputs the movement that the subject wishes to have his or her hand performed.

[0006] One aspect of the present invention is an estimation device that estimates an evaluation index for a movement performed by a subject with his / her hands by inputting the myoelectric potential measured from the upper limbs into an estimation model that takes the myoelectric potential measured from the upper limbs as input and outputs an evaluation index for the movement performed by the subject with his / her hands.

[0007] One aspect of the present invention is an estimation method for estimating a movement that a subject wishes to have his or her hand performed by inputting myoelectric potentials measured from the upper limbs into an estimation model that outputs the movement that the subject wishes to have his or her hand performed.

[0008] One aspect of the present invention is a program that causes a computer to estimate the movement that a subject wishes to have his or her hand performed by inputting the myoelectric potentials measured from the upper limbs into an estimation model that outputs the movement that the subject wishes to have his or her hand perform. Effect of the Invention

[0009] According to the present invention, it is possible to estimate the movement that the hand is about to perform. [Brief description of the drawings]

[0010] [Figure 1]1 is a diagram showing a configuration of an estimation model creation device 1 according to a first embodiment. [Diagram 2] 1 shows an example of measured myoelectric potential signals for different instructed hand movements. [Diagram 3] FIG. 2 is a diagram showing a configuration of an estimation device 4 according to the first embodiment. [Figure 4] 4 is a flowchart showing an operation of the estimation model creation device 1 according to the first embodiment. [Diagram 5] 4 is a flowchart showing an operation of the estimation model creation device 1 according to the first embodiment. [Figure 6] 4 is a flowchart showing the operation of the estimation device 4 according to the first embodiment. [Figure 7] FIG. 13 is a diagram showing the accuracy of the estimation result using an estimation model that estimates the hand movement when the upper limb of the measurement subject is not in a paralyzed state. [Figure 8] FIG. 13 is a diagram showing the accuracy of the estimation result using an estimation model that estimates hand movements when the upper limb of a measurement subject is in a paralyzed state. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] First Embodiment Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. 1 is a diagram showing the configuration of an estimation model creation device 1 according to the first embodiment. The estimation model creation device 1 according to the first embodiment includes a measurement signal receiving unit 11, a feature extracting unit 12, a feature recording unit 13, a model generating unit 14, a model output unit 15, and a storage unit 21.

[0012] The measurement signal receiving unit 11 receives a myoelectric potential (Electromyogram: EMG) measured from the upper limb 2 of the subject by a measuring device 3. The measuring device 3 is attached, for example, to the skin surface of the upper limb 2, and measures the myoelectric potential signal.

[0013] The measurement device 3 may measure myoelectric potential signals at multiple positions of the upper limb 2. For example, the measurement device 3 measures myoelectric potential signals at, for example, the flexor carpi radialis (FCR), the flexor carpi ulnaris (FCU), the abductor pollicis brevis (APB), and the extensor digitorum communis (EDC).

[0014] The measuring device 3 measures myoelectric potential signals for each different hand movement of the upper limb 2 instructed to the subject. The different hand movements include, for example, thumbs up, pinch, fist, extension, spread, flexion, etc. For example, the measuring device 3 measures myoelectric potential signals for specific hand movements by having the subject perform different hand movements in sequence for a certain period of time. Fig. 2 shows an example of the myoelectric potential signal measured for each of the different instructed hand movements. In the example shown in Fig. 2, the hand is instructed to perform the movements in the order of gripping, pinching, bending, stretching, spreading, and thumbs up, and the measuring device 3 measures the myoelectric potential signal for each movement.

[0015] The measurement device 3 may measure myoelectric potential signals of two upper limbs of the same person. One of the two upper limbs may be paralyzed due to an illness such as acute stroke.

[0016] The feature extraction unit 12 extracts features from the myoelectric potential signal for each hand movement and each position where the myoelectric potential signal is measured. The feature is a numerical value or a vector indicating the characteristics of the extracted raw data. There are no particular limitations on the method of extracting the features, and the feature extraction unit 12 may calculate, for example, the maximum fractal length (MFL) or the multi-model deep feature (MMDF) as the feature of the myoelectric potential signal. Note that the feature extraction unit 12 may combine a plurality of feature extraction methods.

[0017] The feature amount recording unit 13 records the extracted feature amount for each different hand movement in the storage unit 21. The feature amount recording unit 13 may record the feature amount separately depending on whether the upper limb of the measurement subject is paralyzed or not. Also, the feature amount recording unit 13 may record the feature amount separately for each person of the measurement subject.

[0018] The model generation unit 14 generates an estimation model based on the recorded feature amount. The generated estimation model is a model that receives the feature amount extracted from the myoelectric potential signal as an input and outputs the hand movement of the upper limb. The model generation unit 14 generates a model that classifies the feature amount by applying, for example, a support vector machine (SVM) to the feature amount for each different hand movement. The model generating unit 14 may generate different estimation models depending on whether the upper limb of the measurement subject is paralyzed or not, and may generate an estimation model for each person who is the measurement subject.

[0019] The model output unit 15 outputs the generated estimation model to the estimation device 4, which will be described later.

[0020] 3 is a diagram showing the configuration of an estimation device 4 according to the first embodiment. The estimation device 4 includes a measurement signal receiving unit 41, a feature extracting unit 42, an estimation unit 43, an estimation result output unit 44, and a storage unit 51. The storage unit 51 stores an estimation model output by the estimation model creation device 1.

[0021] Similar to the measurement signal receiving unit 11, the measurement signal receiving unit 41 receives the myoelectric potential measured by the measuring device 3 from the upper limb 5. Similar to the feature extracting unit 12, the feature extracting unit 42 extracts features from the myoelectric potential signal.

[0022] The estimation unit 43 inputs the feature amount into the estimation model, thereby outputting the estimated upper limb hand movement. The estimation result output unit 44 outputs the upper limb hand movement as the estimated result. In this way, the estimation device 4 can estimate the movement of the upper limb hand that is about to be performed. The output upper limb hand movements are displayed, for example, on a VR headset or AR headset worn by the person being measured, providing visual assistance to the person being measured when undergoing rehabilitation of their paralyzed upper limbs.

[0023] 4 and 5 are flowcharts showing the operation of the estimation model creation device 1 according to the first embodiment. FIG. 4 is a flowchart showing the operation of the first stage of the estimation model creation device 1. First, the measurement signal receiving unit 11 receives the myoelectric potential signal measured by the measurement device 3 (step S11). After that, the feature amount extracting unit 12 extracts the feature amount from the myoelectric potential signal (step S12). The feature amount recording unit 13 records the extracted feature amount in the storage unit 21. At this time, the feature amount recording unit 13 may record the corresponding hand movement, subject information, whether or not the measured upper limb is paralyzed, etc. together with the feature amount. The estimation model creation device 1 increases the number of feature amounts stored in the storage unit 21 by repeating the operations from steps S11 to S13.

[0024] 5 is a flowchart showing the operation of the latter stage of the estimation model creation device 1. The model generation unit 14 generates an estimation model based on the feature amount stored in the storage unit 21 (step S14). The model output unit 15 outputs the generated estimation model to the estimation device 4 (step S15).

[0025] 6 is a flowchart showing the operation of the estimation device 4 according to the first embodiment. The measurement signal receiving unit 41 receives the myoelectric potential signal measured by the measurement device 3 (step S21). After that, the feature amount extracting unit 42 extracts feature amounts from the myoelectric potential signal (step S22). The estimation unit 43 estimates the hand movement from the feature amounts using an estimation model (step S23). The estimation result output unit 44 outputs the estimation result by the estimation unit 43 (step S24).

[0026] 7 and 8 are diagrams showing the estimation results by the estimation device 4. FIG. 7 is a diagram showing the accuracy of the estimation result by the estimation model that estimates the hand movement when the upper limb of the measurement subject is not paralyzed. FIG. 8 is a diagram showing the accuracy of the estimation result by the estimation model that estimates the hand movement when the upper limb of the measurement subject is paralyzed. The accuracy rate, which is the rate at which the actual hand movement (True gesture) and the estimated hand movement (Predicted gesture) are the same, was calculated. SVM was used to create the estimation model. In the results shown in Figure 7, the accuracy rate was 86.6%, and in the results shown in Figure 8, the accuracy rate was 67.0%. Therefore, when the upper limbs are paralyzed, the hand movements can be estimated with an accuracy rate of 67%.

[0027] Second Embodiment The estimation model creation device 1 and the estimation device 4 according to the second embodiment will be described below, but a description of the same points as those of the estimation model creation device 1 and the estimation device 4 according to the first embodiment will be omitted. The feature recording unit 13 according to the second embodiment records the extracted feature and an evaluation index of the hand movement for the feature in the storage unit 21. The evaluation index of the hand movement is an index for evaluating the degree of paralysis, such as FMA (Fugl-Meyer Assessment), Brunnstrom stage, SIAS (Stroke Impairment Assessment Set), and MAS (Modified Ashworth Scale). The model generating unit 14 according to the second embodiment receives the feature extracted from the myoelectric potential signal as an input and generates a model that outputs an evaluation index of the hand movement.

[0028] The estimation unit 43 according to the second embodiment inputs the feature amount into the estimation model, thereby outputting the evaluation index of the hand movement. The estimation result output unit 44 according to the second embodiment outputs the evaluation index of the hand movement as an estimation result. The output evaluation index of the hand movement is displayed, for example, on an external display device. This makes it possible to estimate an index of paralysis of the upper limbs of the person to be measured.

[0029] When one of the two upper limbs of the person being measured is paralyzed and the other is non-paralyzed, the feature recorded by the feature recording unit 13 according to the second embodiment and the feature extracted by the feature extraction unit 42 according to the second embodiment may be a feature extracted from an EMG signal measured from the two upper limbs.

[0030] Other Embodiments Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention. For example, the estimation model may take a feature extracted from the myoelectric potential signal as an input and output a severity. In this case, the model generation unit 14 takes the feature extracted from the myoelectric potential signal as an input and generates a model that outputs a severity. The estimation unit 43 inputs the feature to the estimation model to output a severity.

[0031] The estimation model creation device 1 and the estimation device 4 in the above-mentioned embodiment may be partly or entirely realized by a computer. In that case, a program for realizing the function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes an OS and hardware of peripheral devices. The term "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a recording device such as a hard disk built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or a client in that case. The above-mentioned program may be a program for realizing part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system. The estimation model creation device 1 and the estimation device 4 may also be partly or entirely realized using a programmable logic device such as an FPGA (Field Programmable Gate Array). [Explanation of symbols]

[0032] 1 Estimation model creation device, 11 Measurement signal receiving unit, 12 Feature extraction unit, 13 Feature recording unit, 14 Model generation unit, 15 Model output unit, 21 Memory unit, 2 Upper limb, 3 Measurement device, 4 Estimation device, 41 Measurement signal receiving unit, 42 Feature extraction unit, 43 Estimation unit, 44 Estimation result output unit, 5 Upper limb, 51 Memory unit

Claims

1. By inputting the myoelectric potential measured from the upper limb of a measurement target person into an estimation model that outputs the movement the subject wants to perform with the hand, using the myoelectric potential measured from the upper limb of the subject as an input, the movement the measurement target person wants to perform with the hand is estimated. Estimation device.

2. By inputting the myoelectric potential measured from the upper limb of a measurement target person into an estimation model that outputs an evaluation index of the movement performed by the hand of the subject, using the myoelectric potential measured from the upper limb of the subject as an input, the evaluation index of the movement performed by the hand of the measurement target person is estimated. Estimation device.

3. A feature quantity extraction unit that extracts feature quantities from the electric potential detected from the upper limb of the subject, A feature quantity recording unit that records the feature quantities for each hand movement instructed by the subject, And a model generation unit that generates an estimation model that outputs the movement the subject wants to perform with the hand from the feature quantities based on the feature quantities recorded for each hand movement instructed by the subject. Estimation model creation device.

4. The feature quantity extraction unit extracts a plurality of types of feature quantities from the electric potential detected from the upper limb of the subject, The feature quantity recording unit records the plurality of types of feature quantities for each hand movement instructed by the subject, The model generation unit generates an estimation model that outputs the movement the subject wants to perform with the hand from the plurality of types of feature quantities based on the plurality of types of feature quantities recorded for each hand movement instructed by the subject. The estimation model creation device according to Claim 3.

5. By inputting the myoelectric potential measured from the upper limb of a measurement target person into an estimation model that outputs the movement the subject wants to perform with the hand, using the myoelectric potential measured from the upper limb of the subject as an input, the movement the measurement target person wants to perform with the hand is estimated. Estimation method.

6. Extract feature quantities from the electric potential detected from the upper limb of the subject, Record the feature quantities for each hand movement instructed by the subject, Generate an estimation model that outputs the movement the subject wants to perform with the hand from the feature quantities based on the feature quantities recorded for each hand movement instructed by the subject. Estimation model creation method.

7. Cause a computer to estimate the movement the measurement target person wants to perform with the hand by inputting the myoelectric potential measured from the upper limb of the measurement target person into an estimation model that outputs the movement the subject wants to perform with the hand, using the myoelectric potential measured from the upper limb of the subject as an input. Program.

8. For a computer, Extract feature quantities from the potentials detected from the upper limbs of the subject, Record the feature quantities for each hand movement instructed by the subject, Generate an estimation model that outputs the movement the subject wants to perform with the hand from the feature quantities based on the feature quantities recorded for each hand movement instructed by the subject, Program.

Images