Information processing device, information processing method, and computer program

Abstract

To provide a technology for controlling operation of a controlled device by utilizing muscle synergy, and a technology for supporting predetermined task learning with body motion of a user by utilizing muscle synergy.SOLUTION: An information processing apparatus includes: a holding unit which holds muscle synergy on a predetermined task; an acquisition unit which acquires muscle activity of a user who executes the task from a sensor; a calculation unit which calculates activity with respect to the muscle synergy from the muscle activity acquired by the acquisition unit, by use of the muscle synergy held in the holding unit; and a processing execution unit which executes predetermined processing on the basis of the activity calculated by the calculation unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a computer program. [Background technology]

[0002] Conventionally, a method has been proposed for extracting a combination of multiple covarying muscles (hereinafter referred to as "muscle synergy") and its activity, also called activation, from the muscle activity of a user performing a specified task, for example by using non-negative matrix factorization (NMF) (e.g., Non-Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Non-Patent Document 1] VCK Cheunga, A. Turollab, M. Agostinib, S. Silvonib, C. Bennisc, P. Kasic, S. Paganonic, P. Bonatoc and E. Bizzia, “Muscle synergy patterns as physiological markers of motor cortical damage,” PNAS, vol. 109, no.2012, pp. 14652-14656. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention has been made in consideration of the above circumstances, and one exemplary objective of one aspect of the present invention is to provide a technology for controlling the operation of a controlled device by utilizing muscle synergy, and another exemplary objective is to provide a technology for assisting a user in learning a specific task by moving the body by utilizing muscle synergy. [Means for solving the problem]

[0005] In order to solve the above problems, an information processing device according to one aspect of the present invention includes: a storage unit which stores muscle synergies related to a predetermined task; an acquisition unit which acquires a muscle activity amount of a user performing the task from a sensor; a calculation unit which calculates an activity amount for the muscle synergy from the muscle activity amount acquired by the acquisition unit by using the muscle synergy stored in the storage unit; and a processing execution unit which executes a predetermined process based on the activity amount calculated by the calculation unit.

[0006] Another aspect of the present invention is an information processing method, which includes the steps of: storing muscle synergies related to a predetermined task in a predetermined storage unit; measuring muscle activity of a user performing the task; calculating an activity amount for the muscle synergy from the measured muscle activity amount by using the muscle synergy related to the task stored in the predetermined storage unit; and executing a predetermined process based on the calculated activity amount, by a computer.

[0007] Any combination of the above components, or mutual substitution of the components or expressions of the present invention between methods, devices, systems, etc. are also valid aspects of the present invention. Effect of the Invention

[0008] According to an aspect of the present invention, the motion of a controlled device can be controlled by utilizing muscle synergy, or muscle synergy can be utilized to assist a user in learning a predetermined task by moving his or her body. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an information processing system according to a first embodiment. [Diagram 2] 13 is a flowchart showing a pre-processing operation. [Diagram 3] 13 is a flowchart showing the operation of this process. [Figure 4] FIG. 13 is a diagram illustrating a configuration of an information processing system according to a second embodiment. [Diagram 5] 13 is a flowchart showing the operation of this process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The present invention will be described below based on preferred embodiments with reference to the drawings. The embodiments are illustrative and do not limit the invention, and all features and combinations described in the embodiments are not necessarily essential to the invention. The same or equivalent components, parts, and processes shown in each drawing are given the same reference numerals, and duplicated descriptions are omitted as appropriate.

[0011] (First embodiment) First, an outline of the first embodiment will be described. As described in the background art, muscle synergies and their activity amounts can be extracted from muscle activity amounts. Therefore, for example, when reproducing the movement of a user's wrist and beyond with a prosthetic hand as a controlled device, the muscle activity amounts of each of a plurality of muscles in the forearm are measured, muscle synergies and their activity amounts are extracted from the muscle activity amounts, and the prosthetic hand is controlled based on the extracted muscle synergies and activity amounts. However, in order to extract muscle synergies and their activity amounts, muscle activity amounts of a certain length in time are required, so they cannot be extracted substantially in real time. Therefore, the conventional method cannot cause the controlled device to reproduce the user's movement substantially in real time, that is, to follow the user's movement.

[0012] In contrast, in the first embodiment, as a "pre-processing" before executing the control of the controlled device, the amount of muscle activity when the user is performing a task is measured, muscle synergies are extracted from the muscle activity, and the extracted muscle synergies are stored in the storage unit. Then, as a "main process" for executing the control of the controlled device, the amount of muscle activity when the user is performing a task is measured, and the amount of activity for the muscle synergy is calculated from the amount of muscle activity using the muscle synergy stored in the storage unit, and the controlled device is controlled based on the muscle synergy and the amount of activity. The muscle synergies are extracted in advance (i.e., in the pre-processing), and the process of calculating the amount of activity does not require a long-term amount of muscle activity, as will be described in detail later. Therefore, the amount of activity can be calculated substantially in real time, and the controlled device can be made to reproduce the user's movement substantially in real time, i.e., to follow the user's movement. A specific description will be given below.

[0013] 1 is a diagram showing a configuration of an information processing system 100 according to a first embodiment. The information processing system 100 includes a controlled device 102, a muscle activity amount measuring unit 104 for measuring a muscle activity amount of a user, and an information processing device 106 for controlling the controlled device 102 based on the muscle activity amount measured by the muscle activity amount measuring unit 104. The controlled device 102 and the muscle activity amount measuring unit 104 are connected to the information processing device 106 by wire or wirelessly. In this embodiment, the controlled device 102 is, although not limited to, a prosthetic hand for reproducing a task of moving a user's right wrist and beyond.

[0014] The muscle activity measuring unit 104 measures the amount of muscle activity when the user performs a task in the pre-processing and main processing. In detail, the muscle activity measuring unit 104 includes a plurality of sensors 108_1 to 108_n for measuring the amount of muscle activity of each of a plurality of muscles of the forearm. Each of the plurality of sensors 108_1 to 108_n measures myoelectric potential as the amount of muscle activity. The muscle activity measuring unit 104 calculates the amount of muscle activity M(m1, ..., m n ) to the information processing device 106.

[0015] Functional blocks are shown for the information processing device 106. Each block shown here can be realized in terms of hardware by elements and mechanical devices such as a computer CPU and memory, and in terms of software by a computer program, etc., but here, functional blocks realized by their cooperation are depicted. Those skilled in the art will understand that these functional blocks can be realized in various forms by combining hardware and software. In addition, the functions of multiple functional blocks may be implemented in a single computer, or may be distributed and implemented in multiple computers. For example, the functions of multiple functional blocks may be realized by multiple computers cooperating as a system. In other words, the information processing device 106 may be configured by multiple computers. The same applies to FIG. 4.

[0016] The information processing device 106 includes a communication unit 110 that is responsible for communication with the controlled device 102 and the muscle activity measuring unit 104, a data storage unit 120 that is a storage area for storing various data, and a data processing unit 130 that executes various data processing based on the data stored in the data storage unit 120 or acquired by the communication unit 110. The data storage unit 120 includes a muscle synergy storage unit 121.

[0017] The data processing unit 130 includes a muscle activity amount acquiring unit 131, a muscle synergy extracting unit 132, an activity amount calculating unit 133, and a processing executing unit 134.

[0018] In the pre-processing and the main processing, the muscle activity acquisition unit 131 acquires the muscle activity M (m1, ..., m n ) to get the

[0019] In pre-processing, the muscle synergy extraction unit 132 extracts muscle synergies for a task based on the amount of muscle activity of the user while performing the task measured by the muscle activity measurement unit 104. The muscle synergy extraction unit 132 may extract muscle synergies using a known technique, typically non-negative matrix factorization (NMF). The muscle synergy extraction unit 132 extracts muscle synergies S (S1, ..., S m ) is stored in the muscle synergy storage unit 121.

[0020] In this process, the activity amount calculation unit 133 calculates the muscle synergies S (S1, ..., S m ), the muscle activity amount M(m1, ..., m n ), the activity C(C1, ,C m ) is calculated.

[0021] Although not particularly limited, in this embodiment, the activity amount calculation section 133 calculates the activity amount C by solving the optimization problem of the following equation (1).

number

[0022] The process execution unit 134 calculates the muscle synergies (S1, . . . , S m ), and the activity amount C (C1, ..., C m ) and controls the controlled device 102 based on the result.

[0023] The above is the basic configuration of the information processing system 100. Next, the operation will be described.

[0024] FIG. 2 is a flowchart showing the operation of the pre-processing.

[0025] With multiple sensors 108_1 to 108_n attached to the upper arm, the user executes a task of moving the wrist and beyond. The motion in this task is, for example, moving the wrist up and down and left and right, or clenching and opening the hand with the back of the hand facing vertically upward. The muscle activity amount acquiring unit 131 acquires the amount of muscle activity measured by the muscle activity amount measuring unit 104 during the execution of the task (S10). The muscle synergy extracting unit 132 extracts muscle synergies from the amount of muscle activity acquired by the muscle activity amount acquiring unit 131 (S12). The muscle synergy extracting unit 132 stores the extracted muscle synergies in the muscle synergy storage unit 121 (S14).

[0026] Fig. 3 is a flowchart showing the operation of this process. A series of processes in Fig. 3 is repeatedly executed at a predetermined cycle (for example, a cycle of 100 msec). It is assumed that the user is executing a predetermined task with multiple sensors 108_1 to 108_n attached to his / her arm.

[0027] The muscle activity amount acquiring unit 131 acquires the muscle activity amount measured by the muscle activity amount measuring unit 104 (S20). The activity amount calculating unit 133 calculates the activity amount for the muscle synergy stored in the muscle synergy storing unit 121 from the muscle activity acquired by the muscle activity amount acquiring unit 131 (S22). The process executing unit 134 controls the controlled device 102 based on the activity amount for the muscle synergy calculated by the activity amount calculating unit 133 (S24).

[0028] According to the present embodiment described above, the controlled device 102 is controlled based on the muscle synergies stored in the muscle synergy storage unit 121 and the activity amount calculated by the activity amount calculation unit 133 solving the equation (1). Here, the muscle synergies are stored in the muscle synergy storage unit 121, i.e., data prepared in advance. The process of solving the equation (1) is a process with a relatively low load. Furthermore, the equation (1) only requires the muscle activity amount at that moment, and does not require the muscle activity amount for a certain period of time, unlike the case of calculating muscle synergies using non-negative matrix factorization. Therefore, the equation (1) can be solved substantially in real time. Therefore, the controlled device 102 can be made to reproduce the user's movement substantially in real time, i.e., to follow the user's movement.

[0029] One aspect of the present invention has been described above based on the first embodiment. Next, modified examples related to the first embodiment will be described.

[0030] (First Modification of the First Embodiment) In the first embodiment, the controlled device 102 is a prosthetic arm and the task is to move the wrist and beyond, but the present invention is not limited thereto. For example, the controlled device 102 may be a humanoid robot and the task may be to walk. In this case, the multiple sensors 108_1 to 108_n may be attached to the user so as to measure the amount of muscle activity of various muscles used for walking, including the muscles of the legs and arms.

[0031] (Second Modification of the First Embodiment) In the first embodiment, a case has been described in which the user who performs the pre-processing and the user who performs the main processing are the same, but this is not limited to this, and the users who perform them may be different.

[0032] (Second embodiment) First, an outline of the second embodiment will be described. Non-skilled people, for example, non-skilled people in the world of craftsmen and sports, often learn the skills of experts, that is, the movements of experts in a task, by watching and imitating. This is, of course, an inefficient learning method. As a more efficient learning method, for example, a method is conceivable in which an unskilled person and an expert perform a task at the same time, output a comparison result of the muscle activity of both at that time, and devise a movement in the task while checking it. In other words, a method is conceivable in which an unskilled person devise a movement so that his or her own muscle activity approaches the target, with the muscle activity of an expert as a target. However, this learning method is not realistic because the expert needs to be constantly present with the unskilled person.

[0033] In contrast to this, in the second embodiment, as a "pre-processing" before learning, the amount of muscle activity is measured when a user (e.g., an expert) other than the user who performs this process is performing a task, muscle synergies are extracted from the measured amount of muscle activity, and the extracted muscle synergies are stored in the storage unit. As a "main process" for performing learning, the amount of muscle activity is measured when a user (e.g., a non-expert) is performing a task, and an amount of activity for the muscle synergy is calculated from the measured amount of muscle activity using the muscle synergy stored in the storage unit, and an amount of muscle activity estimated by the muscle synergy of the expert is calculated from the muscle synergy and the calculated amount of activity, and a comparison result between the measured amount of muscle activity and the estimated amount of muscle activity is output. The user devise a movement for the task while checking the comparison result. This enables efficient learning. A specific description will be given below.

[0034] 4 is a diagram showing a configuration of an information processing system 200 according to the second embodiment. The following description will focus on the differences from the first embodiment.

[0035] The information processing system 200 includes a U / I (user interface) unit 112 that is responsible for displaying information and outputting audio. On the other hand, the information processing system 200 does not include a controlled machine.

[0036] In the present embodiment, in the pre-processing, the muscle activity amount acquisition unit 131 acquires muscle activity amounts of a user other than the user executing this process, and the muscle synergy extraction unit 132 extracts muscle synergies from the muscle activity amounts of the other user and stores them in the muscle synergy storage unit 121. In the present process, the activity amount calculation unit 133 calculates activity amounts from the muscle activity amounts of the user executing this process, using muscle synergies based on the other user stored in the muscle synergy storage unit 121.

[0037] In this process, if the muscle activity of the user measured by the muscle activity measuring unit 104 (hereinafter referred to as "actual muscle activity") approaches the muscle activity of another user based on the muscle synergy calculated by the following formula (2) (hereinafter referred to as "estimated muscle activity"), the movements in the task will approach those of the other user. Muscle activity amount (estimated) = S×C (2) Where: S: Muscle synergy based on another user stored in the muscle synergy storage unit 121 C: Activity amount calculated by the activity amount calculation unit 133 It is.

[0038] Therefore, the processing execution unit 134 calculates the amount of muscle activity (estimated) and then outputs a comparison result between the amount of muscle activity (actual) and the amount of muscle activity (estimated). In this embodiment, the processing execution unit 134 outputs the score value (R 2 ) is output as the comparison result. The closer the muscle activity (actual) is to the muscle activity (estimated), the smaller this score value becomes.

number

[0039] The comparison result may be output by directly displaying or graphically displaying the score value on a specified display, or by outputting a voice reading out the score value or a tone corresponding to the score value from a specified speaker. As a modified example, the comparison result may be output by displaying the muscle activity amount (actual) and the muscle activity amount (estimated) in a manner that allows comparison between the two, such as by arranging the time series graphs of the two amounts vertically.

[0040] The above is the basic configuration of the information processing system 200. Next, the operation of the system will be described.

[0041] The operation of the pre-processing is similar to that of the first embodiment. In the pre-processing of the present embodiment, muscle synergies are extracted by measuring muscle activities of a user (e.g., an expert) who is different from the user who executes the main process and who is used as a reference in learning a task.

[0042] Fig. 5 is a flowchart showing the operation of this process. The entire process in Fig. 5 is repeatedly executed at a predetermined cycle. It is assumed that the user is executing a predetermined task with a plurality of sensors 108_1 to 108_n attached to the arm.

[0043] The muscle activity amount acquiring unit 131 acquires the muscle activity amount measured by the muscle activity amount measuring unit 104 (S30). The activity amount calculating unit 133 calculates the activity amount for the muscle synergy stored in the muscle synergy storing unit 121 from the muscle activity acquired by the muscle activity amount acquiring unit 131 (S32). The process executing unit 134 calculates the muscle activity amount (target) obtained by equation (2) (S34), and outputs the result of comparison with the muscle activity amount (actual) (S36).

[0044] The user checks the comparison result and refines the task movement. For example, if the comparison result is a score value calculated by formula (3), the user can refine the task movement to reduce the score value, so that the actual muscle activity approaches the estimated muscle activity. In other words, the user can achieve a task movement similar to that of another user.

[0045] According to the present embodiment described above, by carrying out the task while checking the comparison result, the task can be learned more efficiently. Also, there is no need for another user to be constantly present during learning.

[0046] One aspect of the present invention has been described above based on the second embodiment. Next, a modification related to the second embodiment will be described.

[0047] (First modified example of the second embodiment) In the second embodiment, the case where the user who executes the pre-processing and the user who executes the main process are different has been described, but the present invention is not limited to this, and the users who execute them may be the same. For example, an athlete may execute the pre-processing when he is in good form to extract muscle synergies at that time, and execute the main process when he is in a slump or otherwise in bad form to return to the state when he was in good form. In other words, he may learn by setting his own state when he was in good form (i.e., his past self) as a goal.

[0048] The present invention has been described using specific terms based on the embodiments, but the embodiments merely show one aspect of the principles and applications of the present invention, and many modifications and changes in arrangement are permitted to the embodiments without departing from the spirit of the present invention as defined in the claims. [Explanation of symbols]

[0049] 100 Information processing system, 102 Controlled device, 104 Muscle activity measurement unit, 106 Information processing device, 121 Muscle synergy retention unit, 131 Muscle activity acquisition unit, 133 Activity calculation unit, 134 Processing execution unit.

Claims

1. a storage unit that stores muscle synergies related to a predetermined movement in advance; An acquisition unit that acquires a muscle activity amount of a user performing the movement from a sensor; a calculation unit that calculates an activity amount for a muscle synergy stored in the storage unit from the muscle activity amount acquired by the acquisition unit without extracting a muscle synergy from the muscle activity amount; A processing execution unit that executes a process of causing a controlled device to reproduce the movement of the user substantially in real time based on the amount of activity calculated by the calculation unit; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the muscle synergy stored in the storage unit is a muscle synergy based on a muscle activity amount of the user performing the motion that is measured in advance.

3. storing muscle synergies related to a predetermined movement in a predetermined storage unit in advance; Measuring a muscle activity of a user performing the movement; calculating an activity amount for a muscle synergy stored in the storage unit from the muscle activity amount measured in the measuring step without extracting a muscle synergy from the muscle activity amount; Executing a process of causing a controlled device to reproduce the movement of the user substantially in real time based on the calculated activity amount; An information processing method implemented by a computer.

4. A function of retaining muscle synergies related to a predetermined movement in a predetermined retaining part; A function of measuring the amount of muscle activity of a user performing the above-mentioned movement; a function of calculating an activity amount for a muscle synergy stored in the storage unit from the muscle activity amount measured by the measuring function without extracting a muscle synergy from the muscle activity amount; A function of executing a process of causing a controlled device to reproduce the movement of the user substantially in real time based on the amount of activity calculated by the calculating function; A computer program that enables a computer to realize the above.

Images