Method and device for determining a desired movement of a limb

Abstract

Method for determining a desired movement of a missing limb (12), in which a section of a body surface where at least some of the muscles that move the limb are present is observed, the muscle movements in the section are optically captured (20), where a link between a desired movement of the missing limb and the muscle movement captured in the section is determined as follows: Specifying a movement of the limb (26) and performing the specified movement by the person, whereby, due to the absence of the limb, only the corresponding muscles are activated, Capturing the muscle movements generated as a result of the specified movement (28), Repeating (30) the preceding procedural steps (26) and (28), Determining a link between a given movement and the recorded muscle movements (32), wherein the evaluation (22) of the recorded muscle movements is based on this determined link.

Description

[0001] The invention relates to a method and a device for determining a desired movement of a limb.

[0002] The limbs of the human body are moved by muscles. The muscles of each limb are always located proximally, that is, towards the center of the body, while the limb being moved by the muscle is always located distally from the moving muscles, that is, further away from the body. By recording muscle movement, it is possible to draw conclusions about the intended movement of the corresponding limb. This is also possible if the intended movement is initiated by the muscles, but no actual movement of the limb occurs, for example, due to an amputated limb, limb inhibition, or limb impairment.

[0003] For example, by recording the movement of the muscles in the forearm, conclusions can be drawn about the intended movement of the fingers. This is desirable, for instance, for controlling modern prostheses, i.e., artificial hands.

[0004] Basically, three methods can be distinguished for detecting muscle movement. For example, force-pressure sensors can be used. Here, the movement of the muscle is measured based on the force it exerts on a sensor attached directly to the skin surface. Examples of this are so-called force-sensing resistors or an array of numerous force-measuring elements (so-called tactels). Another method is surface electromyography (EMG). Here, the measurement of the electrical signals of the muscles allows conclusions to be drawn about the intended movement of the limbs. A third method uses ultrasound as an imaging technique, which visualizes the movement of the muscles and allows conclusions to be drawn about the intended movement of the limb.

[0005] A disadvantage of these methods, however, is that they all require the sensors to be attached directly to the skin. This leads to drawbacks, as the sensors must be positioned precisely, and any shifting of the sensors, for example, can result in signal inaccuracies. Furthermore, the contact can damage the skin at the contact points. Therefore, trained personnel are required to attach the sensors, and there is a high potential for error in correct positioning or sensor slippage. Additionally, the duration of the sensor's detection is limited by its direct skin contact.

[0006] Furthermore, signal quality deteriorates over extended periods (e.g., with EMG signals), and there are problems with ultrasound coupling at the skin surface. Additionally, the sensors used are very complex and therefore expensive.

[0007] EP 2 394 574 A1 describes a system for training a speech, swallowing or breathing process, whereby three-dimensional optical data is recorded and the speech, swallowing or breathing process is then analyzed.

[0008] The object of the invention is to create a method and a device for detecting a desired movement of a limb, which operates without contact.

[0009] The problem is solved by the method according to claim 1 and the device according to claim 9.

[0010] In the inventive method for determining a desired movement of a limb, a section of a body surface is observed. For the desired movement to occur, it is not necessary that the limb actually moves. For example, the actual movement of the limb can be prevented or weakened by its absence, inhibition, or obstruction. The limb in question is, in particular, a limb of the human body. The section of the body surface is selected such that at least a portion of the musculature moving the limb is present at the location of the section.

[0011] In the inventive method, no physical contact with the body surface occurs during observation. According to the invention, detectable muscle movements are optically recorded in the selected area. These recorded muscle movements are then evaluated to determine the intended movement of the limb in question. No direct observation of the limb takes place. Instead, a prediction is made, based on the observed muscle movements, as to which movement the person intends to perform.

[0012] The optical detection of muscle movement makes the method contactless. A connection between a sensor and the observed area of ​​the body surface is not required. Specifically, a camera is used for optical detection. In particular, no special requirements need to be placed on the camera, so a camera with a resolution of 1280 x 720 pixels can be considered sufficient. Images are preferably captured at a frequency of 15 frames per second or more. This results in a simple setup with inexpensive components. Problems such as fatigue with EMG sensors and the difficulty of permanently coupling ultrasound sensors to the skin surface are also eliminated in the method according to the invention.Furthermore, any health risks can be ruled out, as there is no direct attachment of sensors and no introduction of energy into the body.

[0013] In particular, this method allows for the simultaneous recording of a large number of muscle movements. Since an entire section of the body surface is observed, there is no point-by-point detection of a single muscle movement, as is the case with conventional and well-known methods. The simultaneous recording of numerous muscle movements allows, on the one hand, a more precise assessment of the intended movement, and on the other hand, it enables the identification of complex movements that require more than one muscle.

[0014] Preferably, the method simultaneously detects a multitude of intentional movements of several limbs. In particular, if the forearm is being observed, the multitude of intentional movements can be the intentional movements of the fingers, which are detected by the method. Thus, in the example of the observed forearm, it is not necessary to limit oneself to the movement of just one finger. Rather, the movements of all fingers can be observed simultaneously.

[0015] Preferably, markers are applied to the observed area of ​​the body surface to record muscle movements. These markers simply simplify the recording and analysis of muscle movements. The markers can be glued onto the area of ​​the body surface, painted on, or, in particular, directly applied to the body surface through tattooing.

[0016] Preferably, emitters and / or reflectors are attached to the observed section of the body surface to detect muscle movements. This simplifies the detection of muscle movements and the determination of the desired movement. In particular, the emitters are IR emitters, so that they can be detected by a conventional camera.

[0017] Preferably, a low-pass filter, a high-pass filter, or a band-pass filter is used when evaluating the recorded muscle movement. This removes, for example, DC components and / or high-frequency components from the signal. In particular, a second-order Butterworth band-pass filter is used for this purpose.

[0018] In particular, the evaluation involves image acquisition, so that the movement of the body surface or the markers attached to it is preferably recorded in all three spatial directions as well as in the three rotational directions – i.e., 6D. However, it is also possible to record only two translational directions – i.e., 2D – or various combinations of translation and rotation. The intended movement of the respective limbs can be determined from the recorded movement of the muscles or the markers attached to the body surface.

[0019] Preferably, a link between muscle movement and intended movement is created or derived by observing a large number of muscle movements generated by a predetermined movement. This allows a muscle movement to be assigned to a predetermined and therefore known intended movement. Thus, an evaluation device creates the link between the observed muscle movement and the known intended movement. This correlation between the recorded muscle movement and the known intended movement is then used in subsequent evaluations to determine unknown intended movements. This process involves a one-time training of the evaluation device to subsequently determine unknown intended movements of the relevant limbs.

[0020] Preferably, the link between muscle movement and intentional movement is established using ridge regression (AE Hoerl and RW Kennard, "Ridge regression: Biased estimation for nonorthogonal problems", Technometrics, Vol. 12, pp. 55-67, 1970), which fits a linear model to the recorded data. This linkage allows, in particular, the derivation of previously unknown intentional movements from observed muscle movement.

[0021] The present invention further relates to a device for detecting a desired movement of a limb using a camera directed at a section of the body surface. The camera detects muscle movement within this section. According to the invention, an evaluation device is connected to the camera. This evaluation device analyzes the optically detected muscle movement. The analysis includes determining the desired movement of the limb. An output device is connected to the evaluation device for displaying the analysis result, and in particular, for displaying the desired movement.

[0022] Preferably, the device is further developed according to the method described above.

[0023] Preferably, the method has features of the device.

[0024] The present invention further relates to the use of the device described above for controlling a prosthesis of the corresponding limb. Here, the output device of the device according to the invention is connected to the prosthesis, which is, for example, a hand prosthesis. By detecting the desired movement, this desired movement can be translated into an actual movement of the prosthesis.

[0025] The invention further relates to the use of the device described above for controlling a machine, in particular a robot. Thus, the desired movements, especially of the hand, can be determined and used within the framework of human-machine interaction to appropriately control the machine and preferably the robot. For example, the robot is an assistance robot that, for instance, grasps and hands a glass to an amputee. However, the device can also be used by able-bodied individuals as an input device for controlling machines.

[0026] The invention further relates to the use of the aforementioned device for controlling and displaying a limb in a virtual reality. For example, hands can be displayed in virtual reality and controlled and moved by capturing muscle movements. Actual movement of the hand, for example, is not required.

[0027] The invention is described below with reference to a preferred embodiment and the attached figures.

[0028] They show: Fig. 1 a device according to the invention for determining a desired movement using the example of the desired movement of a hand, Fig. 2 a schematic sequence of the method according to the invention and Fig. 3. A schematic procedure for determining a link between intended movement and recorded muscle movement.

[0029] According to the invention, the device for detecting a desired movement of a limb includes a camera 10. In the Fig. In the example shown, the limb to be moved is a human hand 12. Camera 10 observes a section of the body surface where at least some of the muscles moving the limb are present. In the hand 12 example, camera 10 observes the forearm 14 to optically record muscle movements. No sensor or similar device touches the forearm.

[0030] Camera 10 is connected to an evaluation device 16. The evaluation device 16 analyzes the images captured by camera 10 and uses this data to determine the muscle movement in the forearm 14. From this determined muscle movement of the muscles in the forearm 14, the evaluation device 16 calculates the desired movement of the hand 12 and / or fingers. This is possible because there is a clear connection between the muscles and the hand or individual fingers, allowing the movement of a specific muscle to deterministically predict which fingers or part of the hand should be moved. This applies to all muscles that always exhibit precisely one direction of movement. Naturally, the interaction of individual muscles to generate a complex movement of a limb is also included.

[0031] An output device 18 is connected to the evaluation device 16. The output device 18 outputs the desired movement, in this example, of the hand 12. The output device 18 can, in turn, be connected, for example, to a prosthesis, which then executes the desired movement. Alternatively, in another application, the evaluation device 18 can be connected to another machine, which then executes a corresponding movement.

[0032] Even if in Fig. 1. An example of the device is shown using the hand; this can of course be applied to any other limb.

[0033] In the method according to the invention, schematically represented in the Fig.2. First, a muscle movement is optically detected. 20. The muscle movement is detected on the body surface within an observed area. This area is located where at least some of the muscles that move the limbs are present. The detected muscle movement is then evaluated. 22. This evaluation includes determining the desired movement of the limb in question. In a further step, 24. The determined movement can be output, for example, to a prosthesis that replaces the limb to be moved or to a machine control system.

[0034] It is possible to train the evaluation device 16 appropriately for evaluation 22. This involves first specifying limb movements to the person 26. The person then performs the specified movements. Whether this involves an actual movement of the limb or, for example, if the limb is missing, merely the activation of the corresponding muscles is irrelevant. The muscle movements generated as a result of the specified movement are then recorded 28. The aforementioned steps are repeated a sufficient number of times 30. Once a sufficiently large number of repetitions have been recorded, a correlation between the specified movement and the recorded muscle movement is determined 32. This could, for example, involve a regression analysis of the movement. The subsequent evaluation 22 is then performed based on this newly determined correlation.This method allows for a relatively simple, personalized evaluation by individually linking limb movement and muscle movement.

Claims

[1] Method for determining a desired movement of a missing limb (12), in which a section of a body surface where at least some of the muscles that move the limb are present is observed, the muscle movements in the section are optically captured (20), where a link between a desired movement of the missing limb and the muscle movement captured in the section is determined as follows: Specifying a movement of the limb (26) and performing the specified movement by the person, whereby, due to the absence of the limb, only the corresponding muscles are activated, Capturing the muscle movements generated as a result of the specified movement (28), Repeating (30) the preceding procedural steps (26) and (28), Determining a link between a given movement and the recorded muscle movements (32), wherein the evaluation (22) of the recorded muscle movements is based on this determined link. [2] Method according to claim 1, wherein a plurality of muscle movements are simultaneously detected. [3] Method according to claim 1 or 2, wherein a plurality of desired movements of several limbs (12) are determined simultaneously. [4] Method according to any one of claims 1 to 3, wherein the forearm (14) is observed and intentional movements of the hand (12) and / or fingers are determined. [5] Method according to any one of claims 1 to 4, wherein markings are applied to the observed section of the body surface to detect muscle movements. [6] Method according to any one of claims 1 to 5, wherein a low-pass filter, a high-pass filter or a band-pass filter is used in the evaluation of the recorded muscle movements. [7] Method according to any one of claims 1 to 6, wherein emitters, in particular IR emitters and / or reflectors, are placed on the observed section of the body surface to detect muscle movements. [8] Method according to any one of claims 1 to 7, wherein an evaluation device is used to create a link between muscle movement and a known intended movement by observing the muscle movement during a plurality of predetermined movements (26) (28) and subsequently, in the evaluation (22) of the recorded muscle movement, this link is used to determine an unknown intended movement. [9] Device for detecting a desired movement of a missing limb, with a camera (10) directed at a section of a body surface to detect muscle movement and an evaluation device (16) connected to the camera (10) for evaluating the optically detected muscle movement, which includes a determination of the desired movement, and an output device (18) connected to the evaluation device (16) for outputting an evaluation result, wherein the device is designed to determine a link between a desired movement and a detected muscle movement by: Specifying a movement of the limb (26) and performing the specified movement by the person, whereby, due to the absence of the limb, only the corresponding muscles are activated, Capturing the muscle movements generated as a result of the specified movement (28), Repeating (30) the preceding procedural steps (26) and (28), Determining a link between a given movement and the recorded muscle movements (32), wherein the evaluation (22) of the recorded muscle movements is based on this determined link. [10] Device according to claim 9 for carrying out the method according to any one of claims 1 to 8. [11] Use of a device according to claim 9 or 10 for controlling a prosthesis of the corresponding limb. [12] Use of a device according to claim 9 or 10 for controlling a machine and in particular a robot. [13] Use of a device according to claim 9 or 10 for controlling a representation of a limb in a virtual reality.

Images