Methods for visualizing multi-channel signals

Abstract

Method for visualizing multiple sensor signals derived via electrodes from a limb or amputation stump to detect muscle activity, comprising the following steps: • Each signal is assigned a display direction and a display amount • The display direction represents exactly one signal • The displayed value represents the intensity of motor activity of either the respective muscle or its nerve. • The direction and magnitude of all signals are simultaneously displayed in real time on a display device as a single graphical object. • On the display device, reference objects that were saved during previous analysis sessions are displayed simultaneously with the object. • Reference contractions are used as reference objects. • Simultaneously with the visualization, a prosthetic or orthotic device is controlled with the signals, whereby the reference objects are used to parameterize the prosthetic or orthotic control.

Description

[0001] The invention relates to a method for visualizing multiple sensor signals, in particular myoelectric signals, which are derived via electrodes from a limb or amputation stump.

[0002] Powered prostheses, meaning prostheses with a motor drive to move components relative to each other, require control signals for the drives to function as desired. One way to achieve this is by recording myoelectric signals via electrodes and, after any necessary amplification, using them as impulses to activate or deactivate the drives via a control unit. To generate meaningful myoelectric signals, it is necessary to contract muscles or residual muscles after amputations. The corresponding signals can then be generated from the contraction pattern, which are used to move the drives. Such myoelectric prosthesis control can be used for people with amputations or dysmelia. In principle, a similar myoelectric control of powered orthoses is also possible.

[0003] The more complex the movements of prostheses or orthoses are intended to be, the greater the number of signals required. While simply opening or closing a gripper requires only two signals, the number of signals needed increases with a corresponding increase in the achievable functions. This increase in the number of signals can be achieved by increasing the number of contraction patterns and / or the number of electrodes. Regardless, it is difficult for the patient to consciously contract muscles or residual muscles without visual feedback. Able-bodied individuals can see how their limbs react to their will to move them in a specific way. When a muscle is contracted, for example, the biceps, the person sees and feels a movement of the forearm.Coordination skills can be improved, in particular, through visible feedback of mental contraction commands in the form of movements. In cases of missing limbs, this visual feedback is absent. Such patients do not see the effect of their intention to move, even though it is present and actually results in muscle contraction or contraction of the remaining muscles. The perception of a differentiated, reproducible effect on the intention to move and muscle contraction is extremely important in rehabilitation measures and in the prosthetic care of patients.

[0004] WO 2007 / 141680 A1 describes a biofeedback procedure in which various electromyographic signals are recorded via sensors. These electromyographic signals are processed and evaluated. Based on this evaluation, light-emitting diodes (LEDs) arranged in an LED array are controlled.

[0005] DE 195 21 464 A1 relates to a method for controlling a knee brake of a transfemoral prosthesis, in which the computer-controlled braking torque can be continuously varied depending on the walking movement of the prosthesis wearer. The control is based on EMG values, pressure values ​​measured in the foot area, the knee angle, and the knee angular velocity.

[0006] EP 1 240 864 A1 relates to a device and a method for the selective training of muscles with at least two signal transducers, from which electromyographic signals are recorded. Depending on the state of tension of the respective muscle, a scalable value assigned to the respective electrode is decreased or increased relative to another remaining scalable value of the other muscle. The electromyographic signals are displayed graphically. Geometric shape parameters, axis dimensions, or the number of vertices can vary according to the state of tension of the respective muscle.

[0007] The object of the invention is to provide a simple and meaningful interpretation of an intention to move, particularly for a missing limb. According to the invention, this object is achieved by a method with the features of the main claim. Advantageous embodiments and further developments of the method are listed in the dependent claims.

[0008] The method for visualizing multiple sensor signals derived from the body to detect muscle activity, such as myoelectric signals recorded via electrodes from a limb or amputation stump, involves assigning a direction and magnitude to each sensor signal. The direction represents a single sensor signal, such as an electrode signal, while the magnitude represents the intensity of that signal—that is, the intensity of the motor activity of the respective muscle or its nerve, for example, the contraction intensity of the muscle associated with the sensor or electrode. The direction and magnitude of all sensor signals are simultaneously displayed in real time on a display device as a single graphical object, so that the intention to move is reflected in recognizable, reproducible forms.The patient receives immediate visual feedback, as all signals are displayed in real time on a display device, particularly a screen. Different intentions to move, such as "clenching the fist" or "flexing the wrist," generate different objects, which can vary in both shape and size. The display of the graphic object, which is shown in real time on a display device depending on the type and intensity of the motor activity or muscle contraction, provides immediate visual feedback of the intention to move. The patient sees what they want to do or would do if the limb were unaffected.

[0009] Preferably, the graphic object is represented as a two-dimensional graph, particularly as a polygon, since such a two-dimensional representation is very easy for a patient to understand. Such simple geometric shapes, i.e., polygons, are significantly clearer than superimposed curves that indicate electrode intensity over time. Furthermore, representing the data as a graphic object ensures high recognizability while simultaneously allowing for the display of a large number of electrode signals. Representing the patient's volition facilitates the reproducibility of the contraction pattern.

[0010] The invention provides that reference objects in the form of reference contractions are displayed on the display device simultaneously with the graphic object. This makes it possible to practice specific target contractions and allows the patient to modify the current contraction until it matches or closely approximates the reference contraction. The reference objects can either be predefined or saved during previous analysis sessions and used to parameterize a prosthesis or orthosis control system. Found or defined reference contractions or reference patterns are displayed simultaneously with the current contraction pattern, enabling both the patient and an orthotist or physician to assess reproducibility and deviations.Simultaneously with the visualization, the prosthesis or orthosis is controlled by the sensor signals, with the reference objects being used to parameterize the prosthesis or orthosis control. This allows the patient to specifically train and further refine their contractions. Additionally, the visualization of the contraction patterns can be performed in parallel with the activation of a prosthesis or orthosis, enabling the simultaneous identification of which object triggers which movement of the prosthesis or orthosis. Thus, any deviation between the actual contraction, the reference contraction, and the action of the prosthesis or orthosis can be visualized and fed back to improve the control quality.

[0011] To increase distinguishability, the reference object is displayed in a different color and / or intensity than the graphic object, making it easier to correlate the current contraction with the desired or reference contraction. It is also possible to highlight the reference object and / or the graphic object in terms of color or intensity if the detected contraction pattern corresponds to or closely approximates the reference pattern. This demonstrates to the patient that the intended movement is sufficiently clear in relation to the contraction.

[0012] The displayed value can be normalized to clearly and immediately show the influence of the contraction intensity on the object shape.

[0013] In addition to deriving sensor signals via surface electrodes, signals from myosensors, neurosensors, pressure sensors and / or strain sensors can also be used to record motor activity.

[0014] The procedure is explained in more detail below with reference to the attached figures. They show: Fig. 1 - a first representation of a representation with reference contraction and derived contraction with four electrodes; Fig. 2 - an anatomical interpretation of the electrode arrangement according to Fig. 1; as well as Fig. 3 - generated reference contractions for eight electrodes.

[0015] In the Fig. Figure 1 is a graphic object 1 that reflects a current contraction of muscle groups. Four electrodes for recording myoelectric signals were arranged on the limb, in this case a forearm. Each electrode represents a vertex 2, 3, 4, 5, around which the graphic object 1 of the current contraction is displayed. In this embodiment, the graphic object 1 is designed as a quadrilateral, with each vertex 2, 3, 4, 5 assigned a display direction. In this embodiment, the display directions correspond to intersecting axes, such that the first electrode is assigned to the axis at 0° originating from the center point, the second electrode 3 to the axis at 90°, the third electrode 4 to the axis at 180°, and the fourth electrode 5 to 270°. The respective vertices 2, 3, 4, 5 are connected to each other by lines, so that the graphic object 1 is a quadrilateral.The distance of vertices 2, 3, 4, and 5 from the center point indicates the intensity of the contraction. If the electrode associated with the first vertex 2 does not receive a myoelectric signal, meaning the muscle associated with that electrode does not contract, and the first vertex 2 lies on or very close to the center point, an intense contraction occurs. If the first vertex 2 moves further outward, the distance indicates an intense contraction. The recorded electrode signals are displayed in real time as graphic object 1, so that the patient receives immediate visual feedback on the current muscle contraction on the display device.

[0016] Also shown is a reference object 10, which is constantly displayed on the display device. This reference object 10 is displayed in a different color and / or intensity than the current contraction of the graphic object 1. In the illustrated embodiment, the current contraction is quite close to the reference object; only the electrode at the first corner point 2 shows an excessive deflection, indicating an excessive contraction of the corresponding muscle.

[0017] In the Fig. Figure 2 shows an anatomical interpretation of the diagram described above. Each vertex 2, 3, 4, 5 is assigned an electrode 20, 30, 40, 50, which is attached to a forearm. The arrangement of the electrodes can be seen in the schematic cross-sectional view through the forearm with the corresponding muscles. This anatomical interpretation of the signals makes it possible to change the arrangement of the electrodes depending on the signals actually measured or to train the patient to practice different movement and muscle contraction patterns. One objective of the invention is to provide patient-adapted control of a prosthesis or orthosis.For this to work, it is necessary that the patient can reproducibly record different contraction patterns, so that, for example, after the mental command "clench fist", a repeatable contraction pattern is generated that leads to a corresponding prosthesis control, so that a prosthetic hand is closed.

[0018] Each electrode 20, 30, 40, 50, with its position and signal intensity, defines a vertex of object 1. All vertices together define the graphic object 1. Object 1 retains its shape for a specific movement or contraction pattern even if the magnitudes are changed uniformly. It scales only in size. A change in shape occurs when the magnitudes are not changed uniformly. The electrode signals are no longer displayed over time, as is known from the prior art, thus reducing complexity for the viewer. Prior art involves displaying the electrode signals using superimposed time graphs. Each graph typically has its own color for easier differentiation. Two or three signals can still be recognized as contraction patterns, but this becomes significantly more difficult with larger numbers of signals and electrodes.Furthermore, a reference contraction cannot be represented using time-dependent curves. Similarly, a graphical separation between contraction strength and contraction type is difficult to achieve. Currently, complex, threshold-based rules are established to classify a contraction pattern, the manual evaluation of which in a time graph is extremely laborious. An assessment of the contractions is only possible via a prosthesis malfunction. In contrast, the presented invention no longer displays time, thereby reducing the complexity of perception for the observer.

[0019] As in the example of the Fig. As shown in Figure 1, this object preferably consists of several vertices, one for each electrode. This polygon can be intuitively interpreted at a glance as a contraction pattern without any technical understanding, training, or other prior knowledge, thus providing visual feedback to the patient's intention to move. The invention restores to the patient the visible effect of their intention to move, enabling them to adjust and selectively differentiate their action, namely the contraction of the muscles. Furthermore, the representation allows for the evaluation and training of the reproducibility of the contraction pattern, enabling reliable classification with a low error rate in prosthesis control and thus an unprecedented level of acceptance of the prosthesis or orthosis.

[0020] If the spatial positioning of the electrodes is in the same order as the signal representation, as described in the Fig. 1 and Fig. 2. If this is the case, conclusions about the anatomy are also possible. For example, the extent of crosstalk between two adjacent signals can be recorded, or contraction patterns can be traced back to their generating muscles using cross-sectional images.

[0021] Furthermore, patients with malformations, who, unlike amputee patients, naturally lack phantom perception, can learn differentiated contractions and thus a will to move thanks to this invention. Through visual feedback, patients with dysmelia can approximate a desired contraction pattern and thus actual movement, ultimately learning a purposeful, differentiated contraction.

[0022] In the Fig.Figure 3 shows five reference contractions generated using eight electrodes. The person generating these contractions can use them to train their contraction patterns.

[0023] Representing a contraction pattern as a single object allows a patient to immediately assess all signals without technical expertise. From this single graphical representation, the patient's intended movement can be directly recognized, either by the patient themselves, by an orthotist, or by a specialist adjusting a prosthesis or orthosis control system. The represented object is preferably a polygon, but can also be a three-dimensional representation whose shape corresponds to the contraction pattern (i.e., the type of desired movement) and whose size corresponds to the contraction strength (i.e., the intensity of muscle tension).

Claims

[1] Method for visualizing multiple sensor signals derived via electrodes from a limb or amputation stump to detect muscle activity comprising the following steps: • Each signal is assigned a display direction and a display amount • The display direction represents exactly one signal • The displayed value represents the intensity of motor activity of either the respective muscle or its nerve. • The direction and magnitude of all signals are simultaneously displayed in real time on a display device as a single graphical object. • On the display device, reference objects that were saved during previous analysis sessions are displayed simultaneously with the object. • Reference contractions are used as reference objects. • Simultaneously with the visualization, a prosthetic or orthotic device is controlled with the signals, whereby the reference objects are used to parameterize the prosthetic or orthotic control. [2] Method according to claim 1, characterized by that the graphic object is represented as a two-dimensional graph. [3] Method according to claim 1 or 2, characterized by , that the reference object is displayed in a different color, pattern and / or intensity than the object. [4] Method according to any of the preceding claims, characterized by , that the displayed amount is normalized. [5] Method according to any of the preceding claims, characterized by , that myoelectric signals, signals from myosensors, neurosensors, pressure sensors and / or strain sensors are used as sensor signals. [6] Method according to claim 2, characterized bythat the graphic object is represented as a polygon.

Images