Method and system for movement-training a movement

Abstract

The invention relates to a method for movement-training a movement of at least one body part of a patient, said body part being provided with an orthopaedic device, said method comprising the following steps: - generating, by means of an electronic computing unit, a digital graphical movement representation of the body part provided with the orthopaedic device depending on a target movement which the body part is supposed to carry out with the provided orthopaedic device, - detecting, by means of a detecting apparatus, a body part movement of the body part of the patient provided with the orthopaedic device, and - superimposing, by an augmented reality device, the generated digital graphical movement representation as an augmented reality onto the reality of the patient perceived in the patient's direction of view in such a manner that the superimposition of the graphical movement representation is positioned depending on the detected body part movement.

Description

[0001] Method and system for movement training of a movement

[0002] The invention relates to a method for movement training of a movement of at least one body part of a patient equipped with an orthopaedic device.

[0003] The invention also relates to a system for this purpose.

[0004] For the purposes of the present invention, orthopedic devices (also referred to as orthopedic devices) include, in particular, orthoses, prostheses, exoskeletons, and possibly also wheelchairs. Orthoses are products that support, assist, protect, and / or restrict the freedom of movement of a patient's body part, such as a joint, in order to prevent excessive strain. Prostheses, on the other hand, do not replace, or no longer replace, the patient's body parts. Exoskeletons are, in particular, mechanical support structures designed to support, assist, and / or protect the patient's main musculoskeletal system.

[0005] In the following, a patient is defined as any user of the orthopedic device. This is therefore the provider of the orthopedic device.

[0006] Every orthotic device is attached to a part of the patient's body. It does not necessarily have to come into contact with the patient's skin. Orthoses and exoskeletons, for example, are often worn over clothing, so that the clothing, such as trousers, is between the orthotic device and the patient's skin. However, a knee orthosis, for example, is attached to the patient's knee or leg. A prosthesis always has an interface element connected to an amputation stump or another body part, which is attached to the respective body part. A leg prosthesis, for example, uses a prosthetic socket that represents the interface between the prosthesis and the amputation stump. In this case, the amputation stump would be the patient's body part.A liner is typically used between the skin surface of the patient's amputation stump and the prosthetic socket to reduce shear forces acting on the skin.

[0007] A prosthetic socket for an amputation stump is typically made of a rigid (hardly deformable) material, such as fiber-reinforced plastic, and forms an important part of the interface between the amputation stump and the prosthesis attached to the socket. Such prosthetic sockets have long been used, particularly for leg prostheses that are to be attached to an amputation stump, such as a femoral stump.

[0008] Prosthetic sockets for leg amputees, in particular, are subjected to particular stresses in daily use. When walking, the patient's entire weight rests on the prosthetic socket, and thus particularly on the amputation stump, which is housed within the prosthetic socket. It is therefore of utmost importance to adapt the prosthetic socket as optimally as possible to the patient's individual circumstances and needs, especially to the shape and geometry of the affected body part.

[0009] During both the assembly and installation of an orthopedic device, the patient is assisted by a technically trained user, such as an orthotist. With their knowledge and experience, the orthotist can make the necessary adjustments to ensure the orthopedic device fits optimally and results in a gentle movement pattern.

[0010] EP 2 153 370 B1 discloses a system for aligning prostheses, in which both movement data of the person fitted with the prosthesis and prosthesis alignment errors are determined from a movement database. These are compared to determine whether the prosthesis corresponds to a target value or requires further adjustment.

[0011] From DE 10 2012 009 507 A1 a method and a device for determining malpositions in the construction of prostheses of the lower extremities is known, wherein inertial measurement data are determined over at least one gait cycle and compared with target values.

[0012] DE 10 2018 128 514 B4 discloses a method for performing a static prosthesis assembly, wherein several components are arranged next to one another. The actual position and actual orientation of the arranged components relative to one another are determined based on recorded positions and orientations of markings and compared with corresponding target values.

[0013] CN 10958157 B discloses a method for a treadmill in which footprints are superimposed onto the user's reality.

[0014] US 2021354023 discloses a device and a method in which targets for the position of joints are displayed to the user during movement by means of augmented reality.

[0015] Against this background, it is the object of the present invention to provide an improved method for movement training for patients who are equipped with orthopaedic equipment.

[0016] The object is achieved according to the invention with the method according to claim 1. Advantageous embodiments of the invention can then be found in the corresponding subclaims.

[0017] According to claim 1, a method for movement training of a movement of at least one body part of a patient which is equipped with an orthopaedic device is proposed, wherein the method comprises the following steps: - generating a digital graphic movement representation of the body part equipped with the orthopaedic device as a function of a target movement which the body part is to perform with the equipped orthopaedic device, by means of an electronic computing unit,

[0018] - detecting a body part movement of the patient's body part equipped with the orthopaedic device by means of a detection device, and

[0019] - Inserting the generated digital graphic movement representation as a reality extension into the reality perceived by the patient in his line of sight by means of a reality extension device in such a way that the position of the insertion of the graphic movement representation depends on the recorded body part movement.

[0020] According to claim 1, a method is proposed for movement training of a movement to be performed by a patient equipped with an orthopedic device. Such an orthopedic device can be, for example, a prosthesis, an orthosis, or even a wheelchair. The movement training is carried out in particular with the body part to which the orthopedic device is attached. This can also include the entire upper or lower extremity and, if necessary, the entire body. Such movement training of a movement serves to help the patient become accustomed to the orthopedic device and perform optimal movements, thus realizing the best possible care and increasing comfort.Such movement training also serves to ensure that the patient performs movements with the orthopedic device that provide optimal support and do not have a negative impact on other parts or elements of the body that are not equipped with the orthopedic device.

[0021] For the movement training according to the invention, a digital graphic movement representation is first generated depending on a target movement. Such a target movement is stored, for example, in a data storage device. By specifying the body part receiving the orthopedic treatment and other boundary conditions, a digital graphic movement representation of the orthopedic treatment, optionally together with the equipped body part, can then be generated, which in particular represents an optimal movement sequence for such an orthopedic treatment. Characteristics of the patient and, if applicable, also characteristics of the injured body part to which the orthopedic treatment is applied, should also be taken into account when creating the digital graphic movement representation.This may, for example, be data on the patient's height, weight and stature, as well as, if applicable, a medical indication and the like.

[0022] The digital graphical movement representation shows the target movement in the form of a movement sequence, with the orthopedic device and, if applicable, the treated body part being part of the graphical movement representation. The orthopedic device and, if applicable, the treated body part are preferably generated in the form of an outline representation or a semi-transparent body representation. The device and body part can be represented in a highly abstract form with a low level of detail. However, a very detailed representation of the device and the body part is also conceivable, provided the corresponding information has been provided to the computing unit in advance.

[0023] A body part movement of the patient's body part equipped with the orthopedic device is now recorded. This can be done, for example, using a camera or other optical sensors. The recording device thus records the movement of the orthopedic device performed by the patient, making it possible to determine how the orthopedic device and the body part it is fitted with move in space and where they are located (absolutely in space or relative to a reference, such as the recording device or the body).

[0024] With the help of a reality augmentation device, the generated movement representation is then overlaid as a reality augmentation into the reality perceived by the patient's line of sight. The position of the overlay of the graphic movement representation depends on the recorded body part movement. This means, in particular, that the digital movement representation is overlaid synchronously with the recorded body part movement. This enables the patient to recognize a deviation between their own body part movement and the overlaid movement representation of the target movement, which can be compensated or balanced by adjusting their own body part movement. This encourages the patient to approach the target movement and thus achieve an optimal movement sequence with the orthopedic device.

[0025] It is particularly advantageous if the digital motion representation is not only displayed synchronously with the recorded body part movement, but also accurately overlaid into the patient's perceived reality, so that the image of the orthopedic device and, if applicable, the body part contained in the digital motion representation are perceived at exactly the spatial positions in the patient's perceived reality where the orthopedic device and the body part should be located according to the target movement. For this purpose, the reality augmentation device typically has sensors that detect the patient's gaze direction, for example, using inertial sensors and / or gaze sensors.

[0026] With the help of the present invention, it is thus possible to overlay an optimal movement sequence of an orthopaedic device into the perceived reality of a patient in such a way that the patient trains this optimal movement sequence by trying to approximate this displayed movement sequence through his own movement.

[0027] According to one embodiment, the body part is a body part of the lower extremity.

[0028] It is conceivable that the orthopedic device is an orthosis to stabilize the lower extremities, whereby gait training is carried out as movement training using the present procedure. However, it is also conceivable that the orthopedic device is a prosthesis that fits the stump of an amputated body part.

[0029] It is also conceivable, however, that the orthopedic device involves a wheelchair, which also indirectly supports the lower extremities for movement. The body parts to be recorded for a movement can then also relate to the upper extremities.

[0030] According to one embodiment, the graphical movement representation includes at least part of a gait cycle.

[0031] Such a gait cycle involves a movement pattern that must be performed on two legs while wearing the orthopedic device. Such a gait cycle can also be an OMT (outcome measurement test), for example.

[0032] According to one embodiment, it is provided that a configuration of the orthopedic supply is obtained and provided to the electronic computing unit, wherein the target movement is generated or selected by the electronic computing unit depending on the provided configuration.

[0033] Individual movement sequences can be trained with the orthopedic device. Different movement modes that are technically supported by the orthopedic device can also be trained. The graphical movement representation is then generated depending on the target movement, also taking into account the specifics of the set movement mode of the orthopedic device. In other words, during movement training, the orthopedic device is placed in an initial movement mode, and this movement mode is communicated to the electronic processing unit, so that the graphical movement representation is created taking the set movement mode into account.

[0034] However, other configurations of the orthopedic device, such as damping values ​​or maximum joint angles, can also be taken into account. According to one embodiment, the target movement is generated or selected by the electronic processing unit depending on a movement type set on the orthopedic device.

[0035] Such a movement type could be walking or running, for example. Another movement type could be climbing stairs. The target movement is the correct movement sequence and / or the placement of the foot on the step. Another movement type could be walking on a ramp, e.g. walking down a ramp. According to one embodiment, for example, the movement of the knee angle through the two body segments of the upper and lower leg can be displayed. Other movement sequences could be sitting down and / or standing up with the help of orthopedic devices. Movement sequences of the upper extremities are also conceivable. For example, the orientation of the arm for different grips can be displayed as a predetermined movement sequence in order to achieve the best possible grip security or to minimize compensatory movement with the shoulder and / or upper body.

[0036] According to one embodiment, it is provided that the graphical movement representation is displayed synchronously with the detected body part movement as a function of the latter.

[0037] The synchronous display of the graphic movement representation includes not only the temporal synchronicity, i.e. the movement sequence of the graphic movement representation is displayed synchronously with the movement sequence of the patient's movement, as already described above, but also the spatial synchronicity, i.e. the movement sequence of the graphic movement representation is displayed in the reality perceived by the patient in such a way that the graphic movement representation overlays the actual movement of the patient, whereby deviations in the actual movement sequence from the target movement become visible.

[0038] According to one embodiment, it is provided that status and / or movement-related data of the orthopaedic supply are recorded during the display of the graphical movement representation and are displayed in the perceived reality by the reality augmentation device.

[0039] This allows the patient to view and perceive the status of their orthopedic device and / or the movement-related data of their orthopedic device during movement training. Conclusions can then be drawn about adjusting the configuration of the orthopedic device, possibly allowing the patient to more closely approximate the target movement through their own movement. Such status and / or movement-related data of the orthopedic device can include, for example, joint angles or forces acting on the orthopedic device.

[0040] According to one embodiment, it is provided that the detected movement sequence is provided to an image recognition device and that movement information of the detected movement sequence is detected by the image recognition device and blended into the perceived reality by the reality extension device.

[0041] Such movement information could, for example, be the joint angle at which the body part with the orthopedic device or the orthopedic device itself is moved in the joint. For example, in the case of a leg amputation below the knee joint, movement of the knee joint will still be possible, which can be recorded as movement information. However, it is also possible to record a joint angle of the orthopedic device itself, for example, the joint angle of an ankle joint in a leg prosthesis with an artificial ankle joint.

[0042] In one embodiment, for example, when walking down a ramp, the user is shown the desired / optimal / generated knee angle as well as the current knee angle in order to build trust in the system and learn or practice the movement and adapt to the target value of the maximum flexion angle. In a further embodiment, the necessary conditions for triggering certain functions (e.g. climbing stairs, sitting down or standing up) could be displayed (e.g. necessary loads and / or position in space, or changes in position and / or accelerations). To assist in learning the movement sequence, the current sensor data of the orthopedic device is displayed in addition to the movement sequence to be achieved. The sensor data can be displayed as absolute values ​​(e.g. force in Newtons), as relative values ​​(e.g. force as a % of body weight) and / or relative values ​​or as a difference to the target value (e.g.% of the required EMG signal for one grip or 3° knee angle difference).

[0043] According to one embodiment, it is provided that the detected movement sequence is provided to the electronic computing unit and that a deviation between the detected movement sequence and the target movement is detected and a graphic representation of this detected deviation is displayed in the perceived reality by the reality extension device.

[0044] This makes it possible to present the patient with a (quantified) measure of the deviation between the patient's actual movement sequence and the target movement sequence, so that the patient knows how much their movement deviates from the target. This also allows progress over time to be documented during continuously recurring movement training.

[0045] According to one embodiment, the graphical movement representation is displayed on a display device worn in front of the patient's eyes.

[0046] Such a display device worn in front of the patient's eyes can be an augmented reality device (AR device) with which information can be projected into the patient's perceived reality by projecting or displaying corresponding images on the transparent surface worn in front of the eyes.

[0047] It is also conceivable that an additional display device is installed in the patient's environment in order to be able to display training-relevant information on projection surfaces in the patient's environment, such as walls, tables and / or floors.

[0048] The object is also achieved according to the invention with the system according to claim 11, wherein the system is configured to carry out the method as described above. For this purpose, the system comprises an electronic computing unit, a detection device, and a reality augmentation device for carrying out the method.

[0049] The invention is explained in more detail using the attached figure as an example. It shows:

[0050] Figure 1 shows a schematic representation of the system for carrying out the method according to the invention;

[0051] Figure 2 schematic representation of a movement training in

[0052] Climbing stairs with external detection device;

[0053] Figure 3 schematic representation of a movement training in

[0054] Stair climbing with glasses-based detection device and reality augmentation device;

[0055] Figure 4 schematic representation of the glasses-based reality augmentation from the patient’s perspective;

[0056] Figure 5 schematic representation of a movement training when grasping objects using a smartphone.

[0057] Figure 1 shows a system 10 comprising a reality augmentation device 11, a detection device 12, and an electronic processing unit 13. The reality augmentation device 11 is designed to be worn on the head 21 of a patient 20, so that something can be visually overlaid into the reality perceived by the patient in their line of sight.

[0058] The patient 20 is provided with a leg prosthesis 30 and performs a movement sequence in order to train the movement with the leg prosthesis 30.

[0059] The electronic processing unit 13 further generates a digital graphic movement sequence of the body part equipped with the orthopedic device, including the orthopedic device, based on a target movement that the patient 20 is to perform with their leg prosthesis 30. This graphic representation, which is created by the electronic processing unit 13, includes, in particular, a target movement sequence of the patient 20 with their leg prosthesis 30.

[0060] Furthermore, the actual movement sequence of the patient 20 with his leg prosthesis 30 is recorded using the recording device 12 and transmitted to the electronic processing unit 13. The electronic processing unit 13 uses image recognition to record the actual movement sequence of the patient 20 and synchronizes it with the digital graphical movement representation of the target movement.

[0061] The electronic processing unit 13 is in communicative connection with the reality augmentation device 11, for example via WLAN, and now sends the synchronized digital movement representation to the reality augmentation device 11 so that the latter can overlay the synchronized movement representation into the reality perceived by the patient 20 in his line of sight.

[0062] Figure 2 shows a schematic representation of an embodiment of a movement training exercise in which the patient 20 is to climb a flight of stairs. The patient 20 is equipped with a leg prosthesis 30. A second person, indicated only schematically, holds the external recording device 12, with which the movement of the patient 20 is recorded during the execution of their movement training. The external recording device 12 then continuously transmits the recorded movement to the reality augmentation device 11, which is designed in the form of a tablet. The electronic processing unit required for synchronization (designated in Figure 1.1.2013) is located in the reality augmentation device 11.

[0063] The tablet then uses the electronic processing unit to generate a graphical motion representation 40, 41 of the movement while climbing stairs. A motion representation 40 of the leg prosthesis and a motion representation 41 in the form of footprints are generated and overlaid into the reality perceived by the patient 20. The overlay is provided via the display of the reality augmentation device 11, which the patient 20 holds in their hands while performing the movement. The motion representations 40, 41, which are generated and synchronized with the recorded movement, are schematically indicated on the stairs in Figure 2.

[0064] Figure 3 shows an embodiment in which the reality augmentation device 11 is designed in the form of a pair of glasses. The glasses also feature the detection device 12, allowing the patient 20 to both detect their own movement and view the generated graphical movement representation 40, 41. The electronic processing unit can be embedded in the glasses or carried as an external device.

[0065] Figure 4 shows a representation of how the patient perceives their reality with the reality augmentation device 11 when performing the movement sequence from Figure 3. With the help of the glasses worn in front of the eyes, the graphic movement representation 40, 41 is superimposed onto the perceived reality so that the patient can see how the movement sequence should actually be performed. In the embodiment of Figure 4, the position of the prosthesis is shown as a first movement representation 40 and, in addition, footprints 41 are shown as a second movement representation. The footprints 41 serve to show the patient where each foot should touch the ground. The represented position of the prosthesis also shows the patient how the prosthesis should best be used for climbing stairs.

[0066] Figure 4 also shows the representation of further status and / or movement-related data 42, which are also displayed as a reality extension into the perceived reality of the patient by the reality extension device 11.

[0067] Finally, Figure 5 shows an embodiment in which the reality augmentation device 11, the detection device 12 and the electronic computing unit 13 are accommodated in a common device in the form of a smartphone 14.

[0068] In the embodiment of Figure 5, an object 50 is to be grasped by a hand prosthesis 31 as an orthopedic device. Using the smartphone 14, which the patient points toward the object 50 with the other hand, the movement of the hand prosthesis 31 is recorded and corresponding graphical movement representations are generated, which are then superimposed on the display as a reality extension.

[0069] 10 systems

[0070] 11 Reality augmentation device 12 Detection device

[0071] 13 electronic computing unit

[0072] 14 smartphones

[0073] 20 patients

[0074] 21 Patient's head 30 orthopaedic supply / leg prosthesis

[0075] 31 orthopaedic supplies / hand prosthesis

[0076] 40 graphic movement representation / leg prosthesis movement

[0077] 41 graphic movement representation / footprints

[0078] 42 status and / or movement-related data 50 Subject

Claims

Patent claims 1. A method for movement training of a movement of at least one body part of a patient (20) equipped with an orthopaedic device (30, 31), the method comprising the following steps: - generating a digital graphic movement representation (40, 41) of the body part equipped with the orthopaedic device (30, 31) as a function of a target movement which the body part with the equipped orthopaedic device (30, 31) is to perform, by means of an electronic computing unit (13), - detecting a body part movement of the body part of the patient (20) which is equipped with the orthopaedic device (30, 31) by means of a detection device, and - Fading in the generated digital graphic movement representation (40, 41) as a reality extension into the reality perceived by the patient (20) in his direction of view by a reality extension device (11) such that the position of the fade-in of the graphic movement representation (40, 41) depends on the detected body part movement.

2. Method according to claim 1, characterized in that the body part is a body part of the lower extremity.

3. Method according to claim 1 or 2, characterized in that the graphic movement representation (40, 41) includes at least part of a gait cycle.

4. Method according to one of the preceding claims, characterized in that a configuration of the orthopaedic supply (30, 31) is obtained and provided to the electronic computing unit (13), wherein the target movement is generated or selected by the electronic computing unit (13) depending on the provided configuration.

5. Method according to one of the preceding claims, characterized in that the target movement is generated or selected by the electronic computing unit (13) as a function of a type of movement set on the orthopaedic device (30, 31).

6. Method according to one of the preceding claims, characterized in that the graphic movement representation (40, 41) is displayed synchronously with the detected body part movement as a function of the latter.

7. Method according to one of the preceding claims, characterized in that status and / or movement-related data (42) of the orthopaedic device (30, 31) are recorded during the display of the graphic movement representation (40, 41) and are displayed in the perceived reality by the reality extension device (11).

8. Method according to one of the preceding claims, characterized in that the detected movement sequence is provided to an image recognition device and that movement information of the detected movement sequence is detected by the image recognition device and is superimposed into the perceived reality by the reality extension device (11).

9. Method according to one of the preceding claims, characterized in that the detected movement sequence is provided to the electronic computing unit (13) and that a deviation between the detected movement sequence and the target movement is recognized and a graphic representation (40, 41) of this detected deviation is superimposed into the perceived reality by the reality extension device (11).

10. Method according to one of the preceding claims, characterized in that the graphic movement representation (40, 41) is displayed on a display device worn in front of the eyes of the patient (20).

11. System (10) for movement training of a movement of at least one Body part of a patient (20) which is equipped with an orthopaedic device (30, 31), wherein the system (10) is designed to carry out the method according to one of the preceding claims.

Images