Rehabilitation exercise feedback system and method using rehabilitation exercise robot

Abstract

According to the present invention, while a user performs exercises using a rehabilitation exercise robot, an exercise image of the user is captured in real time, the user's joint coordinates are derived from the real-time exercise image by using artificial intelligence, and feedback information for correcting exercise posture is displayed together with the real-time exercise image by using augmented reality technology. This allows the user to more intuitively check and correct his or her exercise posture during exercise in real time, thereby enabling more effective rehabilitation.

Description

Rehabilitation exercise feedback system and method using a rehabilitation exercise robot

[0001] The present invention relates to a rehabilitation exercise feedback system and method using a rehabilitation exercise robot, and more particularly, to a rehabilitation exercise feedback system and method using a rehabilitation exercise robot that analyzes a user's exercise data measured by a rehabilitation exercise robot and an exercise video of the user captured by a camera to display feedback information for correcting exercise posture using augmented reality technology, thereby enabling more effective rehabilitation exercise.

[0002] Rehabilitation generally refers to treatment aimed at achieving and maintaining optimal function or improving the quality of life for people with disabilities. Recently, the development of rehabilitation exercise robots, which utilize rapidly advancing robotics technology to assist with rehabilitation training, is on the rise.

[0003] However, conventional rehabilitation exercise robots suffer from the problem of only using signals measured by sensors to drive exercise equipment, failing to provide an evaluation of the rehabilitation exercise itself. To address this issue, technologies that display biofeedback values ​​numerically or graphically in rehabilitation exercise robots are being applied. However, these technologies limit patients' ability to intuitively understand and use these data to correct exercise posture.

[0004] The purpose of the present invention is to provide a rehabilitation exercise feedback system and method using a rehabilitation exercise robot that provides feedback on a user's rehabilitation exercise more efficiently, thereby enabling more effective rehabilitation exercise.

[0005] A rehabilitation exercise feedback system using a rehabilitation exercise robot according to the present invention comprises: a camera for capturing a real-time exercise image of a user placing both feet on footrests of a rehabilitation exercise robot and performing a stepping exercise; an artificial intelligence unit for deriving in real-time the coordinates of each joint of the user's hip joint, knee joint, and ankle joint from the exercise image captured by the camera; a calculation unit for calculating in real-time a leg angle including an angle of an upper leg connecting the hip joint and the knee joint and an angle of a lower leg connecting the knee joint and the ankle joint using the joint coordinates derived by the artificial intelligence unit; a feedback information derivation unit for comparing the leg angle derived by the calculation unit with a preset reference angle range, and deriving leg angle feedback information that guides correction of the user's exercise posture so that the leg angle is within the reference angle range if the leg angle is outside the reference angle range; and a display unit provided in the rehabilitation exercise robot for displaying the exercise image in real-time, and for displaying the leg angle feedback information as an augmented reality image on the exercise image if the leg angle is outside the reference angle range.

[0006] The above leg angle feedback information may include an arrow indicating a direction and magnitude in which at least one of the upper leg and the lower leg should move.

[0007] The above camera may include at least one, and the joint coordinates may include coordinates derived using at least one motion image captured by the camera.

[0008] The above feedback information derivation unit receives a value of the user's knee moment, and if the value of the knee moment is greater than a previously input value, it can derive knee moment feedback information that guides correction of the user's exercise posture to reduce the knee moment.

[0009] The above display unit can display the knee moment feedback information in the exercise video as an augmented reality video.

[0010] The robot may further include a control unit that drives the rehabilitation exercise robot according to at least one of the leg angle feedback information and the knee moment feedback information.

[0011] A rehabilitation exercise feedback method using a rehabilitation exercise robot according to the present invention comprises the steps of: a step in which a camera captures a movement image of the user in real time when the user places both feet on the footrests of the rehabilitation exercise robot and performs a stepping exercise; a step in which an artificial intelligence unit derives in real time the coordinates of each joint of the user's hip joint, knee joint, and ankle joint from the movement image captured by the camera; a step in which a calculation unit calculates in real time a leg angle including an angle of an upper leg connecting the hip joint and the knee joint and an angle of a lower leg connecting the knee joint and the ankle joint using the joint coordinates derived by the artificial intelligence unit; a step in which a feedback information derivation unit compares the leg angle calculated by the calculation unit with a preset reference angle range, and if the leg angle is out of the reference angle range, derives leg angle feedback information that guides correction of the user's movement posture so that the leg angle is within the reference angle range; and a step in which a display unit displays the movement image in real time, and if the leg angle is out of the reference angle range, displays the leg angle feedback information as an augmented reality image on the movement image.

[0012] The above leg angle feedback information may include an arrow indicating a direction and magnitude in which at least one of the upper leg and the lower leg should move.

[0013] The above feedback information derivation unit may further include a step of receiving a value of the user's knee moment, and if the value of the knee moment is greater than a previously input value, deriving knee moment feedback information that guides correction of the user's exercise posture to reduce the knee moment.

[0014] The above display unit can display the knee moment feedback information in the exercise video as an augmented reality video.

[0015] The present invention has the advantage of enabling more effective rehabilitation exercise by allowing the user to more intuitively check and correct his / her exercise form and feedback information while exercising in real time by taking exercise videos in real time while exercising using a rehabilitation exercise robot, deriving the user's joint coordinates from the real-time exercise videos using artificial intelligence, and displaying feedback information for correcting exercise posture together with the real-time exercise videos using augmented reality technology.

[0016] FIG. 1 is a block diagram schematically illustrating the configuration of a rehabilitation exercise feedback system using a rehabilitation exercise robot according to an embodiment of the present invention.

[0017] Figure 2 is a flowchart illustrating a rehabilitation exercise feedback method according to an embodiment of the present invention.

[0018] Figure 3 shows an example of a screen displaying feedback information according to an embodiment of the present invention.

[0019] Hereinafter, embodiments of the present invention will be described with reference to the attached drawings.

[0020] In the embodiments of the present invention, the rehabilitation exercise robot is described as a rehabilitation stepping exercise device that allows a user to place both feet on footrests and perform stepping exercises. However, this is not limited to this, and any exercise device that allows a user to perform leg exercises can be applied.

[0021] FIG. 1 is a block diagram schematically illustrating the configuration of a rehabilitation exercise feedback system using a rehabilitation exercise robot according to an embodiment of the present invention.

[0022] Referring to FIG. 1, a rehabilitation exercise feedback system according to an embodiment of the present invention includes a camera (10), an artificial intelligence unit (20), a calculation unit (30), a feedback information derivation unit (40), and a display unit (50).

[0023] The above camera (10) is installed in the rehabilitation exercise robot or in front of the rehabilitation exercise robot, and captures a real-time video of the user's exercise when the user places both feet on the footrests of the rehabilitation exercise robot and performs a stepping exercise. The camera (10) will be described by way of example using a monocular wide-angle camera. Monocular wide-angle cameras have the advantage of being inexpensive and easy to use in narrow spaces compared to motion capture equipment or expensive RGB-D cameras.

[0024] An image correction unit (not shown) for correcting the motion image captured by the above camera (10) may be further included. The image correction unit (not shown) corrects distortion occurring in a monocular wide-angle camera. The image correction unit (not shown) may be provided in the camera (10), or, of course, may be provided in a terminal (not shown) or server (not shown) described later.

[0025] The above artificial intelligence unit (20) receives a movement video captured by the camera (10), recognizes the user from the movement video using artificial intelligence, and derives the coordinates of each joint of the user's hip joint, knee joint, and ankle joint in real time.

[0026] The above artificial intelligence unit (20) includes an artificial intelligence algorithm that learns pre-built learning data and derives the user's joint coordinates from various movement images.

[0027] The above calculation unit (30) calculates the angle of the upper leg connecting the hip joint and the knee joint using the joint coordinates derived from the artificial intelligence unit (20).

[0028] In addition, the calculation unit (30) calculates in real time a leg angle including the angle between the lower leg connecting the knee joint and the ankle joint.

[0029] The above feedback information derivation unit (40) derives leg angle feedback information and knee moment feedback information.

[0030] The above feedback information derivation unit (40) compares the leg angle calculated by the calculation unit (30) with a preset reference angle range, and if the leg angle is outside the reference angle range, derives leg angle feedback information that guides the user to correct his or her exercise posture so that the leg angle is within the reference angle range.

[0031] The above leg angle feedback information may include an arrow indicating a direction and magnitude in which at least one of the upper leg and the lower leg should move.

[0032] Meanwhile, the feedback information derivation unit (40) can receive the user's knee moment.

[0033] Here, the user's knee moment is calculated by a separate moment calculation unit (not shown) and input to the feedback information derivation unit (40) as an example. However, the present invention is not limited thereto, and it is also possible to calculate the moment using the joint coordinates and the movement data measured by the sensor unit (not shown) equipped in the reusable movement robot in the calculation unit (30). The user's knee moment is input multiple times while the user is exercising.

[0034] The above feedback information derivation unit (40) compares the value of the knee moment, which changes each time, with the value of the knee moment previously input, and if the value of the knee moment increases, derives knee moment feedback information that guides the user to correct his or her exercise posture to reduce the knee moment.

[0035] The above knee moment feedback information includes at least one of the knee moment, whether increased, whether decreased, and decrease induction indication, and may be displayed as letters, numbers, symbols, etc.

[0036] The above display unit (50) is equipped on the rehabilitation exercise robot and displays the user's exercise video in real time, and displays at least one of the leg angle feedback information and the knee moment feedback information together with the exercise video as an augmented reality image.

[0037] In addition, the display unit (50) can also display the respective positions of the hip joint, the knee joint, and the ankle joint in the exercise video as an augmented reality image.

[0038] In addition, the display unit (50) can also be displayed as an augmented reality image by indicating the upper leg and the lower leg with lines or the like in the exercise image.

[0039] The above display unit (50) includes a display panel provided in a position that the user can see while exercising in front of the rehabilitation exercise robot.

[0040] Meanwhile, in this embodiment, the artificial intelligence unit (20), the calculation unit (30), and the feedback information derivation unit (40) are described as being provided in a preset terminal (not shown) by way of example. However, the present invention is not limited thereto, and it is also possible for at least some of the artificial intelligence unit (20), the calculation unit (30), and the feedback information derivation unit (40) to be provided in a server (not shown).

[0041] The above terminal (not shown) may be equipped on the rehabilitation exercise robot, and of course, it may be equipped to enable wired or wireless communication with at least some of the rehabilitation exercise robot, the camera (10), and the display unit (50).

[0042] Meanwhile, in this embodiment, the use of one camera (10) has been described as an example. However, the present invention is not limited thereto, and it is also possible to capture multiple motion images by placing multiple cameras (10) at different locations. When multiple cameras (10) are used, the joint coordinates can be derived as two-dimensional or three-dimensional coordinates by taking into account the distance between the multiple cameras (10), etc.

[0043] Meanwhile, in this embodiment, the rehabilitation exercise feedback system has been described as guiding the leg angle feedback information and the knee moment feedback information through the display unit (50) by way of example. However, the present invention is not limited thereto, and it is also possible to guide the leg angle feedback information and the knee moment feedback information through a sound source such as an alarm or voice message.

[0044] In addition, the rehabilitation exercise feedback system may further include a control unit (not shown) that directly drives the rehabilitation exercise robot according to at least one of the leg angle feedback information and the knee moment feedback information. In this case, the control unit (not shown) controls the movement of a drive unit (not shown) that drives a mechanism such as a footrest (not shown) of the rehabilitation exercise robot, so that the rehabilitation exercise robot can help the user effectively exercise in a more correct posture according to the leg angle feedback information or the knee moment feedback information.

[0045] In addition, it is also possible to provide a vibration unit (not shown) to the foot plate (not shown) so that the control unit (not shown) vibrates the foot plate (not shown) using the vibration unit (not shown) according to the leg angle feedback information and the knee moment feedback information to notify the user.

[0046] The rehabilitation exercise feedback method according to the embodiment of the present invention configured as described above is described as follows.

[0047] Figure 2 is a flowchart illustrating a rehabilitation exercise feedback method according to an embodiment of the present invention.

[0048] Referring to Fig. 2, when a user places both feet on the footrests of the rehabilitation exercise robot and starts stepping, the camera (10) captures a real-time video of the user's movement. (S1)

[0049] The above image correction unit (not shown) corrects distortion of the motion image captured by the camera (10). (S2)

[0050] The above image correction unit (not shown) corrects distortion caused by the wide angle of the camera (10).

[0051] The above artificial intelligence unit (20) recognizes the user from the exercise image corrected by the image correction unit (not shown) and derives the coordinates of each joint of the user's hip joint, knee joint, and ankle joint in real time. (S3)

[0052] The above calculation unit (30) calculates the leg angle in real time using the joint coordinates derived from the artificial intelligence unit (20). (S4)

[0053] Here, the leg angle includes the angle of the upper leg and the angle of the lower leg. The angle of the upper leg refers to the angle at which the upper leg connecting the hip joint and the knee joint is inclined with respect to the vertical direction. The angle of the lower leg refers to the angle at which the lower leg connecting the knee joint and the ankle joint is inclined with respect to the vertical direction.

[0054] The above feedback information derivation unit (40) compares the leg angle calculated by the calculation unit (30) with a preset reference angle range. (S5)

[0055] Here, the reference angle range is preset to the leg angle range when the user exercises in the correct posture. The reference angle range may be set differently depending on the user's height and weight. The reference angle range includes a reference angle range for the upper leg angle and a reference angle range for the lower leg angle, and may be set to be the same or different from each other.

[0056] The above feedback information derivation unit (40) derives leg angle feedback information that guides the user to correct his / her exercise posture so that the leg angle is within the reference angle range when at least one of the angle of the upper leg and the angle of the lower leg is outside the reference angle range (S6).

[0057] The above leg angle feedback information is explained as an example of an arrow indicating the direction and magnitude in which the leg, among the upper and lower legs, that falls outside the reference angle range should move to achieve the correct exercise posture. However, this is not limited to this, and other symbols, letters, numbers, etc. other than the arrows may also be included.

[0058] In addition, the feedback information derivation unit (40) compares the value of the knee moment with a previously input value. (S7)

[0059] In this embodiment, the knee moment is calculated through a separate moment calculation unit (not shown) and input to the feedback information derivation unit (40) as an example.

[0060] That is, the knee moment calculated at the n+1th movement is compared with the knee moment calculated at the nth movement.

[0061] The above feedback information derivation unit (40) derives moment feedback information that guides posture correction to reduce the user's knee moment if the value of the knee moment is greater than the previously input value. (S8)

[0062] Referring to FIG. 3, the moment feedback information is explained as an example including arrows indicating the guide direction and magnitude in which the user should move the leg to reduce the knee moment, and left and right graphs for indicating changes in the left and right knee moments. In FIG. 3, circular dots and rectangular dots indicate the left and right joints of the user, the white lines connecting the joints indicate the current posture of the user, and the guide lines indicated thicker than the white lines indicate the posture in which the user should exercise. However, the present invention is not limited thereto, and the guide direction, magnitude, joints, current posture, guide lines, etc. may be displayed in various ways by differentiating arrows, colors, line thicknesses, symbols, letters, numbers, etc.

[0063] When the leg angle feedback information and the moment feedback information are derived, the display unit (50) displays the leg angle feedback information and the moment feedback information together as an augmented reality image on the exercise image captured in real time by the camera (10). (S9)

[0064] Accordingly, while exercising using the rehabilitation exercise robot, the user can not only check their own movement through the display panel, but also receive feedback information that guides posture correction for proper exercise. In other words, a mirror therapy effect can be achieved.

[0065] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the appended claims.

[0066] According to the present invention, a rehabilitation exercise feedback system using a rehabilitation exercise robot that makes rehabilitation exercise more effective can be manufactured.

Claims

1. A camera that records real-time video of a user stepping on the footplates of a rehabilitation exercise robot; An artificial intelligence unit that derives in real time the coordinates of each joint of the user's hip joint, knee joint, and ankle joint from the motion video captured by the above camera; An operation unit that calculates in real time a leg angle including an angle of the upper leg connecting the hip joint and the knee joint and an angle of the lower leg connecting the knee joint and the ankle joint using the joint coordinates derived from the artificial intelligence unit; A feedback information derivation unit that compares the leg angle calculated by the above calculation unit with a preset reference angle range, and, if the leg angle is outside the reference angle range, derives leg angle feedback information that guides the user to correct their exercise posture so that the leg angle is within the reference angle range; Equipped in the above rehabilitation exercise robot, the exercise image is displayed in real time, and when the leg angle is out of the reference angle range, a display unit is included that displays the leg angle feedback information as an augmented reality image on the exercise image. Rehabilitation exercise feedback system using a rehabilitation exercise robot.

2. In claim 1, The above leg angle feedback information is, Including arrows indicating the direction and magnitude in which at least one of the upper leg and the lower leg should move, Rehabilitation exercise feedback system using a rehabilitation exercise robot.

3. In claim 1, The above camera comprises at least one, The above joint coordinates are coordinates derived using at least one motion image captured by the camera. Rehabilitation exercise feedback system using a rehabilitation exercise robot.

4. In claim 1, The above feedback information derivation unit, The above feedback information derivation unit receives a value of the user's knee moment, and if the value of the knee moment is greater than a previously input value, derives knee moment feedback information that guides correction of the user's exercise posture to reduce the knee moment. Rehabilitation exercise feedback system using a rehabilitation exercise robot.

5. In claim 4, The above display part, Displaying the knee moment feedback information in the above exercise video as an augmented reality video, Rehabilitation exercise feedback system using a rehabilitation exercise robot.

6. In claim 4, Further comprising a control unit that drives the rehabilitation exercise robot according to at least one of the leg angle feedback information and the knee moment feedback information. Rehabilitation exercise feedback system using a rehabilitation exercise robot.

7. A step in which a camera captures a real-time video of the user's movement when the user places both feet on the footboards of the rehabilitation exercise robot and performs a stepping exercise; A step in which the artificial intelligence unit derives in real time the coordinates of each joint of the user's hip joint, knee joint, and ankle joint from the motion video captured by the camera; A step of calculating, in real time, a leg angle including an angle of the upper leg connecting the hip joint and the knee joint and an angle of the lower leg connecting the knee joint and the ankle joint using the joint coordinates derived by the artificial intelligence unit; A step of comparing the leg angle calculated by the calculation unit with a preset reference angle range, and, if the leg angle is outside the reference angle range, deriving leg angle feedback information that guides the user to correct their exercise posture so that the leg angle is within the reference angle range; A step of displaying the exercise image in real time and, when the leg angle is outside the reference angle range, displaying the leg angle feedback information as an augmented reality image on the exercise image, A method for providing feedback on rehabilitation exercise using a rehabilitation exercise robot.

8. In claim 7, The above leg angle feedback information is, Including arrows indicating the direction and magnitude in which at least one of the upper leg and the lower leg should move, A method for providing feedback on rehabilitation exercise using a rehabilitation exercise robot.

9. In claim 7, The above feedback information derivation unit further includes a step of receiving a value of the user's knee moment, and deriving knee moment feedback information that guides correction of the user's exercise posture to reduce the knee moment if the value of the knee moment is greater than a previously input value. A method for providing feedback on rehabilitation exercise using a rehabilitation exercise robot.

10. In claim 9, The above display part, Displaying the knee moment feedback information in the above exercise video as an augmented reality video, A method for providing feedback on rehabilitation exercise using a rehabilitation exercise robot.

Images