Rehabilitation exercise feedback system and method using a rehabilitation exercise robot

Abstract

The present invention has the advantage of enabling more effective rehabilitation exercise by capturing exercise video in real time while a user exercises using a rehabilitation exercise robot, deriving the user's joint coordinates from the real-time exercise video using artificial intelligence, and displaying feedback information for correcting exercise posture together with the real-time exercise video using augmented reality technology, thereby allowing the user to more intuitively check and correct their exercise appearance and feedback information while exercising in real time.

Description

Technology Field

[0001] The present invention relates to a rehabilitation exercise feedback system and method using a rehabilitation exercise robot, and more specifically, to a rehabilitation exercise feedback system and method using a rehabilitation exercise robot that enables more effective rehabilitation exercise by analyzing user exercise data measured by the rehabilitation exercise robot and user exercise video captured by a camera, and displaying feedback information for correcting exercise posture using augmented reality technology. Background Technology

[0002] In general, rehabilitation refers to treatment carried out to achieve and maintain optimal function or improve the quality of life for a person with a disability. Recently, there has been an increase in the development of technology for rehabilitation exercise robots that assist in rehabilitation training using rapidly advancing robotic technology.

[0003] However, conventional rehabilitation exercise robots have a problem in that they only utilize signals measured by sensors to drive the exercise equipment and fail to provide an evaluation of the rehabilitation exercises. To address this issue, technology is being applied to display biofeedback values ​​as numbers or graphs in rehabilitation exercise robots, but there are limitations to patients intuitively understanding this to correct their exercise posture. Prior art literature

[0004] Korean Registered Patent No. 10-1385165 The problem to be solved

[0005] The objective of the present invention is to provide a rehabilitation exercise feedback system and method using a rehabilitation exercise robot that enables more effective rehabilitation exercise by presenting feedback on the user's rehabilitation exercise more efficiently. means of solving the problem

[0006] A rehabilitation exercise feedback system using a rehabilitation exercise robot according to the present invention comprises: a camera that captures in real time an exercise video of a user placing both feet on the footplates of the rehabilitation exercise robot and performing a stepping exercise; an artificial intelligence unit that derives in real time the joint coordinates of the user's hip joint, knee joint, and ankle joint from the exercise video captured by the camera; a calculation unit that calculates in real time a leg angle including the angle of the upper leg connecting the hip joint and the knee joint and the angle of the lower leg connecting the knee joint and the ankle joint using the joint coordinates derived by the artificial intelligence unit; a feedback information derivation unit that compares the leg angle calculated by the calculation unit with a preset reference angle range and derives leg angle feedback information to guide the user's exercise posture correction so that the leg angle is within the reference angle range if the leg angle deviates from the reference angle range; and a display unit provided in the rehabilitation exercise robot that displays the exercise video in real time and, if the leg angle deviates from the reference angle range, displays the leg angle feedback information as an augmented reality image on the exercise video.

[0007] The leg angle feedback information may include an arrow indicating the direction and magnitude in which at least one of the upper leg and the lower leg must move.

[0008] The above camera includes at least one, and the joint coordinates may include coordinates derived using at least one motion image captured by the camera.

[0009] The above feedback information derivation unit receives a value of the user's knee moment, and if the value of the knee moment is greater than a previously received value, it can derive knee moment feedback information that guides the user's exercise posture correction to reduce the knee moment.

[0010] The above display unit can display the knee moment feedback information as the augmented reality image on the exercise image.

[0011] A control unit that drives the rehabilitation exercise robot according to at least one of the leg angle feedback information and the knee moment feedback information may be further included.

[0012] A rehabilitation exercise feedback method using a rehabilitation exercise robot according to the present invention comprises: a step in which a camera captures a video of the user’s exercise in real time when the user places both feet on the footplates of the rehabilitation exercise robot and performs a stepping exercise; a step in which an artificial intelligence unit derives the coordinates of each joint of the user’s hip joint, knee joint, and ankle joint in real time from the exercise video captured by the camera; a step in which a computing unit calculates a leg angle in real time, including the angle of the upper leg connecting the hip joint and the knee joint and the angle of the lower leg connecting the knee joint and the ankle joint, using the joint coordinates derived by the artificial intelligence unit; a step in which a feedback information derivation unit compares the leg angle calculated by the computing unit with a preset reference angle range, and if the leg angle deviates from the reference angle range, derives leg angle feedback information that guides the user’s exercise posture correction so that the leg angle is within the reference angle range; and a step in which a display unit displays the exercise video in real time, and if the leg angle deviates from the reference angle range, displays the leg angle feedback information as an augmented reality image on the exercise video.

[0013] The leg angle feedback information may include an arrow indicating the direction and magnitude in which at least one of the upper leg and the lower leg must move.

[0014] The above feedback information derivation unit may further include a step of receiving a value of the user's knee moment, and if the value of the knee moment is greater than a previously received value, deriving knee moment feedback information that guides the user's exercise posture correction to reduce the knee moment.

[0015] The above display unit can display the knee moment feedback information as the augmented reality image on the exercise image. Effects of the invention

[0016] The present invention has the advantage of enabling more effective rehabilitation exercise by capturing exercise video in real time while a user exercises using a rehabilitation exercise robot, deriving the user's joint coordinates from the real-time exercise video using artificial intelligence, and displaying feedback information for correcting exercise posture together with the real-time exercise video using augmented reality technology, thereby allowing the user to more intuitively check and correct their exercise appearance and feedback information while exercising in real time. Brief explanation of the drawing

[0017] FIG. 1 is a block diagram schematically illustrating the configuration of a rehabilitation exercise feedback system using a rehabilitation exercise robot according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a rehabilitation exercise feedback method according to an embodiment of the present invention. Figure 3 shows an example of a screen displaying feedback information according to an embodiment of the present invention. Specific details for implementing the invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the attached drawings.

[0019] In the embodiments of the present invention, the rehabilitation exercise robot is described as a rehabilitation stepping exercise device in which a user places both feet on footrests and performs stepping exercises. However, it is not limited to this, and can be applied in various ways as long as it is an exercise device that allows a user to perform leg exercises.

[0020] FIG. 1 is a block diagram schematically illustrating the configuration of a rehabilitation exercise feedback system using a rehabilitation exercise robot according to an embodiment of the present invention.

[0021] Referring to FIG. 1, a rehabilitation exercise feedback system according to an embodiment of the present invention includes a camera (10), an artificial intelligence unit (20), a computation unit (30), a feedback information derivation unit (40), and a display unit (50).

[0022] The camera (10) is provided on the rehabilitation exercise robot or is provided in front of the rehabilitation exercise robot, and captures a video of the user exercising in real time when the user places both feet on the footrests of the rehabilitation exercise robot and performs stepping exercises. The camera (10) is described as using a monocular wide-angle camera as an example. A monocular wide-angle camera has the advantage of being inexpensive compared to motion capture equipment or expensive RGB-D cameras, and is easy to use in a narrow space.

[0023] The above-described camera (10) may further include an image correction unit (not shown) for correcting motion images captured by the camera. The image correction unit (not shown) corrects distortion occurring in a monocular wide-angle camera. The image correction unit (not shown) may be provided in the camera (10), and of course, may also be provided in a terminal (not shown) or server (not shown) described later.

[0024] The artificial intelligence unit (20) receives a motion video captured by the camera (10), recognizes the user from the motion video using artificial intelligence, and derives the coordinates of each joint of the user's hip joint, knee joint, and ankle joint in real time.

[0025] The artificial intelligence unit (20) includes an artificial intelligence algorithm that learns pre-established training data and derives the user's joint coordinates from various exercise videos.

[0026] The above calculation unit (30) calculates the angle of the upper leg connecting the hip joint and the knee joint using the joint coordinates derived from the above artificial intelligence unit (20).

[0027] In addition, the calculation unit (30) calculates the leg angle in real time, including the angle between the lower legs connecting the knee joint and the ankle joint.

[0028] The above feedback information derivation unit (40) derives leg angle feedback information and knee moment feedback information.

[0029] The above feedback information derivation unit (40) compares the leg angle calculated by the above calculation unit (30) with a preset reference angle range, and if the leg angle deviates from the reference angle range, derives leg angle feedback information that guides the user's exercise posture correction so that the leg angle is within the reference angle range.

[0030] The leg angle feedback information may include an arrow indicating the direction and magnitude in which at least one of the upper leg and the lower leg must move.

[0031] Meanwhile, the above feedback information derivation unit (40) can receive the user's knee moment.

[0032] Here, the user's knee moment is explained as being calculated through a separate moment calculation unit (not shown) and input into the feedback information derivation unit (40). However, it is not limited to this, and it is also possible to calculate it using the joint coordinates and motion data measured by the sensor unit (not shown) equipped in the recycling exercise robot in the calculation unit (30). The user's knee moment is input multiple times while the user is exercising.

[0033] The above feedback information derivation unit (40) compares the knee moment value that changes each time with the previously received knee moment value, and if the knee moment value increases, derives knee moment feedback information that guides the user's exercise posture correction to reduce the knee moment.

[0034] The above knee moment feedback information includes at least one of the knee moment, whether it increases, whether it decreases, and an indication of inducing a decrease, and may be displayed as characters, numbers, symbols, etc.

[0035] The above display unit (50) is provided in the rehabilitation exercise robot and displays the user's exercise video in real time, and displays at least one of the leg angle feedback information and the knee moment feedback information together as an augmented reality video in the exercise video.

[0036] In addition, the display unit (50) can also display the respective positions of the hip joint, the knee joint, and the ankle joint in the motion video as augmented reality images.

[0037] In addition, the display unit (50) can also display the upper leg and the lower leg as lines, respectively, in the motion video to display them as an augmented reality video.

[0038] The above display unit (50) includes a display panel provided in a position where the user can view it while exercising in front of the rehabilitation exercise robot.

[0039] Meanwhile, in this embodiment, the artificial intelligence unit (20), the computation unit (30), and the feedback information derivation unit (40) are described as being provided in a pre-set terminal (not shown). However, this is not limited thereto, and it is of course possible for at least some of the artificial intelligence unit (20), the computation unit (30), and the feedback information derivation unit (40) to be provided in a server (not shown).

[0040] The above terminal (not shown) may be provided with the rehabilitation exercise robot, and it is also possible to provide it to communicate wired or wirelessly with at least some of the rehabilitation exercise robot, the camera (10), and the display unit (50).

[0041] Meanwhile, in this embodiment, the example was described using one camera (10). However, it is not limited to this, and it is also possible to place multiple cameras (10) at different locations to capture multiple motion images. When multiple cameras (10) are used, the joint coordinates can be derived as two-dimensional or three-dimensional coordinates by considering the distance between the multiple cameras (10).

[0042] Meanwhile, in this embodiment, the rehabilitation exercise feedback system is described as an example of providing the leg angle feedback information and the knee moment feedback information through the display unit (50). However, it is not limited to this, and it is also possible to provide the leg angle feedback information and the knee moment feedback information using a sound source such as an alarm or a voice message.

[0043] Additionally, the rehabilitation exercise feedback system may further include a control unit (not shown) that directly drives the rehabilitation exercise robot according to at least one of the leg angle feedback information and the knee moment feedback information. In this case, the control unit (not shown) controls the movement of a drive unit (not shown) that drives a mechanism such as a footrest (not shown) of the rehabilitation exercise robot, thereby enabling the rehabilitation exercise robot to help the user effectively perform exercises in a more correct posture according to the leg angle feedback information or the knee moment feedback information.

[0044] In addition, it is also possible to provide a vibration unit (not shown) on the footplate (not shown) so that the control unit (not shown) uses the vibration unit (not shown) to vibrate the footplate (not shown) according to the leg angle feedback information and the knee moment feedback information to notify the user.

[0046] The rehabilitation exercise feedback method according to the embodiment of the present invention configured as above is described as follows.

[0047] FIG. 2 is a flowchart illustrating a rehabilitation exercise feedback method according to an embodiment of the present invention.

[0048] Referring to FIG. 2, when a user places both feet on the footplates of the rehabilitation exercise robot and starts a stepping exercise, the camera (10) captures a video of the user's exercise in real time. (S1)

[0049] The above image correction unit (not shown) corrects the distortion of the motion image captured by the camera (10). (S2)

[0050] The above image correction unit (not shown) corrects distortion caused by the wide angle of the camera (10).

[0051] The artificial intelligence unit (20) recognizes the user from the motion image corrected by the image correction unit (not shown) and derives the coordinates of each joint of the user's hip joint, knee joint and ankle joint in real time. (S3)

[0052] The above calculation unit (30) calculates the leg angle in real time using the joint coordinates derived from the above artificial intelligence unit (20). (S4)

[0053] Here, the leg angle includes the angle of the upper leg and the angle of the lower leg. The angle of the upper leg refers to the angle at which the upper leg connecting the hip joint and the knee joint is tilted with respect to the vertical direction. The angle of the lower leg refers to the angle at which the lower leg connecting the knee joint and the ankle joint is tilted with respect to the vertical direction.

[0054] The above feedback information derivation unit (40) compares the leg angle calculated by the above calculation unit (30) with a preset reference angle range. (S5)

[0055] Here, the reference angle range is pre-set as the leg angle range when the user exercises in the correct posture. The reference angle range may also be set differently depending on the user's height and weight. The reference angle range includes a reference angle range for the upper leg angle and a reference angle range for the lower leg angle, and may be set to be the same or different from each other.

[0056] The above feedback information derivation unit (40) derives leg angle feedback information that guides the user's exercise posture correction so that the leg angle is within the reference angle range when at least one of the upper leg angle and the lower leg angle deviates from the reference angle range. (S6)

[0057] The above leg angle feedback information is explained by example as an arrow indicating the direction and magnitude in which the leg that deviates from the reference angle range among the upper leg and the lower leg must move for the correct exercise posture. However, it is not limited to this, and other symbols, letters, numbers, etc. other than the arrow may also be included.

[0058] In addition, the feedback information derivation unit (40) compares the value of the knee moment with a previously input value. (S7)

[0059] In this embodiment, the knee moment is calculated through a separate moment calculation unit (not shown) and input to the feedback information derivation unit (40), as an example.

[0060] In other words, compare the knee moment calculated during the (n+1)th movement with the knee moment calculated during the nth movement.

[0061] The above feedback information derivation unit (40) derives moment feedback information that guides posture correction to reduce the user's knee moment when the value of the knee moment is greater than the previously input value. (S8)

[0062] Referring to FIG. 3, the moment feedback information is described as, for example, an arrow indicating the direction and magnitude in which the user must move their leg to reduce the knee moment. However, it is not limited to this and may include other symbols, letters, numbers, etc., other than arrows.

[0063] When the leg angle feedback information and the moment feedback information are derived, the display unit (50) displays the leg angle feedback information and the moment feedback information together as an augmented reality image on the motion video captured in real time by the camera (10). (S9)

[0064] Therefore, while exercising using the rehabilitation exercise robot, the user can not only check their own exercise movements through the display panel, but also receive feedback information guiding posture correction for proper exercise. In other words, a mirror therapy effect can be obtained.

[0066] The present invention has been described with reference to the embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the technical spirit of the appended claims. Explanation of the symbols

[0067] 10: Camera 20: Artificial Intelligence Department 30: Operation unit 40: Feedback information output unit 50: Display section

Claims

Claim 1 A camera equipped on a rehabilitation exercise robot or equipped in front of the rehabilitation exercise robot, which captures in real time an exercise video of a user placing both feet on the footplates of the rehabilitation exercise robot and performing a stepping exercise; an artificial intelligence unit that derives the coordinates of each joint of the user's hip joint, knee joint, and ankle joint in real time from the exercise video captured by the camera; a calculation unit that calculates in real time a leg angle including the angle of the upper leg connecting the hip joint and the knee joint, and the angle of the lower leg connecting the knee joint and the ankle joint, using the joint coordinates derived by the artificial intelligence unit; a feedback information derivation unit that compares the leg angle calculated by the calculation unit with a preset reference angle range, derives leg angle feedback information to guide the user's exercise posture correction so that the leg angle is within the reference angle range if the leg angle deviates from the reference angle range, receives the value of the user's knee moment, and derives knee moment feedback information to guide the user's exercise posture correction to reduce the knee moment if the knee moment value is greater than the previously received value; and on the rehabilitation exercise robot A display unit provided, which displays the leg angle feedback information and the moment feedback information together as an augmented reality image in the motion video captured in real time by the camera;A rehabilitation exercise feedback system using a rehabilitation exercise robot, comprising: a control unit that drives the rehabilitation exercise robot according to at least one of the leg angle feedback information and the knee moment feedback information; wherein the leg angle feedback information includes an arrow indicating the direction and magnitude in which at least one of the upper leg and the lower leg must move; and wherein the knee moment feedback information includes at least one of the knee moment, whether it increases, whether it decreases, and an indication of inducing a decrease; wherein the display unit includes a display panel provided at a position in front of the rehabilitation exercise robot that a user can view while exercising, and displays the exercise video of the user exercising in real time, and displays at least a portion of the leg angle feedback information, the knee moment feedback information, the respective positions of the hip joint, the knee joint, and the ankle joint, and the lines indicating the upper leg and the lower leg as an augmented reality image on the exercise video. Claim 2 delete Claim 3 A rehabilitation exercise feedback system using a rehabilitation exercise robot according to claim 1, wherein the camera comprises at least one, and the joint coordinates are coordinates derived using at least one motion image captured by the camera. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A step in which, when a user places both feet on the footplates of a rehabilitation exercise robot and performs a stepping exercise, a camera equipped on the rehabilitation exercise robot or equipped on the front of the rehabilitation exercise robot captures a video of the user's exercise in real time; a step in which an artificial intelligence unit derives the coordinates of each joint of the user's hip joint, knee joint, and ankle joint in real time from the exercise video captured by the camera; a step in which a computational unit calculates a leg angle in real time, including the angle of the upper leg connecting the hip joint and the knee joint and the angle of the lower leg connecting the knee joint and the ankle joint, using the joint coordinates derived by the artificial intelligence unit; a step in which a feedback information derivation unit compares the leg angle calculated by the computational unit with a preset reference angle range, and if the leg angle deviates from the reference angle range, derives leg angle feedback information to guide the user's exercise posture correction so that the leg angle is within the reference angle range; and a step in which the feedback information derivation unit receives the value of the user's knee moment calculated by the moment computational unit, and if the value of the knee moment is greater than the previously received value, A step of deriving knee moment feedback information that guides the user's exercise posture correction to reduce the above knee moment;A rehabilitation exercise feedback method using a rehabilitation exercise robot, comprising the step of a display unit displaying the leg angle feedback information and the moment feedback information together as an augmented reality image on an exercise video captured in real time by the camera, wherein the leg angle feedback information includes an arrow indicating the direction and magnitude in which at least one of the upper leg and the lower leg must move, and the knee moment feedback information includes at least one of the knee moment, whether it increases, whether it decreases, and an indication of inducing decrease, and wherein the display unit includes a display panel provided in a position in front of the rehabilitation exercise robot that a user can view while exercising, and displays the exercise video in which the user is exercising in real time, and displays at least a portion of the leg angle feedback information, the knee moment feedback information, the respective positions of the hip joint, the knee joint, and the ankle joint, and the lines indicating the upper leg and the lower leg as an augmented reality image on the exercise video. Claim 8 delete Claim 9 delete Claim 10 delete

Images