A Kinect-based rehabilitation training system for movement disorders

By generating user profiles and adjusting the weights of training parameters, the Kinect system solves the problem of inaccurate motion recognition, enabling personalized rehabilitation training that is suitable for home-based rehabilitation.

CN114973322BInactive Publication Date: 2024-11-12THE FIRST AFFILIATED HOSPITAL OF ZHENGZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210540473.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-17
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing Kinect-based rehabilitation training systems lack personalized adjustments when recognizing user movements, resulting in inaccurate recognition of users with different body types and clothing, and poor versatility. In particular, it is difficult to provide appropriate rehabilitation training parameters for users who are overweight or wearing loose clothing.

Method used

By generating user profiles and using color and depth images captured by Kinect, the width of the Neck, WristRight, and WristLeft skeletal points, as well as the user's height and gender, are calculated. The training parameter weights are adjusted, and the user's input difficulty and skeletal point position information are combined to determine whether the user's actions are qualified.

Benefits of technology

It enables automatic adjustment of rehabilitation training difficulty based on user characteristics, improves the accuracy of motion recognition and the applicability of the system, reduces equipment costs, and is suitable for home use.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114973322B_ABST
    Figure CN114973322B_ABST
Patent Text Reader

Abstract

The present invention provides a Kinect-based rehabilitation training system for movement disorders, which obtains a user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeletal points in the first grayscale image and the second grayscale image, and the user's height, obtains the difficulty input by the user, and calculates the training parameter weight corresponding to the user according to the user portrait, and determines whether the user's action is qualified according to the difficulty, the training parameter weight, the skeletal point position information, and the standard action. The present invention solves the problem that the user's action cannot be adjusted according to the user's physical characteristics, adjusts the rehabilitation training in combination with the difficulty set by the user, and only requires the use of the existing Kinect, without the need for additional equipment, and is very suitable for users who perform rehabilitation training at home.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of Kinect, and in particular to a movement disorder rehabilitation training system based on Kinect. Background Art

[0002] With the aggravation of the aging population in my country, the number of patients paralyzed by stroke, hypertension, etc. is increasing. Production accidents and traffic accidents are also a major factor leading to the loss of limb function. Rehabilitation training for these patients is very necessary, and rehabilitation training is also used in the intervention training of children with autism. Rehabilitation training is a process in which patients restore body functions by performing designated movements or massaging the patient to make the body move. Rehabilitation training in my country is still mainly based on traditional rehabilitation training. Rehabilitation trainers guide patients in rehabilitation movements or assist patients in rehabilitation training with the help of a rehabilitation training device. However, for patients, the current cost of rehabilitation training is relatively high, especially one-on-one rehabilitation training.

[0003] With the development of automatic control technology and computer technology, rehabilitation training robots have emerged. Programs are set in rehabilitation training robots, and robots assist patients in rehabilitation training. However, rehabilitation training robots are relatively expensive, and patients can only undergo rehabilitation training in hospitals or rehabilitation institutions. There are not many practical applications. Later, rehabilitation training manufacturers launched rehabilitation training systems based on virtual reality. Kinect is a visual interactive device developed by Microsoft. Kinect can collect sound, color images, depth images, obtain 25 bone points, and identify human body movements through pattern recognition and other methods. It can also be directly connected to a computer, making it very suitable for use at home. The application of Kinect in rehabilitation training has achieved good results. Compared with traditional rehabilitation training and rehabilitation training robots, the training system based on Kinect is cheaper and more interesting. Patients can undergo rehabilitation training at home. However, patients are different in terms of fatness, height, clothing, gender, etc. For example, when adults and children do the same action, or wear loose clothes and tight clothes, the Kinect-based rehabilitation training system may recognize that the actions of children and loose clothes are not standard, or even recognize them as different actions. The current Kinect-based obstacle rehabilitation training system has poor versatility. Summary of the invention

[0004] In order to solve the above problems, on the one hand, the present invention provides a Kinect-based movement disorder rehabilitation training system, the system comprising the following modules:

[0005] A user portrait generation module is used to adjust the synchronization between the color image and the depth image, obtain the user color image, and convert the user color image into a first grayscale image; obtain the user depth image corresponding to the user color image, and convert the user depth image into a second grayscale image; obtain the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image, as well as the user's height and gender;

[0006] The training action judgment module obtains the difficulty of the user input, and calculates the training parameter weight corresponding to the user according to the user portrait, and judges whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action.

[0007] Preferably, obtaining the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, WristLeft skeleton points in the first grayscale image and the second grayscale image and the height and gender of the user is specifically as follows:

[0008] The first grayscale image and the second grayscale image are respectively executed as follows: obtain the straight line L1 between the Neck and Head skeleton points, obtain the line width perpendicular to the straight line L1 at the Neck skeleton point, and obtain the Neck width; obtain the straight line L2 between the WristRight and ElboWrist skeleton points, obtain the line width perpendicular to the straight line L2 at the WristRight skeleton point, and obtain the WristRight width; obtain the straight line L3 between the WristLeft and ElbowLeft skeleton points, obtain the line width perpendicular to the straight line L3 at the WristLeft skeleton point, and obtain the WristLeft width;

[0009] The average value of the Neck width in the first grayscale image and the Neck width in the second grayscale image is used as the user's Neck width; the average value of the WristRight width in the first grayscale image and the WristRight width in the second grayscale image is used as the user's WristRight width; the average value of the WristLeft width in the first grayscale image and the WristLeft width in the second grayscale image is used as the user's WristLeft width;

[0010] The user's height is obtained through the Head skeleton point, and the user's height, gender, user's neck width, user's wrist right width, and user's wrist left width are used as user portrait parameters.

[0011] Preferably, the training parameter weights corresponding to the user are calculated according to the user portrait, specifically: according to the height and gender of the user, a standard value of the user's neck width, a standard value of the user's wrist right width, and a standard value of the user's wrist left width corresponding to the height and gender are obtained;

[0012] If User neck +User wristright +User wristleft >S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0013] If User neck +User wristright +User wristleft =S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user w1=0;

[0014] If User neck +User wristright +User wristleft <S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0015] Among them, User neck is the user neck width, User wristright is the width of the user WristRight, User wristleft is the user WristLeft width, S neck is the standard value of the user's neck width, S wristright is the standard value of the user's WristRight width, S wristleft It is the standard value of the user's WristLeft width.

[0016] Preferably, judging whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action is specifically as follows:

[0017] The action of the user's skeleton point is obtained, and the similarity between the action of the user's skeleton point and the standard action is calculated, and a new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold. If the similarity is not less than the new similarity threshold, the user action is determined to be qualified, otherwise, the user action is determined to be unqualified.

[0018] Preferably, the new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold, specifically:

[0019] Thr=k·(1+w1)·Thr', wherein Thr is the new similarity threshold, k is the difficulty, and Thr' is the standard similarity threshold.

[0020] Preferably, the user color image is a color image containing only the user; and the user depth image is a depth image containing only the user.

[0021] On the other hand, the present invention further provides a non-volatile computer storage medium, wherein the non-volatile computer storage medium stores instructions, and when the instructions are executed on a processor, the processor executes the following steps:

[0022] S1, adjust the synchronization of the color image and the depth image, obtain the user color image, and convert the user color image into a first grayscale image; obtain the user depth image corresponding to the user color image, and convert the user depth image into a second grayscale image; obtain the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image as well as the user's height and gender;

[0023] S2, training action judgment module, obtains the difficulty of the user input, and calculates the training parameter weight corresponding to the user according to the user portrait, and judges whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action.

[0024] Preferably, obtaining the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, WristLeft skeleton points in the first grayscale image and the second grayscale image and the height and gender of the user is specifically as follows:

[0025] The first grayscale image and the second grayscale image are respectively executed as follows: obtain the straight line L1 between the Neck and Head skeleton points, obtain the line width perpendicular to the straight line L1 at the Neck skeleton point, and obtain the Neck width; obtain the straight line L2 between the WristRight and ElboWrist skeleton points, obtain the line width perpendicular to the straight line L2 at the WristRight skeleton point, and obtain the WristRight width; obtain the straight line L3 between the WristLeft and ElbowLeft skeleton points, obtain the line width perpendicular to the straight line L3 at the WristLeft skeleton point, and obtain the WristLeft width;

[0026] The average value of the Neck width in the first grayscale image and the Neck width in the second grayscale image is used as the user's Neck width; the average value of the WristRight width in the first grayscale image and the WristRight width in the second grayscale image is used as the user's WristRight width; the average value of the WristLeft width in the first grayscale image and the WristLeft width in the second grayscale image is used as the user's WristLeft width;

[0027] The user's height is obtained through the Head skeleton point, and the user's height, the user's Neck width, the user's WristRight width, and the user's WristLeft width are used as user portrait parameters.

[0028] Preferably, the training parameter weights corresponding to the user are calculated according to the user portrait, specifically: according to the height and gender of the user, a standard value of the user's neck width, a standard value of the user's wrist right width, and a standard value of the user's wrist left width corresponding to the height and gender are obtained;

[0029] If User neck +User wristright +User wristleft >S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0030] If User neck +User wristright +User wristleft =S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user w1=0;

[0031] If User neck +User wristright +User wristleft <S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0032] Among them, User neck is the user neck width, User wristright is the width of the user WristRight, User wristleft is the user WristLeft width, Sneck is the standard value of the user's neck width, S wristright is the standard value of the user's WristRight width, S wristleft It is the standard value of the user's WristLeft width.

[0033] Preferably, judging whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action is specifically as follows:

[0034] The action of the user's skeleton point is obtained, and the similarity between the action of the user's skeleton point and the standard action is calculated, and a new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold. If the similarity is not less than the new similarity threshold, the user action is determined to be qualified, otherwise, the user action is determined to be unqualified.

[0035] The Kinect-based rehabilitation training system provided by the present invention solves the problem that the existing rehabilitation training system cannot adjust the motion parameters according to the user's physical characteristics, and can judge whether the user's movements are qualified according to the user's height, fatness and rehabilitation stage. For users with a fat body shape, the difficulty of rehabilitation training is automatically reduced, and the rehabilitation training is adjusted according to the difficulty set by the user. Moreover, it only needs to use the existing Kinect and does not need additional equipment such as a weight scale. Therefore, it is very suitable for users who perform rehabilitation training at home. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 This is a physical picture of Kinect;

[0038] Figure 2 This is a schematic diagram of the skeleton points in Kinect;

[0039] Figure 3 It is a structural schematic diagram of the present invention;

[0040] Figure 4 This is a flow chart of embodiment 3 of the present invention. DETAILED DESCRIPTION

[0041] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the first embodiment, the present invention provides a Kinect-based movement disorder rehabilitation training system, the system comprising the following modules:

[0044] A user portrait generation module is used to adjust the synchronization between the color image and the depth image, obtain the user color image, and convert the user color image into a first grayscale image; obtain the user depth image corresponding to the user color image, and convert the user depth image into a second grayscale image; obtain the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image, as well as the user's height and gender;

[0045] like Figure 1 As shown in the figure, Kinect can collect three kinds of information, namely, video data through a color camera, depth data through an infrared transmitter and an infrared receiver, and audio data through a microphone. Video data includes multiple video frames, that is, multiple images. The images are in RGB format, which is more in line with human vision. The depth map is a map composed of the distance between the object and Kinect. The audio data collects the speech of people around. With the help of the depth map, Kinect can obtain human skeleton points, also known as joint points and bone joint points. In the first version of Kinect, 20 skeleton points can be collected, and in the second version, 25 skeleton points can be collected. Figure 2 shown.

[0046] Color images and depth images are images collected by two different principles. In order to determine user features more accurately, the present invention uses both user color images and user depth images for analysis. The user color image is obtained through KinectManager.Instance.GetUsersClrTex(), and the user depth image is obtained through KinectManger.Instance.GetUsersLblTex(). The user color image and user depth image obtained by the above method are directly obtained by cutout to obtain images containing only the user. Therefore, there is no need to perform cutout, etc.

[0047] Generally speaking, users with fatter bodies also have thicker necks and wrists. The user's fatness or thinness can be judged by the width of the neck and wrists. In addition, since the neck and wrists cannot be covered by clothes, the impact of the user wearing loose clothes can be ruled out.

[0048] The training action judgment module obtains the difficulty of the user input, and calculates the training parameter weight corresponding to the user according to the user portrait, and judges whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action.

[0049] There are many ways to identify user actions, including static recognition methods and dynamic action recognition, such as template matching method, state machine recognition method, etc. The main principle of action recognition is to determine the position relationship of skeleton points and the similarity between the movement of skeleton points and the preset movement points. If the similarity is large, it means that the user is performing a certain movement, otherwise it is judged that the user is not performing the action or the action is not standard.

[0050] The user portrait is obtained according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image as well as the height and gender of the user, specifically as follows:

[0051] The first grayscale image and the second grayscale image are respectively executed as follows: obtain the straight line L1 between the Neck and Head skeleton points, obtain the line width perpendicular to the straight line L1 at the Neck skeleton point, and obtain the Neck width; obtain the straight line L2 between the WristRight and ElboWrist skeleton points, obtain the line width perpendicular to the straight line L2 at the WristRight skeleton point, and obtain the WristRight width; obtain the straight line L3 between the WristLeft and ElbowLeft skeleton points, obtain the line width perpendicular to the straight line L3 at the WristLeft skeleton point, and obtain the WristLeft width;

[0052] The average value of the Neck width in the first grayscale image and the Neck width in the second grayscale image is used as the user's Neck width; the average value of the WristRight width in the first grayscale image and the WristRight width in the second grayscale image is used as the user's WristRight width; the average value of the WristLeft width in the first grayscale image and the WristLeft width in the second grayscale image is used as the user's WristLeft width;

[0053] The user's height is obtained through the Head skeleton point, and the user's height, gender, user's neck width, user's wrist right width, and user's wrist left width are used as user portrait parameters.

[0054] In a specific embodiment, the training parameter weights corresponding to the user are calculated based on the user portrait, specifically: according to the height and gender of the user, the standard value of the user's neck width, the standard value of the user's wrist right width, and the standard value of the user's wrist left width corresponding to the height and gender are obtained;

[0055] If User neck +User wristright +User wristleft >S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0056] If User neck +User wristright +User wristleft =S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user w1=0;

[0057] If User neck +User wristright +User wristleft <S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0058] Among them, User neck is the user neck width, User wristright is the width of the user WristRight, User wristleft is the user WristLeft width, S neckis the standard value of the user's neck width, S wristright is the standard value of the user's WristRight width, S wristleft It is the standard value of the user's WristLeft width.

[0059] The standard values ​​of user Neck width, user WristRight width, and user WristLeft width corresponding to the height and gender can be obtained through a table lookup. For example, for female users with a height between 1.2 and 1.5, the corresponding standard values ​​of user Neck width, user WristRight width, and user WristLeft width are 15 cm, 7 cm, and 7 cm, respectively.

[0060] For example, in one embodiment, it is assumed that the standard values ​​of the user's Neck width, the user's WristRight width, and the user's WristLeft width are 15 cm, 7 cm, and 7 cm, respectively;

[0061] If the Neck width value, WristRight width value, and WristLeft width value of a user are 16 cm, 8 cm, and 9 cm respectively, the first calculation method is used, and w1=0.319;

[0062] If the Neck width value, WristRight width value, and WristLeft width value of a user are 13 cm, 6 cm, and 5 cm respectively, the third calculation method is used for calculation, and w1=-0.479.

[0063] Since the user may wear a watch on his wrist, in another embodiment, if the difference between the user's WristRight width and the user's WristLeft width is greater than a first threshold, the smaller of the two is used as the user's WristRight width and the user's WristLeft width.

[0064] In one embodiment, judging whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action is specifically as follows:

[0065] The action of the user's skeleton point is obtained, and the similarity between the action of the user's skeleton point and the standard action is calculated, and a new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold. If the similarity is not less than the new similarity threshold, the user action is determined to be qualified, otherwise, the user action is determined to be unqualified.

[0066] In one embodiment, the new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold, specifically:

[0067] Thr=k·(1+w1)·Thr', wherein Thr is the new similarity threshold, k is the difficulty, and Thr' is the standard similarity threshold.

[0068] Still taking w1 calculated above as an example, when w1=0.319, Thr=k·1.319·Thr'; when w1=-0.479, Thr=k·0.521·Thr'.

[0069] Since Kinect involves a large amount of data calculation, and the performance of existing computing devices in homes and hospitals is limited, in another specific embodiment, the interval between two video frames and depth image frames is calculated according to the frame rates of the video data stream and the depth data stream, and the action delay time is determined. If the action delay time is greater than the interval, the video frame image size and the depth frame image size are reduced. The reduced video frame and depth image include the complete user. This can reduce the amount of data of color images and depth images processed by the computing device and improve processing efficiency.

[0070] In the process of using Kinect, it is found that the user color image will have a white edge problem, which is caused by inaccurate cutout of the color image. In the user depth image, due to the interval of infrared reception, the user depth image will also have inaccuracies. This problem will lead to inaccurate recognition of bone points. In order to suppress the noise of the user color image and the user depth image mentioned above, in another embodiment, according to the shape of the user in the grayscale image, the first grayscale image and the second grayscale image are aligned, and the overlapping part is recorded. The pixel points of the overlapping part are retained in the user color image to obtain the denoised user color image, and the position of the overlapping part is retained in the user depth image to obtain the denoised user depth image. In this way, the color image and depth image obtained by two different imaging principles are used to denoise the image, which prevents the influence of visible light, infrared light and infrared reception interval.

[0071] In a second embodiment, the present invention further provides a non-volatile computer storage medium, wherein the non-volatile computer storage medium stores instructions, and when the instructions are executed on a processor, the processor executes the following steps:

[0072] S1, adjust the synchronization of the color image and the depth image, obtain the user color image, and convert the user color image into a first grayscale image; obtain the user depth image corresponding to the user color image, and convert the user depth image into a second grayscale image; obtain the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image as well as the user's height and gender;

[0073] S2, training action judgment module, obtains the difficulty of the user input, and calculates the training parameter weight corresponding to the user according to the user portrait, and judges whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action.

[0074] Preferably, obtaining the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, WristLeft skeleton points in the first grayscale image and the second grayscale image and the height and gender of the user is specifically as follows:

[0075] The first grayscale image and the second grayscale image are respectively executed as follows: obtain the straight line L1 between the Neck and Head skeleton points, obtain the line width perpendicular to the straight line L1 at the Neck skeleton point, and obtain the Neck width; obtain the straight line L2 between the WristRight and ElboWrist skeleton points, obtain the line width perpendicular to the straight line L2 at the WristRight skeleton point, and obtain the WristRight width; obtain the straight line L3 between the WristLeft and ElbowLeft skeleton points, obtain the line width perpendicular to the straight line L3 at the WristLeft skeleton point, and obtain the WristLeft width;

[0076] The average value of the Neck width in the first grayscale image and the Neck width in the second grayscale image is used as the user's Neck width; the average value of the WristRight width in the first grayscale image and the WristRight width in the second grayscale image is used as the user's WristRight width; the average value of the WristLeft width in the first grayscale image and the WristLeft width in the second grayscale image is used as the user's WristLeft width;

[0077] The user's height is obtained through the Head skeleton point, and the user's height, gender, user's neck width, user's wrist right width, and user's wrist left width are used as user portrait parameters.

[0078] Preferably, the training parameter weights corresponding to the user are calculated according to the user portrait, specifically: according to the height and gender of the user, a standard value of the user's neck width, a standard value of the user's wrist right width, and a standard value of the user's wrist left width corresponding to the height and gender are obtained;

[0079] If User neck +User wristright +User wristleft >S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0080] If User neck +User wristright +User wristleft =S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user w1=0;

[0081] If User neck +User wristright +User wristleft <S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user

[0082] Among them, User neck is the user neck width, User wristright is the width of the user WristRight, User wristleft is the user WristLeft width, S neck is the standard value of the user's neck width, S wristright is the standard value of the user's WristRight width, S wristleft It is the standard value of the user's WristLeft width.

[0083] Preferably, judging whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action is specifically as follows:

[0084] The action of the user's skeleton point is obtained, and the similarity between the action of the user's skeleton point and the standard action is calculated, and a new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold. If the similarity is not less than the new similarity threshold, the user action is determined to be qualified, otherwise, the user action is determined to be unqualified.

[0085] In a third embodiment, the present invention further provides a Kinect-based rehabilitation training method for movement disorders, which is applied to a Kinect rehabilitation training device, such as Figure 4 As shown, the method comprises the following steps:

[0086] S1, adjust the synchronization of the color image and the depth image, obtain the user color image, and convert the user color image into a first grayscale image; obtain the user depth image corresponding to the user color image, and convert the user depth image into a second grayscale image; obtain the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image as well as the user's height and gender;

[0087] S2, training action judgment module, obtains the difficulty of the user input, and calculates the training parameter weight corresponding to the user according to the user portrait, and judges whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action.

[0088] The device embodiments described above are merely illustrative, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. A person skilled in the art may understand and implement the present invention without creative effort.

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Kinect-based movement disorder rehabilitation training system, characterized in that: The system includes the following modules: A user portrait generation module is used to adjust the synchronization between the color image and the depth image, obtain the user color image, and convert the user color image into a first grayscale image; obtain the user depth image corresponding to the user color image, and convert the user depth image into a second grayscale image; obtain the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image, as well as the user's height and gender; The training action judgment module obtains the difficulty of the user input, and calculates the training parameter weight corresponding to the user according to the user portrait, and judges whether the user action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action; The method of obtaining the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image as well as the height and gender of the user is specifically as follows: executing on the first grayscale image and the second grayscale image respectively: obtaining the straight line L1 between the Neck and Head skeleton points, obtaining the line width perpendicular to the straight line L1 at the Neck skeleton point, and obtaining the Neck width; obtaining the straight line L2 between the WristRight and ElboWrist skeleton points, obtaining the line width perpendicular to the straight line L2 at the WristRight skeleton point, and obtaining the WristRight width; obtaining the straight line L3 between the WristLeft and ElbowLeft skeleton points, obtaining the line width perpendicular to the straight line L3 at the WristLeft skeleton point, and obtaining the WristLeft width; taking the average value of the Neck width in the first grayscale image and the Neck width in the second grayscale image as the user Neck width; taking the average value of the WristRight width in the first grayscale image and the WristRight width in the second grayscale image as the user WristRight width; The average of the WristLeft width in the first grayscale image and the WristLeft width in the second grayscale image is used as the WristLeft width of the user; the height of the user is obtained through the Head skeleton point, and the height, gender, neck width, wristright width and wristleft width of the user are used as user portrait parameters; The training parameter weights corresponding to the user are calculated based on the user portrait, specifically: according to the height and gender of the user, the standard values ​​of the user's neck width, the user's wrist right width, and the user's wrist left width corresponding to the height and gender are obtained; if the user neck +User wristright +User wristleft >S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user If User neck +User wristright +User wristleft =S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user w1=0; If User neck +User wristright +User wristleft <S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user Among them, User neck is the user neck width, User wristright is the width of the user WristRight, User wristleft is the user WristLeft width, S neck is the standard value of the user's neck width, S wristright is the standard value of the user's WristRight width, S wristleft It is the standard value of the user's WristLeft width.

2. The system according to claim 1, characterized in that The determining whether the user's action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action is specifically as follows: The action of the user's skeleton point is obtained, and the similarity between the action of the user's skeleton point and the standard action is calculated, and a new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold. If the similarity is not less than the new similarity threshold, the user action is determined to be qualified, otherwise, the user action is determined to be unqualified.

3. The system according to claim 2, characterized in that The new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold, specifically: Thr=k·(1+w1)·Thr', wherein Thr is the new similarity threshold, k is the difficulty, and Thr' is the standard similarity threshold.

4. The system according to any one of claims 1 to 3, characterized in that: The user color image is a color image that only contains the user; the user depth image is a depth image that only contains the user.

5. A non-volatile computer storage medium having instructions stored therein, characterized in that: When the instruction is executed on the processor, it causes the processor to perform the following steps: S1, adjust the synchronization of the color image and the depth image, obtain the user color image, and convert the user color image into a first grayscale image; obtain the user depth image corresponding to the user color image, and convert the user depth image into a second grayscale image; obtain the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image as well as the user's height and gender; S2, a training action judgment module, obtains the difficulty level input by the user, and calculates the training parameter weight corresponding to the user according to the user portrait, and judges whether the user action is qualified according to the difficulty level, the training parameter weight, the skeleton point position information and the standard action; The method of obtaining the user portrait according to the Neck width, WristRight width, WristLeft width at the Neck, WristRight, and WristLeft skeleton points in the first grayscale image and the second grayscale image as well as the height and gender of the user is specifically as follows: executing on the first grayscale image and the second grayscale image respectively: obtaining the straight line L1 between the Neck and Head skeleton points, obtaining the line width perpendicular to the straight line L1 at the Neck skeleton point, and obtaining the Neck width; obtaining the straight line L2 between the WristRight and ElboWrist skeleton points, obtaining the line width perpendicular to the straight line L2 at the WristRight skeleton point, and obtaining the WristRight width; obtaining the straight line L3 between the WristLeft and ElbowLeft skeleton points, obtaining the line width perpendicular to the straight line L3 at the WristLeft skeleton point, and obtaining the WristLeft width; taking the average value of the Neck width in the first grayscale image and the Neck width in the second grayscale image as the user Neck width; taking the average value of the WristRight width in the first grayscale image and the WristRight width in the second grayscale image as the user WristRight width; The average of the WristLeft width in the first grayscale image and the WristLeft width in the second grayscale image is used as the WristLeft width of the user; the height of the user is obtained through the Head skeleton point, and the height, gender, neck width, wristright width and wristleft width of the user are used as user portrait parameters; The training parameter weights corresponding to the user are calculated based on the user portrait, specifically: according to the height and gender of the user, the standard values ​​of the user's neck width, the user's wrist right width, and the user's wrist left width corresponding to the height and gender are obtained; if the user neck +User wristright +User wristleft >S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user If User neck +User wristright +User wristleft =S neck +S wristright +S wristleft , then the training parameter weight w1 corresponding to the user is 0; if User neck +User wristright +User wristleft <S neck +S wristright +S wristleft , then the training parameter weight corresponding to the user Among them, User neck is the user neck width, User wristright is the width of the user WristRight, User wristleft is the user WristLeft width, S neck is the standard value of the user's neck width, S wristright is the standard value of the user's WristRight width, S wristleft It is the standard value of the user's WristLeft width.

6. The non-volatile computer storage medium of claim 5, wherein: The determining whether the user's action is qualified according to the difficulty, the training parameter weight, the skeleton point position information and the standard action is specifically as follows: The action of the user's skeleton point is obtained, and the similarity between the action of the user's skeleton point and the standard action is calculated, and a new similarity threshold is calculated according to the difficulty, the training parameter weight, and the standard similarity threshold. If the similarity is not less than the new similarity threshold, the user action is determined to be qualified, otherwise, the user action is determined to be unqualified.