Taijiquan rehabilitation training action recognition method and system based on VR

Abstract

The invention belongs to the technical field of limb action recognition, and particularly relates to a shadowboxing rehabilitation training action recognition method based on VR. Comprising the following steps: S1, importing standard shadowboxing guidance actions into a VR system, and constructing shadowboxing movement data; s2, wearing pose sensors on a plurality of limb links of the user, and collecting spatial pose data of each bone joint of the user in the shadowboxing exercise; and S3, comparing the spatial pose data of the user with standard Tai Chi exercise data in a VR system, monitoring the Tai Chi exercise process of the user in real time, identifying the user actions, and correcting errors existing in the exercise process in time. According to the recognition method, the poses of all the bone joints of the user in the shadowboxing exercise process can be accurately recognized, the poses are compared with standard shadowboxing guidance actions, the non-standard actions of the user are corrected, and therefore the rehabilitation effect of exercise and the enthusiasm of the user are improved. The invention further discloses a shadowboxing rehabilitation training action recognition system based on VR.

Description

Technical Field

[0001] The present invention belongs to the technical field of limb movement recognition, and particularly relates to a method and system for recognizing Tai Chi rehabilitation training movements based on VR. Background Technique

[0002] Tai Chi is a traditional sports project in China. Its movements are slow and gentle. Practicing Tai Chi can make the meridians of middle-aged and elderly people, chronic disease groups and other people smooth, the metabolism vigorous, and the physique and function enhanced.

[0003] Tai Chi has a certain curative effect on treating KOA (Knee Osteoarthritis, abbreviated as KOA, which is a chronic degenerative disease mainly characterized by articular cartilage degeneration, secondary osteophyte formation, subchondral bone sclerosis, synovial inflammation, etc.). Due to problems such as high cost and site requirements for professional medical personnel to provide on-site real-time guidance for Tai Chi training, most patients need long-term home rehabilitation training. However, improper movements during Tai Chi training may exacerbate symptoms, and there is currently a lack of real-time guidance for KOA patients during Tai Chi practice. Therefore, there is an urgent need to establish a set of Tai Chi rehabilitation training methods and systems to monitor the process of KOA patients practicing Tai Chi in real time and correct the errors existing in the training process in a timely manner, so as to give full play to the rehabilitation benefits of Tai Chi. Summary of the Invention

[0004] The purpose of the present invention is to address the above-mentioned existing technical problems and provide a method for recognizing Tai Chi rehabilitation training movements based on VR, which can accurately recognize the poses of each bone joint of the user during Tai Chi exercise, compare them with the standard Tai Chi guiding movements, and correct the user's improper movements to improve the rehabilitation effect and user enthusiasm of the exercise.

[0005] In view of this, the present invention provides a method for recognizing Tai Chi rehabilitation training movements based on VR, including the following steps:

[0006] S1. Import the standard Tai Chi guiding movements into the VR system to construct Tai Chi motion data;

[0007] S2. Wear pose sensors on multiple limb segments of the user to collect the spatial pose data of each bone joint of the user during Tai Chi exercise;

[0008] S3. Compare the spatial pose data of the user with the standard Tai Chi motion data in the VR system, monitor the process of the user's Tai Chi exercise in real time, recognize the user's movements, and correct the errors existing in the exercise process in a timely manner.

[0009] In this technical solution, the pose sensor collects the spatial pose data of each bone joint of the user during Tai Chi exercise, compares it with the standard Tai Chi exercise data in the VR system, monitors the user's Tai Chi exercise process in real time, identifies the user's movements, and corrects the existing errors in the exercise process in a timely manner, so as to give full play to the rehabilitation benefits of Tai Chi for KOA patients, reduce the need for on-site guidance by professional medical staff, and reduce medical costs.

[0010] Further, in step S2, pose sensors are respectively worn on the user's forehead, the 10th thoracic vertebra, the sacrum, the upper arm, the forearm, the thigh, the calf, and the dorsum of the foot, for analyzing and obtaining the angles of the wrist joint, elbow joint, shoulder joint, head and neck, trunk, pelvis, hip joint, knee joint, and ankle joint.

[0011] Further, in step S1, the VR system is connected to the PC end through a data cable;

[0012] In step S2, the spatial pose data of each bone joint is processed by the Kalman filtering algorithm and then imported into the PC end.

[0013] Further, the PC end is wirelessly connected to the pose sensor. The PC end first checks whether the data is connected, then calibrates the spatial pose data, performs angle data conversion, then performs secondary calibration on the spatial pose data, and finally imports the angle data into the VR system.

[0014] Further, the VR system is the UNITY3D game engine. First, a virtual character is imported into the UNITY3D game engine, and the moving limb bones are bound. Then, the synchronous movement postures are checked. Finally, the movement synchronization between the user wearing pose sensors on multiple limb links and the virtual character is realized;

[0015] By comparing the actions of the virtual character with the standard Tai Chi guiding actions, the actions of the user are identified, and the user can correct the existing errors in the exercise process in a timely manner;

[0016] The pose sensor is connected to the PC end through Bluetooth or wireless network for data transmission.

[0017] Further, virtual objects, sound effects, and textures are imported into the UNITY3D game engine for constructing different virtual scenes.

[0018] Further, interactive game logic is designed and interactive logic code is written in the UNITY3D game engine to construct a virtual interaction mode and create a multi-scene virtual interaction game.

[0019] The beneficial effects of the present invention are:

[0020] 1. The pose sensor collects the spatial pose data of each bone joint of the user during Tai Chi exercise, compares it with the standard Tai Chi exercise data in the VR system, monitors the user's Tai Chi exercise process in real time, identifies the user's movements, and corrects the existing errors in the exercise process in a timely manner, so as to give full play to the rehabilitation benefits of Tai Chi for KOA patients, reduce the need for on-site guidance by professional medical staff, and reduce medical costs.

[0021] 2. Pose sensors are respectively worn on the user's forehead, 10th thoracic vertebra, sacrum, upper arm, forearm, thigh, calf, and dorsal foot to analyze and obtain the angles of the wrist joint, elbow joint, shoulder joint, head and neck, trunk, pelvis, hip joint, knee joint, and ankle joint, covering the positions of the bone joints involved in the Tai Chi exercise process, and can accurately identify the poses of each bone joint of the user during the Tai Chi exercise, evaluate the completion of the rehabilitation training task and give early warning of danger, and avoid joint injuries caused by irregular Tai Chi exercise, which may further deteriorate the condition of KOA patients and increase pain, resulting in the patient giving up Tai Chi exercise.

[0022] 3. The Kalman filter algorithm is used to process the spatial pose data collected by the pose sensor to improve the accuracy of pose action recognition.

[0023] 4. A large number of patient training samples are collected through the PC side, a suitable neural network algorithm is constructed for training, the classification algorithm is optimized and improved according to the training effect, and the feature vector suitable for rehabilitation behavior recognition is found.

[0024] 5. The UNITY3D game engine can build different virtual scenes and interaction methods, create multi-scene virtual interaction games, and increase the fun, interactivity and immersive experience of rehabilitation training.

[0025] The present invention also provides a VR-based Tai Chi rehabilitation training action recognition system, including:

[0026] A VR system for importing standard Tai Chi guiding actions and constructing Tai Chi exercise data;

[0027] A pose sensor for wearing on multiple limb segments of the user to collect the spatial pose data of each bone joint of the user during Tai Chi exercise;

[0028] A PC side for communicating and processing spatial pose data.

[0029] The VR system is connected to the PC side through a data cable, and the VR system is the UNITY3D game engine;

[0030] The pose sensor is connected to the PC via Bluetooth or wireless network for data transmission. The pose sensors are respectively worn on the user's forehead, 10th thoracic vertebra, sacrum, upper arm, forearm, thigh, calf, and dorsum of the foot for analyzing and obtaining the angles of the wrist joint, elbow joint, shoulder joint, head and neck, trunk, pelvis, hip joint, knee joint, and ankle joint.

[0031] The pose sensors are worn on each limb segment of the user through connecting straps. The pose sensors are used to output the position and angle data of the user in the three-dimensional space in real time, and the pose sensors include a three-axis gyroscope and a three-axis accelerometer. Description of the Drawings

[0032] Figure 1 is a schematic diagram of the steps of a Tai Chi rehabilitation training action recognition method based on VR in the present invention;

[0033] Figure 2 is a detailed schematic diagram of the steps of a Tai Chi rehabilitation training action recognition method based on VR in the present invention;

[0034] Figure 3 is a structural block diagram of a Tai Chi rehabilitation training action recognition system based on VR in the present invention;

[0035] The markings in the figure are shown as:

[0036] 1. VR system; 2. Pose sensor; 3. PC; 4. Data cable; 5. UNITY3D game engine; 6. Bluetooth or wireless network; 7. Wrist joint; 8. Elbow joint; 9. Shoulder joint; 10. Head and neck joint; 11. Trunk; 12. Hip joint; 13. Knee joint; 14. Ankle joint; 15. Three-axis gyroscope; 16. Three-axis accelerometer; 17. Pelvis. Detailed Embodiments

[0037] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0038] In the description of the present application, it should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments of the present application. For the convenience of description, the dimensions of the various parts shown in the drawings are not drawn according to the actual proportional relationship. Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the said technologies, methods, and devices should be regarded as part of the authorization specification. In all the examples shown and discussed here, any specific value should be construed as merely exemplary and not as a limitation. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0039] Embodiment 1:

[0040] This embodiment provides a method for recognizing Tai Chi rehabilitation training actions based on VR, including the following steps:

[0041] S1. Import standard Tai Chi guiding actions into VR system 1 to construct Tai Chi motion data;

[0042] S2. Wear pose sensors 2 on multiple limb joints of the user to collect the spatial pose data of each bone joint of the user during Tai Chi exercise;

[0043] S3. Compare the spatial pose data of the user with the standard Tai Chi motion data in VR system 1, supervise the user's Tai Chi exercise process in real time, recognize the user's actions, and correct the existing errors in the exercise process in a timely manner.

[0044] In this embodiment, the pose sensors 2 collect the spatial pose data of each bone joint of the user during Tai Chi exercise, compare it with the standard Tai Chi motion data in VR system 1, supervise the user's Tai Chi exercise process in real time, recognize the user's actions, and correct the existing errors in the exercise process in a timely manner, so as to give full play to the rehabilitation benefits of Tai Chi for KOA patients, reduce the need for on-site guidance by professional medical staff, and reduce medical costs.

[0045] Embodiment 2:

[0046] This embodiment provides a method for recognizing Tai Chi rehabilitation training actions based on VR. In addition to including the technical solutions of the above embodiment, it also has the following technical features.

[0047] In S2, pose sensors 2 are respectively worn at the user's forehead, 10th thoracic vertebra, sacrum, upper arm, forearm, thigh, calf, and dorsum of the foot, for analyzing and obtaining the angles of the wrist joint 7, elbow joint 8, shoulder joint 9, head and neck 10, trunk 11, pelvis 17, hip joint 12, knee joint 13, and ankle joint 14.

[0048] In this embodiment, the angles of the joints need to be calculated through two adjacent limbs. Therefore, pose sensors 2 are respectively worn at the user's forehead, 10th thoracic vertebra, sacrum, upper arm, forearm, thigh, calf, and dorsum of the foot, covering the bone joint positions involved in the Tai Chi exercise process, which will not affect joint movement, can accurately identify the poses of each bone joint of the user during the Tai Chi exercise process, evaluate the completion of the rehabilitation training task and give early warnings of danger, avoid joint injuries caused by non-standard Tai Chi exercises, prevent the further deterioration of KOA patients and the aggravation of pain, so that patients will give up Tai Chi exercises.

[0049] Embodiment 3: This embodiment provides a VR-based Tai Chi rehabilitation training action recognition method. In addition to including the technical solutions of the above embodiments, it also has the following technical features.

[0050] In S1, the VR system 1 is connected to the PC terminal 3 through the data cable 4;

[0051] In S2, the spatial pose data of each bone joint is processed by the Kalman filter algorithm and then imported into the PC terminal 3.

[0052] In this embodiment, the Kalman filter algorithm is used to process the spatial pose data collected by the pose sensor 2 to improve the accuracy of pose action recognition.

[0053] Embodiment 4: This embodiment provides a VR-based Tai Chi rehabilitation training action recognition method. In addition to including the technical solutions of the above embodiments, it also has the following technical features.

[0054] The PC terminal 3 and the pose sensor 2 are wirelessly connected. The PC terminal 3 first checks whether the data is connected, then calibrates the spatial pose data, performs angle data conversion, then performs secondary calibration on the spatial pose data, and finally imports the angle data into the VR system 1.

[0055] In this embodiment, a large number of patient training samples are collected through the PC terminal 3, a suitable neural network algorithm is constructed for training, the classification algorithm is optimized and improved according to the training effect, and the feature vector suitable for rehabilitation behavior recognition is found.

[0056] Embodiment 5: This embodiment provides a VR-based Tai Chi rehabilitation training action recognition method. In addition to including the technical solutions of the above embodiments, it also has the following technical features.

[0057] The VR system 1 is a UNITY3D game engine 5. First, a virtual character is imported into the UNITY3D game engine 5, and the skeleton of the moving limbs is bound. Then, the synchronous motion posture is checked, and finally, the user wearing multiple limbs and posture sensors 2 and the virtual character are synchronized in motion.

[0058] By comparing the movements of the virtual character with the standard Tai Chi instruction movements, the user's movements can be recognized, and the user can correct the mistakes in the exercise process in time;

[0059] The posture sensor 2 is connected to the PC terminal 3 via Bluetooth or a wireless network 6 for transmitting data.

[0060] Virtual objects, sound effects and textures are imported into the UNITY 3D game engine 5 to construct different virtual scenes.

[0061] The interactive game logic is designed and the interactive logic code is written in the UNITY3D game engine 5 to build a virtual interactive mode to create a multi-scene virtual interactive game.

[0062] In this embodiment, the UNITY3D game engine 5 can construct different virtual scenes and interaction methods, and can create multi-scene virtual interactive games to increase the fun, interactivity and immersive experience of rehabilitation training.

[0063] Embodiment 6:

[0064] The present invention also discloses a VR-based Tai Chi rehabilitation training action recognition system, comprising:

[0065] VR system 1, used to import standard Tai Chi instruction movements and construct Tai Chi movement data;

[0066] Posture sensor 2, used to be worn on multiple limbs of the user to collect spatial posture data of each bone joint of the user during Tai Chi exercise;

[0067] PC side 3 is used for communication and processing of spatial posture data.

[0068] The VR system 1 is connected to a PC terminal 3 via a data cable 4, and the VR system 1 is a UNITY 3D game engine 5;

[0069] The posture sensor 2 is connected to the PC 3 via Bluetooth or a wireless network 6 for transmitting data, and the posture sensor 2 is worn on the user's forehead, 10th thoracic vertebra, sacrum, upper arm, forearm, thigh, calf, and dorsum of the foot for analyzing and obtaining the angles of the wrist joint 7, elbow joint 8, shoulder joint 9, head and neck 10, trunk 11, pelvis 17, hip joint 12, knee joint 13, and ankle joint 14.

[0070] The pose sensor 2 is worn on each limb segment of the user through a connecting strap. The pose sensor 2 is used to output the position and angle data of the user in the three-dimensional space in real time, and the pose sensor 2 includes a three-axis gyroscope 15 and a three-axis accelerometer 16.

[0071] The embodiments of the present application have been described above in conjunction with the accompanying drawings. Without conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. A VR-based Tai Chi rehabilitation training motion recognition method, characterized in that: The following steps are involved: S1, importing standard Tai Chi instruction movements into the VR system (1) to construct Tai Chi movement data; S2, wearing posture sensors (2) on multiple limb segments of the user to collect spatial posture data of each bone joint of the user during Tai Chi exercise; S3. Compare the user's spatial posture data with the standard Tai Chi movement data in the VR system (1), monitor the user's Tai Chi exercise process in real time, identify the user's movements, and correct errors in the exercise process in a timely manner.

2. The VR-based Tai Chi rehabilitation training motion recognition method according to claim 1, characterized in that: In the S2, posture sensors (2) are respectively worn on the user's forehead, 10th thoracic vertebra, sacrum, upper arm, forearm, thigh, calf, and dorsum of the foot to analyze and obtain the angles of the wrist joint (7), elbow joint (8), shoulder joint (9), head and neck (10), trunk (11), pelvis (17), hip joint (12), knee joint (13), and ankle joint (14).

3. The VR-based Tai Chi rehabilitation training motion recognition method according to claim 2 is characterized in that: The VR system (1) in S1 is connected to the PC terminal (3) via a data cable (4); The spatial posture data of each bone joint in S2 is processed by the Kalman filter algorithm and then imported into the PC (3).

4. The VR-based Tai Chi rehabilitation training motion recognition method according to claim 3 is characterized in that: The PC end (3) is connected to the posture sensor (2) via wireless. The PC end (3) first checks whether the data is connected, then calibrates the spatial posture data, performs angle data conversion, then performs secondary calibration of the spatial posture data with synchronous motion data, and finally imports the angle data into the VR system (1).

5. The method for recognizing Tai Chi rehabilitation training movements based on VR according to claim 4, characterized in that: The VR system (1) is a UNITY3D game engine (5), which first imports a virtual character into the UNITY3D game engine (5), binds the moving limb skeleton, and then checks the synchronous movement posture, and finally realizes the movement synchronization between a user wearing multiple limb position sensors (2) and the virtual character; By comparing the movements of the virtual character with the standard Tai Chi instruction movements, the user's movements can be recognized, and the user can correct the mistakes in the exercise process in time; The posture sensor (2) is connected to the PC (3) via Bluetooth or a wireless network (6) for data transmission.

6. The method for recognizing Tai Chi rehabilitation training movements based on VR according to claim 5, characterized in that: Virtual objects, sound effects and textures are imported into the UNITY 3D game engine (5) to construct different virtual scenes.

7. The VR-based Tai Chi rehabilitation training motion recognition method according to claim 6, characterized in that: The interactive game logic is designed and the interactive logic code is written in the UNITY3D game engine (5) to build a virtual interactive mode to create a multi-scene virtual interactive game.

8. A VR-based Tai Chi rehabilitation training motion recognition system, characterized in that: include: VR system (1), used to import standard Tai Chi instruction movements and construct Tai Chi movement data; Position and posture sensors (2) are used to be worn on multiple limbs of the user to collect spatial position and posture data of each bone and joint of the user during Tai Chi exercise; PC side (3), used for communication and processing of spatial posture data.

9. A VR-based Tai Chi rehabilitation training motion recognition system according to claim 8, characterized in that: The VR system (1) is connected to the PC end (3) via a data cable (4), and the VR system (1) is a UNITY 3D game engine (5); The posture sensor (2) is connected to the PC (3) via Bluetooth or a wireless network (6) for data transmission, and the posture sensor (2) is respectively worn on the user's forehead, the tenth thoracic vertebra, the sacrum, the upper arm, the forearm, the thigh, the calf, and the dorsum of the foot for analyzing and obtaining the angles of the wrist joint (7), the elbow joint (8), the shoulder joint (9), the head and neck (10), the trunk (11), the pelvis (17), the hip joint (12), the knee joint (13), and the ankle joint (14).

10. A VR-based Tai Chi rehabilitation training motion recognition system according to claim 9, characterized in that: The posture sensor (2) is worn on each limb of the user through a connecting strap, and the posture sensor (2) is used to output the position and angle data of the user in three-dimensional space in real time, and the posture sensor (2) includes a three-axis gyroscope (15) and a three-axis accelerometer (16).

Images