Burn patient rehabilitation training method and system based on virtual reality

Abstract

The invention discloses a burn patient rehabilitation training method and system based on virtual reality. The method comprises the steps that information of a to-be-trained patient is acquired; the information of the patient to be trained is input into a rehabilitation training model, a rehabilitation training result is obtained, the rehabilitation training model is constructed through a Kinect V2 module and a Leap Motion module, the Kinect V2 module is used for capturing human skeleton points, and the Leap Motion module is used for recognizing hand postures; and in combination with a historical training result, evaluating the rehabilitation training result, obtaining stage rehabilitation suggestions, and completing the current training. The limb rehabilitation training system has the advantages of improving the rehabilitation training efficiency, effect, safety, individuation and the like, and has important significance and application value for limb rehabilitation training.

Description

Technical Field

[0001] The present invention belongs to the technical field of virtual reality, and in particular relates to a rehabilitation training method and system for burn patients based on virtual reality. Background Art

[0002] With the aggravation of aging in our country, the number of patients paralyzed due to stroke, hypertension, etc. is increasing continuously. Accidents, safety incidents, etc. are also a major factor leading to the loss of limb function. It is very necessary to carry out rehabilitation training for these patients.

[0003] The hand is an important organ of the human body, with a delicate structure, complex functions, and is often exposed, making it extremely vulnerable to injury. The incidence of hand burns accounts for 45%-50% of all burns. Especially for burns above deep second degree, it often leads to hand scars, contractures, deformities and functional disorders, restricting joint activities, which not only affects aesthetics but also hand functions. It causes many problems for patients in terms of body, psychology, economy, society and emotion. The hand function accounts for 57% of the individual's functions, and most delicate movements are mainly completed by the hand. The quality of hand function rehabilitation is directly related to the later daily living ability, work, study and quality of life. Therefore, the rehabilitation exercise of hand function is of vital importance.

[0004] Therapeutic exercise is an important means for the rehabilitation treatment of hand burn patients. Under the traditional method, patients complete training through finger flexion, extension and finger separation, etc. Most hospitals use manual measurement methods to evaluate the rehabilitation degree of patients, but only use a protractor to measure the range of motion of patients. The measurement is inaccurate and the operation is complex. The test time is up to 1 hour, and then through various types of evaluation scales, the hand function is evaluated with low accuracy.

[0005] Gesture recognition is a technology for recording the position of the palm and finger movements. Certain progress has been made at home and abroad. Ye Sufen proposed an algorithm based on a particle swarm optimization hybrid kernel function weighted extreme learning machine to solve the problem of poor recognition performance of traditional gesture recognition algorithms for unbalanced gesture data. Yuyang Fu et al. proposed a long-distance gesture recognition algorithm based on Kinect.

[0006] In recent years, gesture recognition technology has been used in hand function rehabilitation training. Among them, the comfort and adaptability of wearable gloves for measurement are not high, and there is also a risk of cross-infection and the possibility of blocking the movement of the subject's hand, especially in the application of burn surgery. Zhang Nieqiang tried to design a gamified hand rehabilitation product for stroke patients. Based on games such as the running game, it was designed in the form of contact hardware such as thin-film pressure-sensing finger sleeves and spring impedance. The contact hardware device is not friendly to the rehabilitation training of hand burn patients. In non-wearable gesture recognition, Zheng Yong et al. designed an augmented reality natural hand rehabilitation interaction system based on Kinect, in which trajectory training and interaction training focused more on the exercise of the upper limb part. Li Weibin designed a natural hand interaction rehabilitation system based on augmented reality. Based on the design of Kinect, the accuracy is not enough to obtain the detailed shape information of the hand.

[0007] The design scheme of the hand burn rehabilitation training system has been slightly involved in China, but the actual products integrating software and hardware are not seen in the market. The schemes of Zhang Nieqiang, Li Weibin, Zhao Yifei, etc. are all in the design stage, and there is no mention of establishing a hand burn rehabilitation training database and conducting personalized quantitative analysis. Among similar products, the rehabilitation training robots are costly, have fixed application places (hospitals / rehabilitation institutions), and the number of applications is extremely small. Summary of the Invention

[0008] To solve the above technical problems, the present invention proposes a virtual reality-based rehabilitation training method and system for burn patients, aiming to adapt to various medical environments and be applicable to the rehabilitation training of burn patients, disabled people, people with motor function disorders, and the recovery of cerebral palsy functions, etc.

[0009] The present invention provides a virtual reality-based rehabilitation training method for burn patients, including:

[0010] Obtaining patient information to be trained;

[0011] Inputting the patient information to be trained into a rehabilitation training model to obtain a rehabilitation training result, wherein the rehabilitation training model is constructed by a Kinect V2 module and a Leap Motion module, the Kinect V2 module is used to capture human bone points, and the Leap Motion module is used to recognize hand postures;

[0012] Combining historical training results to evaluate the rehabilitation training result to obtain stage rehabilitation suggestions and complete the current training.

[0013] Optionally, inputting the patient information to be trained into the rehabilitation training model to obtain a rehabilitation training result includes:

[0014] Setting indicators and parameters for rehabilitation training according to the patient information to be trained;

[0015] Obtain a rehabilitation training plan according to the said indicators and parameters;

[0016] The patient trains according to the said rehabilitation training plan, automatically records the training data, and obtains the rehabilitation training result.

[0017] Optionally, obtaining the rehabilitation training plan includes:

[0018] Use the Leap Motion module to recognize hand postures and formulate 13 kinds of hand rehabilitation training actions;

[0019] Use the Kinect V2 module to capture 25 skeletal points of the human body and formulate 23 kinds of limb rehabilitation training actions;

[0020] Obtain a rehabilitation training plan according to the said hand rehabilitation training actions and limb rehabilitation training actions.

[0021] Optionally, the patient's training according to the said rehabilitation training plan includes:

[0022] Determine the rehabilitation training type according to the said rehabilitation training plan;

[0023] Conduct different mode training according to different parts of the said rehabilitation training type.

[0024] Optionally, the said rehabilitation training type includes: hand training, neck training, shoulder joint training, trunk training, elbow joint training, and hip joint training.

[0025] The present invention also provides a virtual reality-based rehabilitation training system for burn patients, including: a display module, an interaction module, a control module, a storage module, and a mechanical module;

[0026] The said display module is used to display virtual reality scenes and limb rehabilitation training tasks;

[0027] The said interaction module is used to collect the user's actions;

[0028] The said control module is used to control the actions according to the rehabilitation training tasks;

[0029] The said storage module is used to store training data, virtual reality scenes, and data of limb rehabilitation training tasks;

[0030] The said mechanical module is used to integrate an all-in-one machine and peripheral devices to provide a basis for displaying virtual reality scenes.

[0031] Optionally, the said interaction module includes: a hand recognition unit and a limb recognition unit;

[0032] The hand recognition unit is used to recognize the hand movements of the patient by using Leap Motion;

[0033] The limb recognition unit is used to capture the limb movements of the patient by using Kinect V2.

[0034] Compared with the prior art, the present invention has the following advantages and technical effects:

[0035] 1. Improve the efficiency of rehabilitation training: Through virtual simulation technology, the limb rehabilitation training evaluator can provide users with more real and vivid training scenarios and tasks, thereby improving the training interest and enthusiasm of users, and further improving the efficiency of rehabilitation training.

[0036] 2. Improve the effect of rehabilitation training: Through limb recognition algorithms and mechanical technologies, the integrated limb rehabilitation training machine can monitor the limb movements and rehabilitation training effects of users in real time, and automatically adjust the rehabilitation training tasks and difficulties according to the rehabilitation training needs and progress of users, thereby improving the effect of rehabilitation training.

[0037] 3. Improve the safety of rehabilitation training: Through mechanical technologies, the integrated limb rehabilitation training machine can provide users with a more stable and safe training environment, thereby reducing the risks and discomfort of users during the rehabilitation training process.

[0038] 4. Realize personalized rehabilitation training: Through virtual simulation technology and limb movement recognition algorithms, personalized rehabilitation training programs can be provided for users according to their limb movements and rehabilitation training needs, thereby realizing precise and effective rehabilitation training. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application. In the drawings:

[0040] Figure 1 is a flowchart of a virtual reality-based rehabilitation training method for burn patients according to an embodiment of the present invention;

[0041] Figure 2 is an external view of the integrated rehabilitation training machine according to an embodiment of the present invention, wherein 1, built-in host; 2, screen; 3, camera; 4, material; 5, gesture recognition device; 6, limb recognition device; 7, external interface; 8, decorative lamp;

[0042] Figure 3 are the side view and top view of the integrated rehabilitation training machine according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The following will describe the present application in detail with reference to the accompanying drawings and in combination with the embodiments.

[0044] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0045] The present invention proposes a rehabilitation training method for burn patients based on virtual reality, as Figure 1 shown, which specifically includes the following steps:

[0046] Obtain the information of the patient to be trained;

[0047] Input the information of the patient to be trained into the rehabilitation training model to obtain the rehabilitation training result, where the rehabilitation training model is constructed by a Kinect V2 module and a Leap Motion module. The Kinect V2 module is used to capture human bone points, and the Leap Motion module is used to recognize hand postures;

[0048] Combine the historical training results to evaluate the rehabilitation training result, obtain the stage rehabilitation suggestions, and complete the current training.

[0049] Specifically, after the user starts the rehabilitation training all-in-one machine, first log in to the personal account or create a new file through the high-definition touch display screen. The user can select from the preset rehabilitation courses. Once the training mode is selected, the user can follow the action guidance displayed on the screen for interactive training. During this period, the Kinect V2 and Leap Motion sensors capture the user's whole body and hand movements in real time to ensure the accuracy of the movements. After each training session, the system automatically saves all the data for doctors to refer to and analyze.

[0050] Furthermore, inputting the information of the patient to be trained into the rehabilitation training model to obtain the rehabilitation training result includes:

[0051] Set the indicators and parameters of the rehabilitation training according to the information of the patient to be trained;

[0052] Obtain the rehabilitation training plan according to the indicators and parameters;

[0053] The patient trains according to the rehabilitation training plan, and automatically records the training data to obtain the rehabilitation training result.

[0054] Specifically, after the patient enters the rehabilitation training stage, system registration is carried out, a personal file is established according to the rehabilitation type, and the indicators and parameters of rehabilitation training (rehabilitation site, muscle strength, range, speed, etc.) are set to form a personalized database.

[0055] Through a customized rehabilitation training plan, the patient conducts targeted rehabilitation training, and the system automatically records the training data according to the established rehabilitation training parameters to form a summary table.

[0056] Based on the results of the stage rehabilitation training, the system forms a preliminary evaluation report, combines it with medical advice, optimizes the rehabilitation plan in a timely manner, formulates a personalized rehabilitation course, and adjusts the training difficulty according to the user's real-time performance to ensure the effectiveness and safety of the training.

[0057] Furthermore, obtaining the rehabilitation training plan includes:

[0058] Using the Leap Motion module to recognize hand postures, 13 kinds of hand rehabilitation training actions are formulated;

[0059] Using the Kinect V2 module to capture 25 human bone points, 23 kinds of limb rehabilitation training actions are formulated;

[0060] According to the hand rehabilitation training actions and limb rehabilitation training actions, the rehabilitation training plan is obtained.

[0061] Specifically, 13 basic hand rehabilitation training simulation models are established, and based on this model, three-level training modes of primary, advanced, and comprehensive are formed to conduct three-dimensional and personalized training for patients in different rehabilitation stages.

[0062] 5 categories of 23 kinds of limb rehabilitation training simulation models are established, and based on this model, two modes of basic action training and advanced comprehensive training are formed to conduct three-dimensional and personalized training for patients in different rehabilitation stages

[0063] Furthermore, the patient's training according to the rehabilitation training plan includes:

[0064] According to the rehabilitation training plan, determine the rehabilitation training type;

[0065] Carry out different mode training according to different parts of the rehabilitation training type.

[0066] Furthermore, the rehabilitation training types include: hand training, neck training, shoulder joint training, trunk training, elbow joint training, and hip joint training.

[0067] The present invention also provides a virtual reality-based rehabilitation training system for burn patients, including: a display module, an interaction module, a control module, a storage module, and a mechanical module;

[0068] A display module, used to display virtual reality scenarios and limb rehabilitation training tasks;

[0069] An interaction module, used to collect the user's actions;

[0070] A control module, used to control actions according to rehabilitation training tasks;

[0071] A storage module, used to store training data, virtual reality scenarios, and data on limb rehabilitation training tasks;

[0072] A mechanical module, used to integrate the all-in-one machine and peripheral devices, providing a basis for displaying virtual reality scenarios.

[0073] Furthermore, the interaction module includes: a hand recognition unit and a limb recognition unit;

[0074] A hand recognition unit, used to recognize the patient's hand movements using Leap Motion;

[0075] A limb recognition unit, used to capture the patient's limb movements using Kinect V2.

[0076] Specifically, the training system includes: 1. A display module: used to display virtual reality scenarios and limb rehabilitation training tasks.

[0077] 2. An interaction module: used to collect the user's limb movements and feedback them to the control module.

[0078] 3. A storage module: used to store data on virtual reality scenarios and limb rehabilitation training tasks.

[0079] 4. A power supply module: used to supply power to the entire device.

[0080] 5. A hand recognition module: used to recognize the user's hand movements and feedback them to the control module.

[0081] 6. A mechanical module: used to integrate the all-in-one machine and peripheral devices.

[0082] As Figures 2-3 shown, the mechanical module includes: A main body frame: The rehabilitation training all-in-one machine is made of durable aluminum alloy material, providing a stable support platform, and its design takes into account both aesthetics and functionality.

[0083] A high-definition touch screen: Located in the center of the front of the device, it is the main interface for user interaction, supports multi-touch operations, and is convenient for patients and medical staff to use.

[0084] A Kinect V2 sensor: Installed below the screen, used to capture the user's full-body motion data and provide important input information for the system.

[0085] Leap Motion Controller: Located below the screen, it focuses on high-precision tracking of hand movements and complements the functions of the Kinect V2.

[0086] Omnidirectional wheels: The omnidirectional wheels with a locking function at the bottom make the device easy to move and can be firmly fixed in the desired position.

[0087] Embedded computer: As the core processing unit, it runs customized software, is responsible for processing data from various sensors, and drives output devices. The chip is the Intel i5 7th generation, 4 + 128G, with a 1650 graphics card slot. It is used for overall data processing and running Unity. It has a built-in SSD hard drive to store all user data, training history records, and the latest software updates, ensuring data security and access speed. The network interface enables a wired connection between the device and the outside world through the motherboard network port, facilitating data synchronization.

[0088] Sensor bracket: Provides stable support for the Kinect V2 and Leap Motion, ensuring the accuracy of the sensors.

[0089] As Figure 2 shown, the all-in-one device includes: built-in host 1, screen 2, camera 3, material 4, gesture recognition device 5, body recognition device 6, external interface 7, decorative lamp 8. Among them, the built-in host 1 is an i5 7th generation with a 4 + 128G independent graphics card; the screen 2 is a 43-inch capacitive touch all-in-one machine; the material 4 adopts the cold-rolled steel plate + plastic spraying process; the size of the gesture recognition device 5 is 80x30x10mm; the size of the body recognition device 6 is: 250x60x85mm; the external interface 7 is the device power switch, start switch, USB, network port.

[0090] The system uses a high-definition touch display screen and an audio system to provide users with instant guidance and corrective suggestions visually and auditorily, enhancing interactivity and user engagement. The built-in data storage unit and communication module allow automatic saving of training data and generation of reports, and at the same time support doctors to remotely monitor the rehabilitation progress of patients through the network platform, improving the efficiency of medical services.

[0091] The following elaborates on this embodiment in detail:

[0092] In the process of implementing this rehabilitation training system, in this embodiment, the Unity editor was first installed and necessary components were configured to ensure that the development environment could support it. To ensure hardware compatibility, in this embodiment, according to the official Kinect V2 and Leap Motion driver programs, it was ensured that the Windows operating system could correctly recognize and configure these two sensor devices. In addition, the documentation of the two SDKs was carefully studied to understand their functions, limitations, and best practices, laying a solid foundation for subsequent integration work. Then, a new project was created in Unity, and a basic 3D or 2D scene was built, including elements such as the user interface (UI) and camera perspective. To simulate a real rehabilitation environment, in this embodiment, different customized game scenes were added to the scene, enabling patients to experience a more interesting rehabilitation experience in the virtual environment. At the same time, in this embodiment, an intuitive and easy-to-operate user interface was designed to ensure that both patients and medical staff could easily get started.

[0093] To integrate the functions of Kinect V2 and Leap Motion, their respective Unity plugin packages were imported into the project. Initialization scripts were written to ensure that the sensors were correctly activated at startup, and the APIs provided by the SDKs were used to capture the user's motion data. For Kinect V2, the focus was on skeleton tracking; while for Leap Motion, it was hand gesture recognition. For different actions of the patient, key body joints such as fingers, elbows, shoulders, knees, etc. were comprehensively judged to determine whether the patient performed the specified rehabilitation actions. Through programming, the captured actions were mapped to the corresponding models or controllers in the virtual environment, enabling the user's actions to be given real-time feedback on the screen. Performance optimization was carried out to reduce latency and improve the response speed to ensure that the data captured from the sensors could be quickly and accurately transmitted to the processing unit, thus providing a smooth user experience. In particular, in this embodiment, various interaction modes, such as follow-up guidance, sound feedback, etc., were designed for different types of rehabilitation training needs, greatly enriching the content and form of rehabilitation training. To address possible network connection problems, in this embodiment, an offline mode was also added, enabling patients to continue basic rehabilitation training even in the case of unstable network.

[0094] On this basis, in this embodiment, an adaptive algorithm was designed to provide personalized rehabilitation programs. An instant feedback module was developed, which immediately provided visual and auditory cues to users when incorrect actions were detected to help them correct their postures. In addition, the system would record the data of each training and analyze the patient's performance.

[0095] Finally, after completing all development, the application is packaged as a stand-alone executable file for the PC and deployed to medical institutions or other applicable locations. The entire development process has successfully achieved seamless integration of Kinect V2 and Leap Motion in Unity, creating an innovative rehabilitation training solution.

[0096] Construction of virtual reality scenarios: Through virtual simulation technology, realistic limb rehabilitation training scenarios can be constructed, such as grasping, pinching, stretching, etc. These scenarios can be customized according to the user's rehabilitation training needs to improve the pertinence and effectiveness of training.

[0097] Capture and feedback of hand movements: Through the interaction module and the hand recognition module, the user's limb movements can be captured and fed back to the control module. The control module can generate corresponding control signals based on the user's hand movements and rehabilitation training tasks to control the operation of the display module, so as to achieve the automation and intelligence of hand rehabilitation training.

[0098] The above is only a preferred specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A burn patient rehabilitation training method based on virtual reality, characterized in that: include: Obtain information about the patient to be trained; Inputting the patient information to be trained into a rehabilitation training model to obtain rehabilitation training results, wherein the rehabilitation training model is constructed by a Kinect V2 module and a Leap Motion module, the Kinect V2 module is used to capture human skeleton points, and the Leap Motion module is used to recognize hand postures; Combined with historical training results, the rehabilitation training results are evaluated, stage rehabilitation suggestions are obtained, and current training is completed.

2. A burn patient rehabilitation training method based on virtual reality according to claim 1, characterized in that: Inputting the patient information to be trained into the rehabilitation training model, and obtaining the rehabilitation training results includes: According to the patient information to be trained, setting indicators and parameters for rehabilitation training; Obtaining a rehabilitation training plan according to the indicators and parameters; The patient performs training according to the rehabilitation training program, and automatically records training data to obtain rehabilitation training results.

3. A burn patient rehabilitation training method based on virtual reality according to claim 2, characterized in that: Access to rehabilitation training programs includes: Using the Leap Motion module to identify hand gestures, 13 hand rehabilitation training movements were developed; The Kinect V2 module was used to capture 25 bone points of the human body and 23 limb rehabilitation training movements were developed; A rehabilitation training program is obtained according to the hand rehabilitation training movements and the limb rehabilitation training movements.

4. A burn patient rehabilitation training method based on virtual reality according to claim 2, characterized in that: The patient undergoes training according to the rehabilitation training program, including: Determining the type of rehabilitation training according to the rehabilitation training plan; Different training modes are performed according to different parts of the rehabilitation training type.

5. A burn patient rehabilitation training method based on virtual reality according to claim 4, characterized in that: The types of rehabilitation training include: hand training, neck training, shoulder joint training, trunk removal training, elbow joint training and hip joint training.

6. A burn patient rehabilitation training system based on virtual reality, characterized in that: include: Display module, interactive module, control module, storage module and mechanical module; The display module is used to display virtual reality scenes and limb rehabilitation training tasks; The interaction module is used to collect user actions; The control module is used to control the action according to the rehabilitation training task; The storage module is used to store training data, virtual reality scenes and data of limb rehabilitation training tasks; The mechanical module is used to integrate the all-in-one machine and peripheral devices to provide a basis for displaying virtual reality scenes.

7. The burn patient rehabilitation training system based on virtual reality according to claim 6, characterized in that: The interaction module includes: a hand recognition unit and a limb recognition unit; The hand recognition unit is used to recognize the patient's hand movements using Leap Motion; The limb recognition unit is used to capture the patient's limb movements using Kinect V2.

Images