A virtual reality-based rehabilitation system and method

Abstract

The present application provides a rehabilitation system and method based on virtual reality. It performs pose node recognition on the simulated pose and actual pose of a patient based on bidirectional association rules, and then determines the interaction identifier of the patient in the virtual reality interaction behavior. It extracts the behavior characteristics of the target patient and other patients' stress behaviors in the virtual reality interaction from the electromyographic signals of the rehabilitation training, performs confidence association on all the behavior characteristics to obtain the association state quantity of the stress behavior. It determines the difference degree of the interaction response of the target patient and other patients in the virtual reality interaction during the rehabilitation training through the interaction identifier and the association state quantity. It predicts and generates an execution instruction for the virtual reality device based on the difference degree of the interaction response and the current rehabilitation index of the target patient, and then the virtual reality device executes the execution instruction to guide the rehabilitation action. By adopting the solution of the present application, the dynamic adaptive adjustment of the individualized guidance requirements during the rehabilitation training can be realized, thereby improving the adaptability of the rehabilitation training intensity.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology. More specifically, this application relates to a rehabilitation system and method based on virtual reality. Background Art

[0002] Virtual Reality (VR) provides an immersive and interactive environment in rehabilitation training, helping patients perform rehabilitation exercises in a safe and controllable virtual scenario, thereby improving motor function and rehabilitation effect. Through VR, various scenarios and tasks can be simulated to help patients complete repetitive movement exercises in a virtual environment, which is helpful for restoring coordination, balance, and muscle strength. In addition, the real-time feedback function of VR technology can enhance patients' sense of participation and enthusiasm, bringing new improvements to the dullness and monotony in traditional rehabilitation training.

[0003] Patients' rehabilitation needs vary significantly according to their conditions, physical functions, and rehabilitation progress. However, it is difficult for VR rehabilitation systems to adjust the training intensity in real time to meet the needs of individualized guidance. In the prior art, traditional rehabilitation VR systems are usually based on preset training modules and lack the ability to adjust according to the patient's real-time status. They cannot dynamically optimize the training plan according to the patient's rehabilitation feedback at any time. Moreover, traditional rehabilitation VR systems lack a real-time adaptive and efficient data processing and dynamic feedback mechanism, making it difficult to ensure the smoothness of the patient's rehabilitation training process, resulting in a poor rehabilitation training experience for patients and even affecting the rehabilitation effect. Therefore, how to achieve dynamic adaptive adjustment of individualized guidance needs in rehabilitation training and thus improve the adaptability of rehabilitation training intensity has become a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides a rehabilitation system and method based on virtual reality, which can achieve dynamic adaptive adjustment of individualized guidance needs in rehabilitation training, thereby improving the adaptability of rehabilitation training intensity.

[0005] In a first aspect, this application provides a rehabilitation training method based on virtual reality, including the following steps:

[0006] Collect the simulated postures and actual postures of all patients during rehabilitation training from virtual reality devices;

[0007] Based on bidirectional association rules, perform posture node recognition on the simulated posture and actual posture of each patient, and then determine the interaction identifier of each patient in the virtual reality interaction behavior;

[0008] Detect the electromyographic signals of the target patient and other patients during rehabilitation training, and then extract the behavioral characteristics of the stress behaviors of the target patient and other patients during virtual reality interaction. Perform confidence association on the behavioral characteristics of all stress behaviors to obtain the association state quantity of the stress behaviors of the target patient and other patients during rehabilitation training;

[0009] Match the rehabilitation training behaviors of the target patient and other patients through all interaction identifiers and the association state quantity to obtain the difference degree of the interaction responses of the target patient and other patients during virtual reality interaction in rehabilitation training;

[0010] When the target patient uses a virtual reality device for rehabilitation training, predict and generate an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response, and then the virtual reality device executes the execution instruction to guide the rehabilitation action.

[0011] Preferably, based on the bidirectional association rule, perform pose node recognition on the simulated pose and the actual pose of each patient, and then determining the interaction identifier of each patient in the virtual reality interaction behavior specifically includes:

[0012] For each patient, perform feature recognition on the key pose nodes in the simulated pose and the actual pose of the patient to obtain the simulated node features and the actual node features;

[0013] Use the bidirectional association rule algorithm to match the simulated node features and the actual node features, and identify the deviation position and deviation degree between the node features;

[0014] Determine the interaction identifier of the patient in the virtual reality interaction behavior through the deviation position and the deviation degree.

[0015] Preferably, extracting the behavioral characteristics of the stress behaviors of the target patient and other patients during virtual reality interaction specifically includes:

[0016] For each patient, obtain the electromyographic signals of the patient during rehabilitation training;

[0017] Use a band-pass filter to smooth the electromyographic signals to obtain the electromyographic signals after removing motion artifacts;

[0018] Perform time-domain feature extraction and frequency-domain feature extraction on the electromyographic signals after removing motion artifacts respectively to obtain behavioral time-domain features and behavioral frequency-domain features;

[0019] Determine the behavioral characteristics of the stress behaviors of the patient during virtual reality interaction through the behavioral time-domain features and the behavioral frequency-domain features.

[0020] Preferably, confidence association is performed on the behavioral characteristics of all stress behaviors to obtain the association status quantity of the stress behaviors of the target patient and other patients during rehabilitation training, which specifically includes:

[0021] The behavioral characteristics of the target patient are associated and matched with the behavioral characteristics of all other patients to obtain an association matching degree;

[0022] The confidence of the association matching degree is corrected by the average score of the current rehabilitation action to obtain the association status quantity of the stress behaviors of the target patient and other patients during rehabilitation training.

[0023] Preferably, predicting and generating an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response specifically includes:

[0024] Determine the feedback parameters of the virtual reality device during rehabilitation training according to the difference degree of the interaction response;

[0025] Determine the initialization parameters of the virtual reality device during rehabilitation training according to the rehabilitation index;

[0026] Predict the rehabilitation training parameters of the target patient through the feedback parameters and the initialization parameters, and then generate an execution instruction for the virtual reality device according to the rehabilitation training parameters.

[0027] Preferably, a surface electromyography sensor is used to detect the electromyography signals of the target patient and other patients during rehabilitation training.

[0028] Preferably, the virtual reality device is a VR head-mounted device.

[0029] In a second aspect, the present application provides a virtual reality-based rehabilitation system, including:

[0030] An acquisition module, configured to acquire the simulated postures and actual postures of all patients during rehabilitation training from the virtual reality device;

[0031] A processing module, configured to perform posture node recognition on the simulated posture and actual posture of each patient based on a two-way association rule, and then determine the interaction identifier of each patient in the virtual reality interaction behavior;

[0032] The processing module is further configured to detect the electromyography signals of the target patient and other patients during rehabilitation training, and then extract the behavioral characteristics of the stress behaviors of the target patient and other patients in the virtual reality interaction, and perform confidence association on the behavioral characteristics of all stress behaviors to obtain the association status quantity of the stress behaviors of the target patient and other patients during rehabilitation training;

[0033] The processing module is further configured to perform behavior pattern matching on the rehabilitation training behaviors of the target patient and other patients through all interaction identifiers and the associated state quantity, so as to obtain the difference degree of the interaction responses of the target patient and other patients in virtual reality interaction during rehabilitation training;

[0034] The execution module is configured to, when the target patient uses the virtual reality device for rehabilitation training, predict and generate an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response, and then the virtual reality device executes the execution instruction to perform rehabilitation action guidance.

[0035] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned virtual reality-based rehabilitation training method.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned virtual reality-based rehabilitation training method is implemented.

[0037] The technical solution provided by the disclosed embodiments of the present application has the following beneficial effects:

[0038] In the embodiments of the present application, first, the simulated postures and actual postures of all patients during rehabilitation training are collected from the virtual reality device; posture node recognition is performed on the simulated posture and actual posture of each patient based on the bidirectional association rule, and then the interaction identifier of each patient in the virtual reality interaction behavior is determined; the myoelectric signals of the target patient and other patients during rehabilitation training are detected, and then the behavioral characteristics of the stress behaviors of the target patient and other patients in virtual reality interaction are extracted. The behavioral characteristics of all stress behaviors are confidence-associated to obtain the associated state quantity of the stress behaviors of the target patient and other patients during rehabilitation training; the rehabilitation training behaviors of the target patient and other patients are subjected to behavior pattern matching through all the interaction identifiers and the associated state quantity to obtain the difference degree of the interaction responses of the target patient and other patients in virtual reality interaction during rehabilitation training; when the target patient uses the virtual reality device for rehabilitation training, an execution instruction for the virtual reality device is predicted and generated based on the current rehabilitation index of the target patient and the difference degree of the interaction response, and then the virtual reality device executes the execution instruction to perform rehabilitation action guidance.

[0039] It can be seen that the present application determines the difference degree of the interaction response of the target patient and other patients in the virtual reality interaction during the rehabilitation training through all interaction identifiers and associated state quantities, predicts and generates an execution instruction for the virtual reality device based on the current rehabilitation index and the difference degree of the interaction response of the target patient, and then the virtual reality device executes the execution instruction to guide the rehabilitation action; among them, first, based on the bidirectional association rule, the simulated posture and the actual posture of the patient are identified for posture nodes, and the interaction identifier of the patient in the virtual reality interaction behavior is obtained. Through the interaction identifier, the actual performance of the patient in the virtual reality interaction training can be identified, which can provide accurate data support for personalized guidance; then, by detecting the myoelectric signal of the patient and extracting the stress behavior characteristics, the associated state quantity of different patients in the stress behavior is further calculated by using association analysis. Through the associated state quantity, the stress response of different patients during the training can be analyzed, and then the training intensity can be adaptively adjusted according to the same physiological feedback among patients; the difference degree of the interaction response of the target patient and other patients in the virtual reality interaction during the rehabilitation training is determined through all interaction identifiers and associated state quantities. Through the difference degree of the interaction response, the difference degree between the behavior response of the target patient and other patients during the rehabilitation training can be measured, and this difference degree is used as the feedback correction parameter of the rehabilitation system. Based on the dynamic adjustment ability of the feedback correction parameter, the deficiency that the training content and intensity in the traditional rehabilitation VR system are difficult to change flexibly can be made up. Finally, the individualized rehabilitation training parameters are predicted according to the rehabilitation index and the difference degree of the interaction response of the individual. Through the rehabilitation training parameters, the synchronous adjustment of the training content and the patient's rehabilitation process is ensured, and the limitation of the rehabilitation effect caused by excessive or insufficient training is avoided. Finally, the execution instruction of the virtual reality device is generated according to the rehabilitation training parameters, and the execution instruction is executed to guide the rehabilitation action; in summary, the solution of the present application can realize the dynamic adaptive adjustment of the individualized guidance requirements during the rehabilitation training, thereby improving the adaptability of the rehabilitation training intensity. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is an exemplary flowchart of a virtual reality-based rehabilitation training method according to some embodiments of the present application;

[0041] Figure 2 is a virtual reality logic diagram according to some embodiments of the present application;

[0042] Figure 3 is a schematic flowchart of extracting behavior characteristics according to some embodiments of the present application;

[0043] Figure 4 is a schematic structural diagram of a virtual reality-based rehabilitation system according to some embodiments of the present application;

[0044] Figure 5Schematic diagram of a computer device for implementing a virtual reality-based rehabilitation training method according to some embodiments of the present application. Detailed implementation manners

[0045] To better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0046] Refer to Figure 1 , which is an exemplary flowchart of a virtual reality-based rehabilitation training method according to some embodiments of the present application. The virtual reality-based rehabilitation training method 100 mainly includes the following steps:

[0047] In step 101, collect the simulated postures and actual postures of all patients during rehabilitation training from the virtual reality device.

[0048] In specific implementation, the simulated postures and actual postures of patients during rehabilitation training can be collected through sensors in the virtual reality device. The simulated posture refers to the action information data in the virtual environment, and the actual posture refers to the patient's limb movement data captured by the sensing device. It should be noted that the virtual reality device in the present application is a VR head-mounted device. In other embodiments, the virtual reality device can also be other devices, such as a body sensor tracker, which is not limited here. It should also be noted that all patients in the present application refer to all patients who use the virtual reality device for rehabilitation training within a period of time, and here the period of time refers to the time period corresponding to the current moment to the past 3 days.

[0049] In some embodiments, refer to Figure 2, this figure is a virtual reality logic diagram in some embodiments of the present application. This figure shows the application logic flow of a virtual reality (VR) system, aiming to improve the effect of rehabilitation training through VR technology. The process starts with a rehabilitation ability analysis to determine whether a patient needs to use the VR system for rehabilitation training. If it is decided not to use VR, then other strategies will be selected. If it is decided to use VR, the process will enter the preliminary analysis stage, including a detailed analysis of the patient's needs, content, and environment to ensure that the design and implementation of the VR system can meet specific training goals. Then, the process enters the design and implementation stage, which is divided into two main parts: scene development and effective implementation. In scene development, content design, interaction design, experience design, and reflection design are required. Effective implementation includes scene analysis, import strategy, simulation practice, and review and reflection. Through these steps, it can be ensured that the VR system can be correctly introduced and effectively used in the rehabilitation training process. Finally, the effect of the VR system is evaluated through qualitative analysis, quasi-experimental research, and multimodal analysis. The entire process is a cycle, meaning that after the result diagnosis, it may return to the preliminary analysis or design and implementation stage according to the feedback for necessary adjustments and improvements.

[0050] In step 102, based on the bidirectional association rule, pose node recognition is performed on the simulated pose and the actual pose of each patient, and then the interaction identifier of each patient in the virtual reality interaction behavior is determined.

[0051] In some embodiments, the determination of the interaction identifier of each patient in the virtual reality interaction behavior by performing pose node recognition on the simulated pose and the actual pose of each patient based on the bidirectional association rule can be implemented by the following steps:

[0052] For each patient, feature recognition is performed on the key pose nodes in the simulated pose and the actual pose of the patient to obtain the simulated node features and the actual node features;

[0053] The simulated node features and the actual node features are matched using the bidirectional association rule algorithm to identify the deviation positions and deviation degrees between the node features;

[0054] The interaction identifier of the patient in the virtual reality interaction behavior is determined through the deviation positions and the deviation degrees.

[0055] In specific implementation, for each patient, first, the collected simulated posture and actual posture of the patient can be pre-smoothed, and then the smoothed simulated posture and actual posture are converted into a standard feature representation form, such as joint angles, position coordinates, etc. Then, the recognition model is used to identify the key posture nodes of the body (such as shoulders, elbows, knees, ankles, etc.), and the recognition model extracts the node features at these key posture nodes from the standard feature representation form. The set of node features extracted from the simulated posture is used as the simulated node features, and the set of node features extracted from the actual posture is used as the actual node features. Then, the extracted simulated node features and actual node features are input into the bidirectional association rule algorithm for matching to identify the deviation positions (such as shoulders, knee joints) and deviation degrees (such as angle differences, distance differences) between the key nodes. The bidirectional association rule algorithm can detect the change trends of the key nodes in different states (simulated state and actual state), and identify the specific parts and deviation amplitudes of the posture deviation. Among them, in the bidirectional association rule algorithm, the rule setting includes the threshold setting of angles, distances, etc. between the key nodes. For example, when the deviation of a certain joint angle exceeds a certain threshold, it is considered that there is a deviation at this node. Finally, the position label of the part where the deviation occurs (such as shoulders, elbows, knees, etc.) can be given according to the identified deviation position, and then the grade label is given according to the preset rule for the deviation degree (such as the angle deviation exceeds 15 degrees), such as "slight deviation", "moderate deviation", "severe deviation". Then, the set composed of the position label and the grade label is used as the interaction identifier of the patient in the virtual reality interaction behavior.

[0056] It should be noted that the interaction identifier in this application is a performance identifier used to describe the interaction behavior deviation during the virtual interaction of the patient in the rehabilitation training, which can provide a basis for subsequent personalized training adjustment and effect monitoring. In addition, in this application, the bidirectional association rule algorithm is used to identify the behavior deviation between the simulated posture and the actual posture, so that the rehabilitation system can specifically analyze the deviation positions and degrees of the patient in the rehabilitation movements. The generated interaction identifier provides a specific identifier for the movement characteristics of the patient, which helps to formulate a personalized rehabilitation training plan. In addition, through the accumulation of the interaction identifier, the rehabilitation system can gradually record and track the rehabilitation progress of the patient, and adjust the training plan to gradually correct the deviation.

[0057] In step 103, the myoelectric signals of the target patient and other patients during the rehabilitation training are detected, and then the behavioral characteristics of the stress behaviors of the target patient and other patients during the virtual reality interaction are extracted. The behavioral characteristics of all stress behaviors are subjected to confidence association to obtain the association state quantity of the stress behaviors of the target patient and other patients during the rehabilitation training.

[0058] It should be noted that the electromyography (EMG) signal refers to the electrical signal generated during muscle contraction. It is a weak potential fluctuation detected by electrodes and reflects the activity state of muscle fibers under nerve stimulation. Specifically, when implemented, the EMG signals of the target patient and other patients during rehabilitation training can be detected in the following manner: surface EMG sensors can be used to detect the EMG signals of the target patient and other patients during rehabilitation training. When using surface EMG sensors, they can be fixed on the surface of the target muscle groups of the patient (such as the upper arm, leg, etc.). The specific muscle site on the surface of the target muscle group can be selected according to the rehabilitation training goal. Further, the surface EMG sensors can be used to detect the EMG signals of the target patient and other patients during rehabilitation training in real time according to a preset sampling frequency.

[0059] In some embodiments, refer to Figure 3 As shown, this figure is a schematic flowchart of extracting behavioral features in some embodiments of the present application. The behavioral features of the stress behaviors of the target patient and other patients in virtual reality interaction in this embodiment can be implemented by the following steps:

[0060] In step 1031, for each patient, obtain the EMG signal of the patient during rehabilitation training;

[0061] In step 1032, use a band-pass filter to smooth the EMG signal to obtain the EMG signal after removing motion artifacts;

[0062] In step 1033, perform time-domain feature extraction and frequency-domain feature extraction on the EMG signal after removing motion artifacts respectively to obtain behavioral time-domain features and behavioral frequency-domain features;

[0063] In step 1034, determine the behavioral features of the stress behaviors of the patient in virtual reality interaction through the behavioral time-domain features and the behavioral frequency-domain features.

[0064] In specific implementation, first, for each patient, the electromyogram (EMG) signal of the patient during rehabilitation training is obtained by the method of the foregoing embodiment. Then, according to the frequency range of the EMG signal, a suitable band-pass filter is selected. Generally, the effective frequency range of the EMG signal is from 10 Hz to 500 Hz. Therefore, the band-pass range of the filter can be set to 10 Hz to 500 Hz. Then, the band-pass filter is used to process the EMG signal to remove the low-frequency noise (such as motion artifacts) in the signal, and the EMG signal after removing the motion artifacts is obtained. Then, time-domain feature extraction is performed on the EMG signal after removing the motion artifacts to analyze the changes of the EMG signal in the time dimension. The time-domain features of behavior are mainly used to describe the muscle activity intensity and waveform characteristics, and can reflect the stress intensity and fluctuation conditions. In the embodiments of the present application, the time-domain features of behavior include the root mean square value and the average absolute value. Through these time-domain features, the contraction intensity and activity frequency of the muscle can be described, providing basic data for the recognition of stress behavior. Then, the power spectral density of the EMG signal is used as the frequency-domain feature of behavior. Finally, the time-domain features of behavior (such as the root mean square value and the average absolute value) and the frequency-domain features of behavior (such as the power spectral density) are combined to form a multi-dimensional feature vector. This multi-dimensional feature vector is used as the behavior feature of the stress behavior of the patient during virtual reality interaction. Through the behavior feature, the stress state of the muscle can be comprehensively reflected. Through the above method, the behavior features of the target patient and other patients during virtual reality interaction can be obtained.

[0065] In some embodiments, the confidence correlation of the behavior features of all stress behaviors is performed to obtain the correlation state quantity of the stress behavior of the target patient and other patients during rehabilitation training, which can be implemented by the following steps:

[0066] The behavior features of the target patient are correlated and matched with the behavior features of all other patients to obtain the correlation matching degree;

[0067] The confidence of the correlation matching degree is corrected by the average score of the current rehabilitation action to obtain the correlation state quantity of the stress behavior of the target patient and other patients during rehabilitation training.

[0068] In specific implementation, first, the behavioral characteristics of the target patient are matched with those of each other patient to evaluate the similarity between the stress behavior of the target patient in the rehabilitation movement and that of others. The quantified value of this similarity (such as the Euclidean distance) is used as the matching degree between the target patient and each other patient. Through the above method, the matching degrees between the target patient and all other patients can be obtained, and then the average value of all the matching degrees is used as the associated matching degree. Then, the average score of each rehabilitation movement is obtained from the system. The average score represents the average completion quality of the rehabilitation movement. Further, a confidence correction model is initialized, and the associated matching degree is combined with the average score of the current rehabilitation movement through the confidence correction model to adjust the confidence of the matching result. For example, if the average score is greater than a preset first threshold, it indicates that the stress level of most patients in this movement is relatively low, so the confidence of this associated matching degree is reduced. If the average score is less than a preset second threshold, the confidence of this associated matching degree is increased to reflect that this movement is more challenging. Finally, the product of the associated matching degree and the confidence is used as the associated state quantity of the stress behavior of the target patient and other patients in the rehabilitation training.

[0069] It should be noted that the associated state quantity of the stress behavior in this application is an index to measure the degree of difference in the stress behavior association between the target patient and other patients. In addition, a dynamic matching and updating mechanism is adopted in this application. During the rehabilitation training process, as the stress behavior of the patient changes in real time, the rehabilitation system needs to recalculate the associated matching degree to ensure the timeliness of its matching degree.

[0070] It also should be noted that by analyzing the stress behavior characteristics of the patient through the electromyogram signal in this application, the physiological stress and muscle load of the patient during training can be evaluated, so as to avoid potential risks caused by overtraining or stress response. At the same time, the calculation of the associated state quantity enables the stress behavior characteristics of the patient to be compared with others, which can help identify and correct abnormal situations of the patient in the movement and stress response.

[0071] In step 104, the rehabilitation training behaviors of the target patient and other patients are subjected to behavior pattern matching through all the interaction identifiers and the associated state quantity, and the difference degree of the interaction responses of the target patient and other patients in the virtual reality interaction during the rehabilitation training is obtained.

[0072] In some embodiments, the behavior pattern matching of the rehabilitation training behaviors of the target patient and other patients through all the interaction identifiers and the associated state quantity to obtain the difference degree of the interaction responses of the target patient and other patients in the virtual reality interaction during the rehabilitation training can be implemented by the following steps:

[0073] Extract the interaction identifiers of the target patient in the virtual reality interaction behavior;

[0074] Cluster all interaction identifiers to obtain behavior clusters at different levels;

[0075] Screen out the optimal matching behavior cluster according to the similarity between the interaction identifier of the target patient and each behavior cluster;

[0076] Couple and compensate the credibility of the optimal matching behavior cluster according to the associated state quantity to obtain the behavior credibility of the optimal matching behavior cluster;

[0077] Determine the difference degree of the interaction response of the target patient and other patients in the virtual reality interaction during rehabilitation training through the behavior credibility of the optimal matching behavior cluster and the interaction identifier of the target patient.

[0078] It should be noted that the interaction identifier of the target patient in the virtual reality interaction behavior can be extracted by using the method of the foregoing embodiment, which will not be elaborated here.

[0079] When specifically implemented, clustering all interaction identifiers to obtain behavior clusters at different levels can be implemented in the following manner, that is: the K-means clustering algorithm can be used to cluster all interaction identifiers, and all similar interaction identifiers are divided into one cluster, and all obtained clusters are used as behavior clusters, and each behavior cluster represents a specific type of rehabilitation deviation state (that is, the deviation degree between virtual and real is different); screening out the optimal matching behavior cluster according to the similarity between the interaction identifier of the target patient and each behavior cluster can be implemented in the following manner, that is: first calculate the average similarity between the interaction identifier of the target patient and all interaction identifiers in each behavior cluster, and then use the behavior cluster corresponding to the maximum average similarity as the optimal matching behavior cluster; coupling and compensating the credibility of the optimal matching behavior cluster according to the associated state quantity to obtain the behavior credibility of the optimal matching behavior cluster can be implemented in the following manner, that is: obtaining the average deviation degree of the virtual reality interaction behavior corresponding to the optimal matching behavior cluster through the interaction identifiers in the optimal matching behavior cluster, and the reciprocal of the average deviation degree can be used as the credibility of the optimal matching behavior cluster, and then the product between the associated state quantity and the credibility of the optimal matching behavior cluster is used as the behavior credibility of the optimal matching behavior cluster.

[0080] Preferably, in the above embodiment, determining the difference degree of the interaction response of the target patient and other patients in the virtual reality interaction during rehabilitation training through the behavior credibility of the optimal matching behavior cluster and the interaction identifier of the target patient can be implemented in the following steps:

[0081] Obtain all interaction identifiers in the optimal matching behavior cluster;

[0082] Determine the average distance between the interaction identifier of the target patient and all interaction identifiers in the optimal matching behavior cluster;

[0083] Evaluate the behavioral differences between the target patient and other patients based on the average distance and the behavioral credibility of the optimal matching behavior clusters, and obtain the difference degree of the interaction responses of the target patient and other patients in virtual reality interaction during rehabilitation training.

[0084] When specifically implemented, first, all interaction identifiers in the optimal matching behavior clusters can be obtained by using the aforementioned method. Then, the Euclidean distance between the interaction identifier of the target patient and each interaction identifier in the optimal matching behavior clusters can be calculated, and the average value of all Euclidean distances is used as the average distance between the interaction identifier of the target patient and all interaction identifiers in the optimal matching behavior clusters. Finally, the product of the average distance and the behavioral credibility of the optimal matching behavior clusters can be used as the difference degree of the interaction responses of the target patient and other patients in virtual reality interaction during rehabilitation training.

[0085] It should be noted that the difference degree of the interaction response in this application measures the difference between the interaction behavior responses of the target patient and other patients during rehabilitation training. It can provide a key feedback mechanism for the rehabilitation system, help the rehabilitation system adapt to the patient's ability level in real time, and continuously optimize the training plan to make the rehabilitation training more accurate and effective.

[0086] In step 105, when the target patient uses the virtual reality device for rehabilitation training, an execution instruction for the virtual reality device is predicted and generated based on the current rehabilitation index of the target patient and the difference degree of the interaction response, and then the virtual reality device executes the execution instruction to perform rehabilitation action guidance.

[0087] It should be noted that the rehabilitation index is a comprehensive index reflecting the current physical rehabilitation status of the patient. The current rehabilitation index of the target patient can be obtained by the following method, that is: a pre-trained rehabilitation assessment model can be used to output the current rehabilitation index of the target patient. It should be noted that the rehabilitation assessment model in this application is a model trained based on a convolutional neural network according to the motion performance data, stress behavior characteristics, and physiological signal data of historical patients.

[0088] In some embodiments, predicting and generating an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response can be implemented by the following steps:

[0089] Determine the feedback parameters of the virtual reality device during rehabilitation training according to the difference degree of the interaction response;

[0090] Determine the initialization parameters of the virtual reality device during rehabilitation training according to the rehabilitation index;

[0091] Predict the rehabilitation training parameters of the target patient through the feedback parameters and the initialization parameters, and then generate an execution instruction for the virtual reality device according to the rehabilitation training parameters.

[0092] In specific implementation, first, initialize a rehabilitation training model. The rehabilitation training model is a regression model based on decision trees. The current rehabilitation index of the target patient can be used as the initialization parameter of the rehabilitation training model in the virtual reality device, and the difference degree of the interaction response can be used as the feedback parameter of the rehabilitation training model in the virtual reality device. The rehabilitation training model can correct the predicted rehabilitation training actions in real time through the feedback parameter, which can improve the adaptability of the patient's rehabilitation training. Then, the corrected rehabilitation training actions are used as the rehabilitation training parameters of the target patient, and the rehabilitation training parameters are converted into a data stream that the VR device can understand, and this data stream is used as the execution instruction of the virtual reality device.

[0093] It should be noted that the rehabilitation training parameters in this application refer to the training parameters set for the patient during the rehabilitation training process, including specific action types, training intensities, frequencies, amplitudes, and speeds.

[0094] It should also be noted that the virtual reality device executing the rehabilitation training parameters for rehabilitation action guidance in this application means that the virtual reality device executes the rehabilitation training parameters and uses a virtual character (NPC, Non-Player Character) in the virtual reality environment to demonstrate and guide the rehabilitation actions, helping the patient to more intuitively understand and imitate the correct training actions.

[0095] On the other hand, in some embodiments, this application provides a virtual reality-based rehabilitation system. Refer to Figure 4 , this figure is a schematic structural diagram of a virtual reality-based rehabilitation system shown according to some embodiments of this application. The virtual reality-based rehabilitation system 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described as follows:

[0096] The acquisition module 401. In this application, the acquisition module 401 is mainly used to acquire the simulated postures and actual postures of all patients during rehabilitation training from the virtual reality device;

[0097] The processing module 402. In this application, the processing module 402 is used to perform posture node recognition on the simulated postures and actual postures of each patient based on bidirectional association rules, and then determine the interaction identifier of each patient in the virtual reality interaction behavior;

[0098] In this application, the processing module 402 is also used to detect the electromyographic signals of the target patient and other patients during rehabilitation training, and then extract the behavioral characteristics of the stress behaviors of the target patient and other patients in the virtual reality interaction, and perform confidence association on the behavioral characteristics of all stress behaviors to obtain the association state quantity of the stress behaviors of the target patient and other patients during rehabilitation training;

[0099] In this application, the processing module 402 is further configured to perform behavior pattern matching on the rehabilitation training behaviors of the target patient and other patients through all interaction identifiers and the associated state quantity, so as to obtain the difference degree of the interaction responses of the target patient and other patients in virtual reality interaction during rehabilitation training;

[0100] The execution module 403. In this application, the execution module 403 is mainly configured to, when the target patient uses the virtual reality device for rehabilitation training, predict and generate an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response, and then the virtual reality device executes the execution instruction to perform rehabilitation action guidance.

[0101] In addition, this application also provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned virtual reality-based rehabilitation training method.

[0102] In some embodiments, refer to Figure 5 , this figure is a schematic structural diagram of a computer device for implementing the virtual reality-based rehabilitation training method according to some embodiments of this application. The virtual reality-based rehabilitation training method in the above embodiments can be implemented by Figure 5 the computer device shown. The computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0103] The processor 501 can be a general-purpose central processing unit (CPU) or an application specific integrated circuit (ASIC).

[0104] The communication bus 502 can be used to transmit information between the above components.

[0105] The memory 503 can be a read only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or can also be an electrically erasable programmable read only memory (EEPROM), a compact disc read only memory (CD ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 503 can exist independently and be connected to the processor 501 through the communication bus 502. The memory 503 can also be integrated with the processor 501.

[0106] Among them, the memory 503 is used to store the program code for executing the solution of this application, and is controlled by the processor 501 to execute. The processor 501 is used to execute the program code stored in the memory 503. The program code can include one or more software modules. The above-described virtual reality-based rehabilitation training method in the embodiment can be implemented by one or more software modules in the program code in the processor 501 and the memory 503.

[0107] The communication interface 504 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0108] In a specific implementation, as an embodiment, the computer device can include multiple processors, and each of these processors can be a single-core (single CPU) processor or a multi-core (multi CPU) processor. The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0109] The computer device described above may be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device may be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0110] In addition, the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-described rehabilitation training method based on virtual reality is implemented.

[0111] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0112] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A virtual reality-based rehabilitation training method, characterized in that, Including the following steps: Collect the simulated postures and actual postures of all patients during rehabilitation training from the virtual reality device; Based on the bidirectional association rule, identify the posture nodes of the simulated posture and the actual posture of each patient, and then determine the interaction identifier of each patient in the virtual reality interaction behavior; Detect the electromyogram signals of the target patient and other patients during rehabilitation training, and then extract the behavioral characteristics of the stress behaviors of the target patient and other patients in virtual reality interaction. Perform confidence association on the behavioral characteristics of all stress behaviors to obtain the association state quantity of the stress behaviors of the target patient and other patients during rehabilitation training; Match the rehabilitation training behaviors of the target patient and other patients through all the interaction identifiers and the association state quantity to obtain the difference degree of the interaction responses of the target patient and other patients in virtual reality interaction during rehabilitation training; When the target patient uses the virtual reality device for rehabilitation training, predict and generate an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response, and then the virtual reality device executes the execution instruction to guide the rehabilitation action; Among them, the difference degree of the interaction response is a measure of the difference between the interaction behavior response of the target patient during rehabilitation training and that of other patients. Matching the rehabilitation training behaviors of the target patient and other patients through all the interaction identifiers and the association state quantity to obtain the difference degree of the interaction response of the target patient and other patients in virtual reality interaction during rehabilitation training specifically includes: Extract the interaction identifier of the target patient in the virtual reality interaction behavior; Cluster all the interaction identifiers to obtain behavior clusters at different levels; Screen out the optimal matching behavior cluster according to the similarity between the interaction identifier of the target patient and each behavior cluster; Perform coupling compensation on the credibility of the optimal matching behavior cluster according to the association state quantity to obtain the behavior credibility of the optimal matching behavior cluster; Determine the difference degree of the interaction response of the target patient and other patients in virtual reality interaction during rehabilitation training through the behavior credibility of the optimal matching behavior cluster and the interaction identifier of the target patient.

2. The method according to claim 1, characterized in that, Based on the bidirectional association rule, identify the posture nodes of the simulated posture and the actual posture of each patient, and then determine the interaction identifier of each patient in the virtual reality interaction behavior specifically includes: For each patient, perform feature recognition on the key posture nodes in the simulated posture and the actual posture of the patient to obtain the simulated node feature and the actual node feature; Use the bidirectional association rule algorithm to match the simulated node feature and the actual node feature, and identify the deviation position and deviation degree between the node features; Determine the interaction identifier of the patient in the virtual reality interaction behavior through the deviation position and the deviation degree.

3. The method according to claim 1, characterized in that, Extract the behavioral characteristics of the stress behaviors of the target patient and other patients in virtual reality interaction specifically includes: For each patient, obtain the electromyogram signal of the patient during rehabilitation training; Use a band-pass filter to smooth the electromyogram signal to obtain the electromyogram signal after removing the motion artifacts; Extract time-domain features and frequency-domain features from the EMG signals after removing motion artifacts respectively to obtain behavioral time-domain features and behavioral frequency-domain features; Determine the behavioral characteristics of the patient's stress behavior in virtual reality interaction through the behavioral time-domain features and the behavioral frequency-domain features.

4. The method according to claim 1, wherein Perform confidence association on the behavioral characteristics of all stress behaviors to obtain the association status quantity of the target patient and other patients' stress behaviors in rehabilitation training, specifically including: Perform association matching on the behavioral characteristics of the target patient and the behavioral characteristics of all other patients to obtain the association matching degree; Perform confidence correction on the association matching degree through the average score of the current rehabilitation action to obtain the association status quantity of the target patient and other patients' stress behaviors in rehabilitation training.

5. The method according to claim 1, characterized in that, Predict and generate an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response, specifically including: Determine the feedback parameters of the virtual reality device in rehabilitation training according to the difference degree of the interaction response; Determine the initialization parameters of the virtual reality device in rehabilitation training according to the rehabilitation index; Predict the rehabilitation training parameters of the target patient through the feedback parameters and the initialization parameters, and then generate an execution instruction for the virtual reality device according to the rehabilitation training parameters.

6. The method according to claim 1, wherein Use a surface EMG sensor to detect the EMG signals of the target patient and other patients in rehabilitation training.

7. The method according to claim 1, characterized in that, The virtual reality device is a virtual reality head-mounted device.

8. A virtual reality-based rehabilitation system that uses the method described in any one of claims 1 to 7 for rehabilitation training, characterized in that, The system includes: An acquisition module for acquiring the simulated postures and actual postures of all patients in rehabilitation training from the virtual reality device; A processing module for performing posture node recognition on the simulated posture and actual posture of each patient based on bidirectional association rules, and then determining the interaction identifier of each patient in the virtual reality interaction behavior; The processing module is further configured to detect the EMG signals of the target patient and other patients in rehabilitation training, and then extract the behavioral characteristics of the stress behaviors of the target patient and other patients in virtual reality interaction, and perform confidence association on the behavioral characteristics of all stress behaviors to obtain the association status quantity of the target patient and other patients' stress behaviors in rehabilitation training; The processing module is further configured to perform behavior pattern matching on the rehabilitation training behaviors of the target patient and other patients through all the interaction identifiers and the association status quantity to obtain the difference degree of the interaction response of the target patient and other patients in virtual reality interaction during rehabilitation training; An execution module for predicting and generating an execution instruction for the virtual reality device based on the current rehabilitation index of the target patient and the difference degree of the interaction response when the target patient uses the virtual reality device for rehabilitation training, and then the virtual reality device executes the execution instruction to guide the rehabilitation action.

9. A computer device, the computer device includes a memory and a processor, the memory stores code, characterized in that, The processor is configured to obtain the code and execute the virtual reality-based rehabilitation training method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the virtual reality-based rehabilitation training method according to any one of claims 1 to 7.

Images