A motor imagery training system based on real-time feedback of 3D virtual reality technology
Through the 3D virtual reality technology real-time feedback sports imagination training system, the problem of inability to evaluate the accuracy and participation of patients in traditional sports imagination training is solved, and personalized rehabilitation training and better feedback effects are achieved.
Patent Information
- Application Number
- CN202111250833.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-26
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-10-26
AI Technical Summary
In traditional sports imagination training, it is impossible to determine whether the patient is properly executing training instructions and lack of participation assessment and feedback methods, and feedback is limited.
Using a 3D virtual reality technology-based motion imagination training system, through virtual reality wearable modules, signal acquisition modules, signal analysis modules and regulation modules, the physiological signals of patients are collected and analyzed in real time, providing visual feedback and evaluating the patient's motor imagination training participation. Spend.
It achieves the accuracy and participation assessment of sports imagination training, provides personalized rehabilitation training, and enhances the feedback experience and rehabilitation effect of patients.
Smart Images

Figure CN114021604B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of rehabilitation engineering technology, and more specifically, to a motor imagery training system based on 3D virtual reality technology real-time feedback. Background Art
[0002] There are about 2 million new strokes in China every year, and the high disability rate of about 50% of strokes has brought a heavy burden to patients and their families. The symptoms of patients with sequelae of stroke are mainly motor dysfunction, which directly affects the quality of life of patients. Patients in the flaccid paralysis stage lose control of the affected side, voluntary movement disappears, muscle tension is low, and tendon reflexes are weakened or disappeared. Despite this, the patient's brain still retains a complete or partial movement "flow chart". Motor imagery is a mental rehearsal of motor behavior without obvious movement through muscle activity. This therapy requires patients to concentrate their attention and fully participate in the motor imagery process, stimulate the neural network by activating the mirror neurons in the brain, increase the activity of the neural network in the cerebral cortex, and thus accelerate cortical reorganization. Clinically, patients are usually asked to close their eyes and perform motor imagery training according to voice instructions. However, the problems faced in traditional motor imagery training are: 1) It is impossible to determine whether the patient correctly executes the motor imagery instructions during the training process; 2) There is a lack of evaluation methods for the patient's participation in the motor imagery training process; 3) There is a lack of visualization methods and real-time feedback methods for brain changes during motor imagery training.
[0003] The Chinese patent with publication number CN112244774A, published on January 22, 2021, discloses a brain-computer interface rehabilitation training system and method for performing motor training on the limbs of the rehabilitation training recipient according to the active motor intention of the rehabilitation training recipient to achieve rehabilitation. The brain-computer interface rehabilitation training system includes a motor intention induction device, a signal acquisition device, a processing device, a control device, a training device, and an evaluation device. Through the brain-computer interface rehabilitation training system and method of this application, it is possible to enhance the active motor intention induced in the rehabilitation training recipient, improve the sensitivity and accuracy of feature extraction from EEG signals, enable the rehabilitation training movements to accurately follow the active motor intention, and further activate nerve conduction. However, the feedback of this patent is to complete limb training through stimulating movements and control the exoskeleton to perform mechanical movements. The feedback received by the patient is more of a mechanically generated perceptual stimulation. However, the patient's perception of changes in the movement speed and amplitude of the rehabilitation mechanical equipment is limited. Summary of the invention
[0004] The present invention provides a motor imagery training system based on 3D virtual reality technology with real-time feedback, aiming to provide personalized rehabilitation training based on real-time feedback and regulation of physiological signals, and to solve the problem that it is clinically impossible to determine whether the patient correctly executes the training instructions during the motor imagery training process and the degree of participation cannot be determined.
[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:
[0006] A motor imagery training system based on real-time feedback of 3D virtual reality technology includes a wearable virtual reality module, a 3D virtual training scene module, a signal acquisition module, a signal analysis module, a control module and an evaluation module, wherein:
[0007] The wearable virtual reality module presents the motor imagery training in a visual manner according to the regulation of the regulation module;
[0008] The 3D virtual training scene module provides virtual scenes with different training methods and contents. After selecting the training method and content to be performed, the 3D virtual training scene module provides the corresponding virtual training scene to the virtual reality wearable module;
[0009] The signal acquisition module is used to collect physiological signals of the patient and transmit the physiological signals to the signal analysis module;
[0010] The signal analysis module extracts features from the physiological signal to obtain a physiological signal feature value;
[0011] The control module controls the virtual reality wearable module according to the physiological signal characteristic value;
[0012] The evaluation module quantifies the patient's participation in motor imagery training each time according to all the above modules, and evaluates the patient's limb rehabilitation level.
[0013] The present invention applies virtual reality technology to motor imagery therapy, by real-time monitoring and collecting the physiological signals generated by the patient during the motor imagery process, online analyzing the signal changes and linking the signal changes with the virtual reality scene, and feeding back the signal changes to the virtual reality training scene in real time to control the animation effects in the scene, so as to quantify the patient's performance in the motor imagery process, that is, to judge whether the patient correctly executes the motor imagery instructions during the training process, and to evaluate the patient's participation in the motor imagery training process.
[0014] Preferably, the virtual reality wearable module includes wearable glasses, a locator and an interactive device, wherein:
[0015] The wearable glasses are used to present the motor imagery training in a visual manner in a virtual space;
[0016] The locator is used to locate the patient in the virtual space;
[0017] The interactive device interacts with the virtual space.
[0018] Preferably, the interactive device includes a virtual handle and a keyboard and mouse.
[0019] Preferably, the 3D virtual training scene module provides upper limb / lower limb training, bed / sitting training, unilateral / bilateral training, learning / training mode, training scene, and training times.
[0020] Preferably, the signal acquisition module acquires electromyographic signals and electroencephalographic signals generated by the patient during motor imagery.
[0021] Preferably, in the signal acquisition module, myoelectric electrode patches, myoelectric electrode wires and acquisition equipment are used to collect myoelectric signals, wherein the myoelectric electrode patches are attached to the muscles of the patient's bilateral training limbs, one end of the myoelectric electrode wire is connected to the myoelectric electrode patch, and the other end is connected to the acquisition equipment, and two effective acquisition channels are respectively set on the patient's left hand and right hand to respectively collect the myoelectric signals of the flexor pollicis longus and the flexor carpi radialis, wherein the electrodes of the flexor digitorum superficialis on both sides serve as reference electrodes for the left hand and the right hand, the sampling rate is set to 1200Hz, and the bandpass filter is set to 0.5Hz-100Hz; An 80-channel bioelectric signal acquisition system and a 64-channel active electrode system were used to collect EEG signals. The electrode position of the 64-channel active electrode system was positioned using the international standard 10-20 electrode lead. The reference electrode was located at the left earlobe, and the ground electrode was located at the AFZ. There were 18 acquisition channels in total: FC5, FC3, FC1, FC2, FC4, FC6, C5, C3, C1, C2, C4, C6, CP5, CP3, CP1, CP2, CP4, and CP6. The sampling rate was set to 1200 Hz, and the bandpass filter was set to 0.5 Hz-100 Hz.
[0022] Preferably, the signal analysis module analyzes the EEG signal to obtain an EEG activity mapping topography, specifically:
[0023] The EEG signals are first filtered and then downsampled to 256 Hz for the 18 channels. The alpha and beta wave frequency bands are then extracted from the signals and the EEG spectrum density is calculated. Event-related synchronization and event-related desynchronization features are finally extracted, and a topographic map of EEG activity is drawn based on the locations of the electrodes corresponding to the 18 channels.
[0024] Preferably, the signal analysis module analyzes the electromyographic signal to obtain a muscle force value, first filters and denoises the electromyographic signal, uses the root mean square root and the electromyographic integral value as characteristic values in the time domain, and obtains the muscle force value, which indicates the strength of the muscle.
[0025] Preferably, the control module controls the virtual reality wearable module according to the physiological signal characteristic value, specifically:
[0026] When the physiological signal characteristic value is greater than a preset threshold, the control module enables the virtual reality wearable module to visually present the motor imagery training;
[0027] When the physiological signal characteristic value is less than a preset threshold, the control module does not enable the virtual reality wearable module to visually present the motor imagery training.
[0028] Preferably, the preset threshold is obtained by:
[0029] When the patient is in a relaxed state, obtain the relaxed muscle strength value M of the healthy side of the patient. healthy_relax and the relaxed muscle strength value M of the affected side affected_relax ;
[0030] In the fist-clenching state, obtain the maximum muscle strength value M of the patient's healthy side. healthy_max and the maximum muscle strength value of the affected side;
[0031] The weighted average of the muscle strength values in the bilateral relaxation state and the fist clenching state was calculated to obtain the comprehensive relaxation muscle strength value M relax and the comprehensive maximum muscle strength value M max_muscle :
[0032] M relax =(M healthy_relax +M affected_relax ) / 2
[0033] M max_muscle =(M healthy_max +M affected_max ) / 2
[0034] Taking two different percentages of the comprehensive maximum muscle strength value to obtain two thresholds M th1 and M th2 , and store the two thresholds in the 3D virtual training scene module:
[0035] M th1 =M max_muscle ×0.2
[0036] M th2 =M max_muscle ×0.6.
[0037] Preferably, the specific steps of selecting a scene and a training mode for motor imagery training and playing a corresponding virtual animation in a corresponding scene for motor imagery training according to the muscle strength value are as follows:
[0038] Before starting training, show the initial scene;
[0039] Select patient type whether bedridden or seated, limb trained whether upper or lower, unilateral or bilateral limb training, training setting, learning mode or training mode, and number of training cycles;
[0040] Display the start button;
[0041] After pressing the button, switch from the initial scene to the selected training scene;
[0042] After the patient completes the training according to the instructions, he / she will determine whether the current number of training times has reached the number of training cycles. If not, continue to repeat the training. If so, end the training cycle.
[0043] Preferably, a virtual reality wearable module is used to play the corresponding virtual animation, and the virtual reality wearable module includes wearable glasses, a locator and a touch controller. The wearable glasses provide a visual presentation of the virtual training scene, the locator is used to locate the patient's position in the virtual training scene, and the touch controller is used to interact with the virtual training scene.
[0044] The present invention analyzes the collected electromyographic signals to obtain the characteristic values of muscle strength, and feeds them back to the virtual training scene in real time, and feeds back the result animation to the patient in a visual manner through a virtual reality wearable module, so that the patient can understand his performance in this training. The prior art does not regulate the virtual scene, but uses the brain engagement index as a training control parameter to regulate the training device, such as hand exoskeleton equipment, leg training vehicle equipment, etc., and also provides a vibration unit for feedback stimulation action. The present invention regulates the virtual training scene according to the characteristic value, while the prior art is used to regulate the training equipment.
[0045] In terms of evaluation technology, the present invention uses clinical scale questionnaires, EEG activity mapping topography and muscle strength changes to comprehensively evaluate the patient's rehabilitation effect. The existing evaluation technologies include brain engagement index, movement state index, multiple EEG signals and EMG signals.
[0046] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0047] The present invention provides the advantages of personalized training for patients at different stages of stroke. The control module serves as a medium between the data analysis module and the virtual training scene module. The data analysis module can extract the collected electromyographic signal data into the muscle strength characteristic value, and the virtual training scene module provides a scene for inducing the patient to perform motor imagery training. The control module feeds back the muscle strength characteristic value to the virtual scene, determines which training result animation to play, and feeds back the virtual animation to the patient through the virtual reality wearable module. Personalized training is embodied by the fact that the muscle strength of different patients at different times is different. The present invention obtains the patient's physiological signal and extracts its characteristics as the patient's participation in motor imagery training. At the same time, the present invention compares the muscle strength values obtained by the patient in different states and under different training, thereby being able to evaluate the changes in the patient's training limb muscle strength and obtain a better motor imagery training effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 It is a schematic diagram of the system module of the present invention.
[0049] Figure 2 A schematic flow chart of a motor imagery rehabilitation training method provided in this embodiment. DETAILED DESCRIPTION
[0050] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;
[0051] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;
[0052] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0053] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0054] Example 1
[0055] This embodiment provides a motor imagery training system based on 3D virtual reality technology with real-time feedback. Figure 1 As shown, it includes a virtual reality wearable module, a 3D virtual training scene module, a signal acquisition module, a signal analysis module, a control module and an evaluation module, wherein:
[0056] The signal acquisition module is used to collect physiological signals of the patient and transmit the physiological signals to the signal analysis module;
[0057] The signal analysis module extracts features from the physiological signal to obtain a physiological signal feature value;
[0058] The control module controls the 3D virtual training scene module according to the physiological signal characteristic value;
[0059] The 3D virtual training scene module provides virtual scenes with different training methods and contents. After selecting the training method and content to be performed, the 3D virtual training scene module provides the corresponding virtual training scene to the virtual reality wearable module;
[0060] The wearable virtual reality module selects the training method and content according to the 3D virtual training scene module, and presents the motor imagery training in a visual manner;
[0061] The evaluation module quantifies the patient's participation in motor imagery training each time according to all the above modules, and evaluates the patient's limb rehabilitation level.
[0062] The virtual reality wearable module includes wearable glasses, a locator and an interactive device, wherein: the wearable glasses are used to present the motor imagery training in a visual manner in a virtual space;
[0063] The locator is used to locate the patient in the virtual space;
[0064] The interactive device interacts with the virtual space.
[0065] The interactive device includes a virtual handle and a keyboard and mouse.
[0066] The 3D virtual training scene module provides upper limb / lower limb training, bed / sitting training, unilateral / bilateral training, learning / training mode, training scene, and training times.
[0067] The signal acquisition module acquires the electromyographic signals and electroencephalographic signals generated during the patient's motor imagery.
[0068] In the signal acquisition module, myoelectric electrode patches, myoelectric electrode wires and acquisition equipment are used to collect myoelectric signals, wherein the myoelectric electrode patches are attached to the muscles of the patient's bilateral training limbs, one end of the myoelectric electrode wire is connected to the myoelectric electrode patch, and the other end is connected to the acquisition equipment, and two effective acquisition channels are respectively set in the patient's left hand and right hand to respectively collect the myoelectric signals of the flexor pollicis longus and the flexor carpi radialis, wherein the electrodes of the flexor digitorum superficialis on both sides are used as reference electrodes for the left hand and the right hand, the sampling rate is set to 1200Hz, and the bandpass filter is set to 0.5Hz-100Hz; An 80-channel bioelectric signal acquisition system and a 64-channel active electrode system were used to collect EEG signals. The electrode position of the 64-channel active electrode system was positioned according to the international standard 10-20 electrode lead. The reference electrode was located at the left earlobe, and the ground electrode was located at the AFZ. There were 18 acquisition channels in total: FC5, FC3, FC1, FC2, FC4, FC6, C5, C3, C1, C2, C4, C6, CP5, CP3, CP1, CP2, CP4, and CP6. The sampling rate was set to 1200 Hz, and the bandpass filter was set to 0.5 Hz-100 Hz.
[0069] The signal analysis module analyzes the EEG signal to obtain an EEG activity mapping topography, specifically:
[0070] The EEG signals are first filtered and then downsampled to 256 Hz for the 18 channels. The alpha and beta wave frequency bands are then extracted from the signals and the EEG spectrum density is calculated. Event-related synchronization and event-related desynchronization features are finally extracted, and a topographic map of EEG activity is drawn based on the locations of the electrodes corresponding to the 18 channels.
[0071] The signal analysis module analyzes the electromyographic signal to obtain a muscle force value. The electromyographic signal is first filtered and denoised, and the root mean square root and electromyographic integral value are used as characteristic values in the time domain to obtain the muscle force value, which indicates the strength of the muscle.
[0072] The control module controls the 3D virtual training scene module according to the physiological signal characteristic value, specifically:
[0073] When the physiological signal characteristic value is greater than a preset threshold, the control module enables the virtual reality wearable module to visually present the motor imagery training;
[0074] When the physiological signal characteristic value is less than a preset threshold, the control module does not enable the virtual reality wearable module to visually present the motor imagery training.
[0075] The acquisition of the preset threshold is specifically as follows:
[0076] When the patient is in a relaxed state, obtain the relaxed muscle strength value M of the healthy side of the patient.healthy_relax and the relaxed muscle strength value M of the affected side affected_relax ;
[0077] In the fist-clenching state, obtain the maximum muscle strength value M of the patient's healthy side. healthy_max and the maximum muscle strength value of the affected side;
[0078] The weighted average of the muscle strength values in the bilateral relaxation state and the fist clenching state was calculated to obtain the comprehensive relaxation muscle strength value M relax and the comprehensive maximum muscle strength value M max_muscle :
[0079] M relax =(M healthy_relax +M affected_relax ) / 2
[0080] M max_muscle =(M healthy_max +M affected_max ) / 2
[0081] Taking two different percentages of the comprehensive maximum muscle strength value to obtain two thresholds M th1 and M th2 , and store the two thresholds in the 3D virtual training scene module:
[0082] M th1 =M max_muscle ×0.2
[0083] M th2 =M max_muscle ×0.6.
[0084] In the specific implementation process, medical staff determine whether the patient should perform the motor imagery training in bed or sitting according to the patient's current rehabilitation level, and adjust the patient to the appropriate position;
[0085] The electromyographic electrode patch is attached to the muscles of the patient's bilateral training limbs, and one end of the electromyographic electrode wire is connected to the electromyographic electrode patch, and the other end is connected to the acquisition device. In this embodiment, the g.Hiamp physiological signal acquisition device and the passive electrode box with 16 channels of g.tec company, as well as the electromyographic electrode wire and patch are used;
[0086] The flexor pollicis longus and flexor carpi radialis of the patient's bilateral upper limb muscles were selected as effective signals, the flexor digitorum superficialis on both sides were used as reference signals for the left and right hands respectively, and the muscles in the middle of the clavicle were used as ground signals. The sampling rate was set to 1200Hz, the bandpass filter was set to 0.5Hz-100Hz, and the notch filter was set to 48-52Hz;
[0087] Instruct the patient to close his eyes, stay still and relax, and use g.Recorder software to collect the patient's electromyographic signals in a relaxed state for two minutes;
[0088] Instruct the patient to make a fist with his healthy upper limb for 2-3 seconds, then relax for 2-3 seconds, and repeat this 5 times. During this period, g.Recorder software is used for signal acquisition.
[0089] Instruct the patient to make a fist with his affected upper limb for 2-3 seconds, then relax for 2-3 seconds, and repeat this 5 times. During this period, g.Recorder software is used for signal acquisition.
[0090] In this embodiment, the healthy side and the affected side each have two channels with valid signals, corresponding to the electromyographic signals of the flexor pollicis longus and the flexor carpi radialis, respectively. In the preprocessing, the valid signals of the two channels need to be weighted averaged to calculate the only valid signal of the limb;
[0091] The unique effective electromyographic signal of the healthy side and the affected side of the patient is subjected to feature extraction. In the example of the present invention, the electromyographic signal is subjected to feature extraction using the root mean square method. Therefore, in the relaxed state, the relaxed muscle force value M of the healthy side of the patient can be obtained respectively. healthy_relax and the relaxed muscle strength value M of the affected side affected_relax ; In the fist-clenching state, the maximum muscle strength value M of the patient's healthy side can be obtained respectively healthy_max and the maximum muscle strength value M of the affected side affected_max ;
[0092] The weighted average of the muscle strength values in the bilateral relaxation state and the fist clenching state was calculated to obtain the comprehensive relaxation muscle strength value M relax and the comprehensive maximum muscle strength value M max_muscle .
[0093] M relax =(M healthy_relax +M affected_relax ) / 2
[0094] M max_muscle =(M healthy_max +M affected_max ) / 2
[0095] Taking two different percentages of the comprehensive maximum muscle strength value to obtain two thresholds M th1 and M th2 , and modify the two thresholds into the program of the 3D virtual training scene module:
[0096] M th1 =M max_muscle ×0.2
[0097] Mth2 =M max_muscle ×0.6
[0098] Start the VR program and use the handle to select the training content on the virtual main menu interface, including upper limb / lower limb training, bed / sitting training, bilateral / unilateral training, learning / training mode, training scene, number of training, etc. Then put the VR helmet on the patient and adjust the appropriate position and angle;
[0099] Inform the patient of the content and requirements of this training, and prepare to start motor imagery training after the patient understands it;
[0100] While the patient observes the virtual limbs performing actions from the first person perspective in the virtual reality environment, he / she needs to carry out motor imagery training of corresponding content, and there are corresponding voice instructions for prompting. During the training, the signal acquisition module will collect electromyographic signals in real time, and before the end of this motor imagery training, the signal analysis module will perform the same preprocessing and feature extraction on the electromyographic signals to obtain the muscle strength value of the current training, thereby quantifying the participation degree of this motor imagery training and comparing it with the two thresholds M. th1 and M th2 The patient's training performance is determined by comparison and the results are fed back into the virtual reality training scene. The patient repeats the motor imagery training according to the number of training times;
[0101] The current training muscle strength values and performance scores obtained from multiple training sessions were analyzed to evaluate the patient's participation in the motor imagery training process and the rehabilitation level of the trained limb.
[0102] The key point of the present invention is to only extract features from the electromyographic signal as the patient's current muscle strength value, that is, to quantify the patient's participation in motor imagery training and feed it back to the virtual training scene; and the feedback parameter of the prior art is the brain participation index, which is obtained by weighted average calculation of motor imagery idea features, attention intensity idea features, visual induced idea features and electromyographic intensity idea features. Both of these indicators are feedback parameters for patients to train. The main difference is that the present invention focuses on the changes in electromyographic signals, and although the prior art also uses electromyographic signals as characteristic values, it pays more attention to the impact of changes in brain electrical signals. The present invention extracts features from the electromyographic signal by the root mean square method, and considers this to be the patient's muscle strength value for this training. This single parameter feedback can more conveniently regulate the patient's training performance results.
[0103] The inventive point of the present invention is to analyze the different muscle strength values obtained by the patient in different states (relaxed state and clenched fist state) and multiple motor imagery training to evaluate the rehabilitation level of the patient's training limb. The evaluation device of the prior art determines the training evaluation parameters according to the brain engagement index, the movement state index, multiple EEG signals, and electromyographic signals to evaluate the training state. The prior art involves the evaluation of multiple physiological signals, which can comprehensively evaluate the patient's training. Although the present invention only focuses on the parameter of electromyographic signal, there are differences in muscle strength values in different states and different trainings, as well as different thresholds for comparison, which will make the evaluation of the patient's training limb muscle strength more accurate.
[0104] Example 2
[0105] This embodiment provides a motor imagery rehabilitation training method based on embodiment 1, such as Figure 2 :
[0106] Collecting physiological signals of the patient, wherein the physiological signals include electroencephalogram signals and electromyography signals;
[0107] The EEG signal is analyzed to obtain the EEG activity mapping topography, and the EMG signal is analyzed to obtain the muscle force value;
[0108] Select the VR scene and training mode for motor imagery training;
[0109] According to the muscle strength value, the corresponding virtual animation is played in the corresponding motor imagery training scene.
[0110] Select the scene and training mode of motor imagery training, and play the corresponding virtual animation in the corresponding scene of motor imagery training according to the muscle strength value. The specific steps are as follows:
[0111] Before starting training, show the initial scene;
[0112] Select patient type whether bedridden or seated, limb trained whether upper or lower, unilateral or bilateral limb training, training setting, learning mode or training mode, and number of training cycles;
[0113] Display the start button;
[0114] After pressing the button, switch from the initial scene to the selected training scene;
[0115] After the patient completes the training according to the instructions, he / she will determine whether the current number of training times has reached the number of training cycles. If not, continue to repeat the training. If so, end the training cycle.
[0116] The above-mentioned lying in bed or sitting in bed are modes available in each training scenario. The lying in bed mode is suitable for training patients who cannot sit in bed, especially for patients with flaccid paralysis caused by stroke. The sitting in bed mode is used for patients who can sit in bed by themselves or with assistance.
[0117] The above-mentioned upper limb training and lower limb training are different for different stroke patients, and the training movements of upper limbs and lower limbs are very different. Therefore, according to the patient's current recovery situation, the upper limbs and lower limbs are trained in a targeted manner to make them more coordinated;
[0118] Bilateral training and unilateral training. The symptoms of stroke patients are generally hemiplegia. According to the current rehabilitation status of the patient, you can choose to train only the affected limb or train both the affected and healthy sides at the same time.
[0119] Scene selection is to select appropriate training scenes for motor imagery training according to the patient's condition;
[0120] Learning mode and training mode. The function of learning mode is to let patients clearly understand the detailed actions and precautions that need to be completed for motor imagery training. This mode plays voice instructions and displays corresponding text in the scene. The function of training mode is to allow patients to conduct immersive training after clarifying the training requirements and goals. Different from the learning mode, the voice instructions in this mode are short and the interference factors are minimized as much as possible, so that patients can fully participate in motor imagery training;
[0121] The number of training cycles can be set to determine how many times the patient needs to perform motor imagery training in each training cycle;
[0122] Taking the upper limb training scenario as an example, a basketball shooting training scenario and a fruit picking training scenario are provided. The basketball shooting training scenario is bilateral training, while the fruit picking training scenario is unilateral training.
[0123] Shooting training scenario (learning mode):
[0124] Guide VR scene: You can see your arms through VR glasses, now please imagine that these are your hands. And you can see that you are in the basketball court, you need to pick up the basketball in front of you and throw it into the basketball hoop. The stronger your shooting power and the better your performance, the higher your score will be. Now please follow the voice instructions and guidance, concentrate and join in the shooting training.
[0125] Step 1 VR scene: Please slowly raise your arms, stretch your hands forward, and rotate your arms so that the palms of your left and right hands are facing each other. Spread your fingers, then increase the strength of your hands and hold the basketball in front of you.
[0126] Step 2 of VR scene: Continue to hold the basketball tightly with both hands, then bend your arms and move the basketball to your chest.
[0127] Step 3 VR scene: Keep your hands tightly holding the basketball, now slowly lift your arms up to drive the basketball so that the basketball and your nose are at the same height.
[0128] Step 4 VR scene: Now, please continue to hold the basketball tightly and focus on the basketball hoop in front of you. At this time, feel that your arms and hands are full of strength. Use both hands to throw the basketball towards the basketball hoop in front of you.
[0129] Shooting training scene (training mode):
[0130] Guide VR scene: You can see your arms through VR glasses, now please imagine that these are your hands. And you can see that you are in the basketball court, you need to pick up the basketball in front of you and throw it into the basketball hoop. The stronger your shooting power and the better your performance, the higher your score will be. Now please follow the voice instructions and guidance, concentrate, and join in the shooting training.
[0131] Step 1 VR scene: Raise both arms, increase the strength of both hands, and hold the basketball.
[0132] Step 2 VR scene: Hold the basketball tightly and bend your arms.
[0133] The third step of VR scene: raise your arms and move the basketball.
[0134] Step 4 VR scene: Use both hands to shoot.
[0135] Fruit picking training scenario (learning mode):
[0136] Guided VR scene: You can see your arm through VR glasses. Now please imagine that this is your left (right) hand. You can also see that you are in an orchard. There are three different heights of fruits on the fruit trees. You need to hold the scissors in front of you and cut the branches. The more force you apply to the scissors, the higher the fruit you can pick, and the higher the score. Now please follow the voice instructions and guidance, concentrate, and join the fruit picking training.
[0137] Step 1 VR scene: Please stretch out your left (right) hand forward and slowly rotate your arm so that your palm is facing downward.
[0138] Step 2 VR scene: Focus on the height of the scissors in front of you. Slowly raise your left (right) hand so that your palm and the scissors are at the same height.
[0139] Step 3 VR scene: Please slowly rotate your arm so that your palm is facing the scissors handle, open your fingers, and then move your left (right) hand so that the scissors handle is in your palm.
[0140] Step 4 VR scene: Now, feel your left (right) arm and fingers are full of strength, bend your fingers and hold the scissors tightly to cut the branch.
[0141] Fruit picking training scene (training mode):
[0142] Guided VR scene: You can see your arm through VR glasses. Now please imagine that this is your left (right) hand. You can also see that you are in an orchard. There are three different heights of fruits on the fruit trees. You need to hold the scissors in front of you and cut the branches. The more force you apply to the scissors, the higher the fruit you can pick, and the higher the score. Now please follow the voice instructions and guidance, concentrate, and join the fruit picking training.
[0143] Step 1 VR scene: Stretch out your left (right) hand.
[0144] Step 2 VR scene: Raise your left (right) hand.
[0145] Step 3 VR scene: Move your left (right) hand and hold the scissors.
[0146] Step 4 VR scene: Use your left (right) arm and fingers to cut off the branches.
[0147] During the patient's motor imagery, the signal acquisition module will collect the patient's EEG and EMG signals in real time, and the data analysis module will analyze the EEG and EMG signal data before the end of each training session. The final eigenvalue is input into the scene intelligent control module through analysis methods such as filtering and denoising, extracting eigenvalues, and the training results are fed back to the patient in real time in a visual manner. In this embodiment, the collected EMG signals of the healthy side and the affected side are subjected to root mean square processing to obtain the eigenvalue of the muscle force value. And by performing a threshold comparison with the previously obtained maximum muscle force value, the training results are fed back to the virtual scene;
[0148] In this embodiment, in the shooting training scene, there are three results: the basketball cannot hit the rim, the basketball bounces after hitting the rim, and the basketball is thrown into the basket; in the fruit picking scene, there are fruits at three different heights on the fruit tree. When the muscle strength obtained through analysis is stronger, the result animation with better performance is selected to provide visual feedback to the patient;
[0149] After each training session, the virtual scene training module will compare the patient's current training times to see if they have reached the number of times in this training cycle. If not, the patient needs to train again; if so, the patient's training is over and he or she can take off the helmet, while the medical staff can put on the virtual reality wearable module again to reselect the menu function options and prepare for the next training cycle.
[0150] After the patient completes this training, the medical staff can once again use the clinical scale questionnaire to evaluate the patient's current limb level and motor function;
[0151] Finally, the two clinical scale questionnaires were analyzed to evaluate the degree of improvement in the patient's limb level and motor function before and after training; the EEG activity mapping topography obtained from the EEG signal acquisition module can evaluate which areas of the brain are activated by motor imagery, and what changes are caused to the alpha and beta wave frequency bands; the changes in muscle strength obtained from the electromyographic signal acquisition module can be used as an evaluation indicator of the limb rehabilitation level. These evaluation methods can be used to comprehensively evaluate the rehabilitation effect of this training cycle on patients.
[0152] A detailed motor imagery training program of the present embodiment, in terms of scene content, each scene is upper limb training, which is divided into four steps. The patient can complete a complete training action according to the four steps. The training actions of shooting and picking fruits mobilize all the joints of the upper limbs. In addition, two versions of voice instructions and text are provided, so that patients can better understand the actions that need to be completed regarding motor imagery. In addition, considering that the physical conditions of patients in different stages of flaccid paralysis are different, the program also provides the option of choosing to sit in a chair or lie in bed for training. The prior art provides upper limb training and lower limb training for patients who need rehabilitation training. The training content is relatively simple, mainly based on repeated execution of a certain action. During the motor imagery training process of the rehabilitation training recipient, the voice induction module plays the prompt voice related to the task. The training action designed in this embodiment is to allow the various joints of the patient's affected limb to participate, while the prior art imagines repeated actions so as to better extract the active movement intention feature signal.
[0153] Regarding personalized training, this embodiment needs to collect the electromyographic signals of the bilateral upper limbs of each patient in the flaccid paralysis stage of stroke before the experiment, use the root mean square value method to extract the electromyographic signal features and use them as the maximum muscle strength of each patient's current training limb (healthy side and affected side), and take two appropriate thresholds. The strength of the muscle strength analyzed by the patient in subsequent training will be fed back to the virtual scene in real time, and the corresponding result animation will be selected after comparison with the two thresholds to feed back to the patient. The prior art requires patients to learn in advance before training, and perform motor imagination for a period of time while maintaining a static state. The EEG signals and electromyographic signals collected in the process are used as training data to determine the regression parameters and brain movement participation thresholds when the motor imagination features are determined. This embodiment uses electromyographic signals as a manifestation of the strength of the muscle strength when the patient performs motor imagination, while the prior art uses electromyographic signals as an indicator of brain movement participation.
[0154] The same or similar reference numerals correspond to the same or similar components;
[0155] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;
[0156] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A motor imagery training system based on 3D virtual reality technology with real-time feedback, characterized in that: It includes a virtual reality wearable module, a 3D virtual training scene module, a signal acquisition module, a signal analysis module, a control module and an evaluation module, among which: The signal acquisition module is used to collect physiological signals of the patient and transmit the physiological signals to the signal analysis module; The signal analysis module extracts features from the physiological signal to obtain a physiological signal feature value; The control module controls the 3D virtual training scene module according to the physiological signal characteristic value; The 3D virtual training scene module provides virtual scenes with different training methods and contents. After selecting the training method and content to be performed, the 3D virtual training scene module provides the corresponding virtual training scene to the virtual reality wearable module; The wearable virtual reality module selects the training method and content according to the 3D virtual training scene module, and presents the motor imagery training in a visual manner; The evaluation module quantifies the patient's participation in motor imagery training each time according to all the above modules, and evaluates the patient's limb rehabilitation level; The signal analysis module analyzes the EEG signal to obtain an EEG activity mapping topography, specifically: First, the EEG signal is filtered, and then the 18 channels of EEG signals collected are downsampled to 256Hz. Then, the alpha and beta wave frequency bands are extracted from the signal, and the EEG spectrum density is calculated. Finally, the event-related synchronization and event-related desynchronization features are extracted, and the EEG activity mapping topography is drawn according to the location of the electrodes corresponding to the 18 channels. The signal analysis module analyzes the electromyographic signal to obtain a muscle force value, first filtering and denoising the electromyographic signal, using the root mean square root and the electromyographic integral value as the characteristic value in the time domain to obtain the muscle force value, which indicates the strength of the muscle; The control module controls the 3D virtual training scene module according to the physiological signal characteristic value, specifically: When the physiological signal characteristic value is greater than a preset threshold, the control module enables the virtual reality wearable module to visually present the motor imagery training; When the physiological signal characteristic value is less than a preset threshold, the control module does not enable the virtual reality wearable module to visually present the motor imagery training.
2. The motor imagery training system based on 3D virtual reality technology real-time feedback according to claim 1 is characterized in that: The virtual reality wearable module includes wearable glasses, a locator and an interactive device, wherein: The wearable glasses are used to present the motor imagery training in a visual manner in a virtual space; The locator is used to locate the patient in the virtual space; The interactive device interacts with the virtual space.
3. The motor imagery training system based on 3D virtual reality technology real-time feedback according to claim 2 is characterized in that: The interactive device includes a virtual handle and a keyboard and mouse.
4. The motor imagery training system based on 3D virtual reality technology real-time feedback according to claim 3 is characterized in that: The 3D virtual training scene module provides upper limb / lower limb training, bed / sitting training, unilateral / bilateral training, learning / training mode, training scene, and training times.
5. The motor imagery training system based on 3D virtual reality technology real-time feedback according to claim 1 is characterized in that: The signal acquisition module acquires the electromyographic signals and electroencephalographic signals generated during the patient's motor imagery.
6. The motor imagery training system based on 3D virtual reality technology real-time feedback according to claim 5 is characterized in that: In the signal acquisition module, myoelectric electrode patches, myoelectric electrode wires and acquisition equipment are used to collect myoelectric signals, wherein the myoelectric electrode patches are attached to the muscles of the patient's bilateral training limbs, one end of the myoelectric electrode wire is connected to the myoelectric electrode patch, and the other end is connected to the acquisition equipment, and two effective acquisition channels are respectively set in the patient's left hand and right hand to respectively collect the myoelectric signals of the flexor pollicis longus and the flexor carpi radialis, wherein the electrodes of the flexor digitorum superficialis on both sides are used as reference electrodes for the left hand and the right hand, the sampling rate is set to 1200Hz, and the bandpass filter is set to 0.5Hz-100Hz; An 80-channel bioelectric signal acquisition system and a 64-channel active electrode system were used to collect EEG signals. The electrode position of the 64-channel active electrode system was positioned according to the international standard 10-20 electrode lead. The reference electrode was located at the left earlobe, and the ground electrode was located at the AFZ. There were 18 acquisition channels in total: FC5, FC3, FC1, FC2, FC4, FC6, C5, C3, C1, C2, C4, C6, CP5, CP3, CP1, CP2, CP4, and CP6. The sampling rate was set to 1200 Hz, and the bandpass filter was set to 0.5 Hz-100 Hz.
7. The motor imagery training system based on 3D virtual reality technology real-time feedback according to claim 5 is characterized in that: The acquisition of the preset threshold is specifically as follows: When the patient is in a relaxed state, obtain the relaxed muscle strength value M of the healthy side of the patient. healthy_relax and the relaxed muscle strength value M of the affected side affected_relax ; In the fist-clenching state, obtain the maximum muscle strength value M of the patient's healthy side. healthy_max and the maximum muscle strength value of the affected side; The weighted average of the muscle strength values in the bilateral relaxation state and the fist clenching state was calculated to obtain the comprehensive relaxation muscle strength value M relax and the comprehensive maximum muscle strength value M max_muscle : M relax =(M healthy_relax +M affected_relax ) / 2 M max_muscle =(M healthy_max +M affected_max ) / 2 Taking two different percentages of the comprehensive maximum muscle strength value to obtain two thresholds M th1 and M th2 , and store the two thresholds in the 3D virtual training scene module: M th1 =M max_muscle ×0.2 M th2 =M max_muscle ×0.6。
