A method and system for upper and lower limb passive rehabilitation training based on motor imagery brain-computer interface

Abstract

The application relates to the field of rehabilitation, in particular to a passive and active rehabilitation training method and system for upper and lower limbs based on a motor imagery brain-computer interface, which creates two game experiment paradigms of active mode induction and passive mode induction to guide patient training; brain-computer interfaces are used to collect electroencephalogram signals and electromyogram signals, motor intention of a stroke patient is extracted from the electroencephalogram signals, and the two game experiment paradigms are used to guide the patient to actively or passively carry out rehabilitation training, so that the rehabilitation training process of the patient is accelerated. Motor imagery is evaluated by using electromyogram signals in combination with expert experience, and according to the evaluation result, the parameter setting of the experiment paradigm and the parameter setting of the motor auxiliary function in the corresponding mode are adjusted, the parameter setting includes gradually changing the impedance level, the power level and other parameters of the system in the training process, the subjective initiative of the patient is improved, the central nervous stimulation feedback of the patient is strengthened, the muscle strength of the patient is enhanced, and the overall rehabilitation effect of the patient is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rehabilitation, in particular to a passive-active upper and lower limb rehabilitation training method and system based on motor imagery brain-computer interface. BACKGROUND

[0002] Stroke is the leading cause of death among Chinese residents, with high incidence, high mortality and high disability rate. Its common symptoms include a sharp decline in the motor function and sensory function of one side of the body. How to help patients recover limb function is an important research topic in the field of medical rehabilitation. With the development of human-computer intelligence and robot technology, the use of rehabilitation robots, especially upper and lower limb rehabilitation training robot systems, to help patients with limb rehabilitation training has gradually become an important means of rehabilitation treatment. However, motor dysfunction in stroke patients is caused by damage to the patient's neural circuit in the brain. The upper and lower limb rehabilitation training system alone can only provide passive rehabilitation assistance training for the upper and lower limbs of the patient. Due to the lack of brain involvement (i.e., active rehabilitation training of the patient), the rehabilitation effect of the rehabilitation robot system is limited.

[0003] Brain-computer interface refers to a direct connection channel between the brain of a human or animal and an external device that enables information interaction between the human brain and the device. Motor imagery brain-computer interface can collect the brain electrical signals during the patient's motor imagery process to obtain the patient's motor intention, thereby controlling the rehabilitation robot device to assist the patient in completing the corresponding rehabilitation training action. It has great application potential in the field of rehabilitation. Recent research shows that motor imagery brain-computer interface technology combined with rehabilitation robot devices for rehabilitation training can significantly promote the neural remodeling process of the damaged part of the patient, and its rehabilitation effect is much better than that of traditional single motor rehabilitation methods.

[0004] Chinese patent with publication number CN116059588A discloses a passive-active upper and lower limb rehabilitation training device. The device is mainly optimized in structure for the proportion defect of the patient's limbs, so that special patient groups such as poliomyelitis patients can normally use the rehabilitation training device. However, it does not consider the limitations of the upper and lower limb rehabilitation training system for functional recovery of patients with brain defects.

[0005] Chinese patent application with publication number CN116386807A discloses a rehabilitation training method and system based on motor imagery and event-related potentials. The algorithm uses transfer learning to reduce the time and number of initial model training for the patient, while reducing the influence of patient specificity on system classification accuracy. It helps the patient quickly enter an effective rehabilitation training state. The introduction of VR virtual reality environment simulation mirror training optimizes the immersive experience of the patient, while the detection of P300 potential expands the patient's autonomous instruction set, allowing the patient to choose a rehabilitation training mode autonomously, and improving the patient's training enthusiasm.

[0006] The Chinese patent application with the publication number CN115944851A discloses a "VR immersion-based electric stimulation motor imagery rehabilitation training method", which changes the rehabilitation mode and stimulation level according to the setting of the rehabilitation mode to achieve the goal of phased rehabilitation. First, the state of the patient is judged to belong to the level of rehabilitation, and the patient is first provided with motor imagery and mirror movement in the game scene to help the patient learn bionics and cooperative learning through the game picture to establish a classification model. Then, in the online stage, the patient's imagination feedback is judged according to the classification model, and corresponding stimulation is performed according to different training requirements of different levels, which can quickly repair the damaged nerves in the brain and make the patient consolidate muscle memory to achieve the goal of muscle strength training in each stage, thereby achieving the purpose of rehabilitation.

[0007] The above two methods are both through designing different games to guide patients to perform motor imagery and perform result feedback, but the main rehabilitation training object is the upper limbs of the patient, and the lower limb rehabilitation training is not considered, and the system cannot be adjusted according to the training condition of the patient, which affects the overall rehabilitation effect of the patient. SUMMARY

[0008] The purpose of the present application is to provide a motor imagery brain-computer interface-based upper and lower limb active and passive rehabilitation training method and system, which extracts the motor intention of a stroke patient through a motor imagery brain-computer interface, thereby controlling an upper and lower limb rehabilitation training system to assist the patient in completing synchronous training of the upper and lower limbs, and gradually changing the impedance level and assistance level of the system during the training process, thereby improving the patient's subjective initiative, strengthening the patient's central nervous system stimulation feedback, and enhancing the patient's muscle strength, which greatly improves the overall rehabilitation effect of the patient.

[0009] To achieve the above purpose, the present application adopts the following technical solutions:

[0010] A motor imagery brain-computer interface-based upper and lower limb active and passive rehabilitation training method, comprising the following steps:

[0011] Step 1, creating two game experiment paradigms of active mode guiding patient training and passive mode guiding patient training;

[0012] Step 2, collecting the electroencephalogram signals and electromyogram signals generated by the patient in performing the two training game paradigms, and assigning a label to each electroencephalogram signal;

[0013] Step 3, preprocessing the collected electroencephalogram signals and electromyogram signals;

[0014] Step 4, using a pre-trained motor imagery classification model to classify and identify the preprocessed electroencephalogram signals to obtain a motor imagery classification result for judging the patient's intention;

[0015] Step 5, generating a control instruction according to the obtained motor imagination classification result, and performing passive mode guided patient training game experiment paradigm controlled movement auxiliary function or active mode guided patient training game experiment paradigm controlled movement auxiliary function according to the control instruction;

[0016] The passive mode guided patient training game experiment paradigm controlled movement auxiliary function is to drive the patient to perform synchronous upper limb and lower limb movement by using the passive mode guided patient training game experiment paradigm, so as to complete the movement rehabilitation training.

[0017] The active mode control logic controlled movement auxiliary function is to make the patient autonomously perform upper limb and lower limb rehabilitation training synchronized with the active mode guided patient training experiment paradigm based on the patient's motor imagination, and record the electromyographic signal of the patient's movement.

[0018] Step 6, after the patient completes the phase training, the muscle strength and motor imagination of the patient are evaluated in combination with the preprocessed electromyographic signal and expert experience, and the game experiment paradigm parameters and movement auxiliary function parameters in the corresponding mode are adjusted according to the evaluation result.

[0019] Further, the passive mode guided patient training game experiment paradigm in step 1 adopts a racing game, and the active mode guided patient training game experiment paradigm adopts a bicycle riding game, and the game interfaces of the two game experiment paradigms both include a patient simulation role and a guide picture; after setting the initial speed of the racing car or bicycle, different changing game scenes and related props are set to guide the patient to perform corresponding motor imagination to generate brain electrical signals.

[0020] Further, the racing game of the passive mode guided patient training game experiment paradigm is designed as follows:

[0021] In the racing game scene, a car representing the patient, a plurality of NPC cars marked with different speeds, a left brake pedal shape and an accelerator pedal shape are constructed, the car representing the patient and the plurality of NPC cars travel in the same direction, and the patient is guided to perform left / right foot motor imagination to decelerate / accelerate through vehicle racing and obstacle avoidance of the car representing the patient, after determining the patient's movement intention according to the motor imagination classification result, the corresponding pedal shape is pressed dynamically, and the vehicle speed changes, so as to strengthen the feedback to the patient's motor imagination.

[0022] The bicycle riding game of the active mode guided patient training game experiment paradigm is designed as follows:

[0023] A game scene of cycling is built with a character representing the patient, a bicycle, left / right pedal animation, and three driving sections, namely, a flat road section, an uphill road section, and a downhill road section. According to the muscle strength of the patient, each driving section is configured as a different impedance mode (impedance and assistance) of auxiliary movement, and the impedance and assistance levels of the flat road section are set to 0, the impedance level of the uphill road section is increased, and the assistance level of the downhill road section is increased. The patient triggers the character to ride forward / stop through left / right foot movement imagination, and the probability of each section in the game is changed by detecting the upper and lower limb muscle strength of the patient.

[0024] Further, the step 3 of preprocessing the collected electroencephalogram signal includes channel selection, re-reference, filtering, downsampling, and epochs segmentation; and the step 3 of preprocessing the collected electromyogram signal includes filtering, downsampling, and epochs segmentation, to obtain electroencephalogram signal sample data.

[0025] Further, the step 3 of using the pre-trained motor imagination classification model training method includes the following steps:

[0026] (1) Real-time collection of electroencephalogram signals of the patient performing two game experiment paradigms and labeling as sample data; the sample data is preprocessed by the same method as the step 3 of preprocessing the electroencephalogram signal, to obtain electroencephalogram signal sample data;

[0027] (2) Feature extraction of the electroencephalogram signal sample data by using the common spatial pattern algorithm, to obtain motor imagination category features and label information of each electroencephalogram signal sample data;

[0028] (3) Calculation of the normalized covariance matrix of each motor imagination sample data according to the motor imagination category features of each electroencephalogram signal sample data, and then calculation of the mixed spatial covariance matrix;

[0029] (4) Obtaining the whitening feature matrix by using the orthogonal whitening transformation of the mixed spatial covariance matrix, and then transforming the covariance matrix of each motor imagination by using the whitening matrix, and then performing principal vector decomposition to obtain the feature vector matrix of each motor imagination sample;

[0030] (5) Multiplication of the whitening feature matrix and the feature vector matrix obtained in (4) to obtain the spatial filter of all motor imagination samples;

[0031] (6) Multiplication of each motor imagination sample and the corresponding spatial filter, and variance normalization, to obtain the feature vector of each electroencephalogram signal sample data; and generation of the support vector machine classifier based on the Gaussian kernel of each patient according to the feature vector and the corresponding label;

[0032] (7) The model parameters of the support vector machine classifier are trained by completely separating all categories of motor imagination sample data by minimizing classification errors and maximizing classification intervals; thus obtaining a motor imagination classification model for judging the movement intention of a patient.

[0033] A passive rehabilitation training system for upper and lower limbs based on a motor imagination brain-computer interface, comprising:

[0034] An experimental paradigm module for presenting a training game paradigm guided in an active mode or a passive mode to a patient;

[0035] A data acquisition module for acquiring electroencephalogram signals and electromyogram signals generated by a patient in performing a training game paradigm in real time, and transmitting the electroencephalogram signals to a data preprocessing module after being labeled;

[0036] A data preprocessing module for preprocessing the received electroencephalogram signals and electromyogram signals respectively; transmitting the preprocessed electroencephalogram signals to an online test module, and transmitting the preprocessed electromyogram signals to an evaluation module;

[0037] An online test module for classifying the preprocessed electroencephalogram signals using a pre-trained motor imagination classification model to obtain a motor imagination classification result for judging the intention of a patient; and broadcasting the motor imagination classification result of the patient through a udp protocol to convey to a motion control module and an experimental paradigm module under the same network frequency band;

[0038] A motion control module for a rehabilitation training system independently operated for an upper limb and a lower limb, comprising an active mode control logic and a passive mode control logic; generating a control instruction according to the received motor imagination classification result, and executing an active mode guided patient training game experimental paradigm controlled motion assistance function or a passive mode guided patient training game experimental paradigm controlled motion assistance function according to the control instruction. The passive mode guided patient training game experimental paradigm controlled motion assistance function is to drive the patient to perform synchronous upper and lower limb motion by using the passive mode guided patient training game experimental paradigm, to complete the motion rehabilitation training; and the active mode control logic controlled motion assistance function is to enable the patient to independently perform upper and lower limb rehabilitation training synchronized with the active mode guided patient training experimental paradigm based on the motor imagination of the patient, and record the electromyogram signals of the patient, which are transmitted to the evaluation module in sequence through the acquisition module and the preprocessing module;

[0039] An evaluation module for evaluating the muscle strength and motor imagination of a patient after completing a phase of training in combination with the received electromyogram signals and expert experience, and adjusting the parameter settings of the experimental paradigm and the parameter settings of the motion assistance function in the corresponding mode according to the evaluation result.

[0040] Further, the upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface is also provided with a model training module, the model training module extracts features from the received electroencephalogram signal sample data by using common spatial pattern, and obtains motor imagery category features and label information of each end electroencephalogram signal sample data; and a motor imagery classification model is trained according to the motor imagery category features and the label information.

[0041] The upper and lower limb passive rehabilitation training method and system based on motor imagery brain-computer interface provided by the application are created on the basis of combining the brain-computer interface rehabilitation method and the upper and lower limb passive rehabilitation training method, and two game experiment paradigms of active mode guiding patient training and passive mode guiding patient training are matched; the electroencephalogram signal and the electromyogram signal are collected through the brain-computer interface, the motor intention of the stroke patient is extracted by using the electroencephalogram signal, the patient is guided to actively or passively perform rehabilitation training in cooperation with the two game experiment paradigms, and the rehabilitation training process of the patient is accelerated. The motor imagery is evaluated by using the electromyogram signal combined with expert experience, the parameter setting of the experiment paradigm and the parameter setting of the motor auxiliary function in the corresponding mode are adjusted according to the evaluation result, the parameter setting includes gradually changing the impedance level, the power level and other parameters of the system in the training process, the subjective initiative of the patient is improved, the central nervous stimulation feedback of the patient is strengthened, and the muscle strength of the patient is also enhanced, so that the overall rehabilitation effect of the patient is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is the upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface provided by the embodiment;

[0043] Figure 2 is the upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface provided by the embodiment;

[0044] Figure 3 is the upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface provided by the embodiment, which is in the passive mode guided training game paradigm and the schematic diagram;

[0045] Figure 4 is the upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface provided by the embodiment, which is in the active mode guided training game paradigm and the schematic diagram;

[0046] Figure 5 is the electroencephalogram acquisition channel and the electromyogram acquisition channel schematic diagram of the upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface provided by the embodiment, wherein a is the electroencephalogram signal acquisition schematic diagram, and b is the electromyogram signal acquisition schematic diagram. DETAILED DESCRIPTION

[0047] The technical scheme of the application will be described in detail below with reference to the drawings.

[0048] The embodiment provides a kind of upper and lower limb passive rehabilitation training method based on motor imagery brain-computer interface, comprising the following steps:

[0049] Step 1, create two game experiment paradigms of active mode guiding patient training and passive mode guiding patient training. The passive mode guiding patient training game experiment paradigm adopts a racing game, and the active mode guiding patient training game experiment paradigm adopts a bicycle riding game. Both of the two game experiment paradigms are made by E-PRIME and Unity software, and the user subject is a free racing car and bicycle model resource from Asset Store imported in Unity, and the visual interface is provided with a patient simulation role and a guide picture; after setting the initial speed of the racing car or bicycle, the corresponding motor imagery of the patient is guided by setting different change game scenes and related props to generate electroencephalogram signals.

[0050] The passive mode guiding patient training game experiment paradigm is set with reference to Figure 3 The car in the center of the visual interface and marked with an arrow represents the patient, and the arrow indicates the forward direction of the patient. A plurality of npc cars marked with different speeds travel on both sides of the patient's car and move forward in the same direction as the patient's car. The left foot brake pedal shape is set in the lower left of the visual interface, and the right foot accelerator pedal shape is set in the lower right. The vehicle racing and obstacle avoidance of the visual interface guide the patient to perform left / right foot motor imagery to reduce / speed up. After determining the patient's movement intention according to the motor imagery classification result, the corresponding pedal shape is pressed dynamically, and the vehicle speed changes to strengthen the feedback to the patient's motor imagery. The initial speed of the vehicle representing the patient is set to 15 revolutions per minute (rpm) in this embodiment, and the speeds of the other npc cars are set to integers between 20 and 35 rpm. Under the passive model, the motor imagery paradigm has a total duration of 12s, 0-2s is used to guide the patient to perform left / right foot motor imagery through visual pictures and voice prompts, 2-6s is used for the patient to perform motor imagery according to the prompts, 6-12s is used to feedback the motor imagery classification result through visual pictures and voice, and the motor auxiliary joint speed is adjusted, to complete a single trial experiment. The speed change step is 5 rpm each time, the E-PRIME experiment paradigm includes 3 rounds, each round includes 5 blocks, and each block includes 10 experimental trials, and the duration of a single experimental trial is 12s.

[0051] The active mode guiding patient training game experiment paradigm is set with reference to Figure 4 .

[0052] The central figure in the visual interface represents the patient, and the arrow indicates the patient's forward direction. Different road segments such as flat road segments, uphill road segments, and downhill road segments are set in the game, and left / right foot stepping animations are displayed on the lower left and right sides of the visual interface. The patient triggers the figure to ride forward / stop by imagining left / right foot movement. The probability of changing road segments in the game is determined by detecting the patient's upper and lower limb muscle strength. Different road segment designs represent different impedance modes (assist level and impedance level) of the motion assistance module. The assist level and impedance level are both 0 on flat ground, the impedance level increases on uphill road segments, and the assist level increases on downhill road segments. The impedance and assist levels depend on the patient's muscle strength assessment. The impedance level increases on uphill road segments, and the picture prompts the patient to increase limb strength to complete the rehabilitation training action within the specified time. The assist level increases on downhill road segments, and the picture prompts the patient to relax the limbs to relieve limb fatigue. The total duration of the motor imagery paradigm in active mode is 12s. From 0s to 2s, the patient is prompted to perform left / right foot motor imagery through visual images and voice. From 2s to 6s, the patient performs motor imagery according to the prompt. From 6s to 12s, the visual image and voice feedback the classification results of motor imagery, and remind the patient to try to complete the specified rehabilitation training action independently. At the same time, the data acquisition module records the patient's upper and lower limb electromyographic signals during this period, realizing a single trial experiment. The E-PRIME experimental paradigm includes four rounds, each round contains 5 blocks, and each block contains 10 experimental trials. The duration of a single experimental trial is 12s.

[0053] Step 2, real-time acquisition of brain electrical signals and electromyographic signals generated by the patient during the execution of the two training game paradigms, and labeling each segment of the brain electrical signal.

[0054] The device used to collect brain electrical signals in this embodiment is the eego TM mylab 32 lead device manufactured by ANT neuro company of Germany. The device used to collect electromyographic signals is the high-performance electromyographic signal collector myoMUSCLE produced by Noraxon company of the United States. The brain electrical acquisition device includes 1 32-channel electroencephalogram cap, 1 32-channel 16kHz amplifier, and 1 terminal notebook equipped with eego64 brain electrical signal acquisition software. The brain electrical signal acquisition and processing software eego64 is used to present and process the received brain electrical signals in real time, and has an event marking function, i.e. labeling. The time position of the event marking is shown in Figure 2 and Figure 3 The brain cap is connected to the amplifier by wire, which is used to transmit the collected brain electrical data to the terminal notebook through the amplifier.

[0055] This embodiment of the electromyography (EMG) acquisition device consists of three parts: EMG acquisition and processing software, a data receiver, and an EMG data sensor. The EMG data sensor has a sampling frequency up to 4000Hz and is used to acquire and transmit EMG signals in real time. Data transmission between the EMG data sensor and the data receiver is wireless. The EMG acquisition and processing software is used to display and process the EMG signals received by the data receiver in real time.

[0056] Step 3: Preprocess the collected EEG and EMG signals.

[0057] Preprocessing of EEG signals includes: channel selection, rereference, filtering, downsampling, and epoch segmentation. Among these:

[0058] For channel selection processing, this embodiment selects the following electrode channels from the international 10-20 standard lead system: Fp1, Fp2, Fpz, C3, C4, Cz, T7, T8, FC1, FC2, Cp1, Cp2, Cp5, Cp6, as follows. Figure 5 As shown in (a). Rereference processing is used to avoid interference from multi-channel EEG signal activity on single-channel signals caused by volume conduction effects, as well as the loss of single-channel EEG signal information. In this embodiment, the M1 and M2 bilateral mastoid electrode channels are selected as reference channels, and the average value of the bilateral mastoid channel signals is subtracted from the signal of each EEG electrode channel to achieve EEG signal rereference. Filtering processing uses a Butterworth bandpass filter to filter the EEG signal to retain EEG signals in a specific frequency band (8-28Hz). In this embodiment, the upper limit frequency is 40Hz and the lower limit frequency is 5Hz. Downsampling processing is used to reduce the length of the acquired EEG signal to remove redundant data. In this embodiment, downsampling reduces the sampling rate from 1000Hz to 250Hz. Epoch segmentation processing is used to divide the acquired EEG signal into EEG signal data of different experimental cycles according to the labels. Data from 2-6 seconds in the active / passive mode experimental paradigm are taken as the EEG signal data of a single trail.

[0059] Preprocessing of electromyographic signals includes filtering, downsampling, and epoch segmentation. Specifically:

[0060] Filtering is used to retain the myoelectric signal data of a specific (8-28Hz) frequency band. The filtering operation uses a Butterworth band-pass filter with an upper limit frequency of 450Hz and a lower limit frequency of 10Hz; at the same time, a 50Hz filtering operation is performed to remove AC interference. The downsampling operation is used to reduce the length of the collected myoelectric signal, reducing the sampling rate from 2000Hz to 250Hz. Epoch segmentation is used to divide the collected myoelectric signal into different experimental times of myoelectric signal data according to the marker points. In this embodiment, the data of 6-12s in the experimental paradigm of the active / passive mode is taken as the myoelectric signal data of a single trail.

[0061] Step 4, using the pre-trained motor imagery classification model to classify and identify the pre-processed electroencephalogram signal, and obtaining the motor imagery classification result for judging the patient's intention. The pre-trained motor imagery classification model used in step 3 of this embodiment has the following training method:

[0062] Real-time acquisition of the patient's electroencephalogram signal in the two game experimental paradigms and labeling as sample data; the sample data is pre-processed in the same way as the electroencephalogram signal pre-processing in step 3, and after pre-processing, the electroencephalogram signal sample data of a single experiment trail is divided into 1000ms time windows with a window overlap rate of 50%. Then the common spatial pattern algorithm is used to extract features from the divided electroencephalogram signal segments and train the spatial filter.

[0063] Let X1 and X2 be the electroencephalogram data sets under the two types of motor imagery of the left and right feet, i.e. And C is the number of electroencephalogram channels, S is the number of samples collected per channel, m is the number of samples for each type of motor imagery, is a real number set.

[0064] For each type of motor imagery sample, calculate the normalized covariance matrix, and then calculate the mixed space covariance matrix, as shown in formulas (1) and (2):

[0065]

[0066] R = R1 + R2 (2)

[0067] Since the mixed space covariance matrix R is a positive definite matrix, the eigenvalue decomposition theorem is used for eigenvalue decomposition:

[0068] R = UΛU T (3)

[0069] U is the eigenvector matrix, Λ is the diagonal matrix of the corresponding eigenvalues, arranged in descending order of eigenvalues, and the whitening value matrix is:

[0070]

[0071] The covariance matrix of each class of motor imagery is transformed by using the whitening matrix, and then principal vector decomposition is performed to obtain the eigenvector matrix of each class of samples, as shown in formulas (5) and (6).

[0072] S1=PR1P T S2=PR2P T (5)

[0073]

[0074] As can be seen from formulas (5) and (6), the eigenvector matrix of the matrix S1 is equal to the eigenvector matrix of the matrix S2, that is, B1=B2=B. The eigenvector corresponding to the maximum eigenvalue of the matrix S1 makes the matrix S2 have the minimum eigenvalue, and vice versa. Therefore, the projection matrix W is:

[0075] W=BP (7)

[0076] The projection matrix W is the spatial filter corresponding to the CSP algorithm. For each motor imagery sample, the eigenvector of the sample can be obtained by the projection matrix W. The eigenvector obtained in a single trial is taken as a sample (a 1×T vector), and the eigenvectors extracted by CSP and the corresponding motor imagery labels of all trials are used to generate a support vector machine classifier based on a Gaussian kernel for each patient. The model parameters W of the support vector machine classifier are trained by minimizing the classification error and maximizing the classification interval to completely separate the two classes of samples. and B

[0077] In implementation, for the received preprocessed electroencephalogram signal X, the feature F extracted by CSP is calculated first, and then the size of FW+B is compared with 0 to determine whether the patient is left / right foot motor imagery. If FW+B>0, 1 is output to represent left foot motor imagery, and otherwise -1 is output to represent right foot motor imagery.

[0078] Step 5, a control instruction is generated according to the obtained motor imagery classification result, and a passive mode is guided to train the motion auxiliary function under the control of the game experiment paradigm or an active mode is guided to train the motion auxiliary function under the control of the game experiment paradigm.

[0079] In the passive mode, the motion control module is initially set to 15 rpm. The motor imagery paradigm time is 12 s, the system judges the instruction by extracting the CSP feature and the support vector machine classifier, and the animation of the car acceleration / deceleration is displayed in the interface, and the speed of the motion control module is increased / decreased synchronously.

[0080] In the active mode, the patient first performs motor imagery, and then performs upper and lower limb rehabilitation training movement autonomously with maximum strength. The motor imagery paradigm time is 12 s. During motor imagery, the system rotates the joint limit, which cannot be rotated. The system extracts the CSP features and judges the instructions by the support vector machine classifier. If the motor imagery is successful, the system cancloses the joint movement limit, the patient performs autonomous rehabilitation training, and the interface bicycle starts to move at the same joint rotation speed, and continues until the next motor imagery. The electromyographic signals of the rectus femoris muscle, biceps femoris muscle, tibialis anterior muscle, gastrocnemius muscle, biceps brachii muscle and triceps brachii muscle are recorded during the patient's autonomous rehabilitation training, as shown in (b) of the same figure. Figure 5 (b) shown. After completing the phase training, the patient's muscle strength is evaluated in combination with the electromyographic signals and expert experience, and the probability of the uphill and downhill scene changes and the resistance and assistance levels corresponding to the uphill and downhill scenes are introduced according to the muscle strength evaluation results.

[0081] Step 6, after the patient completes the phase training, the patient's muscle strength and motor imagery are evaluated in combination with the electromyographic signals and expert experience, and the game experiment paradigm parameters and motion assistance function parameters in the corresponding mode are adjusted according to the evaluation results.

[0082] In this embodiment, 30 rounds are defined as a phase. For the experimental paradigm module, the patient's motor imagery success rate is defined as the ratio of the number of successful motor imagery times to the total number of experiments within the specified motor imagery time. In the passive and active modes, if the patient's phase motor imagery accuracy is greater than or equal to 90%, the motor imagery time in the experimental paradigm is shortened by 0.5 s, until the patient's motor imagery time is 1 s. For the motion assistance module, the parameter adjustment involves the resistance and assistance levels of the system rotating joint.

[0083] In the active mode, the patient's muscle strength level is defined as 0-5 levels, which is consistent with the definition of medical muscle strength level. According to the amplitude, time and maximum muscle strength of the patient's electromyographic signals during the phase of completing autonomous rehabilitation training, the patient's limb muscle strength is evaluated in combination with expert experience. The resistance and assistance levels of the system joint of the motion assistance module are defined as 0-5 levels. The resistance levels are defined as 0%, 10%, 20%, 30%, 40%, 50% of the patient's current maximum muscle strength, and the assistance levels are defined as 0%, 20%, 40%, 60%, 80%, 100% of the patient's current maximum muscle strength. If the muscle strength level is defined as a level, the uphill scene rehabilitation training joint resistance of the system is defined as a level, the downhill scene rehabilitation training joint assistance level is defined as 5-a level, and the probabilities of flat, uphill and downhill are 50%, (10a)%, (50-10a)%.

[0084] Based on the above method, this embodiment also provides an upper and lower limb passive and active rehabilitation training system based on motor imagery brain-computer interface, as shown inFigure 1 and Figure 2 as shown, comprising:

[0085] An experimental paradigm module is configured to present a training game paradigm in an active mode or a passive mode to the patient.

[0086] A data acquisition module is configured to acquire electroencephalogram signals and electromyogram signals generated by the patient in real time during the execution of the training game paradigm, and to assign labels to the electroencephalogram signals, and then to transmit the electroencephalogram signals and the electromyogram signals synchronously to a data preprocessing module.

[0087] The data preprocessing module is configured to preprocess the electroencephalogram signals and the electromyogram signals, transmit the preprocessed electroencephalogram signals to an online test module, and transmit the preprocessed electromyogram signals to an evaluation module.

[0088] The online test module is configured to use a pre-trained motor imagery classification model to classify the preprocessed electroencephalogram signals to obtain a motor imagery classification result for determining the intention of the patient, and to broadcast the motor imagery classification result of the patient through a udp protocol to convey the motor imagery classification result to a motor control module and the experimental paradigm module in the same network frequency band.

[0089] The motor control module is a rehabilitation training system for an upper limb and a lower limb to operate independently, and includes active mode control logic and passive mode control logic. The motor control module is configured to generate a control instruction according to the received motor imagery classification result, and to execute an active mode guided patient training game experimental paradigm control function or a passive mode guided patient training game experimental paradigm control function according to the control instruction. The passive mode guided patient training game experimental paradigm control function is configured to drive the patient to perform synchronous upper limb and lower limb movement by using the passive mode guided patient training game experimental paradigm, and to complete motor rehabilitation training. The active mode control logic is configured to enable the patient to perform synchronous upper limb and lower limb rehabilitation training based on motor imagery, and to record electromyogram signals of the patient. The electromyogram signals are transmitted to the evaluation module in sequence through the acquisition module and the preprocessing module.

[0090] The evaluation module is configured to evaluate the muscle strength and motor imagery of the patient after the patient completes a stage of training, in combination with the received electromyogram signals and expert experience, and to adjust parameter settings of the experimental paradigm and parameter settings of the motor assistance function in the corresponding mode according to the evaluation result.

[0091] Due to the huge individual difference of the brain electrical signals of each subject, in order to obtain the best matching model for the user, the embodiment sets a model training module for the upper and lower limb passive rehabilitation training system. The model training module receives the preprocessed brain electrical signals, then adopts the common space mode to perform feature extraction on the received brain electrical signal sample data, and obtains the motor imagery category features and label information of each brain electrical signal sample data; and trains the motor imagery classification model according to the motor imagery category features and label information.

[0092] It can be understood that the present application is described by some embodiments, and those skilled in the art know that various changes or equivalent replacements can be made to the features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, the features and embodiments can be modified to adapt to specific conditions and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope of the present application.

Claims

1. A upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface, characterized in that, Comprise: An experimental paradigm module for presenting an active mode guided or passive mode guided training game paradigm to a patient; A data acquisition module for acquiring brain electrical signals and muscle electrical signals generated by the patient in performing the training game paradigm in real time, and after labeling the brain electrical signals, synchronously transmitting them to a data preprocessing module; The data preprocessing module respectively preprocesses the received brain electrical signals and muscle electrical signals; The preprocessed brain electrical signals are transmitted to an online test module, and the preprocessed muscle electrical signals are transmitted to an evaluation module; The online test module uses a pre-trained motor imagery classification model to classify the preprocessed brain electrical signals to obtain motor imagery classification results for judging the patient's intention; The patient's motor imagery classification results are broadcasted through the udp protocol to convey to the motion control module and the experimental paradigm module under the same network frequency band; The motion control module is a rehabilitation training system for an upper limb and a lower limb independently running, including active mode control logic and passive mode control logic; According to the received motor imagery classification results, control instructions are generated, and according to the control instructions, passive mode guided patient training game experimental paradigm controlled motion assistance function or active mode guided patient training game experimental paradigm controlled motion assistance function is executed; The passive mode guided patient training game experimental paradigm controlled motion assistance function is to drive the patient to perform synchronous upper limb and lower limb movement by using the passive mode guided patient training game experimental paradigm, and to complete the motion rehabilitation training; The active mode control logic based motion assistance function is that the patient independently performs upper limb and lower limb rehabilitation training synchronized with the active mode guided patient training experimental paradigm based on the patient's motor imagery, and records the patient's motion electromyography signal, which is transmitted to the evaluation module in turn through the acquisition module and the preprocessing module; The evaluation module is used to evaluate the patient's muscle strength and motor imagery after the patient completes the stage training, combined with the received electromyography signal and expert experience, and adjusts the parameter settings of the experimental paradigm and the parameter settings of the motion assistance function under the corresponding mode.

2. The upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface according to claim 1, characterized in that, The system also has a model training module, which uses common space pattern to extract features from the received brain electrical signal sample data, obtains motor imagery category features and label information of each end brain electrical signal sample data, and trains the motor imagery classification model according to the motor imagery category features and label information.

3. The upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface according to claim 1, characterized in that, The system is configured to perform the following steps: Step 1, create two game experimental paradigms of active mode guided patient training and passive mode guided patient training; Step 2, acquire brain electrical signals and muscle electrical signals generated by the patient in performing the two training game paradigms, and label each brain electrical signal; Step 3, preprocess the acquired brain electrical signals and muscle electrical signals; Step 4, use a pre-trained motor imagery classification model to classify and identify the preprocessed brain electrical signals to obtain motor imagery classification results for judging the patient's intention; Step 5, generating control instructions according to the obtained motor imagery classification results, and performing passive mode guided patient training game experiment paradigm controlled motion auxiliary function or active mode guided patient training game experiment paradigm controlled motion auxiliary function according to the control instructions; The passive mode guided patient training game experiment paradigm controlled motion auxiliary function is to drive the patient to perform synchronous upper and lower limb movement by using the passive mode guided patient training game experiment paradigm, so as to complete the motion rehabilitation training. The active mode controlled logic motion auxiliary function is to enable the patient to autonomously perform upper and lower limb rehabilitation training synchronized with the active mode guided patient training experiment paradigm based on the patient's motor imagery, and record the patient's motor electromyographic signal. Step 6, after the patient completes the phase training, the patient's muscle strength and motor imagery are evaluated in combination with the pre-processed electromyographic signal and expert experience, and the game experiment paradigm parameters and motion auxiliary function parameters in the corresponding mode are adjusted according to the evaluation results.

4. The upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface according to claim 3, characterized in that: The passive mode guided patient training game experiment paradigm in step 1 adopts a racing game, and the active mode guided patient training game experiment paradigm adopts a bicycle riding game, and the game interfaces of the two game experiment paradigms both include a patient simulation role and a guide picture. After setting the initial speed of the racing car or bicycle, different changing game scenes and related props are set to guide the patient to perform corresponding motor imagery to generate electroencephalogram signals.

5. The upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface according to claim 4, characterized in that, The racing game of the passive mode guided patient training game experiment paradigm is designed as follows: In the racing game scene, a car representing the patient, multiple npc cars indicating different speeds, a left brake pedal shape and a throttle pedal shape are constructed, the car representing the patient and the multiple npc cars travel in the same direction, the patient is guided to perform left / right foot motor imagery to slow down / accelerate through vehicle racing and obstacle avoidance, after the patient's movement intention is determined according to the motor imagery classification results, the corresponding pedal shape is pressed dynamically, and the vehicle speed changes to strengthen the feedback to the patient's motor imagery. The bicycle riding game of the active mode guided patient training game experiment paradigm is designed as follows: In the bicycle riding game scene, a character representing the patient and a bicycle, left / right foot pedal animation, and three types of travel sections are constructed, the three types of travel sections are flat road section, uphill road section and downhill road section; each type of travel section is configured as a different impedance mode of auxiliary movement including impedance and power assistance according to the patient's muscle strength, the flat road section impedance and power assistance level is set to 0, the uphill road section impedance level increases, and the downhill road section power assistance level increases; the patient triggers the character to ride forward / stop through left / right foot motor imagery, and the probability of changing various road sections in the game is determined by detecting the patient's upper and lower limb muscle strength.

6. The upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface according to claim 3, characterized in that: The pre-processing of the collected electroencephalogram signals in step 3 includes channel selection, re-reference, filtering, down-sampling, and epochs segmentation; the pre-processing of the collected electromyographic signals includes filtering, down-sampling, and epochs segmentation, to obtain electroencephalogram signal sample data.

7. The upper and lower limb passive rehabilitation training system based on motor imagery brain-computer interface according to claim 3, characterized in that: The pre-trained motor imagery classification model used in step 3 is trained by the following steps: Real-time acquisition of patient performing two game experiment paradigm of electroencephalogram and assign a label, as sample data; the same method as step 3 electroencephalogram signal preprocessing is used to preprocess sample data, and the electroencephalogram signal sample data is obtained; The common space mode algorithm is used for feature extraction of electroencephalogram signal sample data, and the motor imagination category feature and label information of each electroencephalogram signal sample data are obtained; After calculating the normalized covariance matrix of each electroencephalogram signal sample data according to the motor imagination category feature of each electroencephalogram signal sample data, the mixed space covariance matrix is calculated; The whitening feature matrix is obtained by using orthogonal whitening transformation to obtain the mixed space covariance matrix, then the covariance matrix of each motor imagination is transformed by using the whitening matrix, and then the principal vector decomposition is carried out, and the feature vector matrix of each motor imagination sample is obtained; The whitening feature matrix is multiplied with the feature vector matrix obtained in (4) to obtain the spatial filter for constructing all motor imagination samples; Each motor imagination sample is multiplied by the corresponding spatial filter, and the variance normalization is carried out, so that the feature vector of each electroencephalogram signal sample data is obtained; the support vector machine classifier based on Gaussian kernel of each patient is generated according to the feature vector and the corresponding label; By minimizing the classification error and maximizing the classification interval, all categories of motor imagination sample data are completely separated, and the model parameters of the support vector machine classifier are trained, so that the motor imagination classification model for judging the motor intention of the patient is obtained.

Images