Brain-computer interaction system for cognitive-motion dual-task collaborative rehabilitation training

Abstract

The invention discloses a brain-computer interaction system for cognitive-motion dual-task collaborative rehabilitation training. The brain-computer interaction system comprises a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit and a motion training unit, according to the system provided by the invention, cognition-movement collaborative rehabilitation training is realized, compared with a single rehabilitation training mode, the rehabilitation efficiency is improved, a brain-computer interface technology is introduced, the attention and the movement intention of a subject are identified in real time, active movement rehabilitation is realized, the active participation degree of the subject in the rehabilitation training process is enhanced, a closed-loop system is established, and the rehabilitation training efficiency is improved. Mutual promotion of cognitive-motor function rehabilitation is enhanced, and induction of nerve recombination and reconstruction is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical fields of brain-computer interaction, motor function rehabilitation training and cognitive function rehabilitation training, and specifically to a brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training. Background Art

[0002] According to data from the National Bureau of Statistics, at the end of 2023, my country's population aged 60 and over will be 296.97 million, accounting for 21.1% of the national population, and is increasing year by year. The incidence of stroke is growing in tandem with aging. According to the "China Stroke Prevention and Treatment Report (2023)", the current number of stroke patients aged 40 and over in my country has reached 12.42 million, and the onset population is younger. Among stroke survivors, about 75% have sequelae and 40% are severely disabled. In addition, aging has also brought about neurodegenerative diseases such as dementia and mild cognitive impairment, which have brought huge burdens to families and society. According to research statistics from The Lancet, my country is the country with the largest rehabilitation needs in the world, with a total number of 460 million people in need of rehabilitation.

[0003] Brain function remodeling is mainly aimed at patients with neurodegenerative diseases and functional deficits caused by brain damage. Specific training equipment and methods are used to promote the relearning of brain damage or redundant nerves, thereby achieving the recovery or compensation of human functions. The essence of brain function remodeling is mainly based on the plasticity mechanism of the brain. The main rehabilitation training methods currently used generally use simple active or passive therapies, such as exercise, physics, homework and speech, to stimulate the central nervous system of the brain to achieve functional recovery, but the rehabilitation efficiency is low and the rehabilitation effect is limited.

[0004] Motor learning in the rehabilitation process is a process of acquiring skilled movements, and it is also a complex process of sensation / cognition / behavior. Therefore, rehabilitation is often not the rehabilitation of a single function, but a process of rehabilitation of multiple functions such as movement, cognition, and speech, and various rehabilitations affect each other. Among them, cognitive rehabilitation training is a training method aimed at improving individual thinking and perceptual abilities. Cognitive training aims to enhance cognitive functions such as memory, attention, problem solving, logical reasoning, language ability, and spatial perception. And motor rehabilitation training is a training method aimed at restoring the patient's limb motor ability. Studies have shown that aerobic exercise combined with cognitive training can enhance the plasticity of the prefrontal lobe, and can significantly improve the daily cognitive ability and life treatment of the elderly. However, at present, different functional rehabilitations are trained separately, the rehabilitation efficiency needs to be improved, and there is a lack of cognitive-motor synergistic training.

[0005] In addition, the brain-computer interface (BCI) can establish information exchange and communication between the brain and external devices, and can realize the interaction between brain thinking activities and external devices, which provides the possibility for active rehabilitation. Active rehabilitation can increase the patient's active participation, promote neural plasticity and improve rehabilitation effects.

[0006] In the prior art, there is a lack of a rehabilitation training method that combines brain-computer interface and cognitive-motor collaborative training. Summary of the invention

[0007] The purpose of the present invention is to provide a brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, including a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit, and a motor training unit;

[0008] The EEG signal acquisition unit acquires the EEG signal of the subject and transmits it to the EEG signal decoding unit;

[0009] The EEG signal decoding unit decodes the EEG signal, identifies the cognitive training content of the subject, and outputs the decoding result to the motor training unit and the cognitive training interaction unit; the decoding result includes attention concentration, the maximum correlation coefficient representing the logical reasoning cognitive training content, and the maximum correlation coefficient representing the spatial perception cognitive training content;

[0010] The movement training unit generates movement control instructions based on the subject's concentration, gaze target, and movement intention, thereby driving the subject's limbs to perform movement training;

[0011] The cognitive training interaction unit presents an interaction interface of cognitive training and brain-computer interface paradigm, as well as decoding results of the EEG signal decoding unit.

[0012] Furthermore, the signal output end of the EEG signal acquisition unit is connected to the signal input end of the EEG signal decoding unit, the signal output end of the EEG signal decoding unit is connected to the signal input end of the motor training unit, and the signal output end of the EEG signal decoding unit is connected to the signal input end of the cognitive training interaction unit.

[0013] Furthermore, when the cognitive training content is attention cognitive training content, the EEG signal decoding unit processes the EEG signal of the frontal area, calculates the power spectrum density in different frequency bands, and determines the increase or decrease of the subject's attention concentration based on the increase or decrease trend of the power spectrum density in different frequency bands corresponding to multiple consecutive time windows;

[0014] When the cognitive training content is logical reasoning cognitive training content, the EEG signal decoding unit uses a logical reasoning cognitive related decoding algorithm to process the EEG signals of multiple channels in the visual area to obtain a maximum correlation coefficient for characterizing the logical reasoning cognitive training content; the logical reasoning cognitive related decoding algorithm includes a canonical correlation analysis algorithm;

[0015] When the cognitive training content is spatial perception cognitive training content, the EEG signal decoding unit uses a spatial perception cognitive related decoding algorithm to perform spatial filtering processing on the multi-channel EEG signals of the visual area to obtain the maximum correlation coefficient for characterizing the spatial perception cognitive training content; the spatial perception cognitive related decoding algorithm includes a task discriminant component analysis algorithm.

[0016] Furthermore, the EEG signal decoding unit performs bandpass filtering on the EEG signal before processing the EEG signal.

[0017] Further, the motion training unit includes a motion control module and a motion feedback module;

[0018] The motion control module generates a motion control instruction based on the decoding result, and transmits it to the motion feedback module, controlling the motion feedback module to drive the subject's limbs to perform motion training.

[0019] The motion feedback module includes but is not limited to a pneumatic robot and an exoskeleton robot.

[0020] Furthermore, the movement feedback module drives the speed of the subject's movement, and the cognitive training interaction unit shows that the speed of the subject's limb movements is positively correlated with the degree of concentration.

[0021] Furthermore, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions;

[0022] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and presents the corresponding action picture.

[0023] Furthermore, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions;

[0024] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and presents the corresponding action picture.

[0025] Furthermore, the cognitive training interaction unit includes one or more of a display, a virtual reality system, or augmented reality glasses.

[0026] Furthermore, the EEG signal acquisition unit includes a plurality of electrodes, which acquire EEG signals on the scalp surface of the subject;

[0027] The sampling frequency of the EEG signal acquisition unit exceeds 250 Hz.

[0028] The technical effect of the present invention is unquestionable. The system provided by the present invention realizes cognitive-motor coordinated rehabilitation training, improves rehabilitation efficiency compared to a single rehabilitation training model, and introduces brain-computer interface technology to identify the subject's attention and movement intention in real time, realize active movement rehabilitation, enhance the subject's active participation in the rehabilitation training process, establish a closed-loop system, strengthen the mutual promotion of cognitive-motor function rehabilitation, and enhance the induction of neural reorganization and reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A brain-computer interaction rehabilitation training system for cognitive-motor coordination;

[0030] Figure 2 It is a sequence image of ball holding action;

[0031] Figure 3 These are diagrams of the index finger clicking the mouse, the thumb and index finger pinching the dice, the paper cup holding and the ball holding action. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.

[0033] Embodiment 1:

[0034] See also Figure 1-Figure 3 , a brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, including a cognitive training interaction unit, an EEG signal acquisition unit, an EEG signal decoding unit, and a motor training unit;

[0035] The EEG signal acquisition unit acquires the EEG signal of the subject and transmits it to the EEG signal decoding unit;

[0036] The EEG signal decoding unit decodes the EEG signal, identifies the cognitive training content of the subject, and outputs the decoding result to the motor training unit and the cognitive training interaction unit; the decoding result includes attention concentration, the maximum correlation coefficient representing the logical reasoning cognitive training content, and the maximum correlation coefficient representing the spatial perception cognitive training content;

[0037] The movement training unit generates movement control instructions based on the subject's concentration, gaze target, and movement intention, thereby driving the subject's limbs to perform movement training;

[0038] The cognitive training interaction unit presents an interaction interface of cognitive training and brain-computer interface paradigm, as well as decoding results of the EEG signal decoding unit.

[0039] The signal output end of the EEG signal acquisition unit is connected to the signal input end of the EEG signal decoding unit, the signal output end of the EEG signal decoding unit is connected to the signal input end of the motor training unit, and the signal output end of the EEG signal decoding unit is connected to the signal input end of the cognitive training interaction unit.

[0040] When the cognitive training content is attention cognitive training content, the EEG signal decoding unit processes the frontal EEG signal, calculates the power spectrum density in different frequency bands, and determines the increase or decrease of the subject's attention concentration based on the increase or decrease trend of the power spectrum density in different frequency bands corresponding to multiple consecutive time windows; for example, if the power spectrum density in the α frequency band calculated in four consecutive time windows increases and the power spectrum density in the β frequency band decreases, it is considered that the attention concentration decreases, and vice versa, it is considered that the attention concentration increases;

[0041] When the cognitive training content is logical reasoning cognitive training content, the EEG signal decoding unit uses a correlation decoding algorithm such as a canonical correlation analysis algorithm to process the EEG signals of multiple channels in the visual area to obtain a maximum correlation coefficient for characterizing the logical reasoning cognitive training content;

[0042] When the cognitive training content is spatial perception cognitive training content, the EEG signal decoding unit uses a relevant decoding algorithm such as a task discriminant component analysis algorithm to perform spatial filtering processing on multi-channel EEG signals in the visual area to obtain a maximum correlation coefficient for characterizing the spatial perception cognitive training content.

[0043] Before the EEG signal decoding unit processes the EEG signal, it also performs band-pass filtering on the EEG signal.

[0044] The motion training unit includes a motion control module and a motion feedback module;

[0045] The motion control module generates a motion control instruction based on the decoding result, and transmits it to the motion feedback module, controlling the motion feedback module to drive the subject's limbs to perform motion training.

[0046] The motion feedback module includes but is not limited to a pneumatic robot and an exoskeleton robot.

[0047] The speed of the subject's movement driven by the motion feedback module and the cognitive training interaction unit showed that the speed of the subject's limb movements was positively correlated with the degree of concentration.

[0048] The motion control module determines the most relevant action based on the maximum correlation coefficient representing the content of the logical reasoning cognitive training, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions, such as hand movements that represent the numbers 0 to 5 with one hand;

[0049] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and presents the corresponding action picture.

[0050] The motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions, such as clicking a mouse, holding a ball, holding a cup, and squeezing a dice;

[0051] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and presents the corresponding action picture.

[0052] The cognitive training interaction unit includes one or more of a display, a virtual reality system, or augmented reality glasses.

[0053] The EEG signal acquisition unit includes a plurality of electrodes, which acquire EEG signals on the scalp surface of the subject;

[0054] The sampling frequency of the EEG signal acquisition unit exceeds 250 Hz.

[0055] Embodiment 2:

[0056] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, comprising a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit, and a motor training unit;

[0057] The EEG signal acquisition unit acquires the EEG signal of the subject and transmits it to the EEG signal decoding unit;

[0058] The EEG signal decoding unit decodes the EEG signal, identifies the cognitive training content of the subject, and outputs the decoding result to the motor training unit and the cognitive training interaction unit; the decoding result includes attention concentration, the maximum correlation coefficient representing the logical reasoning cognitive training content, and the maximum correlation coefficient representing the spatial perception cognitive training content;

[0059] The movement training unit generates movement control instructions based on the subject's concentration, gaze target, and movement intention, thereby driving the subject's limbs to perform movement training;

[0060] The cognitive training interaction unit presents an interaction interface of cognitive training and brain-computer interface paradigm, as well as decoding results of the EEG signal decoding unit.

[0061] Embodiment 3:

[0062] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as that of Example 2, further, the signal output end of the EEG signal acquisition unit is connected to the signal input end of the EEG signal decoding unit, the signal output end of the EEG signal decoding unit is connected to the signal input end of the motor training unit, and the signal output end of the EEG signal decoding unit is connected to the signal input end of the cognitive training interaction unit.

[0063] Embodiment 4:

[0064] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of embodiments 2-3, and further, when the cognitive training content is attention cognitive training content, the EEG signal decoding unit processes the frontal area EEG signal, calculates the power spectrum density in different frequency bands, and determines the increase or decrease of the subject's attention concentration based on the increase or decrease trend of the power spectrum density in multiple consecutive different frequency bands;

[0065] When the cognitive training content is logical reasoning cognitive training content, the EEG signal decoding unit uses a correlation decoding algorithm such as a canonical correlation analysis algorithm to process the EEG signals of multiple channels in the visual area to obtain a maximum correlation coefficient for characterizing the logical reasoning cognitive training content;

[0066] When the cognitive training content is spatial perception cognitive training content, the EEG signal decoding unit uses a relevant decoding algorithm such as a task discriminant component analysis algorithm to perform spatial filtering processing on multi-channel EEG signals in the visual area to obtain a maximum correlation coefficient for characterizing the spatial perception cognitive training content.

[0067] Embodiment 5:

[0068] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of embodiments 2-4, and further, the EEG signal decoding unit performs bandpass filtering on the EEG signal before processing the EEG signal.

[0069] Embodiment 6:

[0070] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of embodiments 2-5, and further, the motor training unit includes a motor control module and a motor feedback module;

[0071] The motion control module generates a motion control instruction based on the decoding result, and transmits it to the motion feedback module, controlling the motion feedback module to drive the subject's limbs to perform motion training.

[0072] The motion feedback module includes but is not limited to a pneumatic robot and an exoskeleton robot.

[0073] Embodiment 7:

[0074] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of Examples 2-6. Furthermore, the speed at which the subject's movement is driven by the movement feedback module, and the speed at which the subject's limb movements are displayed by the cognitive training interaction unit are positively correlated with the degree of concentration.

[0075] Embodiment 8:

[0076] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of embodiments 2-7, and further, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions;

[0077] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and presents the corresponding action picture.

[0078] Embodiment 9:

[0079] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of embodiments 2-8, further, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions;

[0080] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and presents the corresponding action picture.

[0081] Embodiment 10:

[0082] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of Examples 2-9, and further, the cognitive training interaction unit includes one or more of a display, a virtual reality system, or augmented reality glasses.

[0083] Embodiment 11:

[0084] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, the technical content of which is the same as any one of embodiments 2-10, and further, the electroencephalogram signal acquisition unit includes a plurality of electrodes, which acquire electroencephalogram signals on the scalp surface of the subject;

[0085] The sampling frequency of the EEG signal acquisition unit exceeds 250 Hz.

[0086] Embodiment 12:

[0087] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, comprising: a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit, and a motor training unit;

[0088] The signal output end of the EEG signal acquisition unit is connected to the signal input end of the EEG signal decoding unit, the signal output end of the EEG signal decoding unit is connected to the signal input end of the motor training unit, and the signal output end of the EEG signal decoding unit is connected to the signal input end of the cognitive training interaction unit;

[0089] The cognitive training interaction unit includes a paradigm presentation module for presenting an interaction interface between cognitive training and brain-computer interface paradigm, and presenting the decoding results of the EEG signal decoding unit in a time-sharing manner;

[0090] The EEG signal acquisition unit comprises: an EEG signal acquisition module, which is used to acquire the EEG signal of the subject and output it to the EEG signal decoding unit;

[0091] The EEG signal decoding unit includes: an EEG signal decoding module, which uses different preset machine learning algorithms for different cognitive-motor collaborative rehabilitation training tasks and difficulties, identifies the subject's attention concentration, gaze target, action intention, etc. according to the EEG signal, and outputs the results to the motor training unit and cognitive training interaction unit;

[0092] The motion training unit includes: a motion control module and a motion feedback module, which is used to generate motion control instructions to the motion control module based on the output of the EEG signal decoding unit, and the motion control module controls the motion feedback module to drive the subject's limbs to perform corresponding motion training.

[0093] According to a specific implementation method, in the above-mentioned cognitive-motor coordinated brain-computer interaction rehabilitation training system, the paradigm presentation module is presented by a display, a virtual reality system or augmented reality glasses, and the presentation content includes a paradigm combining cognitive training of different difficulties such as attention, logical reasoning, spatial perception, etc. with a brain-computer interface visual paradigm and the output result of the EEG signal decoding module.

[0094] The sampling frequency of the above-mentioned EEG signal acquisition module exceeds 250 Hz;

[0095] The above-mentioned EEG signal decoding module for receiving the above-mentioned EEG signal adopts different EEG signal decoding methods for presenting different cognitive training contents in the paradigm presentation module: when presenting the attention cognitive training contents, the EEG signal is band-pass filtered, the power spectrum density in different frequency bands is calculated for the EEG signal in the frontal area, and the attention concentration is decoded based on the strength of the power spectrum density in different frequency bands; when presenting the logical reasoning cognitive training contents, the EEG signal is band-pass filtered, and then the EEG signal of the multi-channel in the visual area is processed by a related decoding algorithm such as the canonical correlation analysis (CCA) algorithm to obtain the maximum correlation coefficient; when presenting the spatial perception cognitive training contents, the EEG signal is band-pass filtered, and then the EEG signal of the multi-channel in the visual area is spatially filtered by a related decoding algorithm such as the task discriminant component analysis (TDCA) algorithm, and then the maximum correlation coefficient is obtained;

[0096] The above motion control module and motion feedback module are configured as follows:

[0097] Based on the strength of the attention concentration obtained by the EEG signal decoding module, a control instruction for controlling the speed of the movement feedback module is generated, and the movement feedback module is controlled to drive the subject's limbs to perform movement training at different speeds, and at the same time, the speed of the limb movement in the limb movement video in the paradigm presentation module is controlled: the higher the attention concentration, the faster the movement, and vice versa;

[0098] Based on the maximum correlation coefficient obtained by the CCA obtained by the EEG signal decoding module, control instructions for controlling different actions of the motion feedback module are generated, and the motion feedback module is controlled to drive the subject's limbs to perform corresponding actions, and at the same time, the paradigm presentation module presents the same action picture;

[0099] Based on the maximum correlation coefficient obtained by TDCA obtained by the EEG signal decoding module, control instructions for controlling different actions of the motion feedback module are generated, and the motion feedback module is controlled to drive the subject's limbs to perform corresponding actions, and at the same time, the paradigm presentation module presents the same action picture;

[0100] The motion feedback module includes: a pneumatic robot, an exoskeleton robot, and an end effector.

[0101] Embodiment 13:

[0102] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training comprises a cognitive training interaction unit and an electroencephalogram (EEG) signal acquisition unit, wherein the output end of the EEG signal acquisition unit is connected to the input end of an EEG signal decoding unit, the output end of the EEG signal decoding unit is connected to the input end of a motor training unit, and the output end of the EEG signal decoding unit is connected to the input end of the cognitive training interaction unit.

[0103] The EEG signal acquisition module is in accordance with the international standard 10 / 20 system, with the forehead Fpz as the ground pole and the left earlobe as the reference, to collect the EEG signals of the subject's Fp1, Fp2, AFz, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC4, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P3, Pz, P4, PO7, PO3, POz, PO4, PO8, O1, Oz and O2, a total of 32 channels distributed on the scalp surface of the frontal lobe, parietal lobe, occipital lobe, temporal lobe, etc., with a sampling frequency of 250 Hz or above;

[0104] The cognitive training interaction unit is presented by a display, a virtual reality system or augmented reality glasses;

[0105] According to the preset cognitive scale, cognitive ability test is carried out to obtain cognitive test results, such as Mini-Mental State Examination (MMSE), battery test method (LOTCA software), Wechsler Memory Scale (WMS), Montreal cognitive assessment (MoCA), to screen abnormal cognitive functions in cognitive fields including visual-spatial function, naming, attention, repeated sentences, fluency, abstract ability, delayed recall, orientation, etc.; according to the cognitive test results, the difficulty of the cognitive-motor synergy brain-computer interaction rehabilitation training method is selected; the cognitive-motor synergy brain-computer interaction rehabilitation training method includes three difficulties: Difficulty 1: Attention-motor synergy training brain-computer interaction training, Difficulty 2: Logical reasoning-motor synergy training brain-computer interaction training, Difficulty 3: Spatial perception attention compound-motor synergy training brain-computer interaction training. This implementation case carries out hand motor function rehabilitation training, but the present invention is not limited to hand motor function rehabilitation training.

[0106] Attention-Motion Coordination Training Brain-Computer Interaction Training:

[0107] The paradigm was presented on the presentation module using a frame-variation-based stimulation pattern. The screen refresh rate was 60 Hz, i.e., 60 frames per second. Stimuli were generated by a frame rate reduction (FRR) method. The steps of FRR were as follows: first, a video of an action, such as thumb movement or grasping, was recorded; second, M images of an action cycle were extracted from the video; third, the M images were presented using Psychophysics Toolbox, and the presentation time of each image was controlled. Each image lasted N / 60s (the screen refresh rate was 60 Hz).

[0108] Take the ball holding action as an example. Figure 2 As shown, each frame is extracted from a video. The same image will last for N / 60s (N ≥ 4 in this study), followed by the next different image. Therefore, the designed paradigm frame rate is reduced to 60 / N (the traditional video frame rate is 30Hz). In one motion cycle, there are M images (M = 16 in the figure), so the motion frequency is 60 / (M×N). The normal speed of motion is guaranteed by designing M and N (60 / (M×N) < 1Hz).

[0109] The paradigm presentation module presents the above paradigm all the time, and the EEG signal acquisition module acquires the EEG signal of the subject in real time, intercepts the EEG signal with a time window length of 3s and a time interval sliding window of 0.5s, performs a Butterworth filter of 3-30Hz on the acquired EEG signal, and then processes the EEG signals of the three channels of the frontal area Fp1, Fp2, and AFz, and uses the Welch method to calculate the power spectral density of the two frequency bands of 8Hz to 13Hz and 13Hz to 30Hz, which are recorded as α and β respectively. If the α calculated in four consecutive time windows increases and β decreases, it is considered that the concentration of attention decreases, otherwise it is considered that the concentration of attention increases;

[0110] The motion control module described in this implementation case has been monitoring whether there are control instructions sent in the TCP / IP communication or serial communication channel. The motion feedback module is a pneumatic rehabilitation glove, which realizes the inflation and deflating of the pneumatic glove by controlling the inflation pump, the exhaust pump and the three-way solenoid valve through the single-chip microcomputer, so as to drive the subject's hand to perform reciprocating grasping movements; after receiving the information of reduced concentration transmitted by the EEG signal decoding module, the single-chip microcomputer controls the opening and closing frequency of the inflation pump and the exhaust pump to reduce the speed of the pneumatic glove grasping movement, so as to reduce the frequency of driving the subject's hand to perform grasping movements; at the same time, the ball gripping speed in the paradigm is reduced by outputting instructions to the paradigm presentation module; so that the subject can intuitively feel the reduction in concentration, and by adjusting his own attention, the concentration level is improved to speed up the movement speed;

[0111] Logical reasoning-motor coordination training Brain-computer interaction training:

[0112] The conventional light flicker paradigm based on steady-state visual evoked potential is used to present the paradigm on the presentation module, and a total of six light flicker paradigms with different flicker frequencies (f1, f2, f3, f4, f5, f6) are presented, and the flicker frequency is between 4 and 40 Hz, such as 5 Hz, 6 Hz, 7 Hz, 8 Hz, 9 Hz, and 10H. There is a number in the center of the light flicker paradigm of each frequency, which is 1, 2, 3, 4, 5, and 0 respectively; after the cognitive-motor synergistic rehabilitation training begins, the paradigm presentation module first presents a random arithmetic problem, which is addition, subtraction, multiplication, and division of a certain A two-digit operation, the operation result is between 0 and 5, the arithmetic problem is presented for 3 seconds, during which the subject calculates the result of the arithmetic problem in his mind, and then the paradigm presentation module presents six light flashing paradigms with different flashing frequencies. According to the result calculated by the subject, the subject fixates on the light flashing stimulation paradigm of the corresponding number. If the calculation result is 3, the subject fixates on the light flashing paradigm with the number 3 below. The light flashing paradigm lasts for 5 seconds, and then the paradigm presentation module presents the decoding result of the EEG signal decoding module for 2 seconds; and then the above method is repeated;

[0113] The EEG signal decoding module adopts a canonical correlation analysis algorithm, firstly performs a 3-30 Hz Butterworth filter on the collected EEG signal, then selects the first-order and second-order sine and cosine functions of f1, f2, f3, f4, f5, and f6 as the template function of the algorithm, calculates the correlation coefficient between the EEG signals of the visual areas PO7, PO3, POz, PO4, PO8, O1, Oz, and O2 channels collected by the EEG acquisition module and the template function, and the frequency corresponding to the maximum correlation coefficient is the recognition result (one of 0, 1, 2, 3, 4, and 5), and transmits the result back to the paradigm presentation module to display the number corresponding to the recognition result;

[0114] The motion control module described in this implementation case has been monitoring whether there are control instructions sent in the TCP / IP communication or serial communication channel. The motion feedback module is a pneumatic rehabilitation glove. The microcontroller controls 5 groups of inflating and exhausting air pumps and a three-way solenoid valve to realize the inflation and exhaust of the pneumatic glove, so as to drive the subject to present six movements of numbers 0 to 6 with one hand; when receiving the recognition result from the EEG signal decoding module, such as 3, the pneumatic glove is controlled to drive the subject to pose 3. If the recognition result is inconsistent with the actual result of the arithmetic problem presented by the paradigm presentation module, the pneumatic glove will not move and remain in the initial open state;

[0115] Spatial perception, attention, compound-motor coordination training, brain-computer interaction training:

[0116] The paradigm is presented on the presentation module using a frame-changing stimulation mode. Four actions are presented on the screen. The design method is consistent with that described above. The action has certain task attributes. This case uses four actions: index finger click mouse action, thumb and index finger pinch dice action, paper cup holding action, and ball holding action. Figure 3 As shown, the frequencies of the four actions are different;

[0117] When the cognitive-motor synergistic rehabilitation training begins, the center of the paradigm presentation module first randomly presents pictures of objects related to the four action paradigms. In this case, one of the four objects, a mouse, a dice, a paper cup, and a massage ball, is presented for 3 seconds; then, Figure 3 In the action paradigm shown, each action in the paradigm is performed at a different frequency. The subject looks at the corresponding action based on the object presented in the center in front. If a mouse appears, the subject continues to look at the action paradigm of clicking the mouse with the index finger until the paradigm disappears. The presentation time of the four paradigms is 5 seconds. Finally, the action picture corresponding to the recognition result transmitted by the EEG signal decoding module is presented in the center of the paradigm presentation module. Then the cycle is repeated in the above manner.

[0118] The EEG signal decoding module adopts the TDCA algorithm, firstly collects the EEG signals of the visual areas PO7, PO3, POz, PO4, PO8, O1, Oz and O2 channels collected by the EEG acquisition module under the four action paradigms of the subject as a training set and constructs a spatial filter, uses the constructed spatial filter to process the EEG signals of the visual areas PO7, PO3, POz, PO4, PO8, O1, Oz and O2 channels collected by the EEG acquisition module in the cognitive-motor collaborative rehabilitation training, and calculates the correlation coefficient with the training set data, the frequency corresponding to the maximum correlation coefficient is the recognition result (one of the four actions), and the result is transmitted back to the paradigm presentation module to display the action corresponding to the recognition result;

[0119] The motion control module described in this implementation case has been monitoring whether there are any control instructions sent in the TCP / IP communication or serial communication channel. The motion feedback module is a pneumatic rehabilitation glove, which realizes the inflation and exhaust of the pneumatic glove by controlling 5 groups of inflation air pumps and exhaust air pumps and a three-way solenoid valve through a single-chip microcomputer, so as to drive the subject to present the four corresponding actions in the action paradigm with one hand, namely, the index finger action, the index finger and thumb finger-to-finger action, the light grasping action and the heavy grasping action. When the recognition result from the EEG signal decoding module is received, such as the index finger clicking the mouse action, the pneumatic glove is controlled to drive the index finger of the subject to move. If the recognition result is inconsistent with the prompt task presented by the paradigm presentation module, the pneumatic glove will not move and remain in the initial open state;

[0120] In the embodiments of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

Claims

1. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, characterized by: It includes a cognitive training interaction unit, an EEG signal acquisition unit, an EEG signal decoding unit, and a motor training unit. The EEG signal acquisition unit acquires the EEG signal of the subject and transmits it to the EEG signal decoding unit; The EEG signal decoding unit decodes the EEG signal, identifies the cognitive training content of the subject, and outputs the decoding result to the motor training unit and the cognitive training interaction unit; the decoding result includes attention concentration, the maximum correlation coefficient representing the logical reasoning cognitive training content, and the maximum correlation coefficient representing the spatial perception cognitive training content; The movement training unit generates movement control instructions based on the subject's concentration, gaze target, and movement intention, thereby driving the subject's limbs to perform movement training; The cognitive training interaction unit presents an interaction interface of cognitive training and brain-computer interface paradigm, as well as decoding results of the EEG signal decoding unit.

2. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The signal output end of the EEG signal acquisition unit is connected to the signal input end of the EEG signal decoding unit, the signal output end of the EEG signal decoding unit is connected to the signal input end of the motor training unit, and the signal output end of the EEG signal decoding unit is connected to the signal input end of the cognitive training interaction unit.

3. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: When the cognitive training content is attention cognitive training content, the EEG signal decoding unit processes the frontal EEG signal, calculates the power spectrum density in different frequency bands, and determines the increase or decrease of the subject's attention concentration based on the increase or decrease trend of the power spectrum density in different frequency bands corresponding to multiple consecutive time windows; When the cognitive training content is logical reasoning cognitive training content, the EEG signal decoding unit uses a logical reasoning cognitive related decoding algorithm to process the EEG signals of multiple channels in the visual area to obtain a maximum correlation coefficient for characterizing the logical reasoning cognitive training content; the logical reasoning cognitive related decoding algorithm includes a canonical correlation analysis algorithm; When the cognitive training content is spatial perception cognitive training content, the EEG signal decoding unit uses a spatial perception cognitive related decoding algorithm to perform spatial filtering processing on the EEG signals of multiple channels in the visual area to obtain a maximum correlation coefficient for characterizing the spatial perception cognitive training content; The spatial perception cognition related decoding algorithm includes a task discriminant component analysis algorithm.

4. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 3, characterized in that: Before the EEG signal decoding unit processes the EEG signal, it also performs band-pass filtering on the EEG signal.

5. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The motion training unit includes a motion control module and a motion feedback module; The motion control module generates a motion control instruction based on the decoding result, and transmits it to the motion feedback module, so as to control the motion feedback module to drive the subject's limbs to perform motion training. The motion feedback module includes but is not limited to a pneumatic robot and an exoskeleton robot.

6. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 5, characterized in that: The speed of the subject's movement driven by the motion feedback module and the cognitive training interaction unit showed that the speed of the subject's limb movements was positively correlated with the degree of concentration.

7. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 5, characterized in that: The motion control module determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions; The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and presents the corresponding action picture.

8. The brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 5, characterized in that: The motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform corresponding actions; The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and presents the corresponding action picture.

9. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The cognitive training interaction unit includes one or more of a display, a virtual reality system, or augmented reality glasses.

10. The brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The EEG signal acquisition unit includes a plurality of electrodes, which acquire EEG signals on the scalp surface of the subject; The sampling frequency of the EEG signal acquisition unit exceeds 250 Hz.

Images