A brain-computer interaction system for cognitive-motor dual task coordination rehabilitation training

Abstract

The application discloses a brain-computer interaction system for cognitive-motor dual task cooperative rehabilitation training, which comprises a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit and a motor training unit. The system realizes cognitive-motor cooperative rehabilitation training, improves rehabilitation efficiency compared with a single rehabilitation training mode, introduces brain-computer interface technology, realizes active motor rehabilitation, enhances the active participation of a subject in the rehabilitation training process, establishes a closed loop system, strengthens the mutual promotion of cognitive-motor function rehabilitation, and enhances the induction of neural reorganization and reconstruction.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of brain-computer interaction, motor function rehabilitation training and cognitive function rehabilitation training, and particularly relates to a brain-computer interaction system for cognitive-motor dual-task cooperative rehabilitation training. BACKGROUND

[0002] According to data from the National Bureau of Statistics, there were 296.97 million people aged 60 and above in China at the end of 2023, accounting for 21.1% of the total population, and the number is increasing year by year. Along with the aging, the incidence of stroke is also increasing. According to the China Stroke Prevention and Treatment Report (2023), the number of people with stroke in China is 12.42 million, and the incidence of the population is becoming younger. Among the stroke survivors, about 75% have sequelae, and 40% have severe disabilities. In addition, aging also brings neurodegenerative diseases such as dementia and mild cognitive impairment, which has brought a huge burden to families and society. According to research by The Lancet, China has the largest demand for rehabilitation in the world, with a total of 460 million people in need of rehabilitation.

[0003] Brain function remodeling mainly aims at the functional loss caused by neurodegenerative disease patients and brain injury, and uses specific training equipment and means to promote the relearning of damaged or redundant nerves in the brain, so as to realize the recovery or compensation of human function. The essence of brain function remodeling is mainly based on the plasticity mechanism of the brain. At present, the main rehabilitation training methods generally use simple active or passive therapies such as exercise, physical, occupational and speech therapies to stimulate the central nervous system of the brain to achieve functional recovery, but the rehabilitation efficiency is low and the rehabilitation effect is limited.

[0004] Motor learning in the rehabilitation process is a process of acquiring skillful movement, and is also a composite process of sensation / cognition / behavior. Therefore, rehabilitation is often not a single functional rehabilitation, but a process of multiple functional rehabilitation such as motor, cognitive, speech, etc., and various rehabilitation methods influence each other. Among them, cognitive rehabilitation training is a training method aimed at improving the thinking and perceptual abilities of individuals. Cognitive training aims to enhance cognitive functions such as memory, attention, problem solving, logical reasoning, language ability and spatial perception. Motor rehabilitation training is a training method aimed at restoring the motor ability of patients. Studies have shown that aerobic exercise combined with cognitive training can enhance the plasticity of the prefrontal cortex, and can significantly improve the daily cognitive ability and life treatment of the elderly. However, at present, different functional rehabilitation methods are trained separately, and the rehabilitation efficiency needs to be improved, and there is a lack of cognitive-motor cooperative training.

[0005] In addition, a brain-computer interface (BCI) can establish information exchange and communication between the brain and external devices, and can realize the interaction between brain thinking activities and external devices, thereby providing the possibility of active rehabilitation. Active rehabilitation can improve the active participation of patients to promote neural plasticity and improve rehabilitation effect.

[0006] In the prior art, there is a lack of a rehabilitation training method combining a brain-computer interface and cognitive-motor collaborative training. SUMMARY

[0007] The purpose of the present application is to provide a cognitive-motor dual-task collaborative rehabilitation training brain-computer interaction system, which comprises a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit, and a motor training unit.

[0008] The electroencephalogram signal acquisition unit acquires the electroencephalogram signal of the subject and transmits it to the electroencephalogram signal decoding unit.

[0009] The electroencephalogram signal decoding unit decodes the electroencephalogram signal, identifies the cognitive training content of the subject, and outputs the decoding result to the motor training unit and the cognitive training interaction unit. The decoding result includes attention concentration, the maximum correlation coefficient representing logical reasoning cognitive training content, and the maximum correlation coefficient representing spatial perception cognitive training content.

[0010] The motor training unit generates a motor control instruction based on the attention concentration, gaze target, and action intention of the subject, thereby driving the subject's limbs to perform motor training.

[0011] The cognitive training interaction unit presents an interactive interface of cognitive training and brain-computer interface paradigm, as well as the decoding result of the electroencephalogram signal decoding unit.

[0012] Further, the signal output end of the electroencephalogram signal acquisition unit is connected to the signal input end of the electroencephalogram signal decoding unit, the signal output end of the electroencephalogram signal decoding unit is connected to the signal input end of the motor training unit, and the signal output end of the electroencephalogram signal decoding unit is connected to the signal input end of the cognitive training interaction unit.

[0013] Further, when the cognitive training content is attention cognitive training content, the electroencephalogram signal decoding unit processes the frontal lobe electroencephalogram signal, calculates the power spectral density in different frequency bands, and determines the rise and fall of the subject's attention concentration based on the power spectral density rise and fall trend in different frequency bands corresponding to a plurality of continuous time windows.

[0014] When the cognitive training content is logical reasoning cognitive training content, the electroencephalogram signal decoding unit adopts a logical reasoning cognitive correlation decoding algorithm to process the multi-channel electroencephalogram signals of the visual area to obtain a maximum correlation coefficient for representing the logical reasoning cognitive training content; the logical reasoning cognitive correlation decoding algorithm includes a canonical correlation analysis algorithm.

[0015] When the cognitive training content is spatial perception cognitive training content, the electroencephalogram signal decoding unit adopts a spatial perception cognitive correlation decoding algorithm to perform spatial filtering processing on the multi-channel electroencephalogram signals of the visual area to obtain a maximum correlation coefficient for representing the spatial perception cognitive training content; the spatial perception cognitive correlation decoding algorithm includes a task discriminant component analysis algorithm.

[0016] Further, the electroencephalogram signal decoding unit also performs band-pass filtering on the electroencephalogram signals before processing the electroencephalogram signals.

[0017] Further, the motion training unit includes a motion control module and a motion feedback module.

[0018] The motion control module generates a motion control instruction based on the decoding result and transmits the motion control instruction to the motion feedback module to control the motion feedback module to drive the subject's limbs to perform motion training.

[0019] The motion feedback module includes but is not limited to a pneumatic robot and an exoskeleton robot.

[0020] Further, the speed at which the motion feedback module drives the subject to move is positively correlated with the degree of attention concentration of the subject's limb movement displayed by the cognitive training interaction unit.

[0021] Further, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content and controls the motion feedback module to drive the subject's limbs to perform the corresponding action.

[0022] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content and presents a picture of the corresponding action.

[0023] Further, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content and controls the motion feedback module to drive the subject's limbs to perform the corresponding action.

[0024] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content and presents a picture of the corresponding action.

[0025] Further, the cognitive training interaction unit includes one or more of a display, a virtual reality system, or an augmented reality glasses.

[0026] Further, the electroencephalogram signal acquisition unit comprises a plurality of electrodes which acquire electroencephalogram signals on the scalp of the subject;

[0027] The sampling frequency of the electroencephalogram signal acquisition unit is more than 250 Hz.

[0028] The technical effect of the present application is self-evident. The system provided by the present application realizes cognitive-motor collaborative rehabilitation training, improves rehabilitation efficiency compared with single rehabilitation training mode, introduces brain-computer interface technology, and realizes active motor rehabilitation by real-time recognition of the attention and motor intention of the subject, enhances the active participation of the subject in the rehabilitation training process, establishes a closed loop system, strengthens the mutual promotion of cognitive-motor function rehabilitation, and enhances the induction of neural reorganization and reconstruction. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The cognitive-motor collaborative brain-computer interaction rehabilitation training system;

[0030] Figure 2 The grip ball action sequence image;

[0031] Figure 3 The index finger clicks the mouse action, the thumb and index finger pinch the dice action, the grip paper cup action, and the grip ball action schematic diagram. DETAILED DESCRIPTION

[0032] The present application will be further described below in conjunction with examples, but should not be understood as limiting the above-mentioned subject matter of the present application to the following examples. Various substitutions and modifications can be made according to ordinary technical knowledge and conventional means in the art without departing from the above-mentioned technical idea of the present application, and all should be included in the protection scope of the present application.

[0033] Example 1:

[0034] Referring to Figures 1-3 A cognitive-motor dual task collaborative rehabilitation training brain-computer interaction system, comprising a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit, and a motor training unit;

[0035] The electroencephalogram signal acquisition unit acquires electroencephalogram signals of the subject and transmits them to the electroencephalogram signal decoding unit;

[0036] The electroencephalogram signal decoding unit decodes the electroencephalogram signals, recognizes the cognitive training content of the subject, and outputs the decoding results to the motor training unit and the cognitive training interaction unit; the decoding results include attention concentration, the maximum correlation coefficient representing logical reasoning cognitive training content, and the maximum correlation coefficient representing spatial perception cognitive training content;

[0037] The motion training unit generates a motion control instruction based on the attention concentration degree, the gaze target, and the action intention of the subject, so as to drive the limbs of the subject to perform motion training.

[0038] The cognitive training interaction unit presents an interaction interface of the cognitive training and the brain-computer interface paradigm, and a decoding result of the electroencephalogram signal decoding unit.

[0039] The signal output end of the electroencephalogram signal acquisition unit is connected with the signal input end of the electroencephalogram signal decoding unit, the signal output end of the electroencephalogram signal decoding unit is connected with the signal input end of the motion training unit, and the signal output end of the electroencephalogram signal decoding unit is connected with the signal input end of the cognitive training interaction unit.

[0040] When the cognitive training content is attention cognitive training content, the electroencephalogram signal decoding unit processes the frontal lobe electroencephalogram signal, calculates the power spectral density in different frequency bands, and determines the rise and fall of the attention concentration degree of the subject based on the power spectral density rise and fall trend in different frequency bands corresponding to a plurality of continuous time windows; for example, if the power spectral density in the alpha frequency band calculated in the continuous four time windows is increased and the power spectral density in the beta frequency band is decreased, it is considered that the attention concentration degree is decreased, and vice versa.

[0041] When the cognitive training content is logic reasoning cognitive training content, the electroencephalogram signal decoding unit processes the multi-channel electroencephalogram signal of the visual area by using a correlation decoding algorithm such as a canonical correlation analysis algorithm to obtain a maximum correlation coefficient for representing the logic reasoning cognitive training content.

[0042] When the cognitive training content is spatial perception cognitive training content, the electroencephalogram signal decoding unit performs spatial filtering processing on the multi-channel electroencephalogram signal of the visual area by using a correlation decoding algorithm such as a task discriminant component analysis algorithm to obtain a maximum correlation coefficient for representing the spatial perception cognitive training content.

[0043] The electroencephalogram signal decoding unit also performs band-pass filtering on the electroencephalogram signal before processing the electroencephalogram signal.

[0044] The motion training unit includes a motion control module and a motion feedback module.

[0045] The motion control module generates a motion control instruction based on the decoding result and transmits the motion control instruction to the motion feedback module to control the motion feedback module to drive the limbs of the subject to perform motion training.

[0046] The motion feedback module includes but is not limited to a pneumatic robot and an exoskeleton robot.

[0047] The motion feedback module drives the speed of the subject's movement, and the cognitive training interaction unit displays the speed of the subject's limb movement, which is positively correlated with the degree of attention concentration.

[0048] The motion control module determines the most relevant action based on the maximum correlation coefficient representing the logic reasoning of the cognitive training content, controls the motion feedback module to drive the subject's limbs to perform the corresponding action, such as the hand movement representing the numbers 0-5 with a single hand;

[0049] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception of the cognitive training content, and presents the corresponding action picture.

[0050] The motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception of the cognitive training content, controls the motion feedback module to drive the subject's limbs to perform the corresponding action, such as clicking the mouse action, holding the ball action, holding the cup action, and pinching the dice action;

[0051] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception of the cognitive training content, and presents the corresponding action picture.

[0052] The cognitive training interaction unit includes one or more of a display, a virtual reality system, or augmented reality glasses.

[0053] The electroencephalogram signal acquisition unit includes a plurality of electrodes that acquire electroencephalogram signals on the surface of the subject's scalp.

[0054] The sampling frequency of the electroencephalogram signal acquisition unit exceeds 250Hz.

[0055] Embodiment 2:

[0056] A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training includes a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit, and a motor training unit.

[0057] The electroencephalogram signal acquisition unit acquires the electroencephalogram signals of the subject and transmits them to the electroencephalogram signal decoding unit.

[0058] The electroencephalogram signal decoding unit decodes the electroencephalogram signals, identifies the cognitive training content of the subject, and outputs the decoding results to the motor training unit and the cognitive training interaction unit; the decoding results include the degree of attention concentration, the maximum correlation coefficient representing the logic reasoning of the cognitive training content, and the maximum correlation coefficient representing the spatial perception of the cognitive training content.

[0059] The motor training unit generates motion control instructions based on the subject's degree of attention concentration, gaze target, and action intention, thereby driving the subject's limbs to perform motor training.

[0060] The cognitive training interaction unit presents an interaction interface of cognitive training and brain-computer interface paradigm, and a decoding result of the electroencephalogram signal decoding unit.

[0061] Embodiment 3:

[0062] The brain-computer interaction system for cognitive-motor dual task collaborative rehabilitation training has the technical content same as that of embodiment 2, further, a signal output end of the electroencephalogram signal acquisition unit is connected with a signal input end of the electroencephalogram signal decoding unit, a signal output end of the electroencephalogram signal decoding unit is connected with a signal input end of the motor training unit, and a signal output end of the electroencephalogram signal decoding unit is connected with a signal input end of the cognitive training interaction unit.

[0063] Embodiment 4:

[0064] The brain-computer interaction system for cognitive-motor dual task collaborative rehabilitation training has the technical content same as that of any one of embodiments 2-3, further, when the cognitive training content is attention cognitive training content, the electroencephalogram signal decoding unit processes the frontal lobe electroencephalogram signal, calculates power spectral densities in different frequency bands, and determines the rise and fall of the attention concentration degree of the subject based on the rise and fall trends of the power spectral densities in the continuous multiple different frequency bands.

[0065] When the cognitive training content is logic reasoning cognitive training content, the electroencephalogram signal decoding unit processes the multi-channel electroencephalogram signals in the visual area by using a correlation decoding algorithm such as a canonical correlation analysis algorithm to obtain a maximum correlation coefficient for representing the logic reasoning cognitive training content.

[0066] When the cognitive training content is spatial perception cognitive training content, the electroencephalogram signal decoding unit performs spatial filtering processing on the multi-channel electroencephalogram signals in the visual area by using a correlation decoding algorithm such as a task discriminant component analysis algorithm to obtain a maximum correlation coefficient for representing the spatial perception cognitive training content.

[0067] Embodiment 5:

[0068] The brain-computer interaction system for cognitive-motor dual task collaborative rehabilitation training has the technical content same as that of any one of embodiments 2-4, further, the electroencephalogram signal decoding unit performs band-pass filtering on the electroencephalogram signal before processing the electroencephalogram signal.

[0069] Embodiment 6:

[0070] The brain-computer interaction system for cognitive-motor dual task collaborative rehabilitation training has the technical content same as that of any one of embodiments 2-5, further, the motor training unit comprises a motor control module and a motor feedback module.

[0071] The motion control module generates motion control instructions based on the decoding result and transmits the motion control instructions to the motion feedback module to control the motion feedback module to drive the subject's limbs to perform motion training.

[0072] The motion feedback module includes but is not limited to a pneumatic robot, an exoskeleton robot.

[0073] Embodiment 7:

[0074] A brain-computer interaction system for cognitive-motor dual task cooperative rehabilitation training, the technical content is the same as any one of embodiments 2-6, further, the speed of the motion feedback module driving the subject to move, and the cognitive training interaction unit displays the speed of the subject's limb movement positively correlated with the degree of concentration.

[0075] Embodiment 8:

[0076] A brain-computer interaction system for cognitive-motor dual task cooperative rehabilitation training, the technical content is the same as any one of embodiments 2-7, further, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the logical inference of the cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform the corresponding action.

[0077] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the logical inference of the cognitive training content, and presents the corresponding action picture.

[0078] Embodiment 9:

[0079] A brain-computer interaction system for cognitive-motor dual task cooperative rehabilitation training, the technical content is the same as any one of embodiments 2-8, further, the motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception of the cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform the corresponding action.

[0080] The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception of the cognitive training content, and presents the corresponding action picture.

[0081] Embodiment 10:

[0082] A brain-computer interaction system for cognitive-motor dual task cooperative rehabilitation training, the technical content is the same as any one of embodiments 2-9, further, the cognitive training interaction unit includes one or more of a display, a virtual reality system, or an augmented reality glasses.

[0083] Embodiment 11:

[0084] The technical content of the brain-computer interaction system for cognitive-motor dual task cooperative rehabilitation training is the same as any one of embodiments 2-10, further, the electroencephalogram signal acquisition unit comprises a plurality of electrodes which acquire electroencephalogram signals on the scalp of the subject.

[0085] The sampling frequency of the electroencephalogram signal acquisition unit is more than 250 Hz.

[0086] Embodiment 12:

[0087] The brain-computer interaction system for cognitive-motor dual task cooperative rehabilitation training comprises a cognitive training interaction unit, an electroencephalogram signal acquisition unit, an electroencephalogram signal decoding unit and a motor training unit.

[0088] The signal output end of the electroencephalogram signal acquisition unit is connected with the signal input end of the electroencephalogram signal decoding unit, the signal output end of the electroencephalogram signal decoding unit is connected with the signal input end of the motor training unit, and the signal output end of the electroencephalogram signal decoding unit is connected with the signal input end of the cognitive training interaction unit.

[0089] The cognitive training interaction unit comprises a paradigm presentation module which presents an interactive interface of cognitive training and brain-computer interface paradigm and presents the decoding results of the electroencephalogram signal decoding unit in time.

[0090] The electroencephalogram signal acquisition unit comprises an electroencephalogram signal acquisition module which acquires electroencephalogram signals of the subject and outputs to the electroencephalogram signal decoding unit.

[0091] The electroencephalogram signal decoding unit comprises an electroencephalogram signal decoding module which adopts different preset machine learning algorithms according to different cognitive-motor cooperative rehabilitation training tasks and difficulties, recognizes the attention concentration degree, gaze target, action intention and the like of the subject according to the electroencephalogram signals, and outputs the results to the motor training unit and the cognitive training interaction unit.

[0092] The motor training unit comprises a motor control module and a motor feedback module which generate motor control instructions to the motor control module based on the output of the electroencephalogram signal decoding unit, and the motor control module controls the motor feedback module to drive the limbs of the subject to perform corresponding motor training.

[0093] According to a specific implementation, in the cognitive-motor cooperative brain-computer interaction rehabilitation training system, the paradigm presentation module is presented by a display, a virtual reality system or an augmented reality glasses, and the presentation content is a paradigm combining cognitive training and brain-computer interface visual paradigm of different difficulties including attention, logical reasoning, spatial perception and the like and the output results of the electroencephalogram signal decoding module.

[0094] The sampling frequency of the electroencephalogram signal collection module is more than 250 Hz.

[0095] The electroencephalogram signal decoding module adopts different decoding methods for different cognitive training contents presented by the paradigm presentation module: when attention cognitive training content is presented, the electroencephalogram signal is band-pass filtered, and the power spectral density in different frequency bands is calculated based on the electroencephalogram signal in the frontal region, and the strength of the power spectral density in different frequency bands is used for decoding the attention concentration degree; when logical reasoning cognitive training content is presented, the electroencephalogram signal is band-pass filtered, and then a correlation decoding algorithm such as a canonical correlation analysis (CCA) algorithm is used to process the multi-channel electroencephalogram signal in the visual area to obtain the maximum correlation coefficient; when spatial perception cognitive training content is presented, the electroencephalogram signal is band-pass filtered, and then a correlation decoding algorithm such as a task-discriminant component analysis (TDCA) algorithm is used to perform spatial filtering processing on the multi-channel electroencephalogram signal in the visual area, and then the maximum correlation coefficient is obtained.

[0096] The motion control module and the motion feedback module are configured to:

[0097] Based on the strength of the attention concentration degree obtained by the electroencephalogram signal decoding module, a control instruction for controlling the speed of the motion feedback module is generated, the motion feedback module drives the subject's limbs to perform motion training at different speeds, and the speed of the limb movement in the limb movement video in the paradigm presentation module is controlled: the higher the attention concentration degree, the faster the movement, and vice versa.

[0098] Based on the maximum correlation coefficient obtained by the electroencephalogram signal decoding module based on CCA, a control instruction for controlling different actions of the motion feedback module is generated, the motion feedback module drives the subject's limbs to perform corresponding actions, and the paradigm presentation module presents the same action pictures.

[0099] Based on the maximum correlation coefficient obtained by the electroencephalogram signal decoding module based on TDCA, a control instruction for controlling different actions of the motion feedback module is generated, the motion feedback module drives the subject's limbs to perform corresponding actions, and the paradigm presentation module presents the same action pictures.

[0100] The motion feedback module includes: a pneumatic robot, an exoskeleton robot, and an end effector.

[0101] Embodiment 13:

[0102] The cognitive-motor dual task cooperative rehabilitation training brain-computer interaction system comprises a cognitive training interaction unit, a brain electrical signal acquisition unit, the output end of the brain electrical signal acquisition unit is connected with the input end of a brain electrical signal decoding unit, the output end of the brain electrical signal decoding unit is connected with the input end of a motor training unit, and the output end of the brain electrical signal decoding unit is connected with the input end of the cognitive training interaction unit.

[0103] The brain electrical signal acquisition module collects 32-channel distribution on the scalp surface of the brain frontal lobe, parietal lobe, occipital lobe and temporal lobe according to the international standard 10 / 20 system, with the forehead Fpz as the ground pole and the left ear lobe as the reference, and collects the brain electrical signals of Fp1, Fp2, AFz, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC4, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P3, Pz, P4, PO7, PO3, POz, PO4, PO8, O1, Oz and O2, with a sampling frequency of 250 Hz or above;

[0104] The cognitive training interaction unit is presented by a display, a virtual reality system or augmented reality glasses;

[0105] The cognitive ability is tested according to a preset cognitive scale to obtain a cognitive test result, for example, the abnormal cognitive function in the cognitive fields including visual space function, naming, attention, repeated sentence, fluency, abstract ability, delayed recall and orientation is screened by using the Mini-Mental State Examination (MMSE), the LOTCA software, the Wechsler Memory Scale (WMS) and the Montreal cognitive assessment (MoCA); the difficulty of the cognitive-motor cooperative brain-computer interaction rehabilitation training method is selected according to the cognitive test result; the cognitive-motor cooperative brain-computer interaction rehabilitation training method comprises three difficulties: difficulty one, attention-motor cooperative training brain-computer interaction training, difficulty two, logical reasoning-motor cooperative training brain-computer interaction training, and difficulty three, spatial perception and attention compound-motor cooperative training brain-computer interaction training. The hand motor function rehabilitation training is carried out in the embodiment, but the present application is not limited to the hand motor function rehabilitation training.

[0106] The attention-motor cooperative training brain-computer interaction training comprises the following steps:

[0107] The paradigm was presented on the presentation module using a frame-based stimulation pattern. The screen refresh rate was 60 Hz, i.e., 60 frames per second. Stimuli were generated using a frame rate reduction (FRR) method. The steps of FRR were as follows: first, a video of an action, such as thumb movement or grasping, was recorded; second, M images of an action cycle were extracted from the video; third, the M images were presented using Psychophysics Toolbox, and the presentation time of each image was controlled. Each image lasted N / 60 seconds (the screen refresh rate was 60 Hz).

[0108] Take the ball grip as an example. Figure 2 As shown, each frame is extracted from a video. The same image will last N / 60 seconds (N ≥ 4 in this study), followed by the next different image. Therefore, the designed paradigm frame rate is reduced to 60 / N (traditional video frame rate is 30Hz). In one motion cycle, there are M images (M = 16 in the figure), so the motion frequency is 60 / (M × N). By designing M and N, the normal speed of motion is guaranteed (60 / (M × N) < 1Hz).

[0109] The paradigm presentation module continuously presents the above paradigm, and the EEG signal acquisition module acquires the subject's EEG signal in real time, intercepts the EEG signal with a time window length of 3 seconds and a sliding window with a time interval of 0.5 seconds, performs a 3-30 Hz Butterworth filter on the acquired EEG signal, and then processes the EEG signals of the three channels Fp1, Fp2, and AFz in the frontal area. The power spectral density of the two frequency bands of 8 Hz to 13 Hz and 13 Hz to 30 Hz is calculated using the Welch method, and is denoted as α and β respectively. If the α calculated in four consecutive time windows increases and the β decreases, it is considered that the attention concentration is reduced, and vice versa, it is considered that the attention concentration is improved;

[0110] The motion control module described in this embodiment continuously monitors whether there are control instructions sent in the TCP / IP communication or serial communication channel. The motion feedback module is a pneumatic rehabilitation glove, which realizes the inflation and deflating of the pneumatic glove by controlling the inflation pump, the exhaust pump and the three-way solenoid valve through the single-chip microcomputer, so as to drive the subject's hand to perform reciprocating grasping movements. Upon receiving the information of reduced concentration transmitted by the EEG signal decoding module, the single-chip microcomputer controls the opening and closing frequency of the inflation pump and the exhaust pump to reduce the speed of the pneumatic glove's grasping movement, thereby reducing the frequency of driving the subject's hand to perform grasping movements. At the same time, the speed of gripping the ball in the paradigm is reduced by outputting instructions to the paradigm presentation module. The subject can intuitively feel the reduction in concentration, and by adjusting his own attention, improve the level of concentration, and thus speed up the movement.

[0111] Logic reasoning-movement coordination training brain-computer interaction training:

[0112] The paradigm presentation module presents the paradigm using a traditional steady-state visual evoked potential-based light flicker paradigm. Six light flicker paradigms with different flicker frequencies (f1, f2, f3, f4, f5, f6) are presented, with the flicker frequencies being between 4 and 40 Hz, such as 5 Hz, 6 Hz, 7 Hz, 8 Hz, 9 Hz, and 10 Hz. Each light flicker paradigm has a number in the center, with the numbers being 1, 2, 3, 4, 5, and 0 in sequence. After the cognitive-motor rehabilitation training starts, the paradigm presentation module first presents a random arithmetic question. The arithmetic question is an operation of addition, subtraction, multiplication, or division of two numbers, and the operation result is between 0 and 5. The arithmetic question is presented for 3 seconds, during which the subject calculates the result of the arithmetic question in his mind. Then, the paradigm presentation module presents the six light flicker paradigms with different flicker frequencies. According to the result calculated by the subject, the subject gazes at the light flicker paradigm with the corresponding number. For example, if the calculation result is 3, the subject gazes at the light flicker paradigm with the number 3 below. The light flicker paradigm lasts for 5 seconds, and then the paradigm presentation module presents the decoding result of the EEG signal decoding module for 2 seconds. Then, the above process is repeated.

[0113] The EEG signal decoding module uses a canonical correlation analysis algorithm. First, the collected EEG signals are subjected to 3-30 Hz Butterworth filtering. Then, the sine and cosine functions of the one and two times of the frequencies f1, f2, f3, f4, f5, and f6 are selected as the template functions of the algorithm. The correlation coefficients of the EEG signals collected by the EEG acquisition module and the template functions of the visual area channels PO7, PO3, POz, PO4, PO8, O1, Oz, and O2 are calculated. The frequency corresponding to the maximum correlation coefficient is the recognition result (one of 0, 1, 2, 3, 4, and 5), which is transmitted back to the paradigm presentation module to display the number corresponding to the recognition result.

[0114] In the present embodiment, the movement control module always listens to the TCP / IP communication or serial communication channel for control instructions. The movement feedback module is a pneumatic rehabilitation glove. Five groups of inflation and deflation air pumps and three-way electromagnetic valves are controlled by a single-chip microcomputer to realize the inflation and deflation of the pneumatic glove, so as to drive the subject to present six actions of the numbers 0 to 6 with one hand. When the recognition result transmitted by the EEG signal decoding module is received, such as 3, the pneumatic glove is controlled to drive the subject to assume the posture of 3. If the recognition result is inconsistent with the true result of the arithmetic question presented by the paradigm presentation module, the pneumatic glove does not move and remains in the initial open state.

[0115] Spatial perception attention complex-movement coordination training brain-computer interaction training:

[0116] The paradigm is presented on the presentation module using a frame change-based stimulation mode. Four actions are presented on the screen, and the design method is consistent with the description above. The actions have certain task attributes, and the index finger click mouse action, the thumb and index finger dice pinching action, the paper cup holding action, and the ball holding action are selected in this case, as shown in Figure 3 The frequencies of the four actions are different.

[0117] When the cognitive-motor coordination rehabilitation training starts, the paradigm presentation module first randomly presents an object picture related to the four action paradigms in the center. In this case, one of the four objects, mouse, dice, paper cup, and massage ball, is presented for 3 seconds. Then, the action paradigm shown in Figure 3 is presented. In the paradigm, each action is performed at a different frequency. The subject gazes at the corresponding action based on the object presented in the center before. For example, if a mouse appears, the subject continuously gazes at the action paradigm of the index finger clicking the mouse until the paradigm disappears. The presentation time of the four paradigms is 5 seconds. Finally, the paradigm presentation module presents the action picture corresponding to the recognition result transmitted by the EEG signal decoding module in the center. Then, the cycle is repeated in the above-mentioned manner.

[0118] The EEG signal decoding module uses the TDCA algorithm. First, the EEG signals of the visual area PO7, PO3, POz, PO4, PO8, O1, Oz, and O2 channels collected by the EEG collection module during the gaze at the four action paradigms are collected as the training set and a spatial filter is constructed. The spatial filter is used to process the EEG signals of the visual area PO7, PO3, POz, PO4, PO8, O1, Oz, and O2 channels collected by the EEG collection module during the cognitive-motor coordination rehabilitation training, and the correlation coefficient with the training set data is calculated. The frequency corresponding to the maximum correlation coefficient is the recognition result (one of the four actions). The result is transmitted back to the paradigm presentation module to display the action corresponding to the recognition result.

[0119] In this implementation case, the motion control module always listens for control instructions sent in the TCP / IP communication or serial communication channel. The motion feedback module is a pneumatic rehabilitation glove. Five groups of inflation and deflation air pumps and three-way electromagnetic valves are controlled by a single-chip microcomputer to realize the inflation and deflation of the pneumatic glove, so as to drive the subject to present the corresponding four actions in the action paradigm, i.e., the index finger action, the index finger and thumb action, the light grip action, and the heavy grip action. When the recognition result transmitted by the EEG signal decoding module is received, such as the index finger clicking the mouse action, the pneumatic glove drives the subject's index finger to move. If the recognition result is inconsistent with the prompt task presented by the paradigm presentation module, the pneumatic glove does not move and remains in the initial open state.

[0120] In embodiments of the present application, the processor can be an integrated circuit chip with the processing capability that can be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.

Claims

1. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training, characterized by: It includes cognitive training interaction unit, EEG signal acquisition unit, EEG signal decoding unit and motor training unit; The EEG signal acquisition unit acquires the EEG signal of the subject and transmits it to the EEG signal decoding unit; The EEG signal decoding unit decodes the EEG signal, identifies the cognitive training content of the subject, and outputs the decoding results to the motor training unit and the cognitive training interaction unit; the decoding results include attention concentration, the maximum correlation coefficient representing the logical reasoning cognitive training content, and the maximum correlation coefficient representing the spatial perception cognitive training content; The movement training unit generates movement control instructions based on the subject's attention concentration, gaze target, and movement intention, thereby driving the subject's limbs to perform movement training; The cognitive training interaction unit presents an interactive interface of cognitive training and brain-computer interface paradigm, as well as decoding results of the EEG signal decoding unit; The motion training unit includes a motion control module and a motion feedback module; The motion control module generates motion control instructions based on the decoding results and transmits them to the motion feedback module to control the motion feedback module to drive the subject's limbs to perform motion training; The motion control module determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform the corresponding action; The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the logical reasoning cognitive training content and presents the corresponding action picture; The motion control module determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content, and controls the motion feedback module to drive the subject's limbs to perform the corresponding action; The cognitive training interaction unit determines the most relevant action based on the maximum correlation coefficient representing the spatial perception cognitive training content and presents the corresponding action picture.

2. The brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The signal output end of the EEG signal acquisition unit is connected to the signal input end of the EEG signal decoding unit, the signal output end of the EEG signal decoding unit is connected to the signal input end of the movement training unit, and the signal output end of the EEG signal decoding unit is connected to the signal input end of the cognitive training interaction unit.

3. The brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: When the cognitive training content is attention cognitive training content, the EEG signal decoding unit processes the EEG signal of the frontal area, calculates the power spectrum density in different frequency bands, and determines the increase or decrease of the subject's attention concentration based on the increase and decrease trends of the power spectrum density in different frequency bands corresponding to multiple consecutive time windows; When the cognitive training content is logical reasoning cognitive training content, the EEG signal decoding unit uses a logical reasoning cognitive related decoding algorithm to process the EEG signals of multiple channels in the visual area to obtain a maximum correlation coefficient for characterizing the logical reasoning cognitive training content; the logical reasoning cognitive related decoding algorithm includes a canonical correlation analysis algorithm; When the cognitive training content is spatial perception cognitive training content, the EEG signal decoding unit uses a spatial perception cognitive related decoding algorithm to perform spatial filtering processing on the EEG signals of multiple channels in the visual area to obtain a maximum correlation coefficient for characterizing the spatial perception cognitive training content; The spatial perception cognition related decoding algorithm includes a task discriminant component analysis algorithm.

4. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 3, characterized in that: The EEG signal decoding unit also performs band-pass filtering on the EEG signal before processing the EEG signal.

5. The brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The motion feedback module includes but is not limited to a pneumatic robot and an exoskeleton robot.

6. A brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 5, characterized in that: The motion feedback module drives the speed of the subject's movement, and the cognitive training interaction unit shows that the speed of the subject's limb movements is positively correlated with the degree of attention concentration.

7. The brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The cognitive training interaction unit includes one or more of a display, a virtual reality system, or augmented reality glasses.

8. The brain-computer interaction system for cognitive-motor dual-task collaborative rehabilitation training according to claim 1, characterized in that: The EEG signal acquisition unit includes a plurality of electrodes, which acquire EEG signals from the subject's scalp surface; The sampling frequency of the EEG signal acquisition unit exceeds 250 Hz.

Images