Rehabilitation method and system for spastic cerebral palsy children based on motor imagery brain-computer interface

Abstract

The invention provides a spastic cerebral palsy child rehabilitation method and system based on a motor imagery brain-computer interface. On one hand, specific motor imagery paradigms including the left hemiplegia paradigm, the right hemiplegia paradigm and the double-paralysis paradigm are constructed according to the severity of the spastic cerebral palsy children, and the rehabilitation curative effect of the spastic cerebral palsy children can be effectively improved in a targeted mode. Moreover, a motor imagery brain-computer interface scheme is provided for decoding electroencephalogram information, whether motor imagery needs to be adjusted or not is effectively evaluated, closed-loop feedback is formed, and the reliability of a motor imagery brain-computer interface therapy is ensured. And on the other hand, the rehabilitation method and system combining the motor imagery brain-computer interface with conventional rehabilitation training are favorable for improving the rehabilitation training effect and the rehabilitation participation degree of the patient, so that a more effective and scientific rehabilitation solution is provided for the spastic cerebral palsy children.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation therapy, and in particular to a rehabilitation method and system for children with spastic cerebral palsy based on motor imagery brain-computer interface. Background Art

[0002] Spastic cerebral palsy (SCP) is a movement and posture disorder caused by non-progressive damage to the developing fetus or infant brain, characterized by high muscle tension and severe hyperreflexia. Therefore, improving motor function and self-care ability is one of the important goals of rehabilitation for children with spastic cerebral palsy. Spastic cerebral palsy is usually accompanied by damage to the sensorimotor pathway. Since the brain has significant neuroplasticity during development, how to improve the motor function of children by enhancing the connection of brain neural networks has become a hot topic in current research. At present, the clinical rehabilitation strategy for children with spastic cerebral palsy is mainly based on exercise therapy and occupational therapy, assisted by orthopedic devices and physical factor therapy. The structured rehabilitation strategy mainly uses bottom-up limb training to gradually enhance the plasticity of the affected limb in the cortical motor representation.

[0003] Motor imagery (MI) is a therapeutic method that promotes rehabilitation by repeatedly simulating specific limb movements in the mind without actual movement or muscle stimulation. This therapy helps patients recover motor function by activating neural networks in the brain that are related to actual movement. Unlike traditional rehabilitation training, motor imagery therapy first mobilizes the core area of ​​motor control in the brain, and activates other motor brain areas and the periphery of the body in a top-down manner, thereby improving neural plasticity and promoting motor cortical reorganization. Studies have shown that motor imagery therapy can improve the upper limb motor function of children with hemiplegia and help improve the ability to plan movements. Due to the complex etiology and diverse clinical manifestations of spastic cerebral palsy, it is not limited to the rehabilitation of hemiplegic hand function, but also involves the gait movement of diplegia. In addition, the experimental settings for testing and training adult motor imagery are not completely suitable for children. Different motor imagery paradigms need to be developed for different types of children with spastic cerebral palsy.

[0004] The brain computer interface (BCI) system analyzes brain activity and converts it into control signals for external devices by establishing a direct connection pathway between human brain nerves and external devices. The motor imagery-based brain computer interface (MI-brain-computer interface, MI-BCI) technology can collect electroencephalogram (EEG) signals in real time during motor imagery and provide neural feedback. Motor imagery BCI technology has achieved good results in improving motor dysfunction in adult stroke patients. Although there are differences in the injury mechanism between stroke (ischemia / hemorrhage) and spastic cerebral palsy (trauma), rehabilitation methods can be successfully converted to restore sensorimotor function defects by relying on neural remodeling. Among related technologies, motor imagery BCI technology can improve the motor function rehabilitation of children with cerebral palsy. However, whether motor imagery BCI technology combined with conventional rehabilitation training can promote the recovery of motor dysfunction in children with spastic cerebral palsy still needs further study.

[0005] For example, the Chinese patent with application number 202010939356.7 and invention name "Robot-assisted cerebral palsy rehabilitation expression training system based on brain-computer interface"; and the Chinese patent with application number CN201710999026.5 and invention name "A rehabilitation system and method based on interactive virtual reality and neural electrical stimulation". In the above patents, motor imagery and brain-computer interface technology have limited application in the field of rehabilitation of children with spastic cerebral palsy, and no specific limb rehabilitation method has been designed for them. At the same time, the lack of a specific motor imagery paradigm based on the severity of the disease of children with spastic cerebral palsy makes the applicability and pertinence of rehabilitation training insufficient. Although these two technologies have shown a positive effect in improving the rehabilitation efficacy of children with cerebral palsy, they have not yet been deeply combined with conventional rehabilitation training, and have failed to form a systematic and complete rehabilitation system. Therefore, there is an urgent need for a method of integrating motor imagery brain-computer interface and conventional rehabilitation training to give full play to their respective advantages, so as to provide more effective rehabilitation solutions for children with spastic cerebral palsy. Summary of the invention

[0006] In order to achieve the above-mentioned purpose and other advantages of the present invention, the first purpose of the present invention is to provide a method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface, comprising the following steps: Conduct limb function training, fine motor training and self-care ability training based on conventional rehabilitation training scenarios; Motor imagery training based on motor imagery brain-computer interface specifically includes the following steps: Access to the selected motor imagery paradigm; Present the motor imagery task of the motor imagery paradigm selected to enable children with spastic CP to continuously engage in motor imagery; Collect EEG signals of children with spastic cerebral palsy when they continuously perform motor imagery; The EEG signal is decoded and closed-loop feedback is used to evaluate whether the motor imagery is effective.

[0007] Furthermore, the motor imagery paradigm includes a left hemiplegia paradigm, a right hemiplegia paradigm and a diplegia paradigm.

[0008] Furthermore, the step of performing motor imagery training based on the motor imagery brain-computer interface further includes: In each motor imagery training task, a rest time was set first to prompt children with spastic cerebral palsy to concentrate; After the rest period, a prompt message was given to indicate that the motor imagery task was about to be presented; After the motor imagery task was presented, a rest phase ensued.

[0009] Furthermore, the motor imagery training includes multiple rounds of motor imagery training, each round of motor imagery training includes multiple motor imagery tasks, and the next round begins after a preset time interval.

[0010] Furthermore, the EEG signal is configured to be acquired through a multi-lead EEG cap.

[0011] Furthermore, the decoding of the EEG signal, evaluating whether the motor imagery is effective with closed-loop feedback, and decoding the motor imagery task using a deep learning classification method based on a convolutional neural network and multi-dimensional feature fusion specifically include the following steps: Preprocessing, data amplification and feature extraction of the EEG signal; The effect of the motor imagery task was evaluated using the motor imagery classification model; If the probability of the assessment result is lower than the set threshold, the child is reminded to adjust the motor imagery.

[0012] Furthermore, the method further comprises the steps of: Divide the collected EEG signals into training set and test set; Preprocessing, data amplification and feature extraction of the EEG signals of the subjects in the training set: Combined with the adaptive joint feature matrix, a convolutional neural network structure is used to train the motor imagery classification model; The features extracted from the EEG signals of the test set subjects are used to test the motor imagery classification model, obtain classification results, and evaluate the model performance.

[0013] Furthermore, the preprocessing, data amplification and feature extraction of EEG signals include the following steps: The preset time period after the start of motor imagery is selected through bandpass filtering for subsequent classification, the EEG signal of the preset time before imagery is used as the baseline to remove drift, and then independent component analysis is used to remove artifacts; The preprocessed signal is cropped and amplified by sliding window to divide the time series of motor imagery into multiple segments; Constructing an adaptive frequency-space-time joint feature matrix that integrates multi-domain information includes the following steps: Based on wavelet packet decomposition of sub-band signal energy distribution, active frequency bands are extracted to achieve adaptive spectrum sensing; Adaptively weighting each lead signal based on the activation value and performing a pair of other common spatial mode spatial filtering; Based on multiple features, the temporal information expression is enriched to capture and integrate the nonlinear and non-stationary information of motor imagery EEG signals at different time points; Through the above steps, adaptive frequency-space-time joint feature learning is jointly achieved.

[0014] Furthermore, the step of combining the adaptive joint feature matrix and using a convolutional neural network structure to train the motor imagery classification model includes: The convolutional neural network structure performs three layers of spatial convolution; The convolutional neural network structure performs two layers of temporal convolution; Connect two fully connected layers to integrate features and map the learned feature representation to the sample label space; Exponential normalization is used as the last layer of the network to map the output to the interval (0,1) for binary classification; Calculate the classification accuracy and Kappa coefficient to select the optimal motor imagery classification model; The EEG signals of the test set subjects are preprocessed, data amplified and feature extracted.

[0015] The second object of the present invention is to provide a rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface, comprising a conventional rehabilitation training module and a motor imagery brain-computer interface module, wherein the conventional rehabilitation training module is used to implement limb function training, fine motor training and self-care ability training, and the motor imagery brain-computer interface module is used to implement motor imagery training; wherein, The motor imagery brain-computer interface module includes a motor imagery paradigm selection module, a motor imagery training module, an electroencephalogram device, and an electroencephalogram signal processing module; The motor imagery paradigm selection module is used to select a motor imagery paradigm; The motor imagery training module is used to realize the presentation of the motor imagery task of the selected motor imagery paradigm, so as to enable the children with spastic cerebral palsy to continuously perform motor imagery; The EEG device is used to collect EEG signals of children with spastic cerebral palsy when they continuously perform action imagination; The EEG signal processing module is used to decode the EEG signal and evaluate whether the motor imagery is effective through closed-loop feedback.

[0016] Furthermore, the motor imagery paradigm includes a left hemiplegia paradigm, a right hemiplegia paradigm and a diplegia paradigm.

[0017] Furthermore, the motor imagery training module also includes a rest module, a prompt module, and a rest module: wherein, The rest module is used to set a rest time in each motor imagery training task to prompt the spastic cerebral palsy child to concentrate; The prompt module is used to issue a prompt message after the rest time is over to prompt that a motor imagery task is about to be presented; The rest module is used to enter the rest phase after the motor imagery task is presented.

[0018] Furthermore, the motor imagery training includes multiple rounds of motor imagery training, each round of motor imagery training includes multiple motor imagery tasks, and the next round begins after a preset time interval.

[0019] Furthermore, the EEG device adopts a multi-lead EEG cap.

[0020] Furthermore, the EEG signal processing module uses a deep learning classification method based on convolutional neural network and multi-dimensional feature fusion to decode the motor imagery task, specifically including: Preprocessing, data amplification and feature extraction of the EEG signal; The effect of the motor imagery task was evaluated using the motor imagery classification model; If the probability of the assessment result is lower than the set threshold, the child is reminded to adjust the motor imagery.

[0021] Furthermore, it also includes a motor imagery classification model training module, which is used to divide the collected EEG signals into a training set and a test set; preprocess, data amplify and extract features of the EEG signals of the subjects in the training set: combine the adaptive joint feature matrix and use a convolutional neural network structure to train the motor imagery classification model; test the motor imagery classification model with the features extracted from the EEG signals of the subjects in the test set to obtain classification results and evaluate the model performance.

[0022] Furthermore, the preprocessing, data amplification and feature extraction of EEG signals include: The preset time period after the start of motor imagery is selected through bandpass filtering for subsequent classification, the EEG signal of the preset time before imagery is used as the baseline to remove drift, and then independent component analysis is used to remove artifacts; The preprocessed signal is cropped and amplified by sliding window to divide the time series of motor imagery into multiple segments; Constructing an adaptive frequency-space-time joint feature matrix that integrates multi-domain information includes the following steps: Based on wavelet packet decomposition of sub-band signal energy distribution, active frequency bands are extracted to achieve adaptive spectrum sensing; Adaptively weighting each lead signal based on the activation value and performing a pair of other common spatial mode spatial filtering; Based on multiple features, the temporal information expression is enriched to capture and integrate the nonlinear and non-stationary information of motor imagery EEG signals at different time points; Through the above steps, adaptive frequency-space-time joint feature learning is jointly achieved.

[0023] Furthermore, the combination of the adaptive joint feature matrix and the use of a convolutional neural network structure to train the motor imagery classification model includes: The convolutional neural network structure performs three layers of spatial convolution; The convolutional neural network structure performs two layers of temporal convolution; Connect two fully connected layers to integrate features and map the learned feature representation to the sample label space; Exponential normalization is used as the last layer of the network to map the output to the interval (0,1) for binary classification; Calculate the classification accuracy and Kappa coefficient to select the optimal motor imagery classification model; The EEG signals of the test set subjects are preprocessed, data amplified and feature extracted.

[0024] A third object of the present invention is to provide a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0025] A fourth object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0026] Compared with the prior art, the present invention has the following beneficial effects: On the one hand, the present invention constructs a specific motor imagery paradigm according to the severity of children with spastic cerebral palsy, including a left hemiplegia paradigm, a right hemiplegia paradigm, and a diplegia paradigm, which can improve the rehabilitation effect of children with spastic cerebral palsy in a targeted and effective manner. In addition, the present invention provides a motor imagery brain-computer interface solution to decode electroencephalogram information, effectively evaluate whether motor imagery needs to be adjusted, form a closed-loop feedback, and ensure the reliability of motor imagery brain-computer interface therapy. On the other hand, the rehabilitation method and system constructed by the present invention that combines motor imagery brain-computer interface with conventional rehabilitation training helps to improve the effect of rehabilitation training and the patient's rehabilitation participation, thereby providing a more effective and scientific rehabilitation solution for children with spastic cerebral palsy.

[0027] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. The specific implementation of the present invention is given in detail by the following embodiments and their accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 The schematic diagram of the rehabilitation method and system for children with spastic cerebral palsy based on motor imagery brain-computer interface; Figure 2 This is a flow chart of the method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface of Example 1; Figure 3 The process of performing motor imagery training based on motor imagery brain-computer interface in Example 1 Figure 1 ; Figure 4 The process of performing motor imagery training based on motor imagery brain-computer interface in Example 1 Figure 2 ; Figure 5 This is a flowchart of EEG signal decoding in Example 1; Figure 6 The motor imagery classification model training flow chart of Example 1; Figure 7 This is a flow chart of training a motor imagery classification model using a convolutional neural network structure in combination with an adaptive joint feature matrix in Example 1; Figure 8 This is a flow chart of the preprocessing, data amplification and feature extraction of EEG signals in Example 1; Fig. 9 This is a flow chart of constructing an adaptive frequency-space-time joint feature matrix that fuses multi-domain information in Example 1; Fig.10 This is a schematic diagram of a rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface according to Example 2; Fig.11 This is a schematic diagram of the motor imagery brain-computer interface module of Example 2; Fig.12 This is a schematic diagram of a computer device in Example 3; Fig.13 This is a schematic diagram of a computer-readable storage medium of Example 4. DETAILED DESCRIPTION

[0029] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. It should be noted that, under the premise of no conflict, the embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0030] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0031] The figure numbers in this application are only used to distinguish the various steps in the scheme, and are not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0033] Example 1 A rehabilitation method for children with spastic cerebral palsy based on motor imagery brain-computer interface helps to improve the rehabilitation effect, such as Figure 1 , Figure 2 As shown, the method comprises the following steps: S1. Perform limb function training, fine motor training and self-care ability training based on conventional rehabilitation training scenarios. The daily training time can be set according to actual conditions; for example, set the daily training time to 30 minutes.

[0034] S2, motor imagery training based on motor imagery brain-computer interface, such as Figure 3 As shown, the specific steps include: During the actual imagination training process, children with spastic cerebral palsy are first fitted with EEG equipment, such as a 64-lead EEG cap. Taking the 64-lead EEG cap as an example, conductive paste is injected and needle-type active electrodes are inserted into the corresponding electrode holes to check the accurate placement of the electrodes and signal stability.

[0035] Then, appropriate motor imagery paradigms were selected for children with spastic cerebral palsy; the motor imagery paradigms included left hemiplegia paradigm, right hemiplegia paradigm, and diplegia paradigm.

[0036] S21, acquisition of the selected motor imagery paradigm; S22. Present the motor imagery task of the selected motor imagery paradigm to enable children with spastic cerebral palsy to continuously perform motor imagery; In some embodiments, Figure 4 As shown, the step of performing motor imagery training based on the motor imagery brain-computer interface also includes: S220, in each motor imagery training task, first set a rest time, for example, set a rest time of 3 seconds, to prompt the child with spastic cerebral palsy to concentrate; S221. After the resting time is over, a prompt message is issued to remind the child that a motor imagery task is about to be presented; for example, after a "beep" prompt, a schematic diagram of the motor imagery task is presented, and the child continues to imagine the action within 6 seconds after the schematic diagram appears.

[0037] S222: After the motor imagery task is presented, a rest phase is entered. For example, after the motor imagery task is completed, a 3-second rest phase is entered.

[0038] The motor imagery training includes multiple rounds of motor imagery training, each round of motor imagery training includes multiple motor imagery tasks, and the next round is entered after a preset time interval. For example, each round of motor imagery training includes 40 motor imagery tasks, and the next round is entered after a 5-minute interval, for a total of 80 trainings.

[0039] S23. Collecting EEG signals of children with spastic cerebral palsy when they continuously perform motor imagery; in some embodiments, the EEG signals are configured to be collected by a multi-lead EEG cap.

[0040] S24, decoding the EEG signal, and evaluating whether the motor imagery is effective by closed-loop feedback.

[0041] In some embodiments, Figure 5 As shown, the decoding of the EEG signal, evaluating whether the motor imagery is effective with closed-loop feedback, and decoding the motor imagery task using a deep learning classification method based on a convolutional neural network and multi-dimensional feature fusion specifically include the following steps: S241, preprocessing, data amplification and feature extraction of the EEG signal. The specific steps are described in the following embodiments and will not be repeated here. S242, Evaluating the effect of motor imagery tasks using a motor imagery classification model; S243: If the probability of the evaluation result is lower than the set threshold, the child is reminded to adjust the motor imagery. For example, if the probability of the evaluation result is lower than the set threshold, a 3-second reminder sound "beep" is played to remind the child to adjust the motor imagery.

[0042] In some embodiments, Figure 6 As shown, the steps include: S200, dividing the collected EEG signals into a training set and a test set; for example, dividing the collected EEG signals into a training set and a test set according to a ratio of 8:2.

[0043] S201, preprocessing, data amplification and feature extraction of the EEG signals of the subjects in the training set. The specific steps are described in the following embodiments and will not be repeated here: S202, combining the adaptive joint feature matrix, using a convolutional neural network structure to train the motor imagery classification model, such as Figure 7 As shown, the specific steps include: S2021. The convolutional neural network structure performs three layers of spatial convolution; for example, three convolution kernels [5×1], [10×1], and [15×1] are used to learn the spatial dimension information, following the learning rule from low complexity to high complexity. The number of the three convolution kernels is set to 8, 16, and 32, respectively.

[0044] S2022. The convolutional neural network structure performs two layers of time convolution; for example, [1×2] and [1×4] convolution kernels are set to learn the features provided by the time domain dimension, and the number of convolution kernels is set to 32.

[0045] S2023, connect two fully connected layers to integrate features and map the learned feature representation to the sample label space; S2024, use exponential normalization (softmax) as the last layer of the network to map the output to the interval (0,1) for binary classification; S2025. Calculate the classification accuracy and Kappa coefficient to select the optimal motor imagery classification model; S2026. Preprocessing, data amplification and feature extraction are performed on the EEG signals of the test set subjects. The specific steps are described in the following embodiments and will not be repeated here.

[0046] S203, testing the motor imagery classification model using the features extracted from the EEG signals of the test set subjects to obtain classification results and evaluate model performance.

[0047] In some optional embodiments, such as Figure 8 As shown, the preprocessing, data amplification and feature extraction of EEG signals include the following steps: S210, selecting a preset time period after the start of motor imagery to participate in subsequent classification through bandpass filtering, using the EEG signal of the preset time before imagery as the baseline to remove drift, and then using independent component analysis to remove artifacts; for example, the bandpass filter is set to 0.1-32Hz, selecting the 0-6s period after the start of motor imagery to participate in subsequent classification, using the 0.5s signal before imagery as the baseline to remove drift, and then using independent component analysis to remove artifacts, so as to eliminate obvious artifact components such as electrooculography, eye movement, and head movement; S211, amplifying the preprocessed signal by sliding window cropping, and dividing the time series of motor imagination into multiple segments; for example, the window width can be set to 3s, the step length can be set to 100 sampling points, and the 6s time series can be divided into 16 segments; S212, construct an adaptive frequency-space-time joint feature matrix that integrates multi-domain information, such as Fig. 9 As shown, the specific steps include: S2121. First, based on wavelet packet decomposition (WPD) sub-band signal energy distribution, active frequency bands are extracted to achieve adaptive spectrum sensing. S2122, secondly, adaptively weighting each lead signal based on the activation value and performing "one-versus-rest common spatial pattern" (OVR-CSP) spatial filtering, and realizing the subject's frequency-space feature description through the above two steps; S2123. Based on multiple feature-enriched time domain information expressions, the nonlinear and non-stationary information of motor imagery EEG signals at different time points can be captured and integrated, and the above adaptive frequency-space characteristics can be further supplemented with intuitive information; for example, based on 8 feature-enriched time domain information expressions, including maximum value, peak value, peak-to-peak value, mean value, absolute mean value, square root amplitude, variance and standard deviation, the nonlinear and non-stationary information of motor imagery EEG signals at different time points can be captured and integrated.

[0048] Through the above steps (step S2121 to step S2123), adaptive frequency-space-time joint feature learning is jointly achieved.

[0049] This embodiment constructs a rehabilitation method for spastic cerebral palsy based on a motor imagery brain-computer interface combined with conventional rehabilitation training. According to the degree of motor impairment in children with spastic cerebral palsy, this embodiment designs three motor imagery paradigms: left hemiplegia paradigm, right hemiplegia paradigm and diplegia paradigm to meet the rehabilitation needs of different patients. In the hemiplegia paradigm, the grasping of the left / right upper limbs and the dorsiflexion of the lower limbs are trained in detail, while the diplegia paradigm is for the coordinated training of the dorsiflexion of the lower limbs. This embodiment adds motor imagery brain-computer interface training on the basis of conventional rehabilitation training, and continuously activates the neural network related to actual movement in the brain through repeated imagination, synergizes with conventional rehabilitation therapy, and promotes the recovery of motor function. This rehabilitation system makes up for the shortcomings of the conventional rehabilitation training process, which is single and boring, and the low active participation of patients, and improves the training motivation and intervention effect of patients.

[0050] Example 2 A rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface, using the above method. For a detailed description of the method, please refer to the corresponding description in the above method embodiment, which will not be repeated here. Figure 1 , Fig.10 As shown, the system 300 includes a conventional rehabilitation training module 310 and a motor imagery brain-computer interface module 320. The conventional rehabilitation training module is used to implement limb function training, fine motor training and self-care ability training. The daily training time can be set according to actual conditions; for example, the daily training time can be set to 30 minutes. The motor imagery brain-computer interface module is used to implement motor imagery training; wherein, like Fig.11 As shown, the motor imagery brain-computer interface module 320 includes a motor imagery paradigm selection module 321, a motor imagery training module 322, an electroencephalogram device 323, and an electroencephalogram signal processing module 324; During the actual imagination training process, children with spastic cerebral palsy are first fitted with EEG equipment, such as a 64-lead EEG cap. Taking the 64-lead EEG cap as an example, conductive paste is injected and needle-type active electrodes are inserted into the corresponding electrode holes to check the accurate placement of the electrodes and signal stability.

[0051] Then, appropriate motor imagery paradigms were selected for children with spastic cerebral palsy; the motor imagery paradigms included left hemiplegia paradigm, right hemiplegia paradigm, and diplegia paradigm.

[0052] The motor imagery paradigm selection module is used to select a motor imagery paradigm; The motor imagery training module is used to realize the presentation of the motor imagery task of the selected motor imagery paradigm, so as to enable the children with spastic cerebral palsy to continuously perform motor imagery; In some embodiments, the motor imagery training module further includes a rest module, a prompt module, and a rest module: The rest module is used to set a rest time in each motor imagery training task, for example, a rest time of 3 seconds, to prompt the spastic cerebral palsy child to concentrate; The prompt module is used to issue a prompt message after the rest time is over to prompt that a motor imagery task is about to be presented; for example, after a "beep" prompt, a schematic diagram of the motor imagery task is presented, and the child continues to perform action imagination within 6 seconds after the schematic diagram appears.

[0053] The rest module is used to enter a rest phase after the motor imagery task is presented. For example, after the motor imagery is finished, a 3-second rest phase is entered.

[0054] The motor imagery training includes multiple rounds of motor imagery training, each round of motor imagery training includes multiple motor imagery tasks, and the next round is entered after a preset time interval. For example, each round of motor imagery training includes 40 motor imagery tasks, and the next round is entered after a 5-minute interval, for a total of 80 trainings.

[0055] The EEG device is used to collect EEG signals of children with spastic cerebral palsy when they continuously perform action imagination; optionally, the EEG device uses a multi-lead EEG cap.

[0056] The EEG signal processing module is used to decode the EEG signal and evaluate whether the motor imagery is effective through closed-loop feedback.

[0057] In some embodiments, the EEG signal processing module uses a deep learning classification method based on convolutional neural networks and multi-dimensional feature fusion to decode the motor imagery task, specifically including: Preprocessing, data amplification and feature extraction are performed on the EEG signal. The specific steps are described in the following embodiment and will not be repeated here. The effect of the motor imagery task was evaluated using the motor imagery classification model; If the probability of the evaluation result is lower than the set threshold, the child is reminded to adjust the motor imagery. For example, if the probability of the evaluation result is lower than the set threshold, a 3-second reminder sound "beep" is played to remind the child to adjust the motor imagery.

[0058] In some embodiments, a motor imagery classification model training module is also included, which is used to divide the collected EEG signals into a training set and a test set; for example, the collected EEG signals are divided into a training set and a test set according to a ratio of 8:2.

[0059] The EEG signals of the subjects in the training set are preprocessed, data amplified and feature extracted. The specific steps are described in the following embodiment and will not be repeated here: Combined with the adaptive joint feature matrix, a convolutional neural network structure is used to train the motor imagery classification model, specifically including: The convolutional neural network structure performs three layers of spatial convolution; for example, three convolution kernels [5×1], [10×1], and [15×1] are used to learn the spatial dimension information, following the learning rule from low complexity to high complexity, and the number of the three convolution kernels is set to 8, 16, and 32 respectively.

[0060] The convolutional neural network structure performs two layers of temporal convolution; for example, [1×2] and [1×4] convolution kernels are set to learn the features provided by the time domain dimension, and the number of convolution kernels is set to 32.

[0061] Connect two fully connected layers to integrate features and map the learned feature representation to the sample label space; Exponential normalization (softmax) is used as the last layer of the network to map the output to the (0,1) interval for binary classification; Calculate the classification accuracy and Kappa coefficient to select the optimal motor imagery classification model; The EEG signals of the test set subjects are preprocessed, data amplified and feature extracted. The specific steps are described in the following embodiment and will not be repeated here.

[0062] The features extracted from the EEG signals of the test set subjects are used to test the motor imagery classification model, obtain classification results, and evaluate the model performance.

[0063] In some optional embodiments, the preprocessing, data amplification and feature extraction of EEG signals include: Through band-pass filtering, a preset period after the start of motor imagery is selected for subsequent classification, and the EEG signal of the preset time before imagery is used as the baseline to remove drift, and then independent component analysis is used to remove artifacts; for example, the band-pass filter is set to 0.1-32Hz, and the 0-6s period after the start of motor imagery is selected for subsequent classification, and the 0.5s signal before imagery is used as the baseline to remove drift, and then independent component analysis is used to remove artifacts, so as to eliminate obvious artifact components such as electrooculography, eye movement, and head movement; The preprocessed signal is cropped and amplified by sliding window, and the time series of motor imagination is divided into multiple segments; for example, the window width can be set to 3s and the step length can be set to 100 sampling points, and the 6s time series can be divided into 16 segments; Constructing an adaptive frequency-space-time joint feature matrix that integrates multi-domain information includes the following steps: Firstly, based on the wavelet packet decomposition (WPD) sub-band signal energy distribution, the active frequency band is extracted to achieve adaptive spectrum sensing. Secondly, the signal of each lead is adaptively weighted based on the activation value and a "one-versus-rest" common spatial pattern (OVR-CSP) spatial filtering is performed. Through the above two steps, the frequency-space feature description of the subject is achieved; The time domain information expression is enriched based on multiple features to capture and integrate the nonlinear and non-stationary information of motor imagery EEG signals at different time points, and further intuitively supplement the above adaptive frequency-space characteristics; for example, based on 8 kinds of feature-enriched time domain information expression, including maximum value, peak value, peak-to-peak value, mean value, absolute mean value, square root amplitude, variance and standard deviation, the nonlinear and non-stationary information of motor imagery EEG signals at different time points can be captured and integrated.

[0064] Through the above steps, adaptive frequency-space-time joint feature learning is jointly achieved.

[0065] This embodiment constructs a spastic cerebral palsy rehabilitation system based on a motor imagery brain-computer interface combined with conventional rehabilitation training. According to the degree of motor impairment in children with spastic cerebral palsy, this embodiment designs three motor imagery paradigms: left hemiplegia paradigm, right hemiplegia paradigm and diplegia paradigm to meet the rehabilitation needs of different patients. In the hemiplegia paradigm, the grasping of the left / right upper limbs and the dorsiflexion of the lower limbs are trained in detail, while the diplegia paradigm is for the coordinated training of the dorsiflexion of the lower limbs. This embodiment adds motor imagery brain-computer interface training on the basis of conventional rehabilitation training, and continuously activates the neural network related to actual movement in the brain through repeated imagination, synergizes with conventional rehabilitation therapy, and promotes the recovery of motor function. This rehabilitation system makes up for the shortcomings of the conventional rehabilitation training process, which is monotonous and boring, and the low active participation of patients, and improves the training motivation and intervention effect of patients.

[0066] Example 3 A computer device 400, such as Fig.12 As shown, it includes a memory 410, a processor 420, and a computer program 430 stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface are implemented. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiment, which will not be repeated here.

[0067] Example 4 A computer readable storage medium such as Fig.13 As shown, a computer program is stored thereon, and when the computer program is executed by the processor, the steps of a method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface are implemented. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiment, and no further description is given here.

[0068] Example 5 A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of a method for rehabilitation of children with spastic cerebral palsy based on a motor imagery brain-computer interface. For a detailed description of the method, reference may be made to the corresponding description in the above method embodiment, which will not be repeated here.

[0069] The number of devices and processing scales described here are used to simplify the description of the present invention. Applications, modifications and variations of the present invention will be obvious to those skilled in the art.

[0070] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and implementation modes. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

[0071] The apparatus, computer device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, computer device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, computer device and non-volatile computer storage medium will not be repeated here.

[0072] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software units for implementing the method and structures within the hardware component.

[0073] The systems, devices or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions in various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or more software and / or hardware.

[0074] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0075] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0079] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program units. Generally, program units include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program units may be located in local and remote computer storage media, including storage devices.

[0080] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0081] The above description is only an embodiment of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A rehabilitation method for children with spastic cerebral palsy based on motor imagery brain-computer interface, characterized in that: The following steps are involved: Conduct limb function training, fine motor training and self-care ability training based on conventional rehabilitation training scenarios; Motor imagery training based on motor imagery brain-computer interface specifically includes the following steps: Access to the selected motor imagery paradigm; Present the motor imagery task of the motor imagery paradigm selected to enable children with spastic CP to continuously engage in motor imagery; Collect EEG signals of children with spastic cerebral palsy when they continuously perform motor imagery; The EEG signal is decoded and closed-loop feedback is used to evaluate whether the motor imagery is effective.

2. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 1, characterized in that: The motor imagery paradigm includes a left hemiplegia paradigm, a right hemiplegia paradigm, and a diplegia paradigm.

3. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 1, characterized in that: The step of performing motor imagery training based on the motor imagery brain-computer interface also includes: In each motor imagery training task, a rest time was set first to prompt children with spastic cerebral palsy to concentrate; After the rest period, a prompt message was given to indicate that the motor imagery task was about to be presented; After the motor imagery task was presented, a rest phase ensued.

4. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 3, characterized in that: The motor imagery training includes multiple rounds of motor imagery training, each round of motor imagery training includes multiple motor imagery tasks, and the next round begins after a preset time interval.

5. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 1, characterized in that: The EEG signal is configured to be acquired through a multi-lead EEG cap.

6. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 1, characterized in that: The decoding of the EEG signal, evaluating whether the motor imagery is effective by closed-loop feedback, and decoding the motor imagery task by using a deep learning classification method based on a convolutional neural network and multi-dimensional feature fusion, specifically includes the following steps: Preprocessing, data amplification and feature extraction of the EEG signal; The effect of the motor imagery task was evaluated using the motor imagery classification model; If the probability of the assessment result is lower than the set threshold, the child is reminded to adjust the motor imagery.

7. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 6, characterized in that: Also includes the steps: Divide the collected EEG signals into training set and test set; Preprocessing, data amplification and feature extraction of the EEG signals of the subjects in the training set: Combined with the adaptive joint feature matrix, a convolutional neural network structure is used to train the motor imagery classification model; The features extracted from the EEG signals of the test set subjects are used to test the motor imagery classification model, obtain classification results, and evaluate the model performance.

8. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 6, characterized in that: The preprocessing, data amplification and feature extraction of EEG signals include the following steps: The preset time period after the start of motor imagery is selected through bandpass filtering for subsequent classification, the EEG signal of the preset time before imagery is used as the baseline to remove drift, and then independent component analysis is used to remove artifacts; The preprocessed signal is cropped and amplified by sliding window to divide the time series of motor imagery into multiple segments; Constructing an adaptive frequency-space-time joint feature matrix that integrates multi-domain information includes the following steps: Based on wavelet packet decomposition of sub-band signal energy distribution, active frequency bands are extracted to achieve adaptive spectrum sensing; Adaptively weighting each lead signal based on the activation value and performing a pair of other common spatial mode spatial filtering; Based on multiple features, the temporal information expression is enriched to capture and integrate the nonlinear and non-stationary information of motor imagery EEG signals at different time points; Through the above steps, adaptive frequency-space-time joint feature learning is jointly achieved.

9. A method for rehabilitation of children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 7, characterized in that: The step of combining the adaptive joint feature matrix and using a convolutional neural network structure to train the motor imagery classification model includes: The convolutional neural network structure performs three layers of spatial convolution; The convolutional neural network structure performs two layers of temporal convolution; Connect two fully connected layers to integrate features and map the learned feature representation to the sample label space; Exponential normalization is used as the last layer of the network to map the output to the interval (0,1) for binary classification; Calculate the classification accuracy and Kappa coefficient to select the optimal motor imagery classification model; The EEG signals of the test set subjects are preprocessed, data amplified and feature extracted.

10. A rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface, characterized in that: It includes a conventional rehabilitation training module and a motor imagery brain-computer interface module. The conventional rehabilitation training module is used to realize limb function training, fine motor training and self-care ability training, and the motor imagery brain-computer interface module is used to realize motor imagery training; wherein, The motor imagery brain-computer interface module includes a motor imagery paradigm selection module, a motor imagery training module, an electroencephalogram device, and an electroencephalogram signal processing module; The motor imagery paradigm selection module is used to select a motor imagery paradigm; The motor imagery training module is used to realize the presentation of the motor imagery task of the selected motor imagery paradigm, so as to enable the children with spastic cerebral palsy to continuously perform motor imagery; The EEG device is used to collect EEG signals of children with spastic cerebral palsy when they continuously perform action imagination; The EEG signal processing module is used to decode the EEG signal and evaluate whether the motor imagery is effective through closed-loop feedback.

11. A rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 10, characterized in that: The motor imagery paradigm includes a left hemiplegia paradigm, a right hemiplegia paradigm, and a diplegia paradigm.

12. A rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 10, characterized in that: The motor imagery training module also includes a rest module, a prompt module, and a rest module: The rest module is used to set a rest time in each motor imagery training task to prompt the spastic cerebral palsy child to concentrate; The prompt module is used to issue a prompt message after the rest time is over to prompt that a motor imagery task is about to be presented; The rest module is used to enter the rest phase after the motor imagery task is presented.

13. A rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 12, characterized in that: The motor imagery training includes multiple rounds of motor imagery training, each round of motor imagery training includes multiple motor imagery tasks, and the next round begins after a preset time interval.

14. The motor imagery brain-computer interface-based rehabilitation system for children with spastic cerebral palsy as claimed in claim 10, characterized in that: The electroencephalogram device adopts a multi-lead electroencephalogram cap.

15. The motor imagery brain-computer interface-based rehabilitation system for children with spastic cerebral palsy as claimed in claim 10, characterized in that: The EEG signal processing module uses a deep learning classification method based on convolutional neural network and multi-dimensional feature fusion to decode the motor imagery task, specifically including: Preprocessing, data amplification and feature extraction of the EEG signal; The effect of the motor imagery task was evaluated using the motor imagery classification model; If the probability of the assessment result is lower than the set threshold, the child is reminded to adjust the motor imagery.

16. A rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 15, characterized in that: It also includes a motor imagery classification model training module, which is used to divide the collected EEG signals into a training set and a test set; preprocess, data amplify and extract features of the EEG signals of the subjects in the training set: combine the adaptive joint feature matrix and use a convolutional neural network structure to train the motor imagery classification model; use the features extracted from the EEG signals of the subjects in the test set to test the motor imagery classification model, obtain classification results, and evaluate the model performance.

17. A rehabilitation system for children with spastic cerebral palsy based on motor imagery brain-computer interface as claimed in claim 15, characterized in that: EEG signal preprocessing, data amplification and feature extraction include: The preset time period after the start of motor imagery is selected through bandpass filtering for subsequent classification, the EEG signal of the preset time before imagery is used as the baseline to remove drift, and then independent component analysis is used to remove artifacts; The preprocessed signal is cropped and amplified by sliding window to divide the time series of motor imagery into multiple segments; Constructing an adaptive frequency-space-time joint feature matrix that integrates multi-domain information includes the following steps: Based on wavelet packet decomposition of sub-band signal energy distribution, active frequency bands are extracted to achieve adaptive spectrum sensing; Adaptively weighting each lead signal based on the activation value and performing a pair of other common spatial mode spatial filtering; Based on multiple features, the temporal information expression is enriched to capture and integrate the nonlinear and non-stationary information of motor imagery EEG signals at different time points; Through the above steps, adaptive frequency-space-time joint feature learning is jointly achieved.

18. The motor imagery brain-computer interface-based rehabilitation system for children with spastic cerebral palsy as claimed in claim 16, characterized in that: The method of combining the adaptive joint feature matrix and using a convolutional neural network structure to train the motor imagery classification model includes: The convolutional neural network structure performs three layers of spatial convolution; The convolutional neural network structure performs two layers of temporal convolution; Connect two fully connected layers to integrate features and map the learned feature representation to the sample label space; Exponential normalization is used as the last layer of the network to map the output to the interval (0,1) for binary classification; Calculate the classification accuracy and Kappa coefficient to select the optimal motor imagery classification model; The EEG signals of the test set subjects are preprocessed, data amplified and feature extracted.

19. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images