Method, device and system for limb rehabilitation after stroke based on multi-layer perceptron model

Abstract

The present invention provides a method, device and system for limb rehabilitation of stroke based on a multi-layer perceptron model, the method comprising: prompting a user to perform motor imagery of limb movements; obtaining the user's motor imagery EEG signal; processing the motor imagery EEG signal to obtain sliding window EEG signal data; inputting the sliding window EEG signal data into a trained spatiotemporal multi-layer perceptron neural network model; obtaining the motor imagery identification result output by the spatiotemporal multi-layer perceptron neural network model; guiding a neuromuscular electrical stimulator to perform electrical stimulation treatment on the muscles of the user's limbs according to the motor imagery identification result. The limb rehabilitation method of stroke provided by the present invention can realize accurate acquisition, effective identification and correct classification of EEG signals, and perform electrical stimulation treatment on the hand muscle nerves by identifying the user's action intention, thereby assisting the user in completing rehabilitation training.

Description

Technical Field

[0001] The present invention belongs to the field of brain-controlled rehabilitation training, and in particular relates to a method, device and system for limb rehabilitation of stroke patients based on a multi-layer perceptron model. Background Art

[0002] After a stroke, the brain area supplied by the diseased blood vessels will be damaged, resulting in unilateral or bilateral hand motor function failure. In addition, some patients with hand fractures or limb amputation surgery also need rehabilitation training to restore hand motor function. Hand motor function rehabilitation can be performed through a neuromuscular electrical stimulation device. Neuromuscular electrical stimulation is a type of low-frequency electrotherapy. It is a technology that uses a 20-50Hz low-frequency current through electrodes to stimulate specific muscle groups to cause them to twitch or contract, thereby achieving "functional" repair. Neuromuscular electrical stimulation technology can be used to perform functional training on muscles with neurodysfunction, exercise to maintain muscle mass, increase joint mobility, promote voluntary muscle control, reduce spasms, and promote plastic changes in the brain. The key point of rehabilitation training is to restore the natural synchronization of brain nerve signals and hand movements, so it is necessary to combine the conscious signals of the human brain to cooperate with the rehabilitation movements of the hands.

[0003] Electroencephalogram (EEG) records the activity of the cerebral cortex and is a general reflection of the activity of the brain nerve cells in the cerebral cortex. It contains a large amount of physiological and pathological information, reflecting the activity state and thinking of the human brain. EEG signal detection technology can identify the rehabilitation movements that the patient currently wants to perform, drive the rehabilitation equipment to assist the patient in completing the movements, and speed up the rehabilitation process. However, EEG signal data has the characteristics of large data volume, many features, and fast changes. After completing data collection, manual analysis often wastes a lot of time and misses some important information. The existing EEG processing method cannot achieve accurate acquisition, effective identification and correct classification of EEG signals, which is not conducive to quickly judging the patient's thoughts and assisting the patient in completing rehabilitation training. Summary of the invention

[0004] In view of this, the present invention aims to propose a method, device and system for limb rehabilitation after stroke based on a multi-layer perceptron model, so as to solve the problem of poor rehabilitation training effect caused by the inability to effectively identify and correctly classify EEG signals in existing limb rehabilitation methods after stroke.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for limb rehabilitation after stroke based on a multi-layer perceptron model, comprising:

[0007] Prompt the user to perform motor imagery of body movements;

[0008] Acquire the user's motor imagery EEG signal;

[0009] Preprocessing the motor imagery EEG signals to obtain EEG signals corresponding to five frequency bands: θ, α, β, γ1, and γ2;

[0010] Using a sliding window, data segmentation is performed on the EEG signals corresponding to the five frequency bands of θ, α, β, γ1, and γ2 to obtain sliding window EEG signal data;

[0011] Input the sliding window EEG signal data into a trained spatiotemporal multilayer perceptron neural network model; the spatiotemporal multilayer perceptron neural network model includes a branch subnetwork, a series layer, and a classification layer, wherein five branch subnetworks are set corresponding to the five frequency bands of θ, α, β, γ1, and γ2, and the inputs of the five branch subnetworks are all the sliding window EEG signal data, and the outputs of the five branch subnetworks are connected in series through the series layer and then input into the classification layer; wherein each branch subnetwork extracts the electrode channel relationship characteristics and the time domain characteristics of the sliding window EEG signal data respectively through two multilayer perceptron layers;

[0012] Obtaining a motor imagery recognition result output by the spatiotemporal multilayer perceptron neural network model;

[0013] The neuromuscular electrical stimulator is guided to perform electrical stimulation treatment on the user's muscles according to the motor imagery recognition result.

[0014] Furthermore, the prompting the user to perform motor imagination includes: using pictures of body movements to prompt the user to perform motor imagination corresponding to the body movements.

[0015] Furthermore, the step of obtaining the user's motor imagery EEG signal includes:

[0016] The motor imagery EEG signals of a total of sixteen electrodes, A1, T5, O1, FP1, FCz, Oz, C4, P4, P3, C3, Fz, Cz, Pz, FP2, O2, and T6, were obtained in the 10-20 international standard lead system.

[0017] Furthermore, the motor imagery EEG signal is preprocessed to obtain EEG signals corresponding to five frequency bands of θ, α, β, γ1, and γ2, including:

[0018] The motor imagery EEG signal time series is bandpass filtered in five frequency bands through a Butterworth filter to obtain an EEG signal for realizing motor imagery action recognition; wherein the five frequency bands are θ, α, β, γ1, and γ2.

[0019] Furthermore, the EEG signals corresponding to the five frequency bands of θ, α, β, γ1, and γ2 are segmented using a sliding window to obtain sliding window EEG signal data, including:

[0020] The EEG brain wave signals in the five frequency bands of θ, α, β, γ1 and γ2 are respectively segmented through a sliding window with a time length of 4 seconds to obtain sliding window EEG signal data; wherein the sliding step length of the sliding window is 4 seconds.

[0021] Furthermore, the branch sub-network includes:

[0022] Data input layer; the input data is the sliding window EEG signal data;

[0023] The first transposition layer is used to transpose the sliding window EEG signal data to obtain an EEG signal channel correlation matrix;

[0024] The first multi-layer perceptron layer is used to obtain the relationship characteristics between the electrode channels of the EEG signal; wherein the first multi-layer perceptron layer includes two fully connected layers with 1024 neurons and an activation layer with a sigmoid activation function;

[0025] A second transposition layer, used to transpose the matrix output by the first multi-layer perceptron layer to obtain a temporal correlation matrix of the EEG signal;

[0026] A second multi-layer perceptron layer; used to obtain the time domain variation characteristics of the EEG signal; wherein the second multi-layer perceptron layer includes two fully connected layers with 1024 neurons and an activation layer with a sigmoid activation function;

[0027] Maximum pooling layer; the pooling kernel size is 2×2;

[0028] Fully connected layer; select ReLU as the activation function, select L2 norm as the regularization term, and set the L2 norm to 0.0008.

[0029] Furthermore, the method further comprises:

[0030] Constructing the spatiotemporal multilayer perceptron neural network model using the motor imagery EEG signal and the motor imagery recognition result corresponding to the motor imagery EEG signal;

[0031] Constructing the spatiotemporal multilayer perceptron neural network model includes:

[0032] A corresponding motor imagery recognition result is set for each motor imagery EEG signal to form a sample, and multiple samples of each user form a sample set;

[0033] Inputting all samples in the sample set into the spatiotemporal multilayer perceptron neural network model;

[0034] Keras was used to perform full-supervision training on the spatiotemporal multilayer perceptron neural network model. The model learning rate was set to 0.0025, and a total of 1000 cycles of cyclic training were performed. The automatic early stopping function was set. The batch size was 2000.

[0035] The trained spatiotemporal multilayer perceptron neural network model is obtained.

[0036] In a second aspect, an embodiment of the present invention further provides a stroke limb rehabilitation device based on a multi-layer perceptron model, comprising:

[0037] A prompt module, used to prompt the user to perform motor imagination of body movements;

[0038] An acquisition module, used to acquire the user's motor imagery EEG signal;

[0039] A preprocessing module is used to preprocess the motor imagery EEG signal to obtain EEG signals in five frequency bands: θ, α, β, γ1, and γ2;

[0040] A segmentation module is used to segment the EEG brain electrical signals of the five frequency bands of θ, α, β, γ1, and γ2 using a sliding window to obtain sliding window EEG signal data corresponding to the EEG brain electrical signals of the five frequency bands of θ, α, β, γ1, and γ2;

[0041] An input module, used to input the sliding window EEG signal data into the trained spatiotemporal multilayer perceptron neural network model respectively;

[0042] An identification module, used to obtain the motor imagery identification result output by the spatiotemporal multilayer perceptron neural network model;

[0043] The guiding module is used to guide the neuromuscular electrical stimulator to perform electrical stimulation treatment on the user's muscles according to the motor imagery recognition result.

[0044] Furthermore, the device also includes:

[0045] A construction module, used to construct the spatiotemporal domain multilayer perceptron neural network model using the motor imagery EEG signal and the motor imagery recognition result corresponding to the motor imagery EEG signal;

[0046] The building blocks are used to:

[0047] A corresponding motor imagery recognition result is set for each motor imagery EEG signal to form a sample, and multiple samples of each user form a sample set;

[0048] Inputting all samples in the sample set into the spatiotemporal multilayer perceptron neural network model;

[0049] Keras was used to perform full-supervision training on the spatiotemporal multilayer perceptron neural network model. The model learning rate was set to 0.0025, and a total of 1000 cycles of cyclic training were performed. The automatic early stopping function was set. The batch size was 2000.

[0050] The trained spatiotemporal multilayer perceptron neural network model is obtained.

[0051] In a third aspect, an embodiment of the present invention further provides a stroke limb rehabilitation system based on a multi-layer perceptron model, comprising:

[0052] Any of the multi-layer perceptron model-based limb rehabilitation devices for stroke provided in the above embodiments;

[0053] Multi-channel high-precision EEG data acquisition instrument and hand neuromuscular electrical stimulator.

[0054] Compared with the prior art, the stroke limb rehabilitation method, device and system based on the multi-layer perceptron model described in the present invention have the following advantages:

[0055] The stroke limb rehabilitation method based on the multi-layer perceptron model provided by the present invention can achieve accurate acquisition, effective identification and correct classification of EEG brain electrical signals, and by identifying the user's movement intention, electrically stimulate the hand muscle nerves to assist the user in completing rehabilitation training; compared with traditional electrical stimulation therapy, the present method can perform real-time electrical stimulation according to the patient's intention, effectively promoting the feedback rehabilitation of the neural connection between the patient's brain and hands.

[0056] The present invention also provides a stroke limb rehabilitation system based on a multi-layer perceptron model, which can effectively identify and correctly classify motor imagery EEG signals and realize the hand motor function rehabilitation training function. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0058] Figure 1 is a flow chart of a stroke limb rehabilitation method based on a multi-layer perceptron model provided in the first embodiment of the present invention;

[0059] Figure 2 A schematic diagram of the structure of a multi-channel high-precision EEG acquisition instrument in a stroke limb rehabilitation method based on a multi-layer perceptron model provided in the first embodiment of the present invention;

[0060] Figure 3 A schematic diagram of the structure of a multilayer perceptron neural network model in the spatiotemporal domain in the stroke limb rehabilitation method based on the multilayer perceptron model provided in the first embodiment of the present invention;

[0061] Figure 4 A schematic diagram of the structure of a multi-layer perceptron layer in the spatiotemporal domain multi-layer perceptron neural network model of the stroke limb rehabilitation method based on the multi-layer perceptron model provided in the first embodiment of the present invention;

[0062] Figure 5 A schematic diagram of the structure of a hand neuromuscular electrical stimulator in a stroke limb rehabilitation method based on a multi-layer perceptron model provided in the first embodiment of the present invention;

[0063] Figure 6 is a flow chart of a stroke limb rehabilitation method based on a multi-layer perceptron model provided in Embodiment 2 of the present invention;

[0064] Figure 7 A schematic diagram of the structure of a stroke limb rehabilitation device based on a multi-layer perceptron model provided in Embodiment 3 of the present invention;

[0065] Figure 8 This is a schematic diagram of the structure of a stroke limb rehabilitation system based on a multi-layer perceptron model provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0066] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0067] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0068] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0069] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0070] Embodiment 1

[0071] Figure 1 This is a flow chart of a stroke limb rehabilitation method based on a multi-layer perceptron model provided in the first embodiment of the present invention. This embodiment can be applied to using a multi-channel high-precision EEG acquisition instrument to collect the user's motor imagery EEG signal, and identify the user's limb movement intention, send the identified movement command to a neuromuscular electrical stimulator, and use the neuromuscular electrical stimulator to perform electrical stimulation treatment on the user's hand muscles to help the user perform limb movement function rehabilitation; this method can be executed by a stroke limb rehabilitation device based on a multi-layer perceptron model, and specifically includes the following steps:

[0072] S110, prompting the user to perform motor imagery of body movements;

[0073] Exemplarily, a display device such as a display or screen may be used to display a motor imagery prompt interface, which may be used to display a picture of the current action to remind the user to perform motor imagery, and to remind the user to take a rest after the stimulation is completed.

[0074] In actual application, pictures of limb movements can be used to prompt users to perform motor imagination of corresponding limb movements; the motor imagination prompt interface can display pictures of corresponding hand movements, prompting users to make motor imaginations corresponding to the pictures, stimulating the user's brain activity, facilitating the subsequent collection of the user's motor imagination EEG signals, improving the accuracy of identifying the user's movement intentions, and thereby guiding the hand neuromuscular electrical stimulator to more accurately stimulate specific positions of the user's limbs.

[0075] S120, obtaining a motor imagery EEG signal of the user;

[0076] Exemplarily, a multi-channel high-precision EEG acquisition device may be used to acquire the user's motor imagery EEG signals. Figure 2 This is a structural diagram of a multi-channel high-precision EEG acquisition instrument in a stroke limb rehabilitation method based on a multi-layer perceptron model provided in the first embodiment of the present invention, see Figure 2 The multi-channel high-precision EEG acquisition instrument includes: a brain electrode cap and its adapter cable for collecting EEG signals, which are connected in sequence, a high-precision EEG signal acquisition circuit for amplifying and converting EEG signals, an ARM processor and a WIFI wireless data transmission circuit for controlling the acquisition of EEG signals and transmitting EEG signals to an action intention recognition module, and a system power supply circuit respectively connected to the high-precision EEG signal acquisition circuit and the ARM processor.

[0077] The described brain electrode cap and its adapter cable obtain the user's EEG brain electrical signals corresponding to the user's A1, T5, O1, FP1, FCz, Oz, C4, P4, P3, C3, Fz, Cz, Pz, FP2, O2, T6 electrodes of the brain electrode cap; the electrode distribution of the brain electrode cap conforms to the 10 / 20 international standard lead. It is connected to the high-precision brain electrical signal acquisition circuit through the adapter cable and DSUB37 interface for brain electrical signal acquisition and transmission;

[0078] The high-precision EEG signal acquisition circuit is composed of several bioelectric signal acquisition chips that integrate a high common mode rejection ratio analog input module for receiving voltage signals collected by an EEG cap, a low-noise programmable gain amplifier (PGA) for amplifying brain voltage signals, and a high-resolution synchronous sampling analog-to-digital converter (ADC) for converting analog signals into digital signals. The sampling rate is 250Hz.

[0079] The ARM processor and WIFI wireless data transmission circuit are integrated by the main control chip SimpleLink CC3200. The ARM processor adopts the Cortex-M4 core, which is used to adjust the acquisition mode and parameters of the high-precision EEG signal acquisition circuit and control the transmission mode and speed of the WIFI wireless data transmission circuit; the output of the WIFI wireless data transmission circuit is connected to the input end of the action intention recognition module, works in Soft AP mode, and has a maximum transmission rate of 54Mbps, transmitting the EEG signal data to the action intention recognition module; the system power supply circuit has an input voltage of 5V, is powered by a USB cable, and provides the working voltage of different chips in the system through a voltage conversion module.

[0080] In practical applications, the user's motor imagery EEG signals can be obtained through a multi-channel high-precision EEG collector, that is, the signals of the sixteen electrodes A1, T5, O1, FP1, FCz, Oz, C4, P4, P3, C3, Fz, Cz, Pz, FP2, O2, and T6 in the 10-20 international standard lead system. These sixteen electrodes are commonly used electrodes in the motor imagery paradigm, and the motor imagery signals are related to them.

[0081] S130, preprocessing the motor imagery EEG signal to obtain EEG signals corresponding to five frequency bands: θ, α, β, γ1, and γ2;

[0082] Exemplarily, the motor imagery EEG signal time series can be bandpass filtered in five frequency bands through a Butterworth filter to obtain an EEG signal for realizing motor imagery action recognition; wherein the five frequency bands are θ, α, β, γ1, and γ2.

[0083] In actual application, the user's EEG signal time series can be band-pass filtered through a Butterworth filter and divided into five frequency bands: θ (4-7 Hz), α (8-12 Hz), β (13-30 Hz), γ1 (31-50 Hz) and γ2 (51-85 Hz), to obtain an EEG signal that can be used to realize motor imagery action recognition. Where L represents the time length, C represents the frequency band number, and G represents the number of channels.

[0084] S140, using a sliding window to segment the EEG signals corresponding to the five frequency bands of θ, α, β, γ1, and γ2 to obtain sliding window EEG signal data;

[0085] Exemplarily, the EEG signals in the five frequency bands of θ, α, β, γ1, and γ2 can be segmented by a sliding window with a time length of 4 seconds to obtain sliding window EEG signal data; wherein the sliding step length of the sliding window is 4 seconds, and the length of 4 seconds can better ensure the balance between acquisition time and the amount of useful information.

[0086] In practical applications, it is necessary to analyze the EEG signals in five frequency bands. The data is segmented by sliding windows with a time length of 4s, and the sliding step of the sliding window is 4s, and a series of sliding window EEG signal data matrices are obtained. Among them, C represents the number of frequency bands, g represents the number of sampling time points, and j represents the number of channels.

[0087] S150, inputting the sliding window EEG signal data into the trained spatiotemporal multilayer perceptron neural network model; Figure 3This is a schematic diagram of the structure of the spatiotemporal domain multilayer perceptron neural network model in the stroke limb rehabilitation method based on the multilayer perceptron model provided in the first embodiment of the present invention, see Figure 3 The spatiotemporal multilayer perceptron neural network model includes a branch subnetwork, a series layer, and a classification layer. Five branch subnetworks are set corresponding to the five frequency bands of θ, α, β, γ1, and γ2. The inputs of the five branch subnetworks are all the sliding window EEG signal data. The outputs of the five branch subnetworks are connected in series through the series layer and then input into the classification layer; wherein each branch subnetwork extracts the electrode channel relationship characteristics and time domain characteristics of the sliding window EEG signal data respectively through two multilayer perceptron layers; of the two multilayer perceptron layers of each branch subnetwork, the former is used to extract low-level (channel association) features, and the latter is used to extract high-level (time association) features based on the low-level features, thereby providing better input for the subsequent classification layer and improving the accuracy of the motor imagery recognition results output by the classification layer.

[0088] Figure 4 The structural diagram of the multi-layer perceptron layer in the spatiotemporal domain multi-layer perceptron neural network model of the stroke limb rehabilitation method based on the multi-layer perceptron model provided in the first embodiment of the present invention is shown in FIG. Figure 4 , the branch sub-network includes:

[0089] Data input layer; the input data is the sliding window EEG signal data;

[0090] The first transposition layer is used to transpose the sliding window EEG signal data to obtain an EEG signal channel correlation matrix;

[0091] The first multi-layer perceptron layer is used to obtain the relationship characteristics between the electrode channels of the EEG signal; wherein the first multi-layer perceptron layer includes two fully connected layers with 1024 neurons and an activation layer with a sigmoid activation function;

[0092] A second transposition layer, used to transpose the matrix output by the first multi-layer perceptron layer to obtain a temporal correlation matrix of the EEG signal;

[0093] A second multi-layer perceptron layer; used to obtain the time domain variation characteristics of the EEG signal; wherein the second multi-layer perceptron layer includes two fully connected layers with 1024 neurons and an activation layer with a sigmoid activation function;

[0094] Maximum pooling layer; the pooling kernel size is 2×2;

[0095] Fully connected layer; select ReLU as the activation function, select L2 norm as the regularization term, and set the L2 norm to 0.0008.

[0096] In this embodiment, two transposition layers are set to achieve two transpositions in order to correspond to the input of the multi-layer perceptron, so that the multi-layer perceptron layer can be associated and trained in the channel dimension / time dimension.

[0097] In practical applications, the spatiotemporal multilayer perceptron neural network model includes five large branches, each of which corresponds to a frequency band of signals θ, α, β, γ1 and γ2; each branch includes the following connected in sequence:

[0098] Data input layer, the input data is the user's EEG signal data matrix

[0099] The transposition layer transposes the EEG signal data matrix to obtain the data matrix:

[0100] Multilayer perceptron layer, the number of perceptrons is j, and the input data is the corresponding row of the data matrix The output matrix is Includes one fully connected layer with 1024 neurons;

[0101] Activation layer, the activation function is sigmoid; a fully connected layer, 1024 neurons;

[0102] Transpose layer, transpose the multi-layer perceptron layer output matrix, and obtain the data matrix as

[0103] Multilayer perceptron layer, the number of perceptrons is g, and the input data is the corresponding row of the data matrix The output matrix is It includes: one fully connected layer with 1024 neurons; one activation layer with sigmoid activation function; one fully connected layer with 1024 neurons;

[0104] Maximum pooling layer, the pooling kernel size is 2×2;

[0105] For the fully connected layer, ReLU is selected as the activation function, L2 norm is selected as the regularization term, L2 norm is set to 0.0008, and the output is defined as Un, n is the corresponding frequency band.

[0106] The outputs Un of the fully connected layers of the five frequency band branches are connected in series to form a series layer, and the series layer is connected to a classification layer. The output of the classification layer is the motor imagery recognition result output by the spatiotemporal multilayer perceptron neural network model, that is, the action instruction result. The spatiotemporal multilayer perceptron neural network model is encapsulated by a program package and can be directly installed and run on a computer or a host computer.

[0107] S160, obtaining the motor imagery recognition result output by the spatiotemporal multilayer perceptron neural network model;

[0108] Exemplarily, by acquiring the user's motor imagery EEG signal and processing it through a spatiotemporal multi-layer perceptron neural network model, a motor imagery identification result identified based on the motor imagery EEG signal can be obtained, that is, an action instruction result; the action instruction result can indicate the part of the limb that the user wants to move, thereby facilitating guiding the neuromuscular electrical stimulator to stimulate the muscles of the limb that needs to move.

[0109] S170. Guiding a neuromuscular electrical stimulator to perform electrical stimulation treatment on the muscles of the user's limbs according to the motor imagery recognition result; the neuromuscular electrical stimulator may be a hand neuromuscular electrical stimulator, and those skilled in the art may also select a neuromuscular electrical stimulator for a corresponding part to treat the patient according to actual treatment needs.

[0110] For example, Figure 5 This is a schematic diagram of the structure of the hand neuromuscular electrical stimulator in the stroke limb rehabilitation method based on the multilayer perceptron model provided in the first embodiment of the present invention, see Figure 5 The hand neuromuscular electrical stimulator includes: a stimulation signal generating circuit for muscle electrical stimulation, a power supply circuit, a main control circuit, a Bluetooth wireless communication circuit, and a low-battery indication circuit connected in sequence. The electrical stimulation signal generating circuit can generate square wave signals of 5Hz, 10Hz, 15Hz, 20Hz, and 25Hz, and the maximum open-circuit output voltage amplitude is 120V, which can be adjusted steplessly. The input voltage of the power supply circuit is 6V, which is powered by 4 No. 5 batteries, and the operating voltage of different chips in the system is provided by the voltage conversion module. The main control circuit is controlled by an ARM processor, and the chip selects STM32F103C8T6, which is used to adjust the electrical stimulation frequency of the electrical stimulation generating circuit and control the transmission mode and speed of the Bluetooth wireless data transmission circuit. The Bluetooth wireless communication circuit adopts the BLE4.2 protocol to receive action instructions from the action intention recognition module. The low-battery indication circuit is used to prompt the user to replace the battery when the battery power is too low.

[0111] During actual application, the operator can also set the electrical stimulation parameters of the hand neuromuscular electrical stimulator in advance according to treatment needs to meet the rehabilitation training requirements of different users; the parameter settings of the hand neuromuscular electrical stimulator can also be integrated into the motor imagery prompt interface. During use, the operator can use the motor imagery prompt interface to design the parameters of the hand neuromuscular electrical stimulator and display limb movement pictures to remind the user to perform corresponding motor imagery.

[0112] This embodiment uses a motor imagery prompt interface to set parameters and provide action prompts before the user performs rehabilitation training, and collects the user's motor imagery EEG signals through a multi-channel high-precision EEG collector; then, the motor imagery EEG signals are processed and input into a spatiotemporal multilayer perceptron neural network model to obtain a motor imagery recognition result of the user's intended action; finally, according to the motor imagery recognition result of the user's intended action, a hand neuromuscular electrical stimulator is guided to perform electrical stimulation treatment on the muscles of the user's limbs, thereby helping the user to rehabilitate the motor function of the hand, which is beneficial to improving the user's rehabilitation training effect.

[0113] Embodiment 2

[0114] Figure 6 It is a flow chart of a stroke limb rehabilitation method based on a multilayer perceptron model provided in Example 2 of the present invention. This embodiment is optimized on the basis of the above embodiment. In this embodiment, the following steps are added: constructing the spatiotemporal multilayer perceptron neural network model using the motor imagery EEG signal and the motor imagery identification result corresponding to the motor imagery EEG signal; constructing the spatiotemporal multilayer perceptron neural network model, including: setting a corresponding motor imagery identification result for each motor imagery EEG signal to form a sample, and multiple samples of each user constitute a sample set; inputting all samples in the sample set into the spatiotemporal multilayer perceptron neural network model; using Keras to perform fully supervised training on the spatiotemporal multilayer perceptron neural network model, with the model learning rate set to 0.0025, performing a total of 1000 cycles of cyclic training, and setting an automatic early stopping function; wherein, the batch size is 2000; and obtaining a trained spatiotemporal multilayer perceptron neural network model.

[0115] Accordingly, the drone brain control method based on the EEG acquisition headband provided in this embodiment specifically includes:

[0116] S210. Construct the spatiotemporal multilayer perceptron neural network model using the motor imagery EEG signal and the motor imagery identification result corresponding to the motor imagery EEG signal; construct the spatiotemporal multilayer perceptron neural network model, including: setting the corresponding motor imagery identification result for each motor imagery EEG signal to form a sample, and multiple samples of each user constitute a sample set; inputting all samples in the sample set into the spatiotemporal multilayer perceptron neural network model; using Keras to perform fully supervised training on the spatiotemporal multilayer perceptron neural network model, with the model learning rate set to 0.0025, performing 1000 cycles of cyclic training in total, and setting an automatic early stopping function; wherein, the batch size is 2000; and obtaining a trained spatiotemporal multilayer perceptron neural network model.

[0117] For example, in order to improve the motion recognition accuracy of the spatiotemporal multilayer perceptron neural network model, it is necessary to train the spatiotemporal multilayer perceptron neural network model. The corresponding training requires the use of a sample set. A sample is formed by setting a corresponding motor imagery recognition result for each motor imagery EEG signal. Multiple samples of each user form a sample set. In actual application, in order to improve the generalization ability of the spatiotemporal multilayer perceptron neural network model for different users, samples of multiple different users can also form a sample set.

[0118] S220, prompting the user to perform motor imagery of body movements;

[0119] S230, obtaining a motor imagery EEG signal of the user;

[0120] S240, preprocessing the motor imagery EEG signal to obtain EEG signals corresponding to five frequency bands: θ, α, β, γ1, and γ2;

[0121] S250, using a sliding window to segment the EEG signals corresponding to the five frequency bands of θ, α, β, γ1, and γ2 to obtain sliding window EEG signal data;

[0122] S260, inputting the sliding window EEG signal data into a trained spatiotemporal multilayer perceptron neural network model; the spatiotemporal multilayer perceptron neural network model includes a branch subnetwork, a series layer, and a classification layer, wherein five branch subnetworks are set corresponding to the five frequency bands of θ, α, β, γ1, and γ2, and the inputs of the five branch subnetworks are all the sliding window EEG signal data, and the outputs of the five branch subnetworks are connected in series through the series layer and then input into the classification layer; wherein each branch subnetwork extracts the electrode channel relationship characteristics and the time domain characteristics of the sliding window EEG signal data respectively through two multilayer perceptron layers;

[0123] S270, obtaining the motor imagery recognition result output by the spatiotemporal multilayer perceptron neural network model;

[0124] S280, guiding the neuromuscular electrical stimulator to perform electrical stimulation treatment on the muscles of the user's limbs according to the motor imagery recognition result.

[0125] This embodiment constructs the spatiotemporal multilayer perceptron neural network model by utilizing the motor imagery EEG signal and the motor imagery identification result corresponding to the motor imagery EEG signal, and uses samples to fully train the spatiotemporal multilayer perceptron neural network model, which can improve the accuracy of the motor imagery identification result output by the spatiotemporal multilayer perceptron neural network model, facilitate the precise stimulation of the user's limb muscles by the hand neuromuscular electrical stimulator, accelerate the user's rehabilitation training progress, and provide rehabilitation training effects.

[0126] Embodiment 3

[0127] Figure 7 The schematic diagram of the structure of the limb rehabilitation device for stroke based on the multi-layer perceptron model provided in the third embodiment of the present invention is as follows. The limb rehabilitation device for stroke analyzes the motor imagery EEG signal of the user through the spatiotemporal multi-layer perceptron neural network model in the above embodiment to obtain the motor imagery recognition result of the user's movement intention. The hand neuromuscular electrical stimulator performs targeted electrical stimulation on the muscles of the limb that the user intends to move according to the motor imagery recognition result to achieve limb rehabilitation training for stroke. Figure 7 As shown, the device comprises:

[0128] Prompt module 710, used to prompt the user to perform motor imagination of body movements;

[0129] An acquisition module 720 is used to acquire the motor imagery EEG signal of the user;

[0130] A preprocessing module 730 is used to preprocess the motor imagery EEG signal to obtain EEG signals in five frequency bands: θ, α, β, γ1, and γ2;

[0131] The segmentation module 740 is used to segment the EEG brain electrical signals of the five frequency bands of θ, α, β, γ1, and γ2 using a sliding window to obtain sliding window EEG signal data corresponding to the EEG brain electrical signals of the five frequency bands of θ, α, β, γ1, and γ2;

[0132] An input module 750 is used to input the sliding window EEG signal data into the trained spatiotemporal multilayer perceptron neural network model;

[0133] An identification module 760 is used to obtain the motor imagery identification result output by the spatiotemporal multilayer perceptron neural network model;

[0134] The guiding module 770 is used to guide the neuromuscular electrical stimulator to perform electrical stimulation treatment on the user's muscles according to the motor imagery recognition result.

[0135] Based on the above embodiments, the device further includes:

[0136] A construction module, used to construct the spatiotemporal domain multilayer perceptron neural network model using the motor imagery EEG signal and the motor imagery recognition result corresponding to the motor imagery EEG signal;

[0137] The building blocks are used to:

[0138] A corresponding motor imagery recognition result is set for each motor imagery EEG signal to form a sample, and multiple samples of each user form a sample set;

[0139] Inputting all samples in the sample set into the spatiotemporal multilayer perceptron neural network model;

[0140] Keras was used to perform full-supervision training on the spatiotemporal multilayer perceptron neural network model. The model learning rate was set to 0.0025, and a total of 1000 cycles of cyclic training were performed. The automatic early stopping function was set. The batch size was 2000.

[0141] The trained spatiotemporal multilayer perceptron neural network model is obtained.

[0142] Exemplarily, the above modules can all be run in a host computer to achieve data reception, processing and distribution.

[0143] The stroke rehabilitation device provided in this embodiment prompts the user to perform motor imagination of corresponding actions and obtains the user's motor imagination EEG signals. After processing the motor imagination EEG signals, the signals are input into a spatiotemporal multilayer perceptron neural network model for analysis, thereby obtaining the user's motor imagination recognition results for guiding a hand neuromuscular electrical stimulator. Combined with the neuromuscular electrical stimulator, precise stimulation of the user's limb muscles can be achieved, thereby improving the user's rehabilitation training effect.

[0144] The stroke limb rehabilitation device based on the multilayer perceptron model provided in the embodiment of the present invention can execute the stroke limb rehabilitation device method based on the multilayer perceptron model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0145] Embodiment 4

[0146] Figure 8 This is a schematic diagram of the structure of a stroke limb rehabilitation system based on a multi-layer perceptron model provided in the fourth embodiment of the present invention, see Figure 8 The stroke limb rehabilitation system based on the multi-layer perceptron model includes any of the stroke limb rehabilitation devices based on the multi-layer perceptron model provided in the above embodiments; a multi-channel high-precision EEG acquisition instrument and a hand neuromuscular electrical stimulator; wherein the multi-channel high-precision EEG acquisition instrument realizes data connection with the hand neuromuscular electrical stimulator through the stroke limb rehabilitation device.

[0147] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A limb rehabilitation device for stroke patients based on a multi-layer perceptron model, characterized in that: include: A prompt module, used to prompt the user to perform motor imagery of body movements; An acquisition module, used to acquire the user's motor imagery EEG signal; A preprocessing module is used to preprocess the motor imagery EEG signal to obtain , , , , EEG signals in five frequency bands; The segmentation module is used to use a sliding window to segment the , , , , The EEG signals of the five frequency bands are segmented separately to obtain , , , , Sliding window EEG signal data corresponding to five frequency bands of EEG signals; An input module is used to input the sliding window EEG signal data into a trained spatiotemporal multilayer perceptron neural network model; the spatiotemporal multilayer perceptron neural network model includes a branch subnetwork, a series layer, and a classification layer, and the branch subnetwork corresponds to , , , , Five frequency bands are set, and the inputs of the five branch sub-networks are all the sliding window EEG signal data. The outputs of the five branch sub-networks are connected in series through the series layer and then input into the classification layer; wherein each branch sub-network extracts the electrode channel relationship characteristics and time domain characteristics of the sliding window EEG signal data respectively through two multi-layer perceptron layers; the branch sub-network includes: Data input layer; the input data is the sliding window EEG signal data; The first transposition layer is used to transpose the sliding window EEG signal data to obtain an EEG signal channel correlation matrix; The first multi-layer perceptron layer is used to obtain the relationship characteristics between the electrode channels of the EEG signal; wherein the first multi-layer perceptron layer includes two fully connected layers with 1024 neurons and an activation layer with a sigmoid activation function; A second transposition layer, used to transpose the matrix output by the first multi-layer perceptron layer to obtain a temporal correlation matrix of the EEG signal; A second multi-layer perceptron layer; used to obtain the time domain variation characteristics of the EEG signal; wherein the second multi-layer perceptron layer includes two fully connected layers with 1024 neurons and an activation layer with a sigmoid activation function; Maximum pooling layer; the pooling kernel size is 2×2; Fully connected layer; select ReLU as the activation function, select L2 norm as the regularization term, and set the L2 norm to 0.0008; An identification module, used to obtain the motor imagery identification result output by the spatiotemporal multilayer perceptron neural network model; The guiding module is used to guide the neuromuscular electrical stimulator to perform electrical stimulation treatment on the user's muscles according to the motor imagery recognition result.

2. The device according to claim 1, characterized in that The method of prompting the user to perform motor imagery of body movements includes: Use pictures of body movements to prompt users to imagine the corresponding body movements.

3. The device according to claim 1, characterized in that The step of obtaining the user's motor imagery EEG signal comprises: The user's motor imagery EEG signals are obtained, namely, the signals of the sixteen electrodes A1, T5, O1, FP1, FCz, Oz, C4, P4, P3, C3, Fz, Cz, Pz, FP2, O2, and T6 in the 10-20 international standard lead system.

4. The device according to claim 1, characterized in that The motor imagery EEG signal is preprocessed to obtain , , , , The EEG signals corresponding to the five frequency bands include: The motor imagery EEG signal time series is band-pass filtered in five frequency bands through a Butterworth filter to obtain an EEG signal for realizing motor imagery action recognition; wherein the five frequency bands are , , , , .

5. The device according to claim 1, characterized in that Use a sliding window to , , , , The EEG signals corresponding to the five frequency bands are segmented to obtain sliding window EEG signal data, including: Regarding the , , , , The EEG signals of the five frequency bands are segmented by sliding windows with a time length of 4 seconds to obtain sliding window EEG signal data; wherein the sliding step length of the sliding window is 4 seconds.

6. The device according to claim 1, characterized in that The device also includes: A construction module, used to construct the spatiotemporal domain multilayer perceptron neural network model using the motor imagery EEG signal and the motor imagery recognition result corresponding to the motor imagery EEG signal; The building blocks are used to: A corresponding motor imagery recognition result is set for each motor imagery EEG signal to form a sample, and multiple samples of each user form a sample set; Inputting all samples in the sample set into the spatiotemporal multilayer perceptron neural network model; Keras was used to perform full-supervision training on the spatiotemporal multilayer perceptron neural network model. The model learning rate was set to 0.0025, and a total of 1000 cycles of cyclic training were performed. The automatic early stopping function was set. The batch size was 2000. The trained spatiotemporal multilayer perceptron neural network model is obtained.

7. A stroke limb rehabilitation system based on a multi-layer perceptron model, characterized in that: include: The stroke limb rehabilitation device based on the multilayer perceptron model as described in any one of claims 1 to 6; Multi-channel high-precision EEG data acquisition instrument and neuromuscular electrical stimulator.

Images