Stroke hand rehabilitation training method, device and system

Abstract

The present invention discloses a method, device and system for hand rehabilitation training for stroke, the method comprising: obtaining EEG signals from forty electrodes; performing fast Fourier transform on the EEG signals, and extracting theta, alpha, and beta frequency bands in the EEG spectrum as EEG features; projecting the positions of the 40-lead electrodes from a three-dimensional space to a two-dimensional plane; normalizing the spectrum power value corresponding to each electrode, and matching it with the electrode position to obtain a discrete image, and obtaining two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β based on the discrete image; converting the two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β into two-dimensional color EEG signal images, and inputting the two-dimensional color EEG signal images into a trained motion signal convolutional neural network model; obtaining a motor imagery result output by the motion signal convolutional neural network model; and guiding a hand rehabilitation device to perform rehabilitation treatment according to the motor imagery result.

Description

Technical Field

[0001] The present invention relates to the technical field of stroke rehabilitation, and in particular to a stroke hand rehabilitation training method, device and system. Background Art

[0002] Electroencephalography (EEG), a physiological monitoring method for recording electrical activity in the brain, continuously collects changes in brain potentials using electrodes placed near the cerebral cortex. Recent advances in digital technology have greatly facilitated the collection and analysis of EEG signals, enabling temporal resolution down to milliseconds or even better. In contrast, other non-invasive cognitive neuroscience techniques, besides EEG, are only cranial magnetic resonance spectroscopy (MRS) and magnetoencephalography (MEG) capable of acquiring data at this sampling rate. Furthermore, because EEG is noise-free, it can more accurately reflect the brain's response to sound stimuli. The rich information contained in EEG signals has been successfully extracted and applied to a wide range of fields, including epilepsy diagnosis, sleep disorder diagnosis, anesthesia depth assessment, emotion recognition, and motor imagery. The development of brain-computer interfaces (BCIs) has further expanded the application of EEG signals to include brain-controlled peripherals. The recent invention of brain-controlled wheelchairs and prosthetic devices has opened up new prospects for rehabilitation for people with disabilities.

[0003] The incidence of stroke has been gradually increasing in my country in recent years. The causes are diverse, and the patient population is vast. After the onset of the disease, blockage or damage to blood vessels in the brain can impair the function of certain brain regions, leading to partial limb weakness or even paralysis. To ensure that stroke patients can resume daily functional hand movements and ensure coordination and synchronization between their brain's intentions and corresponding hand movements, it is necessary not only to restore hand motor function but also to synchronize rehabilitation training with brain consciousness. This is where EEG-based rehabilitation therapy devices come into being.

[0004] In the field of EEG signal research, researchers have proposed numerous methods for identifying, classifying, and predicting EEG signals to overcome the signal coupling inherent in the complex human brain. These include methods for extracting classic EEG features, time-frequency domain analysis of EEG signals, Bayesian methods, and, in recent years, deep learning methods, which have been gaining widespread attention. Extracting distinct features from EEG signals and effectively integrating the collected multi-dimensional state information to more accurately determine the subject's current motor intentions is a challenging task. Creating a method that consistently performs well across diverse subjects will facilitate the application and widespread adoption of motor imagery recognition systems in real-world scenarios. Summary of the Invention

[0005] The embodiments of the present invention provide a method, device and system for hand rehabilitation training for stroke patients, so as to solve the technical problem in the prior art that it is impossible to judge the current movement intention of the subject based on a large number of types of EEG signals.

[0006] In a first aspect, an embodiment of the present invention provides a method for hand rehabilitation training after stroke, comprising:

[0007] Acquire EEG signals from forty electrodes;

[0008] Performing fast Fourier transform on the EEG signal, and then extracting theta, alpha, and beta frequency bands in the EEG spectrum as EEG features;

[0009] Project the positions of the 40-lead electrodes from 3D space to a 2D plane;

[0010] Normalizing the spectral power values corresponding to each electrode and matching them with the electrode positions to obtain discrete images, and obtaining two-dimensional grayscale EEG signal images in three frequency bands of θ, α, and β based on the discrete images;

[0011] Converting the two-dimensional grayscale EEG signal images of the three frequency bands of θ, α, and β into two-dimensional color EEG signal images, and inputting the two-dimensional color EEG signal images into the trained motion signal convolutional neural network model;

[0012] Obtaining a motion imagery result output by the motion signal convolutional neural network model;

[0013] The hand rehabilitation device is guided to perform rehabilitation treatment according to the motor imagery results.

[0014] In a second aspect, an embodiment of the present invention further provides a stroke hand rehabilitation training device, comprising:

[0015] An acquisition module is used to acquire EEG signals from forty electrodes;

[0016] A lifting module is used to perform fast Fourier transform on the EEG signal, and then extract the θ, α, and β frequency bands in the EEG spectrum as EEG features;

[0017] A projection module, used to project the positions of the 40-lead electrodes from a 3D space to a 2D plane;

[0018] A matching module is used to normalize the spectral power values corresponding to each electrode and match them with the electrode positions to obtain discrete images, and to obtain two-dimensional grayscale EEG signal images in three frequency bands of θ, α, and β based on the discrete images;

[0019] A conversion module is used to convert the two-dimensional grayscale EEG signal images in the three frequency bands of θ, α and β into two-dimensional color EEG signal images, and input the two-dimensional color EEG signal images into the trained motion signal convolutional neural network model;

[0020] An output module, configured to obtain a motion imagery result output by the motion signal convolutional neural network model;

[0021] The guidance module is used to guide the hand rehabilitation equipment to perform rehabilitation treatment according to the motor imagery results.

[0022] In a third aspect, an embodiment of the present invention further provides a stroke hand rehabilitation training system, comprising:

[0023] Any one of the stroke hand rehabilitation training devices provided in the above embodiments;

[0024] High-precision 40-channel EEG acquisition equipment and hand rehabilitation equipment.

[0025] The stroke hand rehabilitation training method, device and system provided by the embodiments of the present invention obtain EEG signals from forty electrodes; perform fast Fourier transform on the EEG signals, and then extract the θ, α, and β frequency bands in the EEG spectrum as EEG features; project the positions of the 40-lead electrodes from a three-dimensional space to a two-dimensional plane; normalize the spectral power values corresponding to each electrode and match them with the electrode positions to obtain discrete images, and obtain two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β based on the discrete images; convert the two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β into two-dimensional color EEG signal images, and input the two-dimensional color EEG signal images into a trained motion signal convolutional neural network model; obtain the motion imagery results output by the motion signal convolutional neural network model; and guide the hand rehabilitation equipment to perform rehabilitation treatment based on the motion imagery results. Using positional projection, electrodes at different locations can be projected onto a single plane. Normalization generates discrete images, and through image conversion, a color EEG signal image corresponding to the three key frequency bands is formed. This allows for the integration of multiple EEG signal features into a single image, facilitating identification by neural network models without missing important information. The identification results can be used to guide patients in rehabilitation training. This also facilitates the construction of convolutional neural network models for motion signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0027] Figure 1 1 is a flow chart of a stroke hand rehabilitation training method provided in Example 1 of the present invention;

[0028] Figure 2 This is a schematic diagram of the structure of a high-precision 40-lead EEG acquisition device in the stroke hand rehabilitation training method provided in Example 1 of the present invention;

[0029] Figure 3 This is a schematic diagram of the minimum interpolation unit in the stroke hand rehabilitation training method provided in Example 1 of the present invention;

[0030] Figure 4 Schematic diagram showing the process and results of converting EEG time domain signals into two-dimensional color images in the stroke hand rehabilitation training method provided in Example 1 of the present invention

[0031] Figure 5 1 is a flow chart of a stroke hand rehabilitation training method provided in Example 2 of the present invention;

[0032] Figure 6 This is a diagram of the internal structure of the CELL in the stroke hand rehabilitation training method provided in Example 2 of the present invention;

[0033] Figure 7 A schematic diagram of the basic architecture of a one-shot model in the stroke hand rehabilitation training method provided in the second embodiment of the present invention;

[0034] Figure 8 A schematic diagram of network gradient propagation in the stroke hand rehabilitation training method provided in the second embodiment of the present invention;

[0035] Figure 9 A schematic diagram of a network basic unit CELL obtained by searching the final network structure in the stroke hand rehabilitation training method provided in the second embodiment of the present invention;

[0036] Figure 10 This is a schematic diagram of the structure of a hand rehabilitation training device for stroke provided in Example 3 of the present invention;

[0037] Figure 11 This is a structural diagram of the stroke hand rehabilitation training system provided in Example 4 of the invention. DETAILED DESCRIPTION

[0038] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0039] Example 1

[0040] Figure 1This is a flow chart of a hand rehabilitation training method for stroke patients provided in Example 1 of the present invention. This embodiment is applicable to situations where EEG signals from forty electrodes are used to identify movement signals to guide rehabilitation. The method can be performed by a hand rehabilitation training device for stroke patients and specifically includes the following steps:

[0041] S110, acquiring EEG signals from forty electrodes.

[0042] For example, a high-precision 40-lead EEG acquisition device may be used to acquire EEG signals from forty electrodes. Figure 2 This is a structural diagram of a high-precision 40-lead EEG acquisition device in the stroke hand rehabilitation training method provided in Example 1 of the present invention, see Figure 2 The high-precision 40-lead EEG acquisition equipment includes: a brain electrode cap and its adapter cable for collecting EEG brain electrical signals, a high-precision bioelectric signal acquisition circuit for amplifying and converting EEG signals, an isolation circuit for distinguishing and isolating analog signals from digital signals to ensure the purity of EEG signals (, an STM32 processor for controlling the bioelectric signal acquisition module and reading and transmitting its collected data, a USB communication circuit for transmitting EEG brain electrical signals to a host computer, and a power supply circuit for powering the bioelectric signal acquisition circuit and the STM32 processor respectively. The brain electrode cap and its adapter cable collect EEG brain electrical signals from different brain regions of the user, and are connected to the Y2 series circular electrical connector interface and the bioelectric signal acquisition module through the adapter cable for collecting and transmitting bioelectric signals;

[0043] The bioelectric signal acquisition module is composed of several bioelectric signal acquisition chips that integrate a high common-mode rejection ratio analog input module for receiving bioelectric signals collected by a brain electrode cap, a low-noise programmable gain amplifier (PGA) for amplifying biovoltage signals, and a high-resolution synchronous sampling analog-to-digital converter (ADC) for converting analog signals into digital signals. The STM32 processor is used to control the acquisition mode, parameters, and working status of the bioelectric signal acquisition module, and control the transmission mode and transmission speed of the USB communication circuit.

[0044] The output of the USB communication circuit is connected to the input of the motor imagery learning module and the motor imagery discrimination module, and is set as a high-speed transmission device based on the engineering foundation of the HID device, with a maximum transmission rate of 480Mbps, to transmit the brain electrical signal data to the motor imagery learning module (5) and the motor imagery discrimination module (6);

[0045] The power supply circuit has an input voltage of 5V and is powered by the USB port on the PC. The voltage conversion module provides the operating voltage for different chips in the system without the need for an additional power adapter.

[0046] The independently developed high-precision 40-lead EEG acquisition equipment can obtain the user's EEG signals corresponding to the 40 electrodes of the brain electrode cap: FP1, FP2, AF7, AF3, AFz, AF4, AF8, F3, F7, Fz, F4, F8, FT7, FC3, FCz, FC4, FT8, T7, C3, Cz, C4, T8, TP7, CP3, CPZ, CP4, TP8, P7, P3, Pz, P4, P8, PO7, PO3, POz, PO4, PO8, O1, O2, Oz; the electrode distribution of the brain electrode cap conforms to the 10 / 20 international standard lead.

[0047] S120 , performing fast Fourier transform on the EEG signal, and then extracting theta, alpha, and beta frequency bands in the EEG spectrum as EEG features.

[0048] First, the collected EEG signals need to be preprocessed, downsampling the sampling frequency from 1000Hz to 250Hz to reduce the input dimensionality and improve algorithm analysis efficiency. The program then automatically calls the EEGLab toolbox in Matlab to apply a bandpass filter to retain the 0.2-50Hz signal data and remove signals in other frequency bands to reduce noise. Through EEGlab preprocessing and independent component correlation analysis (ICA), blink artifacts and oculomotor interference are removed from the EEG signals.

[0049] When a user engages in motor imagery, the θ, α, and β frequency bands of the brain's EEG signal contain the most characteristic information about motor imagery. First, a fast Fourier transform is performed on the preprocessed motor imagery EEG signal. Then, the θ, α, and β frequency bands in the EEG spectrum are extracted as EEG features for analysis.

[0050] S130 , projecting the positions of the 40-lead electrodes from the 3D space to a 2D plane.

[0051] To obtain a two-dimensional image of the brain's spatial activity, the positions of the 40 electrodes were projected from three-dimensional space onto a two-dimensional plane. This transformation preserves the relative spatial positions of adjacent electrodes.

[0052] Exemplarily, the projecting of the positions of the 40-lead electrodes from a 3D space to a 2D plane may include: establishing a 2D polar coordinate system, selecting a position on the spherical surface of the human brain as the coordinate origin of the polar coordinates; calculating the distances of other points on the spherical surface relative to the coordinate origin, and the angles relative to the coordinate origin; converting the current polar coordinate system into a Cartesian coordinate system, thereby obtaining a 2D projection of the human brain. First, a 2D polar coordinate system is established, and a position on the spherical surface is selected as the "center point" of the projection plane, i.e., the coordinate origin of the polar coordinates. The distances ρ (i.e., the arc length on the sphere) of other points on the earth relative to the coordinate origin, as well as the angles θ relative to the coordinate origin, are then calculated. Subsequently, the current polar coordinate system is converted into a Cartesian coordinate system to obtain a 2D projection of the human brain.

[0053] It should be noted that the above steps may also be performed before S120 without affecting the final motor imagery result determination.

[0054] S140 , normalizing the spectral power values corresponding to the electrodes and matching them with the electrode positions to obtain discrete images, and obtaining two-dimensional grayscale EEG signal images of three frequency bands of θ, α, and β based on the discrete images.

[0055] Exemplarily, it may include: normalizing in the following manner;

[0056]

[0057] t represents the spectrum power value, P t This is the normalized result;

[0058] The spectral power values corresponding to each electrode are interpolated to ultimately obtain a 32×32 image; two-dimensional grayscale EEG signal images of the three frequency bands of θ, α, and β are obtained based on the discrete images corresponding to the θ, α, and β electrodes. Optionally, the interpolation of the spectral power values corresponding to each electrode may include: triangulating the points of the electrode channel in the two-dimensional coordinate system to form a temporary triangular irregular network; defining a bivariate polynomial on each triangle to create a surface composed of a series of triangular surface patches, where the lines connecting the triangle vertices and the centroid divide the triangle into three sub-triangles, and the defined bivariate cubic polynomial is as follows:

[0059]

[0060] Among them, c ij The function value f and the first-order partial derivative f at each vertex x , f y , and the normal derivatives of the midpoints of the three sides A total of 12 parameters are determined; interpolation is performed based on the interpolation surface of the temporary triangle.

[0061] Optionally, after obtaining the two-dimensional plane projection of the EEG electrodes, the spectral power values corresponding to each electrode are normalized and matched with the electrode positions to obtain a discrete image. The normalization formula is as follows:

[0062]

[0063] t represents the spectrum power value, P t This is the normalized result. The normalization process in the above formula ensures that when the spectral power value is used to generate the corresponding pixel value, the value remains within the range of 0 to 255.

[0064] Figure 3 This is a schematic diagram of the minimum interpolation unit in the stroke hand rehabilitation training method provided in Example 1 of the present invention, as shown in FIG. Figure 3 As shown, the spectral power values on the scalp can be interpolated to finally obtain a 32×32 image. Before interpolation, the points of the electrode channels in the two-dimensional coordinate system are triangulated to form a temporary triangular irregular network. A bivariate polynomial is defined on each triangle to create a surface composed of a series of triangular surface patches. The lines connecting the triangle vertices and the centroid divide the triangle into three sub-triangles. The defined bivariate cubic polynomial is as follows:

[0065]

[0066] Among them, c ij The function value f and the first-order partial derivative f at each vertex x , f y , and the normal derivatives of the midpoints of the three sides These 12 parameters are determined.

[0067] S150, converting the two-dimensional grayscale EEG signal images of the three frequency bands of θ, α and β into two-dimensional color EEG signal images, and inputting the two-dimensional color EEG signal images into the trained motion signal convolutional neural network model.

[0068] Figure 4 This is a schematic diagram showing the process and results of converting EEG time domain signals into two-dimensional color images in the stroke hand rehabilitation training method provided in Example 1 of the present invention. Figure 4 Repeat the above image mapping process for the three different frequency bands of motor imagery, θ, α, and β, to obtain two-dimensional grayscale EEG signal images for the three frequency bands; then convert the two-dimensional grayscale EEG signal images for the three frequency bands into two-dimensional color EEG signal images. Alternatively, a single-frequency domain image can be treated as a grayscale image of a single channel. Images in multiple frequency domains can be treated as different RGB channels and mixed to obtain a color image. The resulting color image is input into the trained convolutional neural network model for motion signals.

[0069] S160, obtaining a motion imagery result output by the motion signal convolutional neural network model.

[0070] The trained motion signal convolutional neural network model is used to output the corresponding motion imagery discrimination results.

[0071] S170: guiding the hand rehabilitation device to perform rehabilitation treatment according to the motor imagery result.

[0072] Currently, the device supports nine rehabilitation movements: flexion, extension, two-finger pinch, three-finger pinch, thumb bend, index finger bend, middle finger bend, ring finger bend, and pinky finger bend. Each of these movements is accomplished by electrical stimulation and motor vibration of the corresponding finger joints, enabling movement guidance.

[0073] This embodiment obtains EEG signals from forty electrodes; performs fast Fourier transform on the EEG signals, and then extracts the θ, α, and β frequency bands in the EEG spectrum as EEG features; projects the positions of the 40-lead electrodes from a 3D space to a 2D plane; normalizes the spectral power values corresponding to the electrodes and matches them with the electrode positions to obtain discrete images, and obtains two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β based on the discrete images; converts the two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β into two-dimensional color EEG signal images, and inputs the two-dimensional color EEG signal images into a trained motion signal convolutional neural network model; obtains a motion imagery result output by the motion signal convolutional neural network model; and guides a hand rehabilitation device to perform rehabilitation treatment based on the motion imagery result. Using positional projection, electrodes at different locations can be projected onto a single plane. Normalization generates discrete images, and through image conversion, a color EEG signal image corresponding to the three key frequency bands is formed. This allows for the integration of multiple EEG signal features into a single image, facilitating identification by neural network models without missing important information. The identification results can be used to guide patients in rehabilitation training. This also facilitates the construction of convolutional neural network models for motion signals.

[0074] Example 2

[0075] Figure 5 A flowchart of a hand rehabilitation training method for stroke patients provided in Example 2 of the present invention. This embodiment is optimized based on the above embodiment. In this embodiment, the method may further include the following steps: constructing the motion signal convolutional neural network model using the color EEG signal image and the corresponding motion label.

[0076] Accordingly, the stroke hand rehabilitation training method provided in this embodiment specifically includes:

[0077] S210, constructing the motion signal convolutional neural network model using the color EEG signal image and the corresponding motion label.

[0078] Since the EEG signals of each patient vary greatly, it is not suitable to use a single motion signal convolutional neural network model to distinguish all patients. If it is used, large errors will occur, which will affect the effect of rehabilitation training. Therefore, in this embodiment, a corresponding motion signal convolutional neural network model is established for each patient. However, establishing a corresponding motion signal convolutional neural network model requires a lot of calculations and adjustments, which is time-consuming and labor-intensive, and cannot facilitate the rapid establishment of a corresponding motion signal convolutional neural network model for each patient.

[0079] Therefore, in this embodiment, the motion signal convolutional neural network model can be constructed by the color EEG signal image and the corresponding motion label. Exemplarily, the stored user's motor imagery data is first preprocessed, and the sampling frequency is downsampled from 1000Hz to 250Hz to reduce the input dimension and improve the efficiency of algorithm analysis. After that, the program will automatically call the bandpass filter of the EEGLab toolbox in Matlab to retain the signal data of 0.2-50Hz and remove other frequency band signals to reduce noise. Through the preprocessing of EEGlab, and using Independent Component Correlation Algorithm (ICA) to eliminate the flicker artifacts (Blink artifacts) and electrooculogram signal interference in the EEG signal. For the brain active area of human brain motor imagery, the main characteristic frequency band of the EEG signal, the corresponding feature extraction is performed on the obtained EEG data respectively. The 4s data obtained after the user's motor imagery starts is used as the judgment basis for the user's brain state at that time, and the corresponding action is used as the data category to label the EEG data.

[0080] The construction of the motion signal convolutional neural network model may include: constructing a network structure search space suitable for EEG signal analysis, where the network is composed of modular units connected in sequence; setting the learning rate to 0.1, performing 200 cycles of cyclic training, and a batch size of 128, to find the optimal convolutional neural network structure for the current user's different motor imagery, and obtaining a unique deep convolutional neural network model structure for each person; and sending each user's two-dimensional color EEG signal image into the unique deep convolutional neural network model structure in turn, using Pytorch to perform fully supervised training on this deep convolutional neural network model, setting the model learning rate to 0.01, performing 200 cycles of cyclic training, and a batch size of 128, to obtain a motion signal deep convolutional neural network model.

[0081] Figure 6 This is a diagram of the internal structure of the CELL in the stroke hand rehabilitation training method provided by Example 2 of the present invention. Figure 6 The modular unit structure is a directed acyclic graph with four nodes, where each node can only have two inputs and one output. Nodes within each unit can use the outputs of the previous two units as input, or they can use the outputs of different nodes within the same unit as input. Each input is processed by selecting an appropriate operation within a pre-defined operation space to obtain a feature map for the current node. Different colored lines in the directed acyclic graph represent different operations; not all operations are shown in the figure due to spatial constraints.

[0082] The search space for the following eight types of operations was set: depthwise-separable convolution with kernel sizes of 3×3 and 5×5, dilated convolution with kernel sizes of 3×3 and 5×5, max pooling with a size of 3×3, average pooling with a size of 3×3, skip connections to reduce network complexity, and no connections. The number of kernels for all convolution operations was set to 16.

[0083] When the network is finally constructed, it first processes the input image using a standard convolution operation. The inputs to the first and second units are then extracted from the output of this convolution. Once all units are stacked, the network is connected to a fully connected layer at the end for classification.

[0084] A network structure search algorithm based on reinforcement learning can be used. Specifically,

[0085] The network infrastructure can be constructed as a one-shot model. Figure 7 This is a schematic diagram of the basic architecture of the one-shot model in the stroke hand rehabilitation training method provided in the second embodiment of the present invention. Figure 7 As shown in Figure 2, in this model, all possible operations on each edge of the network are listed. In network structure search, the network graph containing all possible operation edges is called the parent network, and the subgraph finally searched is called the subnetwork.

[0086] And build an attention mechanism model based on the one-shot architecture, and establish the concept of network structure weight. Different from the neural transformation weights in traditional neural networks, the network structure weights are used to represent the importance of each operation, and the two exist simultaneously in the one-shot model. On this basis, the mother network is directly trained as a super network containing all possible structures, and the sub-networks are no longer trained separately. The structural weights and network neural transformation weights in the network are optimized at the same time. For example, 3×3 convolution, 5×5 convolution and maximum pooling are trained at the same time, and the sum of the outputs of the three is used as the output of the current module. The different structural operations are sorted during the training process. When the mother network training converges, the optimal network architecture can be obtained by selecting the largest one to retain according to the structural weights of each operation.

[0087] Specifically, the steps of the network structure search algorithm based on reinforcement learning can be as follows: In the network structure search algorithm, each change in the network structure can be regarded as a state transition in reinforcement learning, so the sampling process of a subnetwork can be transformed into:

[0088] p(τ)=Π t π(a t |s t )

[0089] p(τ) represents the probability distribution of the state sequence sampled by reinforcement learning under the sampling strategy π. This sequence is connected to form a sampling subnetwork.

[0090] By leveraging the policy gradient algorithm in reinforcement learning, the quality of the final network structure is used as the performance metric J(θ) of the policy gradient method. This eliminates the need for a black-box model and instead uses gradients as a guide. However, the sub-network accuracy is a constant and non-differentiable, so we discard it and instead use the differentiable sub-network loss as the performance metric for the network structure. Both network loss and network accuracy reflect the network's ultimate performance, so this change does not alter the optimization goal of "finding the optimal architecture."

[0091] According to the policy gradient algorithm optimization objective described below, the objective function of the network structure search algorithm based on policy gradient can be expressed as:

[0092]

[0093] Among them L ω That is, it represents the network loss with ω as the neural transformation weight.

[0094] After obtaining the final optimization goal of the policy gradient algorithm, it is necessary to replace the states and actions in reinforcement learning with the sampling process of the network structure. Since each sampling operation can be approximately considered independent of each other, the probability distribution p(τ) of the state sequence is replaced by:

[0095] p(τ)=Π i p(z i )

[0096] where Z i Represents the network structure encoding (one-hot encoding) obtained by sampling in step i. Here, Z is used to directly represent the entire structure encoding of the entire sub-network:

[0097] p(τ)=p(Z)

[0098] Therefore, the final policy gradient algorithm optimization objective expression is:

[0099]

[0100] In essence, the reinforcement learning strategy here is a probability distribution about the network structure, and θ is the constituent parameter of this probability distribution. The entire network search process uses this probability distribution to sample different sub-networks and continuously optimizes this probability distribution through the loss of the sub-networks to make it close to the optimal architecture.

[0101] The probability distribution of the network structure is a discrete distribution, so it is impossible to directly obtain the performance metric—the gradient of the sub-network loss function with respect to the probability distribution parameter θ. Therefore, a method is needed to sample the probability distribution without losing the gradient information of the probability distribution parameter θ. Here, we introduce the reparameterization method commonly used in reinforcement learning to obtain the gradient information of the parameters in the discrete distribution. The specific process is as follows:

[0102]

[0103] Among them, Z i,j It is a one-hot variable after serialization, representing the type of operation selected on the edge (i, j). i,j Represents the Z i,j Gumbel random variables of the same dimension are used as noise in the sampling process to ensure randomness. The generation process is shown in the following formula.

[0104] G i,j =-log(-log(U i,j ))

[0105] U i,j Is with Z i,jContinuous uniformly distributed random variables of the same dimension. λ is the softmax temperature. The purpose of this operation is to ensure that the gradient information of the discrete distribution parameter θ is not destroyed during the sampling process. However, since the Gumbel argmax operation is not differentiable, the Gumbel softmax is used to obtain the final sampling result, which is a smooth approximation of the one-hot vector. After such continuous sampling, the gradient information of the parameter θ in the discrete structural probability distribution can be obtained, and then the gradient of the performance metric with respect to the probability distribution parameter θ can be calculated. During the training process, the softmax temperature λ gradually tends from 1 to 0, and finally a fixed one-hot variable is obtained, which is the final result of the structure search.

[0106] Through the above derivation, we successfully combined the updating process of reinforcement learning strategy with the updating process of network neural transformation weights, making the two proceed synchronously. Figure 8 This is a schematic diagram of network gradient propagation in the stroke hand rehabilitation training method provided by the second embodiment of the present invention. Figure 8 shown.

[0107] Figure 9 This is a schematic diagram of the network basic unit CELL obtained by searching the final network structure in the stroke hand rehabilitation training method provided in the second embodiment of the present invention. Figure 9 As shown, the number of convolution kernels is 16, and each convolution kernel uses ReLU as the activation function. The network is composed of three basic units connected in series, and adaptive global average pooling is performed at the end. The pooling layer is followed by a fully connected layer, which uses ReLU as the activation function and the L2 norm as the regularization term, with the L2 norm set to 0.0009. The fully connected layer is followed by a classification layer, which outputs the recognition results of the motor imagery discrimination module. The classification layer outputs the recognition results of motor imagery for each of the nine motor imagery movements.

[0108] S220, obtaining EEG signals from forty electrodes.

[0109] S230 , performing fast Fourier transform on the EEG signal, and then extracting theta, alpha, and beta frequency bands in the EEG spectrum as EEG features.

[0110] S240, projecting the positions of the 40-lead electrodes from the 3D space to a 2D plane.

[0111] S250, normalizing the spectral power values corresponding to the electrodes and matching them with the electrode positions to obtain discrete images, and obtaining two-dimensional grayscale EEG signal images of the three frequency bands of θ, α and β based on the discrete images.

[0112] S260, converting the two-dimensional grayscale EEG signal images of the three frequency bands of θ, α and β into two-dimensional color EEG signal images, and inputting the two-dimensional color EEG signal images into the trained motion signal convolutional neural network model.

[0113] S270, obtaining the motion imagery result output by the motion signal convolutional neural network model.

[0114] S280: guiding the hand rehabilitation equipment to perform rehabilitation treatment according to the motor imagery result.

[0115] This embodiment adds the following steps: constructing the motion signal convolutional neural network model using the color EEG signal image and the corresponding motion labels. Using network structure search and an automatically optimized gradient descent algorithm, and leveraging image characteristics, the motion signal convolutional neural network model for different patients is rapidly constructed. This avoids the extensive work required to construct the motion signal convolutional neural network model.

[0116] Example 3

[0117] Figure 10 This is a structural diagram of a stroke hand rehabilitation training device provided in Example 3 of the present invention, as shown in FIG. Figure 10 As shown, the device includes:

[0118] An acquisition module 310 is used to acquire EEG signals from forty electrodes;

[0119] The module 320 is used to perform fast Fourier transform on the EEG signal, and then extract the θ, α, and β frequency bands in the EEG spectrum as EEG features;

[0120] A projection module 330 is used to project the positions of the 40-lead electrodes from a 3D space to a 2D plane;

[0121] Matching module 340, configured to normalize the spectral power values corresponding to each electrode and match them with the electrode positions to obtain discrete images, and to obtain two-dimensional grayscale EEG signal images in three frequency bands of θ, α, and β based on the discrete images;

[0122] a conversion module 350 for converting the two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β into two-dimensional color EEG signal images, and inputting the two-dimensional color EEG signal images into the trained motion signal convolutional neural network model;

[0123] An output module 360 is used to obtain a motor imagery result output by the motion signal convolutional neural network model;

[0124] The guidance module 370 is used to guide the hand rehabilitation device to perform rehabilitation treatment according to the motor imagery result.

[0125] The stroke hand rehabilitation training device provided in this embodiment obtains EEG signals from forty electrodes; performs fast Fourier transform on the EEG signals, and then extracts the θ, α, and β frequency bands in the EEG spectrum as EEG features; projects the positions of the 40-lead electrodes from a three-dimensional space to a two-dimensional plane; normalizes the spectral power values corresponding to each electrode and matches them with the electrode positions to obtain discrete images, and obtains two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β based on the discrete images; converts the two-dimensional grayscale EEG signal images in the three frequency bands of θ, α, and β into two-dimensional color EEG signal images, and inputs the two-dimensional color EEG signal images into a trained motion signal convolutional neural network model; obtains the motion imagery results output by the motion signal convolutional neural network model; and guides the hand rehabilitation device to perform rehabilitation treatment based on the motion imagery results. Using positional projection, electrodes at different locations can be projected onto a single plane. Normalization generates discrete images, and through image conversion, a color EEG signal image corresponding to the three key frequency bands is formed. This allows for the integration of multiple EEG signal features into a single image, facilitating identification by neural network models without missing important information. The identification results can be used to guide patients in rehabilitation training. This also facilitates the construction of convolutional neural network models for motion signals.

[0126] Based on the above embodiments, the segmentation module is used to:

[0127] The audio is divided into a number of audio segments of equal length.

[0128] Based on the above embodiments, the matching module includes:

[0129] A normalization unit, used for performing normalization in the following manner;

[0130]

[0131] t represents the spectrum power value, and P_t is the normalized result;

[0132] The interpolation unit is used to interpolate the spectrum power values corresponding to each electrode to finally obtain a 32×32 image;

[0133] The obtaining unit is used to obtain two-dimensional grayscale EEG signal images of three frequency bands of θ, α and β according to the discrete images corresponding to the θ, α and β electrodes.

[0134] Based on the above embodiments, the projection module includes:

[0135] An establishment unit is used to establish a two-dimensional polar coordinate system, and a position on the spherical surface of the human brain is selected as the coordinate origin of the polar coordinate;

[0136] A calculation unit, used to calculate the distance ρ of other points on the spherical surface relative to the coordinate origin, and the angle θ relative to the coordinate origin;

[0137] The conversion unit is used to convert the current polar coordinate system into a Cartesian coordinate system, that is, to obtain a two-dimensional projection of the human brain.

[0138] Based on the above embodiments, the device further includes:

[0139] A construction module, configured to construct the motion signal convolutional neural network model using the color EEG signal image and the corresponding motion label;

[0140] The building blocks are used to:

[0141] Construct a network structure search space suitable for EEG signal analysis, where the network consists of sequentially connected modular units;

[0142] The learning rate is set to 0.1, and a total of 200 cycles of cyclic training are performed with a batch size of 128. The optimal convolutional neural network structure for different motor imagery of the current user is found, and the unique deep convolutional neural network model structure of each person is obtained.

[0143] The two-dimensional color EEG signal image of each user is then fed into the unique deep convolutional neural network model structure in turn. This deep convolutional neural network model is fully supervised trained using Pytorch. The model learning rate is set to 0.01, and a total of 200 cycles of cyclic training are performed with a batch size of 128 to obtain a deep convolutional neural network model for motion signals.

[0144] The stroke hand rehabilitation training device provided by the embodiment of the present invention can execute the stroke hand rehabilitation training method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0145] Example 4

[0146] Figure 11 This is a structural diagram of the stroke hand rehabilitation training system provided by the fourth embodiment of the invention, see Figure 11 The stroke hand rehabilitation training system comprises:

[0147] Any one of the stroke hand rehabilitation training systems provided in the above embodiments;

[0148] High-precision 40-channel EEG acquisition equipment and hand rehabilitation equipment.

[0149] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A hand rehabilitation training method for stroke, characterized in that: include: Acquire EEG signals from forty electrodes; Performing fast Fourier transform on the EEG signal, and then extracting theta, alpha, and beta frequency bands in the EEG spectrum as EEG features; Project the positions of the 40-lead electrodes from 3D space to a 2D plane; Normalizing the spectral power values corresponding to each electrode and matching them with the electrode positions to obtain discrete images, and obtaining two-dimensional grayscale EEG signal images in three frequency bands of θ, α, and β based on the discrete images; Converting the two-dimensional grayscale EEG signal images of the three frequency bands of θ, α, and β into two-dimensional color EEG signal images, and inputting the two-dimensional color EEG signal images into the trained motion signal convolutional neural network model; Obtaining a motion imagery result output by the motion signal convolutional neural network model; guiding the hand rehabilitation device to perform rehabilitation treatment according to the motor imagery results; Projecting the positions of the 40-lead electrodes from the 3D space to the 2D plane includes: Establish a two-dimensional polar coordinate system and select a position on the human brain sphere as the origin of the polar coordinates; Calculate the distance ρ of other points on the sphere relative to the coordinate origin, and the angle θ relative to the coordinate origin; Convert the current polar coordinate system into the Cartesian coordinate system, that is, obtain the two-dimensional projection of the human brain; The spectral power values corresponding to the electrodes are normalized and matched with the electrode positions to obtain discrete images, and two-dimensional grayscale EEG signal images of three frequency bands of θ, α and β are obtained based on the discrete images, including: Normalization is performed in the following way; t represents the spectrum power value, P t This is the normalized result; The spectral power values corresponding to each electrode are interpolated to finally obtain a 32×32 image; According to the discrete images corresponding to the θ, α and β electrodes, two-dimensional grayscale EEG signal images of the three frequency bands of θ, α and β are obtained; The interpolating the spectrum power values corresponding to the electrodes includes: triangulate the points of the electrode channel in the two-dimensional coordinate system to form a temporary triangular irregular network; Define a bivariate polynomial on each triangle to create a surface consisting of a series of triangular patches. The lines connecting the triangle vertices and the centroid divide the triangle into three sub-triangles. The defined bivariate cubic polynomial is as follows: Among them, cij is composed of the function value f and the first-order partial derivative f of each vertex x , f y , and the normal derivatives of the midpoints of the three sides A total of 12 parameters are determined; Interpolate based on the interpolation surface of the temporary triangles.

2. The method according to claim 1, characterized in that The method further comprises: Constructing the motion signal convolutional neural network model using the color EEG signal image and the corresponding motion label; The constructing of the motion signal convolutional neural network model includes: Construct a network structure search space suitable for EEG signal analysis, where the network consists of sequentially connected modular units; The learning rate is set to 0.1, and a total of 200 cycles of cyclic training are performed with a batch size of 128. The optimal convolutional neural network structure for different motor imagery of the current user is found, and the unique deep convolutional neural network model structure of each person is obtained. The two-dimensional color EEG signal image of each user is then fed into the unique deep convolutional neural network model structure in turn. This deep convolutional neural network model is fully supervised trained using Pytorch. The model learning rate is set to 0.01, and a total of 200 cycles of cyclic training are performed with a batch size of 128 to obtain a deep convolutional neural network model for motion signals.

3. The method according to claim 2, characterized in that The modular unit structure is: A directed acyclic graph with 4 nodes, and each node can only have two inputs and one output. The nodes within each unit can use the outputs of the previous two units as input, or the outputs of different nodes within the same unit as input.

4. A hand rehabilitation training device for stroke, characterized in that: include: An acquisition module is used to acquire EEG signals from forty electrodes; A lifting module is used to perform fast Fourier transform on the EEG signal, and then extract the θ, α, and β frequency bands in the EEG spectrum as EEG features; A projection module, used to project the positions of the 40-lead electrodes from a 3D space to a 2D plane; A matching module is used to normalize the spectral power values corresponding to each electrode and match them with the electrode positions to obtain discrete images, and to obtain two-dimensional grayscale EEG signal images in three frequency bands of θ, α, and β based on the discrete images; A conversion module is used to convert the two-dimensional grayscale EEG signal images in the three frequency bands of θ, α and β into two-dimensional color EEG signal images, and input the two-dimensional color EEG signal images into the trained motion signal convolutional neural network model; An output module, configured to obtain a motor imagery result output by the motion signal convolutional neural network model; A guidance module, configured to guide the hand rehabilitation device to perform rehabilitation treatment according to the motor imagery result; The projection module includes: An establishment unit is used to establish a two-dimensional polar coordinate system, and a position on the spherical surface of the human brain is selected as the coordinate origin of the polar coordinate; A calculation unit, used to calculate the distance ρ of other points on the spherical surface relative to the coordinate origin, and the angle θ relative to the coordinate origin; A conversion unit, used to convert the current polar coordinate system into a Cartesian coordinate system, that is, to obtain a two-dimensional projection of the human brain; The matching module includes: A normalization unit, used for performing normalization in the following manner; t represents the spectrum power value, P t This is the normalized result; The interpolation unit is used to interpolate the spectrum power values corresponding to each electrode to finally obtain a 32×32 image; an obtaining unit for obtaining two-dimensional grayscale EEG signal images of three frequency bands of θ, α and β according to discrete images corresponding to the θ, α and β electrodes; The interpolation unit is used to: triangulate the points of the electrode channel in the two-dimensional coordinate system to form a temporary triangular irregular network; Define a bivariate polynomial on each triangle to create a surface consisting of a series of triangular patches. The lines connecting the triangle vertices and the centroid divide the triangle into three sub-triangles. The defined bivariate cubic polynomial is as follows: Among them, cij is composed of the function value f and the first-order partial derivative f of each vertex x , f y , and the normal derivatives of the midpoints of the three sides A total of 12 parameters are determined; Interpolate based on the interpolation surface of the temporary triangles.

5. The device according to claim 4, characterized in that The device further comprises: A construction module, configured to construct the motion signal convolutional neural network model using the color EEG signal image and the corresponding motion label; The building blocks are used to: Construct a network structure search space suitable for EEG signal analysis, where the network consists of sequentially connected modular units; The learning rate is set to 0.1, and a total of 200 cycles of cyclic training are performed with a batch size of 128. The optimal convolutional neural network structure for different motor imagery of the current user is found, and the unique deep convolutional neural network model structure of each person is obtained. The two-dimensional color EEG signal image of each user is then fed into the unique deep convolutional neural network model structure in turn. This deep convolutional neural network model is fully supervised trained using Pytorch. The model learning rate is set to 0.01, and a total of 200 cycles of cyclic training are performed with a batch size of 128 to obtain a deep convolutional neural network model for motion signals.

6. A stroke hand rehabilitation training system, characterized in that: include: The stroke hand rehabilitation training device according to any one of claims 4-5; High-precision 40-channel EEG acquisition equipment and hand rehabilitation equipment.

Images